Announcement

Collapse
No announcement yet.

[howto] Make your forum search-engine spiderable

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • [howto] Make your forum search-engine spiderable

    Mods, I'm posting this here because it doesn't really seem to fit in any of the other forums. I'm hoping it can be moved to vB FAQ.

    I got this idea from fastforward's hack at vB.org. This does not modify your actual PHP files though. The only downside of this hack is that WOL will just show a bunch of Unknown Location links when users are viewing a forum or viewing a thread. All other aspects of WOL will still function.

    Also different to fastfoward's release is that (1) this one works (I never got his working successfully), and (2) broken images are a nonissue as this doesn't emulate directories, just a ton of flat files.

    Requirements: Apache with mod_rewrite enabled (if your phpinfo page contains the text "mod_rewrite" then it's installed)
    Time required: 20 minutes, plus backup time

    If you mess up you can grandly break your forums! Do a database backup first, and if it doesn't work, revert the backup.

    1. Turn your forums off in the admin CP
    2. Back up your database (seriously)
    3. Download the attached .htaccess file. Please don't quote its contents; I don't want unlicensed users to be able to see its contents.
    4. Upload it to your forums' root directory. If you already have a .htaccess file there, just append this one's contents to the existing one.
    5. In the templates forumhome_forumbit_level1_nopost, forumhome_forumbit_level1_post, forumhome_forumbit_level2_nopost, forumhome_forumbit_level2_post, forumdisplay_forumbit_level1_nopost, forumdisplay_forumbit_level1_post, forumdisplay_forumbit_level2_nopost, and forumdisplay_forumbit_level2_post (eight in total), replace:
    Code:
    <a href="forumdisplay.php?s=$session[sessionhash]&forumid=$forum[forumid]">
    With:
    Code:
    <a href="$bburl/forumid$forum[forumid]session$flathash.html">
    7. In forumdisplaybit, replace:
    Code:
    <a href="showthread.php?s=$session[sessionhash]&threadid=$thread[threadid]">
    with:
    Code:
    <a href="$bburl/threadid$thread[threadid]session$flathash.html">
    8. In your phpinclude template, add the following:
    PHP Code:
    $useflathashes 0// Change to 1 to turn on
    if ($useflathashes)
    {
        
    $flathash $session[sessionhash];

    This disables sessionhashes in the flat URLs. This will cause problems for a few people; namely those behind a proxy, but Google won't index your site unless sessionhashes are gone.
    9. Reopen your forums in the admin CP. If all goes well, links like showthread.php?s=sessionhash&threadid=threadid will be equivalent to threadidthreaidsessionsessionhash.html. Note that these .html files don't actually exist! They are effectively aliases to the proper pages.
    10. Submit your forums to Google and wait for Mr. Bot to come-a-knocking!
    Last edited by filburt1; Sat 19 Oct '02, 8:31am.
    --filburt1, vBulletin.org/vBulletinTemplates.com moderator
    Web Design Forums.net: vB Board of the Month
    vBulletin Mail System (vBMS): webmail for your forum users

  • #2
    Re: [howto] Make your forum search-engine spiderable

    I actually run an extended version of this that encompasses my whole site which involves hacking most of my PHP files. I have removed sessionhash altogether though. My site appears to work fine without it. I'll PM you the URL to my site for you to have a look.

    Edited: Thanks Filburt for answering my question.
    Last edited by Erwin; Mon 14 Oct '02, 5:35pm.
    Avatar Chat

    Comment


    • #3
      Replied.

      WOOHOO!

      Mr. Googlebot is back and this time going into the individual forums!
      --filburt1, vBulletin.org/vBulletinTemplates.com moderator
      Web Design Forums.net: vB Board of the Month
      vBulletin Mail System (vBMS): webmail for your forum users

      Comment


      • #4
        Nice hack or "template modification" filburt... very nice indeed! I will see about installing it on my forums later on today
        -
        Visit the Web Scripts Directory @ http://www.scriptz.com
        -
        PHP, CGI, Perl, ASP, JavaScript, CFML, Python and more!

        -

        Comment


        • #5
          Updated to include a way to turn off sessionhashes for the flat URLs
          --filburt1, vBulletin.org/vBulletinTemplates.com moderator
          Web Design Forums.net: vB Board of the Month
          vBulletin Mail System (vBMS): webmail for your forum users

          Comment


          • #6
            Hi filburt,

            that sounds really good, since I tried to get fastforwards hack running, too - no chance.

            I have a notice from my hoster, who says I need RewriteCond(itions) - without that conditions the RewriteEngine won't work with their servers.
            So... do you have the appropriate RewriteCond for your Rewrites, so that I can use this?
            Thanks,
            -Tom
            http://www.mcseboard.de/images/buttons/lastpost.gif www.MCSEboard.de
            German Windows Server & IT Pro Community dedicated to Windows Client & Server Systems. MVPs inside

            Comment


            • #7
              It took me two hours to learn regexs and how to use mod_rewrite, so I can't say that I know how to use RewriteCond yet
              --filburt1, vBulletin.org/vBulletinTemplates.com moderator
              Web Design Forums.net: vB Board of the Month
              vBulletin Mail System (vBMS): webmail for your forum users

              Comment


              • #8
                Okay, fair enough
                Will see, if I can get it working...
                thx,
                -Tom
                http://www.mcseboard.de/images/buttons/lastpost.gif www.MCSEboard.de
                German Windows Server & IT Pro Community dedicated to Windows Client & Server Systems. MVPs inside

                Comment


                • #9
                  Re: [howto] Make your forum search-engine spiderable

                  Originally posted by filburt1
                  The only downside of this hack is that WOL will just show a bunch of Unknown Location links when users are viewing a forum or viewing a thread.
                  Originally posted by fastforward at vB.org
                  easypeasy
                  In online.php : Change the 'default' block of the CASE statement on or around line 626 to look like this:

                  PHP Code:
                  default:
                      if (
                  preg_match ("/t([0-9]+)\.html/"$filename$match)) {
                         
                  $userinfo[threadid] = $match[1];
                         
                  $userinfo[activity] = 'showthread';
                         
                  $threadids .= ",$userinfo[threadid]";
                      } elseif (
                  preg_match ("/f([0-9]+)/"$filename$match)) {
                         
                  $forumid $match[1];
                         
                  $forumids .= ",$forumid";
                         
                  $userinfo[activity] = 'forumdisplay';
                         
                  $userinfo[forumid] = $forumid;
                      } elseif (
                  preg_match ("/robots\.txt/"$filename)) {
                         
                  $userinfo[activity] = 'spider';
                      } else {
                        
                  $userinfo[activity] = 'unknown';
                      }
                    } 
                  You're my Prince of Peace
                  And I will live my life for You

                  Comment


                  • #10
                    Ah, but that would be hacking
                    --filburt1, vBulletin.org/vBulletinTemplates.com moderator
                    Web Design Forums.net: vB Board of the Month
                    vBulletin Mail System (vBMS): webmail for your forum users

                    Comment


                    • #11
                      Oops! Busted!
                      You're my Prince of Peace
                      And I will live my life for You

                      Comment


                      • #12
                        Originally posted by filburt1
                        Updated to include a way to turn off sessionhashes for the flat URLs
                        By doing so, will this allow users behind Proxy servers to visit the site properly?

                        I really want to install this update, but I'm worried that at least a good % of my possible users will get lock out becasue they come from behind a proxy server (i.e. bussiness users)

                        Thanks!!!!!!!!!!!!!!

                        Comment


                        • #13
                          You'll have to turn on $useflathashes to 1 but it's iffy as to whether bots will crawl the site.
                          --filburt1, vBulletin.org/vBulletinTemplates.com moderator
                          Web Design Forums.net: vB Board of the Month
                          vBulletin Mail System (vBMS): webmail for your forum users

                          Comment


                          • #14
                            Maybe the VB Team should get clever and include this option in vB3, or even better, figure out a way to make it crawlable without causing the drawbacks! ::drools:: would be so sweet.
                            Gaming Forums: Necrotalk.com | TorBoard.com NEW!

                            Comment


                            • #15
                              Originally posted by nocturnix
                              Maybe the VB Team should get clever and include this option in vB3, or even better, figure out a way to make it crawlable without causing the drawbacks! ::drools:: would be so sweet.
                              There is a way to make it crawlable without "drawbacks". It's called mod_Rewrite. But the vb developers don't want to use it cause it doesn't work on all servers (only apache). I've been considering releasing a complete spiderable hack (or template mod) for vbulletin v2, but since vb3 is almost here, i've decided to wait. Once vb3 comes out i'll see about doing it and releasing it
                              -
                              Visit the Web Scripts Directory @ http://www.scriptz.com
                              -
                              PHP, CGI, Perl, ASP, JavaScript, CFML, Python and more!

                              -

                              Comment

                              widgetinstance 262 (Related Topics) skipped due to lack of content & hide_module_if_empty option.
                              Working...
                              X