Announcement

Collapse
No announcement yet.

[howto] Make your forum search-engine spiderable

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • #16
    BTW filburt, has google spidered your site yet?
    -
    Visit the Web Scripts Directory @ http://www.scriptz.com
    -
    PHP, CGI, Perl, ASP, JavaScript, CFML, Python and more!

    -

    Comment


    • #17
      It has crawled threads but oddly enough they don't show up when I search...
      --filburt1, vBulletin.org/vBulletinTemplates.com moderator
      Web Design Forums.net: vB Board of the Month
      vBulletin Mail System (vBMS): webmail for your forum users

      Comment


      • #18
        Originally posted by nocturnix
        Maybe the VB Team should get clever and include this option in vB3, or even better, figure out a way to make it crawlable without causing the drawbacks! ::drools:: would be so sweet.
        vBulletin v3 has an 'archive' feature, which makes your site perfectly crawlable.

        Comment


        • #19
          I get the 403 "Forbidden" message from Apache when I upload the .htaccess file. Does anyone know what might be going on? I'm running the site on my own dedicated server.
          Last edited by LanciaStratos; Sun 1 Dec '02, 5:05pm. Reason: Just wanted to include the dedicated server bit...

          Comment


          • #20
            Ah, never mind, I figured out what my problem was (just had to add Options +FollowSymlinks to the top of my .htaccess file). This is a great modification, filburt, thank you! I do have one question, though - why is there a need to include the sessionid in the URL if it is disabled?

            Comment


            • #21
              I first got the 403 error. Then, after reading your post, I added "Options +FollowSymLinks" to .htaccess. I then got 404 "File not found" errors. The RewriteEngine is loaded in Apache, so I tried everything else I could think of, fiddling with httpd.conf, restarting httpd, searching other support boards for possible solutions, reverting and then reediting the templates. I HATE defeat, but after spending two hours in the late afternoon (with 150-200 "current" users showing), I figured I better retreat to fight another day, reverting the templates and reopening the board.

              Anybody got any ideas? This is an Ensim server, which might have something to do with the problem.

              Comment


              • #22
                Actually, one of my moderators just complained about not being able to merge two threads, getting a "You have not specified a valid URL for the thread that you want to merge" error. Turned out I had forgotten to revert the forumdisplaybit template, and the .html symbolic links were working for threads. I hadn't been able to get past the forum listing without a 404 when I had all the templates modified. So the problem may not have been the .htaccess file.

                Nevertheless, even if I figure it out, moderators not being able to merge the symbolic linked threads could be a problem.

                Comment


                • #23
                  Originally posted by Don Black
                  Actually, one of my moderators just complained about not being able to merge two threads, getting a "You have not specified a valid URL for the thread that you want to merge" error. Turned out I had forgotten to revert the forumdisplaybit template, and the .html symbolic links were working for threads. I hadn't been able to get past the forum listing without a 404 when I had all the templates modified. So the problem may not have been the .htaccess file.

                  Nevertheless, even if I figure it out, moderators not being able to merge the symbolic linked threads could be a problem.
                  That is a good point, one which I don't have a workaround for yet.
                  --filburt1, vBulletin.org/vBulletinTemplates.com moderator
                  Web Design Forums.net: vB Board of the Month
                  vBulletin Mail System (vBMS): webmail for your forum users

                  Comment


                  • #24
                    Originally posted by Don Black
                    Actually, one of my moderators just complained about not being able to merge two threads, getting a "You have not specified a valid URL for the thread that you want to merge" error. Turned out I had forgotten to revert the forumdisplaybit template, and the .html symbolic links were working for threads. I hadn't been able to get past the forum listing without a 404 when I had all the templates modified. So the problem may not have been the .htaccess file.

                    Nevertheless, even if I figure it out, moderators not being able to merge the symbolic linked threads could be a problem.
                    Don, have you checked to make sure mod_rewrite is enabled? I'm running an Ensim server as well, and the Options +FollowSymlinks at the beginning of my .htaccess file was all that was needed.

                    As for thread merging, although I don't have a technical solution, you can still do it and your moderators should easily be able to as well. Teach them how to identify and remember/copy the threadid, and then when you are asked for the URL of the thread you want to merge into, simply enter threadid=# (keeping in mind that the # sign will be the threadid number). That's all vB looks for in the form, so that's just a low-tech workaround that might help you out.

                    Comment


                    • #25
                      Filburt, I've been playing around with the htaccess file and I've got a few questions:

                      1. Would removing the sessionid text and (.*) from the URL harm anything? If they are disabled in the phpinclude, I can't figure out why they still need to be mentioned in the URL.

                      2. Multiple-paged threads and forumdisplays, beyond the first page, are not affected by this modification. Although I've found a semi-fix for threads, which are the most important, will Google still go through the multiple forumdisplay pages to spider all the .html threads it finds, or will it just stop and only crawl the first page?

                      Comment


                      • #26
                        Originally posted by LanciaStratos
                        Filburt, I've been playing around with the htaccess file and I've got a few questions:

                        1. Would removing the sessionid text and (.*) from the URL harm anything? If they are disabled in the phpinclude, I can't figure out why they still need to be mentioned in the URL.

                        2. Multiple-paged threads and forumdisplays, beyond the first page, are not affected by this modification. Although I've found a semi-fix for threads, which are the most important, will Google still go through the multiple forumdisplay pages to spider all the .html threads it finds, or will it just stop and only crawl the first page?
                        1. You'd have to modify the other rewrite stuff. I included the phpinclude switch to make it easier.
                        2. It will only crawl the first page at present though I'm working on a way for it to go through all threads
                        --filburt1, vBulletin.org/vBulletinTemplates.com moderator
                        Web Design Forums.net: vB Board of the Month
                        vBulletin Mail System (vBMS): webmail for your forum users

                        Comment


                        • #27
                          Originally posted by filburt1
                          1. You'd have to modify the other rewrite stuff. I included the phpinclude switch to make it easier.
                          I see, thanks!

                          2. It will only crawl the first page at present though I'm working on a way for it to go through all threads
                          Sweet! I'm guessing that it doesn't involve modification of the Page Navigation templates...? (Because they are universally used all over the place like forums, threads, memberlist, and who knows where else!)

                          Comment


                          • #28
                            Originally posted by LanciaStratos
                            I see, thanks!

                            Sweet! I'm guessing that it doesn't involve modification of the Page Navigation templates...? (Because they are universally used all over the place like forums, threads, memberlist, and who knows where else!)
                            No, the pagenavs will have to be modified to reflect the new flat URLs, although I haven't looked at that specifically, yet. I'm going to wait until January 1 because I can't modify my forums until the BOTM contest is over.
                            --filburt1, vBulletin.org/vBulletinTemplates.com moderator
                            Web Design Forums.net: vB Board of the Month
                            vBulletin Mail System (vBMS): webmail for your forum users

                            Comment


                            • #29
                              Originally posted by filburt1
                              No, the pagenavs will have to be modified to reflect the new flat URLs, although I haven't looked at that specifically, yet. I'm going to wait until January 1 because I can't modify my forums until the BOTM contest is over.
                              Well, I've spent nearly all day playing around with the Page Navigation templates (just for multiple paged threads) and I have yet to discover a practical solution. I modified the htaccess to include the &pagenumber variable, and it works if you type in the URL directly, but it's impossible to edit the links into the templates to work with each unique threadid because you can't insert $thread[threadid] into the page nav templates (I should not that it is possible, but only by inserting a / into the flat URL and tricking the system into thinking it is a directory. However, that breaks virtually every link and image on the page.).

                              I think the secret lies in the $address variable inside the page nav templates, but I can't find it anywhere in the template set or in any of the PHP scripts.

                              On a side note, I'm playing around with this using a different template set so it won't affect my board. You might want to do that too while your board is under review (good luck, btw)!

                              Comment


                              • #30
                                can you paste the code for the .htaccess file, because attatchment doesnt work, then i can just create it, thanks...

                                Comment

                                widgetinstance 262 (Related Topics) skipped due to lack of content & hide_module_if_empty option.
                                Working...
                                X