Announcement

Collapse
No announcement yet.

Request for an official thread/resource for Web Spiders/Bots list.

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Request for an official thread/resource for Web Spiders/Bots list.

    As the question says.
    TechArena - TechArena Community - News - Download - Video - Guide - Review

  • #2
    Have you tried this thread:

    http://www.vbulletin.com/forum/showt...xml-quot-files)

    While it's not official, there's a fairly comprehensive list in there...
    Vote for:

    - *Admin Settable Paid Subscription Reminder Timeframe*
    -
    *PM - Add ability to reply to originator only*
    - Add Admin ability to auto-subscribe users to specific channel(s)
    - "Quick Route" Interface...

    Comment


    • #3
      You would have to use the list that Trevor pointed out. We do not intend nor did we ever intend to update or maintain this beyond the default list provided in the software download. Moving the list to XML was done at the behest of the community for community maintenance and expansion.
      Translations provided by Google.

      Wayne Luke
      The Rabid Badger - a vBulletin Cloud demonstration site.
      vBulletin 5 API - Full / Mobile
      Vote for your favorite feature requests and the bugs you want to see fixed.

      Comment


      • #4
        Can you explain why you will not update the default xml in vbulletin to the latest spider definitions?

        I always forget this with upgrades and that thread is not always easily found. I dont understand why IB doesnt save us from the hassle.
        I buy 420 forums

        Comment


        • #5
          Originally posted by Alfa1 View Post
          Can you explain why you will not update the default xml in vbulletin to the latest spider definitions?
          Because it was never our intention to track every obscure spider on the internet but the most common search engines. Maintaining, updating and verifying such a list would be a full time job for at least one person. The benefits do not justify the costs of doing so.
          Translations provided by Google.

          Wayne Luke
          The Rabid Badger - a vBulletin Cloud demonstration site.
          vBulletin 5 API - Full / Mobile
          Vote for your favorite feature requests and the bugs you want to see fixed.

          Comment


          • #6
            Originally posted by Wayne Luke View Post
            Because it was never our intention to track every obscure spider on the internet but the most common search engines. Maintaining, updating and verifying such a list would be a full time job for at least one person. The benefits do not justify the costs of doing so.
            Wayne, you are absolutely correct here.
            But VB team can atleast plan to use the list of bots identified by VB product users(end-users can contribute to same), VB team will only require to compile this list @ add the same to every new product release.

            This will just enhance the product, as no other forum software provides this feature..
            TechArena - TechArena Community - News - Download - Video - Guide - Review

            Comment


            • #7
              Originally posted by SaN-DeeP View Post
              Wayne, you are absolutely correct here.
              But VB team can atleast plan to use the list of bots identified by VB product users(end-users can contribute to same), VB team will only require to compile this list @ add the same to every new product release.

              This will just enhance the product, as no other forum software provides this feature..
              support.

              also, I hope this gets added soon: http://tracker.vbulletin.com/browse/VBIV-6601

              Comment


              • #8
                Originally posted by Wayne Luke View Post
                Because it was never our intention to track every obscure spider on the internet but the most common search engines. Maintaining, updating and verifying such a list would be a full time job for at least one person. The benefits do not justify the costs of doing so.
                You dont need to track them. Thats already being done for you. You could just implement the spider listing from the thread you link to. That shouldn't be much effort. There are volunteers for this and if needed more volunteers could be found.
                I buy 420 forums

                Comment


                • #9
                  Any reason you just don't chown/chmod the file so your ftp user can't overwrite it?

                  Comment

                  widgetinstance 262 (Related Topics) skipped due to lack of content & hide_module_if_empty option.
                  Working...
                  X