Announcement

Collapse
No announcement yet.

Googlebots

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • [Forum] Googlebots

    How can I stop googlebots from trying to log into or look at my ftp?

    They keep getting blocked by my server, here is a couple of the block notices I get:

    66.249.72.78 # lfd: 66.249.72.78 (US/United States/crawl-66-249-72-78.googlebot.com), 1 distributed ftpd attacks on account [anonymous] in the last 300 secs - Sun Oct 2 11:52:00 2011
    66.249.68.169 # lfd: (ftpd) Failed FTP login from 66.249.68.169 (US/United States/crawl-66-249-68-169.googlebot.com): 1 in the last 300 secs - Tue Oct 4 06:29:51 2011
    My Mods also available at OzzModz.com

    New vBulletin Spider Definitions, vBulletin Spiders List Hits 1000 Spiders!
    Stop Spam Now The era of Big Spam is over.

    Vote to bring back hooks in vB5, http://tracker.vbulletin.com/browse/VBV-10333

  • #2
    You need to create a robots.txt file that can control search engine bots. Do a search here or at www.vbulletin.org for robots.txt for examples.
    Translations provided by Google.

    Wayne Luke
    The Rabid Badger - a vBulletin Cloud customization and demonstration site.
    vBulletin 5 Documentation - Updated every Friday. Report issues here.
    vBulletin 5 API - Full / Mobile
    I am not currently available for vB Messenger Chats.

    Comment


    • #3
      I have searched and see no way of blocking the ftp, as the ftp is not a file in the forum root, or part of the site index.
      My Mods also available at OzzModz.com

      New vBulletin Spider Definitions, vBulletin Spiders List Hits 1000 Spiders!
      Stop Spam Now The era of Big Spam is over.

      Vote to bring back hooks in vB5, http://tracker.vbulletin.com/browse/VBV-10333

      Comment


      • #4
        Googlebots won't be getting into your FTP. They are only interested in HTTP.
        Translations provided by Google.

        Wayne Luke
        The Rabid Badger - a vBulletin Cloud customization and demonstration site.
        vBulletin 5 Documentation - Updated every Friday. Report issues here.
        vBulletin 5 API - Full / Mobile
        I am not currently available for vB Messenger Chats.

        Comment


        • #5
          If that were the case how come I keep getting the csf blocks on them saying they are trying to access it?

          Here is a full list From Sept 21st to now.

          66.249.71.226 # lfd: 66.249.71.226 (US/United States/crawl-66-249-71-226.googlebot.com), 1 distributed ftpd attacks on account [anonymous] in the last 300 secs - Wed Sep 21 21:53:14 2011
          66.249.71.86 # lfd: 66.249.71.86 (US/United States/crawl-66-249-71-86.googlebot.com), 1 distributed ftpd attacks on account [anonymous] in the last 300 secs - Sun Sep 25 02:57:42 2011
          66.249.72.113 # lfd: (ftpd) Failed FTP login from 66.249.72.113 (US/United States/crawl-66-249-72-113.googlebot.com): 1 in the last 300 secs - Sun Sep 25 12:52:47 2011
          66.249.67.228 # lfd: (ftpd) Failed FTP login from 66.249.67.228 (US/United States/crawl-66-249-67-228.googlebot.com): 1 in the last 300 secs - Tue Sep 27 12:13:52 2011
          66.249.71.206 # lfd: (ftpd) Failed FTP login from 66.249.71.206 (US/United States/crawl-66-249-71-206.googlebot.com): 1 in the last 300 secs - Thu Sep 29 01:17:54 2011
          66.249.71.0/24 # lfd: (NETBLOCK) 66.249.71.0/24 has had more than 4 blocks in the last 86400 secs - Thu Sep 29 03:59:35 2011
          66.249.71.80 # lfd: (ftpd) Failed FTP login from 66.249.71.80 (US/United States/crawl-66-249-71-80.googlebot.com): 1 in the last 300 secs - Thu Sep 29 03:59:36 2011
          66.249.72.138 # lfd: (ftpd) Failed FTP login from 66.249.72.138 (US/United States/crawl-66-249-72-138.googlebot.com): 1 in the last 300 secs - Sun Oct 2 09:21:07 2011
          66.249.67.73 # lfd: (ftpd) Failed FTP login from 66.249.67.73 (US/United States/crawl-66-249-67-73.googlebot.com): 1 in the last 300 secs - Mon Oct 3 09:51:10 2011
          66.249.68.169 # lfd: (ftpd) Failed FTP login from 66.249.68.169 (US/United States/crawl-66-249-68-169.googlebot.com): 1 in the last 300 secs - Mon Oct 3 08:58:09 2011
          66.249.72.0/24 # lfd: (NETBLOCK) 66.249.72.0/24 has had more than 4 blocks in the last 86400 secs - Sun Oct 2 11:52:00 2011
          66.249.72.78 # lfd: 66.249.72.78 (US/United States/crawl-66-249-72-78.googlebot.com), 1 distributed ftpd attacks on account [anonymous] in the last 300 secs - Sun Oct 2 11:52:00 2011
          66.249.68.169 # lfd: (ftpd) Failed FTP login from 66.249.68.169 (US/United States/crawl-66-249-68-169.googlebot.com): 1 in the last 300 secs - Tue Oct 4 06:29:51 2011
          My Mods also available at OzzModz.com

          New vBulletin Spider Definitions, vBulletin Spiders List Hits 1000 Spiders!
          Stop Spam Now The era of Big Spam is over.

          Vote to bring back hooks in vB5, http://tracker.vbulletin.com/browse/VBV-10333

          Comment


          • #6
            Yeah.. Most likely someone posing as Googlebot. You'll need to contact your hosting company for support with FTP as it isn't related to vBulletin.
            Translations provided by Google.

            Wayne Luke
            The Rabid Badger - a vBulletin Cloud customization and demonstration site.
            vBulletin 5 Documentation - Updated every Friday. Report issues here.
            vBulletin 5 API - Full / Mobile
            I am not currently available for vB Messenger Chats.

            Comment


            • #7
              I don't believe it is someone posing as googlebot as the ip addy's resolve back to google, I guess I will have to some more research on the issue.

              Thanks for your time Wayne!
              My Mods also available at OzzModz.com

              New vBulletin Spider Definitions, vBulletin Spiders List Hits 1000 Spiders!
              Stop Spam Now The era of Big Spam is over.

              Vote to bring back hooks in vB5, http://tracker.vbulletin.com/browse/VBV-10333

              Comment


              • #8
                I'm having the exact same problem and my firewall is setup to instantly ban on wrong login so have to remove them from my banlist all the time

                Time: Thu Dec 15 16:42:04 2011 +0100
                IP: 66.249.71.217 (US/United States/crawl-66-249-71-217.googlebot.com)
                Failures: 1 (ftpd)
                Interval: 300 seconds
                Blocked: Permanent Block

                Log entries:

                Dec 15 16:42:03 server2 pure-ftpd: ([email protected]) [WARNING] Authentication failed for user [anonymous]
                Time: Thu Dec 15 16:41:49 2011 +0100
                IP: 66.249.71.152 (US/United States/crawl-66-249-71-152.googlebot.com)
                Failures: 1 (ftpd)
                Interval: 300 seconds
                Blocked: Permanent Block

                Log entries:

                Dec 15 16:41:49 server2 pure-ftpd: ([email protected]) [WARNING] Authentication failed for user [anonymous]
                Sat-Television.com Forum:
                http://www.satfriends.com/

                Sat-Television.com Upload Center:
                http://www.satfriends.com/ulc.php

                Comment


                • #9
                  Well googlebot won't be trying to log into your FTP unless you have a link to it somewhere on a webpage. FTP access is not relevant to vBulletin support. You should contact your hosting company about switching to a secure form of FTP.
                  Translations provided by Google.

                  Wayne Luke
                  The Rabid Badger - a vBulletin Cloud customization and demonstration site.
                  vBulletin 5 Documentation - Updated every Friday. Report issues here.
                  vBulletin 5 API - Full / Mobile
                  I am not currently available for vB Messenger Chats.

                  Comment


                  • #10
                    We RARELY use ftp on our site so we disable it when we are not using it which plugs that possible security hole.

                    So consider disabling ftp when you are not using it if that is a concern for you.

                    -- Rik

                    Comment


                    • #11
                      I would really prefer not to disable ftp. I have no problem with googlebot attempting to crawl whatever it wants, but once csf bans googlebot, it can't crawl my actual forum, which is a problem.

                      Comment

                      widgetinstance 262 (Related Topics) skipped due to lack of content & hide_module_if_empty option.
                      Working...
                      X