Announcement

Collapse
No announcement yet.

Trend Micro crawling my forums?

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Trend Micro crawling my forums?

    This was mentioned previously but the discussion seemed to veer more towards permissions than why Trend Micro would be trying to crawl my forums. There are currently nine IP addresses registered to Trend Micro trying to access my forums. They can't see anything and get the permissions denied error but how can I make them stop trying?

  • #2
    I wouldn't worry about it honestly.. its just a spider.

    Comment


    • #3
      Originally posted by JamieFry View Post
      This was mentioned previously but the discussion seemed to veer more towards permissions than why Trend Micro would be trying to crawl my forums. There are currently nine IP addresses registered to Trend Micro trying to access my forums. They can't see anything and get the permissions denied error but how can I make them stop trying?
      If you areally dont want them to have access (and your forums run on *nix) add their IP to the htaccess file

      Comment


      • #4
        In this case when their are only 9 IP's crawling your probably right its not a bit issue.

        We just fixed a major slowdown caused by spiders. We were running 150-250 users and 200-400 guests. A bit a research revealed that a ton of the guests were from the same IP block and this block resolved to inktomisearch.com We blocked them via a robots.txt and our performance jumped and our guests total dropped to an average of 29.

        Comment


        • #5
          I have seen over and over that Yahoo (Inktomi) completely ignores the robots.txt directives that they recommend you use. They swarm over forums with no self-discipline or coordination. Eventually, I banned Yahoo/Inktomi from my website. But some people don't want to be unlisted from Yahoo.

          Comment


          • #6
            In may case they did completely obey the robots file.

            Comment


            • #7
              I banned all but Google from my forum.

              Comment


              • #8
                WHen I was looking intoi the "guests" issue I found one lonely google bot. He can stay he's well behaved.

                Comment


                • #9
                  I have robots.txt set to deny all spiders - my forums are completely closed to non-members. Trend Micro is impervious to robots.txt.

                  I banned the IP block through cPanel. They are gone. I guess I was really just trying to figure out why an anti-virus company would crawl my site. I read somewhere that the new anti-fraud toolbar associated with Trend Micro's latest software is the culprit.

                  Comment


                  • #10
                    robots.txt is a recommendation for search engines who like to play nice and respect these wishes, ... it isn't a rule set that can prevent access.. That's what .htaccess or firewalls or the tools from cpanel are for

                    Comment


                    • #11
                      Originally posted by Zachery View Post
                      I wouldn't worry about it honestly.. its just a spider.
                      I know I am bumping an old thread but these might be a lot more than just spiders.

                      http://www.projecthoneypot.org/ip_216.104.15.138

                      They have a lot of comment spam bots it seems.

                      216.104.15.138
                      216.104.15.142
                      216.104.15.130
                      216.104.15.134
                      150.70.84.151
                      150.70.84.24
                      150.70.84.152

                      Bad news
                      Last edited by jonsidneyb; Sun 5 Sep '10, 6:06pm.

                      Comment

                      widgetinstance 262 (Related Topics) skipped due to lack of content & hide_module_if_empty option.
                      Working...
                      X