No announcement yet.

Yahoo Slurp Spiders

  • Filter
  • Time
  • Show
Clear All
new posts

  • Yahoo Slurp Spiders

    How is it that 20-40 different yahoo slurp spiders can crawl my threads or thread content when it's disabled for guests(non-members)viewing...
    This just doesn't make sense to me. I even have my search friendly archive disabled. Yet when I look in who's online, there they are, albeit in front of stop signs but still the thread title is listed as if they're viewing it....??
    When I log off of my forum and try to click on one of the boards, I can't get in or see anything, as it should be per my setup and I only get a members only message.....How is it that thread titles are listed next to the slurp spiders in users online......?? Most perplexing
    Last edited by Herc; Wed 11 Oct '06, 10:54pm.

  • #2
    That sounds normal. Spiders are guests, your guests don't have access, and the spiders are listed with stop signs indicating exactly that. Spiders are good at finding links, but they are subject to the same permissions as guests.


    • #3
      Thanks, Jake..


      • #4
        One of my mods just noted the following message with a spider:

        Yahoo! Slurp Spider Modifying Password

        What does THAT mean? This is a first time for this that we're aware of. Any ideas?



        • #5
          Like Jake said, Its good at finding links its probabily found profile.php?do=editpassword
          It's harmless I'v seen them playing Pacman on my site :P


          widgetinstance 262 (Related Topics) skipped due to lack of content & hide_module_if_empty option.