Help Needed Quickly Please - Host Change

[lnxcode]

Mike,

Look inside your public_html and you will see a file called .htaccess, all you need to do is:

User-agent: *
Disallow: /

It's mentioned more here:

http://www.hosticananswers.com/quest...7B47%7DFolders

[encryption]

I've never found the need to do it so I can't accurately answer that.

Sorry.

EDIT: I imagine that since these spiders are assigned a range of IPs within a subnet, you could technically block out the whole subnet or that IP Block and get a similar result.

[thesportszone]

Yea ask them to set up the .htaccess page for you and see if that helps your load. However that should still not be the way to go (imo of course).

well I have read that you can set it up to block certain IP address's but there is no way I can block all slurp spider ip's correct?

[encryption]

Yea ask them to set up the .htaccess page for you and see if that helps your load. However that should still not be the way to go (imo of course).

[thesportszone]

You probably just have to ask them, that's all. It's not something that would necessarily be done automatically, other than maybe to add a host's specific requirements for a customer.

Peace,
Gene

I will ask them and see what they say.

Thanks

[thesportszone]

If the .htaccess file has been setup by your webhost, check http://the-sportszone.net/forum/online.php and see the tail ending pages if you see any bots on there (I cant see it cause guest access to that page is disabled)

Oh that page yes thats how I know there are so many bots.

lets see if I can paste this here:

Yahoo! Slurp Spider
Viewing Thread
Week 14 Predictions
74.6.20.34
11:18 AM Google Spider
Viewing Archives
66.249.72.68
11:10 AM Guest
Viewing Forum
NFL - National Football League
67.169.217.222
11:20 AM Yahoo! Slurp Spider
Viewing Thread
2007 National League Pennant
74.6.23.164
11:11 AM Yahoo! Slurp Spider
Viewing Thread
Will Bonds pass Ruth? Aaron?
74.6.25.12
11:19 AM Guest
Viewing Who's Online
12.147.199.226
11:06 AM Guest
Viewing Thread
Take the Wonderlic Test For Yourself
143.250.2.10
11:17 AM Yahoo! Slurp Spider
Viewing Thread
Cars and your systems
74.6.23.73
11:13 AM Yahoo! Slurp Spider
Viewing Thread
TSZ Treasure Island Trivia
74.6.22.38
11:09 AM Yahoo! Slurp Spider
Viewing Thread
What makes the Bengals a non contender
74.6.28.140
11:15 AM Yahoo! Slurp Spider
Viewing Thread
What was the Last Thing You bought?
74.6.22.30
11:06 AM Yahoo! Slurp Spider
Viewing Thread
NFL News Roundup: Vikes extend win streak to four
74.6.27.116
11:14 AM Yahoo! Slurp Spider
Viewing Thread
hi
74.6.74.154
11:10 AM Yahoo! Slurp Spider
Viewing Thread
Boxing News: Taylor barely wins decision over Hopkins
74.6.73.229
11:20 AM Yahoo! Slurp Spider
Viewing Forum
Fantasy Hockey
74.6.27.45
11:16 AM Yahoo! Slurp Spider
Viewing Thread
Mike Vick, Really?
74.6.70.84
11:12 AM Yahoo! Slurp Spider
Viewing Thread
DosChico in the house
74.6.22.19
11:07 AM Yahoo! Slurp Spider
Viewing Thread
TSZ quick promo on PowerHouseRock.net
74.6.26.224
11:18 AM Yahoo! Slurp Spider
Viewing Thread
Post Milestones
74.6.24.122
11:10 AM Yahoo! Slurp Spider
Viewing Thread
74.6.25.141
11:20 AM Google AdSense Spider
Viewing Thread
Patrick Leaving ESPN
66.249.72.68
11:20 AM Yahoo! Slurp Spider
Viewing Thread
Top 10 Signings of 2007 Free Agency
74.6.70.85
11:12 AM Yahoo! Slurp Spider
Viewing Thread
NBA Vote Off Game
74.6.70.238
11:19 AM Yahoo! Slurp Spider
Viewing Thread
NFL network
74.6.70.233
11:15 AM Yahoo! Slurp Spider
Viewing Thread
Southern Hip Hop Discussion
74.6.20.168
11:20 AM Yahoo! Slurp Spider
Viewing Thread
Dynasty Baseball I League
74.6.19.211

[Gene Steinberg]

Not that I was told maybe they did and just didnt tell me?

You probably just have to ask them, that's all. It's not something that would necessarily be done automatically, other than maybe to add a host's specific requirements for a customer.

Peace,
Gene

[encryption]

If the .htaccess file has been setup by your webhost, check http://the-sportszone.net/forum/online.php and see the tail ending pages if you see any bots on there (I cant see it cause guest access to that page is disabled)

[thesportszone]

lol I just read that your host is hostican... didn't they offer to setup the .htaccess file for you ?

EDIT: If you're stuck on staying where you are, the only other drastic and undesirable thing you can do is disable guest access to the forums entirely. That's sure to get rid of the guest/bot/spider problem

I really want to just get rid of the yahoo slurp spiders but then again not sure its just them.

[thesportszone]

lol I just read that your host is hostican... didn't they offer to setup the .htaccess file for you ?

Not that I was told maybe they did and just didnt tell me?

[encryption]

lol I just read that your host is hostican... didn't they offer to setup the .htaccess file for you ?

EDIT: If you're stuck on staying where you are, the only other drastic and undesirable thing you can do is disable guest access to the forums entirely. That's sure to get rid of the guest/bot/spider problem

[thesportszone]

Under Usergroup Manager >> Unregistered / Not Logged In >> Click Edit

Scroll down to Forum Searching Permissions

Can Search Forums >> Yes / No

However you shouldn't have to disable searching on your board to reduce load. Thats how people stumble upon your site and able to find the content relevant to them.

already had it set to no. Thanks.

I am so lost on this .htaccess thing its not even funny.

[encryption]

Under Usergroup Manager >> Unregistered / Not Logged In >> Click Edit

Scroll down to Forum Searching Permissions

Can Search Forums >> Yes / No

However you shouldn't have to disable searching on your board to reduce load. Thats how people stumble upon your site and able to find the content relevant to them.

[thesportszone]

Mike you got a link to your site ? Also, try disabling bot/guest searching of the boards, that could help with the load

www.the-sportszone.net/forum thanks for the help. Where do I find the bot/guest searching? is that in the user group options?

[thesportszone]

Mike,

The issue is that your memory usage and cpu usage is off the charts you're sitting around 23 ~ 24 CPU constantly and about ~18 on memory. That's the main issue. If you have that many search bots and you know this for a fact you should block them via your .htaccess with

http://diveintomark.org/archives/200..._to_go_to_hell

If you know this for a fact you can't just leave the issue and think that it will resolve itself.

Ted?

Alright I think that this needs to be done but how do I go about this? I was pretty much given this site and am not the coding wiz that some of you guys are I am more of the learning type of guy.