Too many guests?

[slinky]

I've just taken a look at my server load averages:

Server Load Averages17.34 17.77 20.24 | 59 Users Online (1 members and 58 guests)

I find it hard to believe that I have this many guests, especially since you can't read the messages completely until you register. I'm wondering if there is lingering session information that is causing a perceptible slowdown due to phantom members.

I'm sure others have seen this symptom... anyone?

Mucho thanks and hello to all.

[Steve Machol]

This is almost certainly one or more search engines spidering your site.

[slinky]

Steve -- Thanks for the reply. Could it be possible that I continuously have 20+ search engines spidering the site? Once again, here's my readout from my forums:

4.68 4.47 4.36 | 32 Users Online (1 members and 31 guests)

I'm assuming that I'll need to find out who is spidering the site and purchase a good log analyzer (the free ones are OK but a little sparse.)

[Steve Machol]

Yes, that's certainly possible. The search engines have been on my site almost continuously for many months.

[pieman666]

You could use the spider tracker that is included in vB. Or to quickly check go to your currently online page (online.php) and then select at the bottom user agent Yes and click display. This will show the useragents for all connected people, that should make it clear if they are spiders.

Check out
http://www.vbulletin.com/forum/showt...t=spiders+list

which is lists of spiders and there identification strings, if you go to

vBulletin Options > Whos online options > Enable Spider Display > Yes

and then copy and past the two lists from the above thread into the relevant boxes below, when you go to the whois online page it will explictly say which spiders are spidering your site.

Stats would be good if you wanted to see how much of the site is spidered and how many search engine referals you are getting etc.

http://www.analog.cx/ is a free analysis program that is quick and should give you the major info you may need.

[slinky]

Pieman -- many thanks and much appreciated. That's exactly what I was looking for. It's hard to believe I have 20+ spiders on my site at almost any given moment.

An investigation into the slowdown that I have been experiencing revealed that some @#$&ing hackers had somehow gotten into my var/tmp directory and placed a few files there. I spotted this quickly. Right now they've done it again and are obviously exploiting some script that may have been newly installed. I'll post here any information I have for the benefit of others.

Regards to all.

[slinky]

I've actually cleaned out all the invasive files. I've turned on the enable spidering and adding in the lists. Unfortunately I still don't see the "guest" visitors. Somehow I had 495 visitors in December... weird. I'll update you if and when I see more activity. Thanks again for your assistance.

[slinky]

if you go to

vBulletin Options > Whos online options > Enable Spider Display > Yes

and then copy and past the two lists from the above thread into the relevant boxes below, when you go to the whois online page it will explictly say which spiders are spidering your site.

1) It seems like my site is not being spidered by more than 10-15 spiders. Still cannot explain all these extra guests. They are "viewing" threads but not listed. Perhaps they are unknown spiders?

2) Pieman -- The spidering thread and list of spiders was pasted but doesn't seem to work. I did turn on the enable spider display but I don't see anything. Is there something I might be missing? I did (a) enabled spider display (b) pasted in the extended list. The online.php DOES work and I do see Google and slurp...

[pieman666]

You won't see most of the spiders as they are relativly rare incomparison to the likes of Yahoo, Google and MSN. I sometimes catch site of one in online.php but it's a rarity.

As to your huge number of guests, are they coming from a simular IP range? Do they seem to stay a long time or just a few mins? If you do have access to weblogs run them through analog or some other prog and you will see how your referals etc are going, what search terms people are using to find you etc...

Your domain name is a good one and may well be 'catching' a lot of I know i'll type in the thelaw and see what I get...

Google has made a pretty good effort at spidering you
http://www.google.co.uk/search?as_q=...w.com&safe=off

so I would imagine you will quite a few referals.

analyse the stats is a good start. analog is free and quite easy to use (drag and drop the log files onto it and then open the report )

Oh one other thing, the Yahoo spider has been known to 'blitz' a site sending hundreds of spiders to do it's work. This could explain your very high most users online.

[slinky]

Superb. Thanks so much for the info pieman. It seems google and yahoo spider me a few times a day. They don't appear on "who's online" but do appear in online.php -- and for several sessions. This may explain also how I had several hundred visitors at one time as well.

MANY thanks again and appreciate the recommendations. I'm actually going to be purchasing a good offline analyzer since I enjoy the ability to analyze the logs with a little greater clarity and depth than analog.

Regards...

[pieman666]

if you want to pay then there are some very thourough programs out there, I have a coule of links I'll post when I get into work tomorrow. Cost is around $500 which maybe too much, if so then I'd be intrested if you find anything that you pay for that is better than analog as I couldn't.

[pieman666]

http://www.webtrends.com/Default.aspx and http://www.clicktracks.com/products have powerful analysis software that will allow you to look at your users habits, see how effective search engines are for you, see if ad's are being sucsessful etc.

As I said before I really couldn't find anything less than around $500 that did more than analog (the person who wrote analog works for clicktracks now) if you do I'd be intrested to hear about it (or if anyone else knows of something)

[brandon6976]

I've had very similar issues. is it possible to limit the amount of sessions 1 ip can have ?

[Steve Machol]

There is no function to do this. This requires modifying the code. We cannot officially support code modifications or forums running modified code, however can try searching or asking for help with this over at www.vbulletin.org.

[brandon6976]

hmm is it a planned feature eventually I would have a great interest in seeing this as I'm sure would other people. With these search engines at times they're sending waaay to many sessions at once.

My ISP obviously isn't ammusedand I feel like I'm between a rock and a hard place