[Forum] Server issues with 4.2.0

[Huskermax]

To start it off my BBS crashed with 12,000 users on it last night off and on for about 6 hrs. I was getting these DB errors:

Database error in vBulletin 4.2.0:

Invalid SQL:

INSERT IGNORE INTO session
(sessionhash, userid, host, idhash, lastactivity, location, styleid, languageid, loggedin, inforum, inthread, incalendar, badlocation, useragent, bypass, profileupdate, apiclientid, apiaccesstoken)
VALUES
('7c8e5903ec00859c8b949dccd48b9040', 0, '108.65.202.7', 'b5d34411950cc69e35ac26bb37890587', 1346555056, '/vbbs/showthread.php?22631-Kick-Catch-Interference', 0, 0, 0, 0, 0, 0, 0, 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:14.0) Gecko/20100101 Firefox/14.0.1', 0, 0, 0, '');

MySQL Error : The table 'session' is full
Error Number : 1114
Request Date : Saturday, September 1st 2012 @ 10:04:16 PM
Error Date : Saturday, September 1st 2012 @ 10:04:16 PM
Script : http://www.huskermax.com/vbbs/showth...h-Interference
Referrer :
IP Address : 108.65.202.7
Username :
Classname : vB_Database
MySQL Version :

This error lead to this questions from my server people?

Whoa. MySQL Error : The table 'session' is full

Looking quickly, I see 12,864 sessions in that table... the bigger problem is, this almost surely means the database itself is getting too big.


Can you find out from HostGator if there is indeed a table size limit? Please also ask them if there is a database size limit overall. You should get two precise answers back.

There is now limit on the table size limit or database size limit on our server. There is a limit to the disk size. We have a HostGator dedicated server. We are in there top price range of there dedicated servers.

I don't know if this is anything you can access, but I see some directories that might be worthy of pruning:


1.7G ./home/huskermax/tmp



1.8G ./home/huskermax/mail/new <- that one makes no sense to me. Perhaps vbulletin has been using a local mailer and is clogging an admin mailbox that goes nowhere...?



4.0G ./var/log

Can you set up a cron job for this? Are these system logs? Does this do anything to the post or thread history. I want to try and keep all my threads and post history intact so people can search them. For some reason my users get real pissed when I prune posts and threads. Even if they are over a year old. Can I have an archive server?


Below is the anwser I got when my server company looked in to my my BBS was crashing based off the error above.

Hello,

We would be happy to assist you with this issue and after running a couple of tests on your server it does appear that part of this issue is with the version of PHP that is currently installed and being used. Currently you have PHP 5.2.14 installed and although this should work there are some known performance issues with this version of PHP 5.2. You should have this upgraded to at least PHP 5.2.17 (the last release of the PHP 5.2 branch) as this will most likely help to resolve part of this issue.



We also noticed that currently your server is using Apache 2.2.16 and although this should work there are known performance issues with using this version of Apache. We highly recommend that you upgrade to version 2.2.22 as this will help with memory management for Apache as well as performance. To have PHP and Apache upgraded on your server this work will take between 45 minutes and 2 hours depending on whether any errors are encountered or not. Please keep in mind that as we will be replacing both Apache and PHP there is a chance that if the Apache web server restarts prior to the completion of the rebuild process that your sites will not be available. Please advise us of the best time to complete this work and we will be happy to assist you with this.


Now we also noticed that currently your server is hosting roughly 1076 domains on it and there appear to be a large number of custom php.ini files. Please keep in mind that if any of these custom php.ini files have large memory limits set in them they will cause server performance issues. The following is a partial printout of the custom php.ini files that are currently on your server:



/home/huskermax/popcornpedia.com/php.ini
/home/huskermax/bearspedia.com/php.ini
/home/huskermax/playoffpedia.com/php.ini
/home/huskermax/motocrosspedia.com/php.ini
/home/huskermax/therapypedia.com/php.ini
/home/huskermax/whitesoxpedia.com/php.ini
/home/huskermax/visionpedia.com/php.ini


Again the above is only a partial list. Please keep in mind that if a website does not need more than the default PHP Memory Limit there is no need to create a custom php.ini file as the Global values will be used. For those websites that do need to have a PHP memory limit higher than the 64M that is set be default then you would provide those domains with their own php.ini file.


We also noticed that currently your server is using a 32bit Operating System and you currently have 8GB of RAM in your server. Now we do see that you are using the PAE Kernel to allow the Operating System to access ALL 8GB of RAM but this does not allow Apache, MySQL or any of the other services to utilize the 8GB of RAM your server has in it. The upgrade to a 64bit OS would allow all of your services to utilize the 8GB of RAM that your server has. This would also improve performance as well.

To better assist you with this issue we have made a couple of slight adjustments to that Apache web server configuration and have restarted the Apache web server. The changes that we made were to better assist Apache with memory management but again as the version of Apache that is installed does have some known issues we recommend upgrading this to version 2.2.22.

Just a tad bit over my head

My question is based off the error we are seeing what do other board owners do with that much data? To keep there board up and running? Is above a good start? Do I need to start put my BBS on more then one server, do I need to prune?

Thanks for any help that can be provided!

[Zachery]

I don't see what having 12,000 sessions in your session table has to do with anything. Its the number of sessions in the table. I don't know if I like or trust your server people so far in this conversation.
Normally the MySQL default table size for session is about 16mb. 12,000 records sounds about right. Did you have a huge spike of traffic last night that pushed you into having 12,000 bots/users/guests?

It really just sounds like your cleanup tasks are not running

What are the last run times from the hourly cleanup tasks?
AdminCP > Scheduled tasks > Task manager


Now, I don't see how any custom php.ini files possibly allowing php to have more memory could cause "major performance issues" php only uses as much memory as it needs per script execution. It generally doesn't take more memory than it needs. Or Servers I've seen with 2gb memory limtis would fall over dead quick fast and in a hurry.

You should upgrade apache to the latest, and PHP to the latest 5.3. If you're on vB3, you might want to disable error display for your site.

As for the ram thing, he is half right and half wrong. While your 32bit OS and programs are limited in their ram usage. It is 2gb per application, but your server still has all 8gb available to it, just no more than 2gb per program. Which could in theory be a problem.







H

[Huskermax]

Thanks a lot that is a huge help.

Here are screen shots of task manager, are they set up right? We do not use the blog. We do use articles or cms.

We did have a spike, this is a sports bbs and yesterday was game day. We had about 5,000 connections to 9,000 during the game. The bbs crashed one time during the game then after the game we shot up to 12,000 and the server load was running around 9.0z

The bbs crashed one time after the game. This is over a 8 hr time period.

So we have been trying to figure out what is causing this.

[Zachery]

That all looks good, you just need more memory for your tmp tables

max_heap_table_size = 32M

[Huskermax]

That all looks good, you just need more memory for your tmp tables

max_heap_table_size = 32M

Thanks a lot for the help. I will have our server people work on it.