Announcement

Collapse
No announcement yet.

vb4 forum sporadically slow/unusable for several minutes throughout the day

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • [Forum] vb4 forum sporadically slow/unusable for several minutes throughout the day

    Hi there! For quite some time now we've been experiencing what I would describe as a sporadic but site crushing latency that occurs multiple throughout the day but then resolves itself after a few minutes. I've done some investigation into these periods of lag and can confirm the following:

    - Everything else on the web server continues to function normally. Other pages not associated with vb4 load quickly and without issue.
    - This doesn't appear to be a problem with the MySQL server as we have some other applications that use the same database/server but continue to function normally while vb4 experiences a period of slowness
    - Non-vB4 PHP scripts that pull information out of the vB4 database continue to function normally and quickly
    - This doesn't appear to be any kind of DoS attack or related to bots or user activity on the site (check server logs very thoroughly)

    The only thing I can think of is that possibly there is some scheduled task a part of vBulletin that is utterly crushing the site and I was wondering if maybe you agreed and could propose some suggestions for which task it might be? Part of my reasoning for this is that the site itself is pretty large... so, it seems plausible that some process takes forever to complete due to the shear amount of content. Does anyone have any suggestions as to what might be causing this?

    Note: We don't plan on deleting any posts or users, we keep everything pretty much forever. If there is a scheduled task causing this I can at least move it to a better period of the day (or something).

    Our site is here:
    http://www.nma-fallout.com/forum.php

  • #2
    Do you run backups at a set time each day?

    Comment


    • #3
      Hiya donald1234, that's a good suggestion... alas, I do run a SQL backup once every day but that happens once a day during off hours. These site timeouts happen multiple times through the day and only last maybe 2-5 minutes.

      Comment


      • #4
        Is there anything showing up in your error_logs (if you don't know where they are, ask your host) during this time?

        Please don't PM or VM me for support - I only help out in the threads.
        vBulletin Manual & vBulletin 4.0 Code Documentation (API)
        Want help modifying your vbulletin forum? Head on over to vbulletin.org
        If I post CSS and you don't know where it goes, throw it into the additional.css template.

        W3Schools <- awesome site for html/css help

        Comment


        • #5
          There aren't any associated errors in the PHP error_log, just the usual benign ones that appear throughout the day. Any other suggestions? I checked the scheduled task log but the only thing running when this is happening is usually the processing of queue'd e-mails. Most of the time there are no e-mails to process or very few (less than 15) so I don't think it's related, plus I doubt that would hose the whole vBulletin platform.

          It only affects vBulletin... all non-vBulletin scripts and database queries function fine.

          Comment


          • #6
            Have you logged onto the server and run "top" to see if anything else is running at this time? Have you checked your Scheduled Task log to see if something is running at the time of the problem? (Not all Scheduled Tasks are set to log their actions. Make sure "Log Entries" is set to Yes for all your tasks while looking into this issue.

            Please don't PM or VM me for support - I only help out in the threads.
            vBulletin Manual & vBulletin 4.0 Code Documentation (API)
            Want help modifying your vbulletin forum? Head on over to vbulletin.org
            If I post CSS and you don't know where it goes, throw it into the additional.css template.

            W3Schools <- awesome site for html/css help

            Comment


            • #7
              This is increasing in frequency and resulting in a lot of double-posting and other poor user experiences. I have checked the scheduled task log multiple times and not found any related tasks. I have also done a fair amount of vb4 optimization to try and reduce lag and load times which work, when the site works, but otherwise does not affect these timeouts.

              I have also run top when this vb-only latency is occurring (once again, all other parts of the site, scripts and database queries, work flawlessly - it is only vBulletin which frequently becomes un-usable for 1-3 minutes at a time) and there doesn't appear to be any offending process, though I did note that the brief moment just before the forum starts working again the mysqld service will spike up to 80% CPU usage (likely to fulfill all the pending requests). This is what top looks like when the site is in an unusable state:

              Comment


              • #8
                Really at a loss here, I can't imagine what would completely hose the site for that long. I've tried adding caching, disabling SQL heavy features like automatic thread marking, displaying online users, etc. As a side note RE: the top command, we have an 8 core server, so even if it measured at over 100% for any command I would expect that not to be overwhelming as I believe it could theoretically go as high as 800%.

                Comment


                • #9
                  One of your cores is at 94% this will be what's causing the problem, I get the exact same issue on an 8 core server when a large forum does a backup, just one core at 90ish% during the backup, you have already ruled out backups as the cause, do you have other tasks? for eg cron.

                  Comment


                  • #10
                    Thank you for the reply! However, if you are referring to the 94.1% I believe that is a number indicating idle CPU time and not CPU load.

                    Comment


                    • #11
                      Sounds very much like a scheduled task issue, I dont see any reply that you checked the tasks to see if one is running periodically.

                      I had an issue like this for a while, turned out to be a stats product I had installed running some housekeeping every 15 minutes, on an ever growing table. This was hogging mysql resources.
                      Baby, I was born this way

                      Comment


                      • #12
                        Originally posted by Paul M View Post
                        Sounds very much like a scheduled task issue, I dont see any reply that you checked the tasks to see if one is running periodically.

                        I had an issue like this for a while, turned out to be a stats product I had installed running some housekeeping every 15 minutes, on an ever growing table. This was hogging mysql resources.
                        Thank you for the reply! I did however make two separate posts indicating I have checked the scheduled tasks on the site and if the top command is to be believed I am not consuming a significant amount of resources. Unless there are tasks that do not show in the tasks log then I am no longer inclined to believe it is a problem with tasks. I am preparing to disable all add-ons and hooks to see if this improves performance and if so will turn them on sequentially until the problem recurs, though I am not hopeful this will flush out the issue.

                        Any other suggestions would be appreciated.

                        Comment


                        • #13
                          Disabled hooks and the site is a lot faster... still times out randomly. Example of the kind of thing user's experience here. When they or I try to post occasionally it will 'spin' for a while and then go nowhere or do nothing. When the site is working it probably takes 1-2 seconds after clicking "reply" for the message to show up:



                          When the site doesn't work it spins and then stops but does nothing. During these times you can open the forum in a new tab and it will be stuck at loading with a white screen. Once the lag disappears the site will load immediately.

                          Comment


                          • #14
                            This is pretty much what users and myself see, this has been going on for a few minutes now:



                            Can go to other large database heavy applications on the same server, like the old forum and it works perfectly during this time:
                            http://www.nma-fallout.com/forum/index.php?archive=true



                            Top once again shows an under use of resources... like they aren't being used at all basically:

                            Comment


                            • #15
                              I am still having problems with latency on my site, though I have been able to improve performance somewhat through a battery of optimizations to vBulletin and MySQL but the issue is ultimately unresolved. I have found at least one specific area that causes site latency and that is the use of the search feature (disabled for guests). Our database large (over a million posts) and the searchcore_text table is 1.5gigs. Depending on what you search it may lock the site up for 20-30 seconds. Is there a way to optimize this search feature so it doesn't hinder the rest of the site? I attempted to add indexes to the searchcore_text table but it appears they already exist by default. Surely other people have encountered numerous issues getting vBulletin to function at an acceptable speed? Is the size of our site that unusual? We're almost ready to move off of vBulletin at this point.

                              Comment

                              Related Topics

                              Collapse

                              • Rivka
                                If You Were Building your First Site Right Now, Would You Use vB4 or vB5?
                                by Rivka
                                Ok I am just now learning and building a site. I have been reading and learning my install of vb4 for the last 2 months, i have a lot of learning to go. I currently do not know when My site will go live,...
                                vb4 - Stick with what's already released & works
                                66.67%
                                26
                                Both - set them up side by side, if the vB5 is gold by the time you want to go life, great, otherwise fall back to vB4
                                10.26%
                                4
                                vB5 - Might as well put the time and effort into setting it all up once
                                23.08%
                                9
                                Fri 28 Sep '12, 12:21pm
                              Working...
                              X