Announcement

Collapse
No announcement yet.

Problem with rebuilding similar threads

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Problem with rebuilding similar threads

    I run a bigger forum, with 6.6 million posts and 500k threads.

    I just turned the similar threads feature on, and I'm finding that reindexing dies consistently on thread #9, which happens to be a thread in our "evidence locker" (where we keep threads we had to take action on.) This is a hidden forum, which is probably part of the problem.

    Is there a feature to toggle somewhere to tell vB to ignore hidden forums when rebuilding the similar thread index?
    The High Road

  • #2
    No sorry, there is no such option.

    This may be a server time and/or memory limit. To increase your limits edit your includes/config.php file and add these lines right under the <?php line:

    ini_set('memory_limit', -1);
    ini_set("max_execution_time", "2880");
    Steve Machol, former vBulletin Customer Support Manager (and NOT retired!)
    Change CKEditor Colors to Match Style (for 4.1.4 and above)

    Steve Machol Photography


    Mankind is the only creature smart enough to know its own history, and dumb enough to ignore it.


    Comment


    • #3
      It looks like that did it. Or at least, the process didn't stop where it did before, and it seems to be chugging along happily (if not as quickly as I'd prefer.)

      Thanks, Steve.
      The High Road

      Comment


      • #4
        Posting a follow-up for people who run into the same problem I do.

        Steve's fix worked to a point. Once implemented the immediate problem went away, but I'd eventually run out of memory and the board would start swapping (and it never swaps). Increasing memory by 2.5 times wasn't really a help, either.

        The problem at this point seemed to be that multiple Apache processes were each consuming 1.6g+ of memory, so the machine would run out of available memory. The fix here was to change MaxRequestsPerChild to something much lower than normal. This forces apache processes to respawn sooner, and respawning releases the memory that was otherwise tied up in those processes.

        So now the final problem is determining the optimal number of threads to process per page. Steve's fix gives a timeout of 2880 seconds, which means that if it takes longer than 48 minutes to complete the threads listed on a particular page then the page will timeout. Repeatedly.

        So, 1 per page works. On my board 6 per page doesn't quite work, so now I'm working it at 4 threads per update, and it's going OK. Load isn't going higher than double normal load, so no worries there.

        The final pointer: if you get a significant part of the processing done and need to restart where you left off, use this:

        <your board's url>admincp/misc.php?do=updatesimilar&startat=<thread number to start at>&pp=<# of threads to process per page>

        Much simpler, though you'll still need to login every hour to keep it running.
        Last edited by dzeanah; Fri 8 Oct '10, 10:21am. Reason: Update on what ended up working
        The High Road

        Comment

        Loading...
        Working...
        X