Announcement

Collapse
No announcement yet.

Problem with rebuilding similar threads

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • dzeanah
    replied
    Posting a follow-up for people who run into the same problem I do.

    Steve's fix worked to a point. Once implemented the immediate problem went away, but I'd eventually run out of memory and the board would start swapping (and it never swaps). Increasing memory by 2.5 times wasn't really a help, either.

    The problem at this point seemed to be that multiple Apache processes were each consuming 1.6g+ of memory, so the machine would run out of available memory. The fix here was to change MaxRequestsPerChild to something much lower than normal. This forces apache processes to respawn sooner, and respawning releases the memory that was otherwise tied up in those processes.

    So now the final problem is determining the optimal number of threads to process per page. Steve's fix gives a timeout of 2880 seconds, which means that if it takes longer than 48 minutes to complete the threads listed on a particular page then the page will timeout. Repeatedly.

    So, 1 per page works. On my board 6 per page doesn't quite work, so now I'm working it at 4 threads per update, and it's going OK. Load isn't going higher than double normal load, so no worries there.

    The final pointer: if you get a significant part of the processing done and need to restart where you left off, use this:

    <your board's url>admincp/misc.php?do=updatesimilar&startat=<thread number to start at>&pp=<# of threads to process per page>

    Much simpler, though you'll still need to login every hour to keep it running.
    Last edited by dzeanah; Fri 8 Oct '10, 10:21am. Reason: Update on what ended up working

    Leave a comment:


  • dzeanah
    replied
    It looks like that did it. Or at least, the process didn't stop where it did before, and it seems to be chugging along happily (if not as quickly as I'd prefer.)

    Thanks, Steve.

    Leave a comment:


  • Steve Machol
    replied
    No sorry, there is no such option.

    This may be a server time and/or memory limit. To increase your limits edit your includes/config.php file and add these lines right under the <?php line:

    ini_set('memory_limit', -1);
    ini_set("max_execution_time", "2880");

    Leave a comment:


  • dzeanah
    started a topic Problem with rebuilding similar threads

    Problem with rebuilding similar threads

    I run a bigger forum, with 6.6 million posts and 500k threads.

    I just turned the similar threads feature on, and I'm finding that reindexing dies consistently on thread #9, which happens to be a thread in our "evidence locker" (where we keep threads we had to take action on.) This is a hidden forum, which is probably part of the problem.

    Is there a feature to toggle somewhere to tell vB to ignore hidden forums when rebuilding the similar thread index?
Loading...
Working...
X