Announcement

Collapse
No announcement yet.

Web Wiz import very slow

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Web Wiz import very slow

    Hello,

    OK first off:

    Clean import
    Source - Web Wiz 7.9 (MSSQL database), 9000 users, 900,000 posts in 82,000 threads
    Target - VB 3.5.3

    I've done a few test runs now, and I think i've managed to iron out all the little niggles from the process. The only problem is that it seems to be taking far too long to complete the import process, judging by some of the comments i've seen in the forum migration experience thread (amongst others). The import post module takes about 550min to complete (that's over 9 hours)

    I'm doing the test imports on a local machine from source data constructed from a recent backup of the current forum database. Machine spec is:

    Win XP SP2
    2.8GHz P4
    768MB RAM
    Apache 2.0.54
    PHP 4.4.0
    MySQL 4.1.12a-nt
    MSSQL Server 2005 Express

    So far i've tried increasing the PHP memory limit in php.ini (to 100M), which doesn't seem to have made much of a difference. Any other suggestions are welcome.

    Cheers

  • #2
    Yea that's a long time !

    Hmmmm, how long is it taking per page ? how many posts per page are you doing ?

    With 100M for PHP, try 20,000 posts per page etc.
    I wrote ImpEx.

    Blog | Me

    Comment


    • #3
      So far i've only been doing 1000 posts per page, it's been taking about 30seconds per page plus a biiiiiiiig delay after the last page. I'll set it off on another run tomorrow at 20,000 per page and see how it goes. Would I also benefit from similarly increasing the items/page for the other modules as well?

      To be honest I don't think any of the modules are running as fast as they could/should (Import Users takes 4min at 500/page, Threads takes 40min at 500/page, Import PMs takes 45min @ 500/page (32,000 PMs)), which lead me to think that maybe my config could do with some tweaking. MSSQL is completely new to me so that is just running exactly how it comes out of the box, and to be fair server config in general isn't my specialist subject...

      Cheers for the help

      Comment


      • #4
        Yes, that is very slow.

        Is the MS-SQL server local as well ?
        I wrote ImpEx.

        Blog | Me

        Comment


        • #5
          Yup all runnning on the same machine.

          20,000 posts per cycle isn't going too well, it occasionally hangs at the end of a cycle. It does look as though it could be running a bit faster though, so i'll have a go with 10 or 15,000 next.

          Comment


          • #6
            OK, trying it with 10,000 posts per cycle drops the time down to 420 minutes, which is faster but still not cooking with gas.

            If I open a support ticket for this is it a problem if i'm not the main contact on the license? The license is in our club treasurers details, i'm one of the additional adresses?

            Comment


            • #7
              OK, a quick update just in case anyone else has similar issues in the future. It appears that MySQL had installed itself with the default table type set to InnoDB, so all the vB tables had been created as InnoDB. I set the MySQL default to MyISAM and started again and the import times are right down, from 9 hours or so for the posts to just 1 hour

              Comment

              widgetinstance 262 (Related Topics) skipped due to lack of content & hide_module_if_empty option.
              Working...
              X