Announcement

Collapse
No announcement yet.

Optimizing MySQL for a large import

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • sweepr
    New Member
    • Jan 2010
    • 2
    • 4.0.0

    Optimizing MySQL for a large import

    I'll be migrating my existing live Web Wiz Forum to vBulletin soon (MSSQL to MySQL).

    I've done a number of test runs with both DBs on my local machine and it takes some time to do the imports (easily 12 hours for 500k+ posts in a test database).

    My live DB has close on 974,493 Posts and 65,622 Topics so by past experience this could take up to 24 hours running continuously?

    I saw this post by Zachery in another topic and was curious about what kind of optimization can be done in MySQL (and MSSQL) to speed things up.

    Originally posted by Zachery
    Jerry has done massive imports in a short period of time. I've done at least one half million post import with the right setup in 30-45 min. Impex is only as fast as you configure the mysql to be.
    Original thread

    I'd think hardware would also play a role and this is where I may be somewhat limited. My local machine is a Core 2 Duo 1.86Ghz with 2GB ram (Windows 7). And have another similarly spec'd machine to use if necessary.

    1. Any advice on optimal MySQL and MSSQL settings to speed things up?
    2. And what are the experiences with using two local machines on a Gigabit LAN (one for the source MSSQL DB and one for the target MySQL DB) vs having both on one machine?

    Advice would be much appreciated!
  • fatherofspeed
    New Member
    • Nov 2010
    • 29

    #2
    I'd like to get an answer to this as well.

    Comment

    • Typhon
      Member
      • Nov 2000
      • 74
      • 3.7.x

      #3
      Same; I'm looking to import a 3 million post forum into a 5 million post forum and it always hangs on users and posts. If I could get hardware specs and configurations, that would be great. I wouldn't even mind dumping both systems and outsourcing it for someone else to do the import. LOL! Thanks!
      ::Typhon::
      www.realmworx.com

      Comment

      widgetinstance 262 (Related Topics) skipped due to lack of content & hide_module_if_empty option.
      Working...
      😀
      🥰
      🤢
      😎
      😡
      👍
      👎