Announcement

Collapse
No announcement yet.

ImpEx failing after 3.1M posts imported - now showing blank page and cannot restart

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • hoover
    replied
    Originally posted by motowebmaster View Post
    On one of my former sites it took me roughly six import attempts, on a test environment (and copy of the database), before the end result was good enough to move forward with. Each import attempt took a few hours (on a dual Quad-Core Server), but it was worth the effort.

    Check to see if php error reporting is activated on the site you are using to do the import. It has been a while since I've used ImpEx but it may be the reason why the pages are blank.
    thanks for all of the advice.

    I re-ran the post import 3 times. on the third try It died with only aprox 24K posts to go (good enough for me!). Since my site was offline for the most part - after my phpbb was thrashed by hackers, my visitors were more than happy to see it back on line - even sans 24K recent posts. The vbulletin software did a great job of filling the missing post gap with duplicate entries from the previous comment/post so it was no big deal.

    My advice to others...

    • read the install guides completely before doing anything.
    • optimize your server / edit mysql tables per 'large site' import instructions, etc... bump up memory in php.ini
    • FOLLOW the post-install instructions in the quick guide (I wish there was step in ImpEx that prompted the post sync & stuff!)
    • seek help in this forum if you have trouble... you guys were great.

    thanks all.

    http://uselessjunk.net/

    Leave a comment:


  • motowebmaster
    replied
    On one of my former sites it took me roughly six import attempts, on a test environment (and copy of the database), before the end result was good enough to move forward with. Each import attempt took a few hours (on a dual Quad-Core Server), but it was worth the effort.

    Check to see if php error reporting is activated on the site you are using to do the import. It has been a while since I've used ImpEx but it may be the reason why the pages are blank.

    Leave a comment:


  • Zachery
    replied
    yay, at least we figured some parts of the issue out.

    Leave a comment:


  • hoover
    replied
    Originally posted by Zachery View Post
    Make sure in the Forum manager that each forum is set to display threads for all time, now just the last 30 days.

    OMG! I feel like a moron. I was looking past the obvious and got so focused on data import & integrity issues that I didn't notice the 'all time' settings for threads in the admin control area. It was also compounded by the fact that the import job bombed on anything after Aug 2008. I'm new to Vbulletin... blah.

    Ok.. thanks for the help. Now I need to flush the post data and try my import again. I followed the instructions for optimizing the import for large forums, so I'll keep my fingers crossed and pray it picks everything up this time!

    Leave a comment:


  • Zachery
    replied
    Make sure in the Forum manager that each forum is set to display threads for all time, now just the last 30 days.

    Leave a comment:


  • hoover
    replied
    Originally posted by Zachery View Post
    Did you follow all of the after the import directions?

    yes, I followed everything as instructed.

    Some of the clean-up activities were not intuitive after completing the ImpEx install. I found these instructions in the ImpEx quick guide:
    In the maintenance section of the adminCP, run the Update counters for the threads, then the forums. Re-order the forums and delete the default Main Forum if need be, then save the display order to rebuild the forum cache.
    Long story short... counters (thread & post totals) are finally displaying accurate data after the clean-up activities but none of the threads are being displayed in the discussion groups. I tried to do the 'rebuild forum cache'. It completed in about 2 seconds, so I'm not sure what it is really rebuilding.

    Leave a comment:


  • Zachery
    replied
    Did you follow all of the after the import directions?

    Leave a comment:


  • hoover
    replied
    Originally posted by Ian99 View Post
    might be useful for Jerry to know which software / version you are importing from

    Also bear in mind that Impex is probably not supported on 3.8 until it goes gold.
    Debian server... importing from phpbb2.x to the latest version of vbulletin.

    Attached is a screen-shot of my database threads. You will see the one at the bottom is a new test post from today - entered via the new install of phpbb. it is displayed correctly in the forum, and the counter is working. All of the other imported threads are in the DB, but they don't show in the forum via a browser.
    Attached Files

    Leave a comment:


  • Ian99
    replied
    might be useful for Jerry to know which software / version you are importing from

    Also bear in mind that Impex is probably not supported on 3.8 until it goes gold.

    Leave a comment:


  • hoover
    replied
    More issues with imported threads / posts.... None of the threads are being displayed in the forum.

    While I can see 3M imported posts + 300K threads when I look in my Database, none are visible in the forum when I view it via a browser. Do I need to merge the imported threads into a new discussion or something?

    note: announcements and sticky posts are being displayed in the imported forums, but the thread counts and post counts for the forum are all 0 - and all of the regular threads are MIA.

    Leave a comment:


  • hoover
    replied
    UPDATE: After restarting servers, and after trying to load the page about 50 times, the ImpEx menu finally came back.

    It shows:
    (module)009 Import post [ redo module ] (success)3090298 (failed)120659 (time)1054 min(s)

    My revised question is now::::::

    How can I pick up the missing 300K++ missing posts without starting over -- or running a dupe checking job that will take hours?

    note: 010-Polls seemed to work fine as the ImpEx tool allowed me to move-on past the posts import even though it never really seemed to finish.

    (exact post count: old phpbb forum = 3,457,370 Import total thus far: 3,210,957(success+fail) gap: 246,413)

    Leave a comment:


  • ImpEx failing after 3.1M posts imported - now showing blank page and cannot restart

    I'm 3/4 of the way through a large forum migration. ImpEx got through 75K users, but it failed on the Posts import.

    I have a total of 3.5M threads. The importer was aprox 6 hours++ into the job - converting 1000 posts at time. When I woke this a.m. to check the progress, the admin menu was loaded on the left, but ImpEx was giving me a blank white page (looks like a browser timeout of sorts maybe).

    How can I get Impex working again? How can I get the center-frame ImpEx menu to load?

    I hope to resume the process without commanding the script to check 3M posts for dupes - is this possible?
widgetinstance 262 (Related Topics) skipped due to lack of content & hide_module_if_empty option.
Working...
X