wget a large forum.

This topic is closed.
  • Time
  • Show
Clear All
new posts
  • prose88
    New Member
    • Dec 2005
    • 1

    wget a large forum.

    I am attempting to wget a large forum for future litigation. I need to login to get the information we require.

    This is what I currently use:
    wget --post-data 'forceredirect=1&do=login&vb_login_md5password=HASH!&vb_login_username=USERNAME&url=/forums/index.php&vb_login_password=&cookieuser=1' -r --save-cookies cookies.txt --keep-session-cookies -o site.log -N -E -b -k -R *login.php?do=logout* http://www.SITE.com/forums/login.php

    It is successful for anywhere between 2,000 - 25,000 posts. Then I start getting 400 Bad Request errors. Can anyone help me narrow this problem down? Is it likely a session timeout? Apache stopping the mirror? Or is it vbulletin code that is set in place to stop it?
  • Zachery
    Former vBulletin Support
    • Jul 2002
    • 59097

    Wrong place to ask how to steal forum content


    widgetinstance 262 (Related Topics) skipped due to lack of content & hide_module_if_empty option.