Announcement

Collapse
No announcement yet.

SMF 2.0 RC4 -> vB 4.1.1 -> attachments file system

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • SMF 2.0 RC4 -> vB 4.1.1 -> attachments file system

    Hi..

    My situation:

    We're using the SMF 2.0 RC4 forum, with approx 4000 members and about 130.000 attachments. I'm now trying to move our forum to the latest vB version (4.1.1). Importing trough impex, everything works well, except the attachments.

    We have been using SMF from version 1.x.x, so about half of the 130.000 attachments are saved using the non encrypted way, the other half encrypted.

    I had to slightly adjust the 010.php file, toget impex to read most (99%) of the attachments. (used part of the SMF code here)

    The first testimport, I left the attachmentlocation to the database, but it grows quickly to a 7+GB mysql, so I started again from scratch and selected "file system" as the attachment location.

    When importing to the database, alle files get imported, but when using the file system (to disk) all i get is Attachment not imported

    Another strange fact is, when importing to the database, after a refresh, mysql just stops working, with mysql.sock errors... All a bit strange..

    I will keep trying, and if I find a solution I will post it here, but if someone else has a solution, please let me know. (downgrading to get this working should not be a solution)

  • #2
    A little more information:

    When I'm importing the 130.000 attachments, it keeps importing them, until around 90.000 files, then the importer just stops (mysql hangs I think)

    When I try to refresh the page, I only get a vbulletin database error, wich point to the mysql.sock file.

    The attachements table is at this point around 7,2Gb. I'm really stuck here...

    Oh yeah, the server is my own llinux Ubuntu 10.4 VPS with 1Gb ram and more than enough hard disk space..


    I really hope some of the vBulletin programmers looks into this, because at this moment I can't do anything with the new forum.

    Comment


    • #3
      Have you tried increasing the php and mysql memory limits, caches, and execution time to help cope with the large amount of attachment data?

      Comment


      • #4
        Originally posted by Zachery View Post
        Have you tried increasing the php and mysql memory limits, caches, and execution time to help cope with the large amount of attachment data?
        I can't reach the server anymore, but with those attachment numbers above, wich settings would you use? I can only do a clean install again and start over. I will then try installing & configuring php / mysql with the numbers you say and get back to you...

        Any other ideas why only the import to database function works and not the filesystem part? I prefer to use the filesystem attachments, because there are so many of them...

        Comment


        • #5
          Can you provide a link to your phpinfo, also go into AdminCP > Maintenance > Diagnostics > look at the mysql information, and provide me with the varible list.

          Comment


          • #6
            Originally posted by Zachery View Post
            Can you provide a link to your phpinfo, also go into AdminCP > Maintenance > Diagnostics > look at the mysql information, and provide me with the varible list.
            I can't get into vBulletin itself anymore, because of a database error, wich I cannot seem to solve. Probably due to the large database.

            The phpinfo is uploaded to my other webserver, since the testserver isn't world-reachable.

            link http://www.houseofdreams.be/phpinfo.pdf

            Comment


            • #7
              What is the database error?

              Memory limit looks okay on php, though up the max_execution_time in php to 300 seconds, at least for the import.

              Comment


              • #8
                Originally posted by Zachery View Post
                What is the database error?

                Memory limit looks okay on php, though up the max_execution_time in php to 300 seconds, at least for the import.
                I'll gladly try it and get back to you, but I think it has something to do with mysql, since it's completely down at the moment and I cannot get it to start...

                The cleaner.php error log:

                ImpEx Database error

                mysql error: Link-ID == false, connect failed

                mysql error:

                Date: Tuesday 25th 2011f January 2011 05:56:25 PM
                Database: bimmerboard
                MySQL error: Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)

                Comment


                • #9
                  that means that mysql is down or unreachable.

                  So the entire forum is down?

                  Comment


                  • #10
                    Originally posted by Zachery View Post
                    that means that mysql is down or unreachable.

                    So the entire forum is down?
                    Yeah, I get the standard vBulletin "database error". I'm currently allready configuring a new server (from a template, so it's not a big deal) but this time with a larger vhd, because the last item in the mysql error log was something like "disk full"

                    I will let you know how things go...

                    Any ideas why the import fails when i select "filesystem attachments" ? Then I don't need that much space in our database to store them. My idea at the moment to circumvent this, was to let impex import them to the database and then switching again to filesystem, but I can't get this far at the moment...

                    Regards,

                    Comment


                    • #11
                      You need to store attachments in the database for Impex to work. After that you can move them to the file system if you want.
                      Steve Machol, former vBulletin Customer Support Manager (and NOT retired!)
                      Change CKEditor Colors to Match Style (for 4.1.4 and above)

                      Steve Machol Photography


                      Mankind is the only creature smart enough to know its own history, and dumb enough to ignore it.


                      Comment


                      • #12
                        Originally posted by Steve Machol View Post
                        You need to store attachments in the database for Impex to work. After that you can move them to the file system if you want.
                        Ah ok, cool At least I was on the right path... Again, I will post my progress here.

                        Comment


                        • #13
                          Originally posted by Houseofdreams View Post
                          Yeah, I get the standard vBulletin "database error". I'm currently allready configuring a new server (from a template, so it's not a big deal) but this time with a larger vhd, because the last item in the mysql error log was something like "disk full"

                          I will let you know how things go...

                          Any ideas why the import fails when i select "filesystem attachments" ? Then I don't need that much space in our database to store them. My idea at the moment to circumvent this, was to let impex import them to the database and then switching again to filesystem, but I can't get this far at the moment...

                          Regards,
                          Right click, view source, and the real error message will be there.

                          Comment


                          • #14
                            Code:
                            <!--
                            Database error in vBulletin :
                            
                            mysql_connect(): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)
                            /var/www/includes/class_core.php on line 314
                            
                            MySQL Error   : 
                            Error Number  : 
                            Request Date  : Tuesday, January 25th 2011 @ 05:40:06 PM
                            Error Date    : Tuesday, January 25th 2011 @ 05:40:06 PM
                            Script        : http://10.10.10.199/
                            Referrer      : 
                            IP Address    : 10.10.10.107
                            Username      : 
                            Classname     : vB_Database
                            MySQL Version : 
                            -->

                            Comment


                            • #15
                              That means mysql is down or otherwise unreachable. Are you sure mysql is up and taking connections?

                              Comment

                              widgetinstance 262 (Related Topics) skipped due to lack of content & hide_module_if_empty option.
                              Working...
                              X