Announcement

Collapse
No announcement yet.

Can the Photo Albums handle 1 million pictures?

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • bigwater
    replied
    godaddy?

    All bets are off then.

    I'm surprised you have a site at all.

    Leave a comment:


  • Suiram
    replied
    Originally posted by bigwater View Post
    1) At the user level directory. You currently have something like /home/myusername/public_html/forums. Place the attachments directory at /home/myusername/public_html/attachments, or whatever you want to name it.
    i don't. when i use ftp i go straight into the root of my domain.
    there i have only a few files that were placed there by my host like php.ini, one or two others and a couple of forum mailer form files, and a cgi, php_uploads, stats and _db_backups directories. everything else is mine.

    i own the domain.
    i use linux shared hosting with godaddy...

    Leave a comment:


  • bigwater
    replied
    1) At the user level directory. You currently have something like /home/myusername/public_html/forums. Place the attachments directory at /home/myusername/public_html/attachments, or whatever you want to name it.

    2) When a new user signs up permissions will be granted based on the permissions of the forum it inherits the permissions from. Once /home/myusername/public_html/attachments has 777, all directories that are created under it will have 777. You just have to apply the permissions one time when you create it. Nothing further will need to be done.

    2b) Yes, you will get thousands of directories... if you get thousands of users. One directory for each user... but the directories are structured in such a fashion that it's very easy to find a user's directory (on the rare occasion you should need to). If a user's number is 3968, the directory would be home/public_html/attachments/3/9/6/8 based on my previous naming convention. All the attachments for that user would be in that directory. Of course you'll have the attached files for user 396 in the directory /home/public_html/attachments/3/6/8 along with the directiories 1-9 under that for user 3961-3969. It sounds complicated, but it isn't. Not at all. Don't be scared. You're never more than 10 directory levels deep from any other directory level.

    The beauty of storing attachments in the file system is that they get cached, so when you have a popular thread that is getting a lot of hits, the attachments are pulled out of the memory of the server rather than having to be rendered by a sql query. For small forums, the database way doesn't really produce a performance hit, but once you get a lot of attachments it can seriously affect response time and create real headaches on backups.

    Leave a comment:


  • Suiram
    replied
    Originally posted by bigwater View Post
    Well you define what the directory will be when you make the change. Suggested policy is to move it above the forum level for security purposes.

    The process of moving files from the db to the fs will create a multi level directory based on your users id number. For example, user 1 will have all of their files in a directory called /1 under whatever you decide to name the directory for your files. User 119's files will be under a subdirectory 1/1/9. User 3267 under /3/2/6/7, etc. You'll need to chmod the entire directory to 777 once it's built... but again, it goes where you tell it to go.
    1.) i don't have access with my current host to directories before (above?) the root of my domain. is that what you meant ? the "deepest" i can go is the root of w?w.mydomain.com. is that not good enough ?

    2.) ....once it's built, you say ? but what happens, for example, when new users sign up and add pics ? that means a new directory will be created. will i have to constantly 777 new sub directories ?? can't i just 777 the parent directory initially, and then any new sub directory will inherit 777 as well ???

    2b.) not with my crappy site, but in general, that means large sites would have thousand's of directories for each user. that now, sounds pretty bad. bad as in cluttered. ftp-ing thousands of directories with pics could get wild. ahhhh, maybe having one HUGE directory would be better (as an optional alternative at least) that could have filename+userid appended to them. so that way each user's pics would still be pretty easily identifiable selectable.

    the current vb software itself is a pretty large "tree" as is, there's no way i could cope with thousands of extra directories. (mentally anyway)

    Leave a comment:


  • bigwater
    replied
    Well you define what the directory will be when you make the change. Suggested policy is to move it above the forum level for security purposes.

    The process of moving files from the db to the fs will create a multi level directory based on your users id number. For example, user 1 will have all of their files in a directory called /1 under whatever you decide to name the directory for your files. User 119's files will be under a subdirectory 1/1/9. User 3267 under /3/2/6/7, etc. You'll need to chmod the entire directory to 777 once it's built... but again, it goes where you tell it to go.

    Leave a comment:


  • Suiram
    replied
    ok, before i attempt to move to f/s please tell me exactly which directory on the server should be 777'ed. if there is more then one, then directories.

    and so i got this straight, once i'm using the f/s for pics, which pics if ANY will still be placed in the db. (i assume none)

    also do all the pics from a database get dumped into one directory ?? so you may end up with a directory with hundreds/thousands of pics ?

    also what happens in the future if a user wants to upload a pic (say png) that has the same name as an already attached pic ? will he be prompted to change/alter the name ?

    thanks,

    Leave a comment:


  • bigwater
    replied
    Originally posted by Suiram View Post
    if true, when will that happen ?? as soon as i make the change in admincp ??
    so if a site that has several year's worth of attached pics does this, the site is down for a while ? or what happens ? surely a large site would need some downtime ? or.......????
    This does work very transparently. One of my client's database started thrashing after about 10GB of binary attachments (pictures) had piled up in the DB. He contacted me wondering what was wrong, and all I could do for him was close my eyes and punch the button. It went amazingly fast, probably only took 2-3 minutes, and went completely un noticed by users, or at least if they noticed they never said anything. It's made a tremendous difference in the performance of his server, and backups are so much easier now that we're just pulling incrementals (new stuff, stuff that changed) every night instead of having to dump the entire db.

    In reality storing images in the file system should be the default way it's done, but I understand why Jellsoft chose db as the default way... it's compatable with more server setups... but if you can put the binaries in the file system, by all means, do.

    Leave a comment:


  • Zachery
    replied
    The move is done without much if any down time. The files are copied into the file system. once the copy is done you remove the files from the database. The users shouldnt even notice.

    Leave a comment:


  • Suiram
    replied
    Originally posted by glennybee View Post
    No...the pics will be copied from the database to the filesystem and then deleted from the database. All subsequent pics will be uploaded to the filesystem and all previously stored pics will be stored in the filesystem.

    The same applied to avatars, profile pics and attachments.
    if true, when will that happen ?? as soon as i make the change in admincp ??
    so if a site that has several year's worth of attached pics does this, the site is down for a while ? or what happens ? surely a large site would need some downtime ? or.......????

    Leave a comment:


  • glennybee
    replied
    No...the pics will be copied from the database to the filesystem and then deleted from the database. All subsequent pics will be uploaded to the filesystem and all previously stored pics will be stored in the filesystem.

    The same applied to avatars, profile pics and attachments.

    Leave a comment:


  • Suiram
    replied
    really ??
    so hundres of pictures will somehow and automatically go from the db to the specified folder ??? i find that hard to believe. i think after a switch from db to fs only from then on will pics be placed into the folder.

    Leave a comment:


  • goyo
    replied
    Originally posted by Suiram View Post
    so if i use the database now, and later switch to file system, will only newly added pics be used in the file system (so that way i'll have some in the db and the rest from then on in the fs) or will the database spew the old pics into the file system too ?
    You'll have all your pics in the file system only (you'll still have your attachment table with some important info but physical location of the files is in the fs).

    Leave a comment:


  • Suiram
    replied
    so if i use the database now, and later switch to file system, will only newly added pics be used in the file system (so that way i'll have some in the db and the rest from then on in the fs) or will the database spew the old pics into the file system too ?

    Leave a comment:


  • goyo
    replied
    Originally posted by TNCclubman View Post
    When I hit what amount is a good time to switch?
    Sorry for the interruption...

    Just let you know that I'm dealing with large amounts of pictures for many years now...mysql not meant to be store binary files so I don't recommend storing several 100.000 media in database in any circumstances...

    As far as I know vB's 3.8 album feature only stores max. 1000 files/folder then it creates a new folder so file system will not become corrupt because of large number of files in one folder...

    Leave a comment:


  • Jake Bunce
    replied
    Yes. If load becomes a problem then you can change the storage method.

    Leave a comment:

Related Topics

Collapse

Working...
X