Moving attachments from database to filesystem

[Todi]

I want to stop using the database to store my attachments, because i'd like to keep the backups smaller and make the files easier to link to. However, when i go into the Attachments -> Attachment Storage Type settings, and try to Move Items from Database into File System, it tells me i can't because:

Your server has PHP's SAFE_MODE enabled. You can not move attachments to files while SAFE_MODE is enabled. Please contact your host and ask them to consider disabling it if you wish to use the file system for attachments.

And since there's not much i can do about that, i can accept it. However, why can't i choose that any new attachments that are added can be saved as files instead? I can't find a setting for it anyway. Perhaps there was one when i initially installed the forum, but now there is not. The actual creation of files on the server works, i've tried with other php-based applications (media-wiki) and they can do it without problem.

Advice?

[Marco van Herwaarden]

Either all (new & old) attachments are stored on the file system, or none.

[feldon23]

It's not a matter of being able to avoid Safe Mode in 1 case but not another. Safe Mode annoyingly and unnecessarily controls and diverts how file uploads are handled.

[Todi]

Ok, so.

1. Is there any way for me to fix this? Copy the DB to a different server which doesn't have SAFE MODE, and do it there, and then move it all back perhaps?

2. If SAFE MODE is on, does that mean vBulletin simply can't handle files instead of db-files? If so, why can other appliances do it?

[Zachery]

1. No
2. I believe there is a change that should let you store your files in the file structor instead of the db but its not as well optmized as the non safemode version.

Safemode does not allow folders to be created. Its not vBulletin, its your server.

[feldon23]

It's not good to have 1,000 or 5,000 files in 1 folder. Because of SAFEMODE, vB cannot create subfolders to divide up the files.

[Marsupilami]

It's not good to have 1,000 or 5,000 files in 1 folder. Because of SAFEMODE, vB cannot create subfolders to divide up the files.

Well then, why not use an old cache folder trick, and simply create some folders during install, like:

cd SomePath
mkdir vbAttachmentDir
cd vbAttachmentDir
mkdir 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
chmod 777 -R .
chown user:group -R .

...and then let VB use these folders however it likes to store the attachements.

SAFEMODE is a reality out on the internet. It won't go away.

Our provider enforces SAFEMODE according to their security strategy, and it costs us plenty to have attachments in the database, instead of using plain disk storage.

Not being able to store attachments on disk is probably the biggest disappointment on VB from my behalf. I calculated on being able to store attachments on disk.

Never mind the estethics - Solve the problem.

[Zachery]

Solving the problem would be best to find a new host who does not insist on safemode or get a dedicated setup.

We currently store attachments in the filesystem as

/path/u/s/e/r/i/d/attachid.attach

the only way I could see this working in safe mode is manually creating folders up to a few hundered thousand if not over a few million.

Because safemode is a realitly we do offer the ability to store attachments in the database.

[Marsupilami]

Solving the problem would be best to find a new host who does not insist on safemode or get a dedicated setup.

We currently store attachments in the filesystem as

/path/u/s/e/r/i/d/attachid.attach

the only way I could see this working in safe mode is manually creating folders up to a few hundered thousand if not over a few million.

Because safemode is a realitly we do offer the ability to store attachments in the database.

So you are suggesting that i do this as a hack instead?

Since the estethics in the storage path coding seems to be more holy than customer satisfaction.

[Paul M]

SAFEMODE is a reality out on the internet. It won't go away.

Actually it will, safemode is being phased out of php (in php6).

Never mind the estethics - Solve the problem.

The problem is your server/php setup, therefore you (or your host) need to fix it.

[Marsupilami]

Then i will do the hack, instead of just "sitting the problem out".
How hard can it be? Spreading files named by unique IDs from the database over a set of cache directories.

Wish me luck!

[feldon23]

SAFE MODE = False Sense of Security Mode

Host updated all the software and safe_mode mysteriously turned itself on. Joomla stopped working and we couldn't figure out why until we noticed that safe_mode was running.

[Zachery]

Then i will do the hack, instead of just "sitting the problem out".
How hard can it be? Spreading files named by unique IDs from the database over a set of cache directories.

Wish me luck!

Its not aesthetics that we store the attachments in that way. Its to avoid issues with some operating systems. Believe it or not some of our users have hundreds of thousands of attachments, some operating systems have a max limit of 65k files in a directory, they hit this and it caused problems. Storing them in /u/s/e/r/i/d/ makes it easy to locate a specific users attachments both for the system and for you yourself.

[Marsupilami]

Yes, i understand why you store them in that way. Solves problems for big sites, and has some scaling and spreading mechanisms in it. Good algoritm.

But still, on some systems small frags of directories also eats its share of inodes and blocks, and i still think a stacking mechanism with a flatter directory structure and with a spreading mechanism handled in the database is lots more efficient on things like inodes, block fillage, cache utilisation. I think VB could do with a more sophisticated data storage layer.