Announcement

Collapse
No announcement yet.

Datastore in filesystem

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • RedWingFan
    replied
    Originally posted by Scott MacVicar View Post
    The idea is that you use eAccelerator / APC with the file data store to take advantage of storing PHP files in compiled memory.

    This reduces the network usage + memory usage of the script as that data would have needed to have been fetched and instantiated.
    OK, that makes sense. Thanks!

    We had APC installed yesterday, but it gave one of our two vB forums a hard time. Once we get our other issues straightened out, we'll have to work at configuring APC to get it to cooperate.

    Leave a comment:


  • Scott MacVicar
    replied
    The idea is that you use eAccelerator / APC with the file data store to take advantage of storing PHP files in compiled memory.

    This reduces the network usage + memory usage of the script as that data would have needed to have been fetched and instantiated.

    Leave a comment:


  • RedWingFan
    replied
    It looks like we have the same query running.

    I can understand if vB has to hit the database to update the cache, but looking at mytop output above, there are several hits to the database. I never see any UPDATE queries to datastore, but mytop can't catch everything (it is just taking a snapshot every few seconds).

    If I knew better how it worked, I would be able to determine if I even have it activated properly. The only file I'm seeing is /includes/datastore/datastore_cache.php, and it was last updated at 1:22am EDT. At least now it is not an empty file (it has our forum's settings in it, and is 3624 lines long), so I can only assume it's working properly.

    Leave a comment:


  • rossco_2005
    replied
    Originally posted by Zachery View Post
    Do you have any hacks or addons that have data inside of the datastore table? and are you sure it is writing files to the correct directory?>
    This is the query that is running for the datastore still:
    Code:
    SELECT *
    FROM datastore
    WHERE title IN ('attachmentcache','cron','profilefield','userstats','birthdaycache','maxloggedin','iconcache','eventcache','mailqueue')
    I don't think any hacks I have installed use the datastore...

    Leave a comment:


  • RedWingFan
    replied
    Hi there...

    Originally posted by Zachery View Post
    Do you have any hacks or addons that have data inside of the datastore table? and are you sure it is writing files to the correct directory?>

    Is the correct directory /includes/datastore? If so, what file(s) are written? I can check those. (If there were a file permissions problem, I'm sure vB would have thrown out an error message, no?)

    The lone plugin I was using is more of a template change. (It inserts a separator for sticky and normal threads.) It's disabled, but I could uninstall it completely if needed. The rest of the vB 3.6 installation is completely original--everything was uploaded new on Saturday morning. I even created a new default style and deleted all the old ones. No modifications yet--we're waiting for Gold before I do any more serious work to it.

    FWIW, here's a snapshot from my modified version of mytop (it omits some unnecessary columns so I can see more of the queries):

    Code:
    MySQL on localhost (4.1.20-log)                                      up 0+01:26:56 [18:40:50]
     Queries Total: 68,150         Avg/Sec: 13.07  Now/Sec: 7.26  Slow: 1,806
     Cache Hits   : 23,024         Avg/Sec: 4.41  Now/Sec: 0.00  Ratio: 102.49%
     Threads Total: 46        Active: 45    Cached: 137
     Key Efficiency: 99.10%  Bytes in: 22,782,141  Bytes out: 877,537,097
    
          Id      Time    Cmd Query or State
          --      ----    --- ----------
        4788        27  Query SELECT announcement.announcementid, startdate, title, announcement.
        4791        27  Query SELECT post.*, post.username AS postusername, post.ipaddress AS ip,
        4794        27  Query SELECT IF(visible = 2, 1, 0) AS isdeleted, thread.* FROM thread AS
        4800        27  Query SELECT filename, attachment.postid, attachment.userid, attachmentid
        4799        27  Query SELECT filename, attachment.postid, attachment.userid, attachmentid
        4802        27  Query SELECT cron.* FROM cron AS cron LEFT JOIN product AS product ON (cr
        4803        27  Query SELECT * FROM datastore WHERE title IN ('attachmentcache','cron','p
        4804        27  Query SELECT * FROM datastore WHERE title IN ('attachmentcache','cron','p
        4806        26  Query SELECT * FROM datastore WHERE title IN ('attachmentcache','cron','p
        4808        25  Query SELECT * FROM datastore WHERE title IN ('attachmentcache','cron','p
        4809        25  Query SELECT * FROM datastore WHERE title IN ('attachmentcache','cron','p
        4792        24  Query REPLACE INTO datastore (title, data, unserialize) VALUES ('cron', '
        4810        23  Query SELECT * FROM datastore WHERE title IN ('attachmentcache','cron','p
        4811        22  Query SELECT * FROM datastore WHERE title IN ('attachmentcache','cron','p
        4812        22  Query SELECT * FROM datastore WHERE title IN ('attachmentcache','cron','p
        4813        22  Query SELECT * FROM datastore WHERE title IN ('attachmentcache','cron','p
        4814        21  Query SELECT * FROM datastore WHERE title IN ('attachmentcache','cron','p
        4815        21  Query SELECT * FROM datastore WHERE title IN ('attachmentcache','cron','p
        4816        21  Query SELECT * FROM datastore WHERE title IN ('attachmentcache','cron','p
        4817        20  Query SELECT * FROM datastore WHERE title IN ('attachmentcache','cron','p
        4818        20  Query SELECT * FROM datastore WHERE title IN ('attachmentcache','cron','p
        4790        18  Query SELECT dateline, thumbnail_dateline, filename, filesize, visible, a
        4819        14  Query SELECT * FROM datastore WHERE title IN ('attachmentcache','cron','p
        4820        12  Query SELECT * FROM datastore WHERE title IN ('attachmentcache','cron','p
        4821        12  Query SELECT * FROM datastore WHERE title IN ('attachmentcache','cron','p
        4822        10  Query SELECT * FROM datastore WHERE title IN ('attachmentcache','cron','p
        4784         8  Query INSERT IGNORE INTO session (sessionhash, userid, host, idhash, last
        4786         8  Query SELECT forumid, moderatorid, permissions FROM moderator WHERE useri
        4823         8  Query SELECT * FROM datastore WHERE title IN ('attachmentcache','cron','p
        4824         8  Query SELECT * FROM datastore WHERE title IN ('attachmentcache','cron','p
        4825         7  Query SELECT * FROM datastore WHERE title IN ('attachmentcache','cron','p
        4826         6  Query SELECT * FROM datastore WHERE title IN ('attachmentcache','cron','p
        4807         5  Query SELECT * FROM style WHERE (styleid = 36 AND userselect = 1) OR styl
        4827         5  Query SELECT * FROM datastore WHERE title IN ('attachmentcache','cron','p
        4783         3  Query SELECT * FROM mailqueue ORDER BY mailqueueid LIMIT 10
        4798         3  Query SELECT filename, attachment.postid, attachment.userid, attachmentid
        4805         3  Query SELECT * FROM style WHERE (styleid = 36 AND userselect = 1) OR styl
        4828         3  Query SELECT * FROM datastore WHERE title IN ('attachmentcache','cron','p
        4797         2  Query (SELECT permissions, FIND_IN_SET(forumid, '2,-1') AS pos, forumid F
        4796         2  Query (SELECT permissions, FIND_IN_SET(forumid, '2,-1') AS pos, forumid F
        4795         1  Query (SELECT permissions, FIND_IN_SET(forumid, '2,-1') AS pos, forumid F
        4801         1  Query (SELECT permissions, FIND_IN_SET(forumid, '2,-1') AS pos, forumid F
        4829         1  Query SELECT * FROM datastore WHERE title IN ('attachmentcache','cron','p
        4502         0  Query show full processlist
        4830         0  Query SELECT * FROM datastore WHERE title IN ('attachmentcache','cron','p
        4831         0  Query SELECT * FROM datastore WHERE title IN ('attachmentcache','cron','p
    Thing is, I'll see a couple dozen queries like this, then I won't see datastore again for maybe 15-20 minutes.

    Leave a comment:


  • Zachery
    replied
    Do you have any hacks or addons that have data inside of the datastore table? and are you sure it is writing files to the correct directory?>

    Leave a comment:


  • RedWingFan
    replied
    Well, I just re-enabled storing it in the filesystem, but I see in mytop that we're still getting a couple dozen hits on the datastore table in a row (SELECT queries). I'm thinking it's not working correctly either...although it's not crashing anything as of yet. I will admit I'm not seeing as many hits to datastore as I used to, but maybe if I could read a good explanation of what it does and how it works, I could better determine if it's helping us or not.

    Leave a comment:


  • rossco_2005
    replied
    What does this even improve?
    I've got just as many queries on a page with or without the cache.

    Leave a comment:


  • Scott MacVicar
    replied
    You need to view the admincp, its tied in to only use that to stop potential race conditions on the forum which would result in it getting constantly updated

    Leave a comment:


  • RedWingFan
    replied
    Originally posted by Marco van Herwaarden View Post
    Bit 1 (the 'x') means something different for directories then for regular files.
    True, but I've "killed" directories before by not having the "X" bit set; in other words, on other applications, the directory was also inaccessible for reading, even if "R" was set. That's why I'm wary of directory permissions.

    How can I tell if the filesystem-based datastore is working? The default file is still in there, even after I've gone to tools.php and updated the datastore caches.

    Leave a comment:


  • Marco van Herwaarden
    replied
    Bit 1 (the 'x') means something different for directories then for regular files.

    File x-premission: Execute
    Directory x-permission: List/search permission (the permission to list the directory contents)

    Leave a comment:


  • RedWingFan
    replied
    That did the trick. Thanks!

    Shouldn't the directory be 777, though? I think you need the eXecute permission for it to be a directory...

    Leave a comment:


  • Floris
    replied
    You need to create a datastore folder inside the includes folder, set it to 666 and place .htaccess to prevent writing to it.

    Leave a comment:


  • RedWingFan
    started a topic Datastore in filesystem

    Datastore in filesystem

    I upgraded to 3.6.0RC3 over the weekend on our live forum, and it went extremely well. In config.php, I see there's an option to have the datastore in the filesystem, rather than in the database. I can change this in config.php, but do I need to set up any special file or directory permissions (like, chmod 777) for this to work? Where is the file stored? The instructions don't mention it, unless I missed it.

Related Topics

Collapse

Working...
X