Announcement

Collapse
No announcement yet.

12 minutes after opening -- 500 people online

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Negligible? Interesting... in most 3 tier systems I have built, persistent connections make a huge difference since getting a DB connection can take over 1 second to establish in oracle. Hmmmm.....



    Originally posted by Martz
    That is down to your configuration.

    From includes/config.php

    // ****** PERSISTENT CONNECTIONS ******
    // This option allows you to turn persistent connections to MySQL on or off.
    // The difference in performance is negligible for all but the largest boards.
    // If you are unsure what this should be, leave it off.
    // 0 = Off; 1 = On
    $usepconnect = 0;

    Comment


    • Originally posted by peterpaine
      Negligible? Interesting... in most 3 tier systems I have built, persistent connections make a huge difference since getting a DB connection can take over 1 second to establish in oracle. Hmmmm.....
      You'd be best asking these questions directly in the mySQL forum, since I think this is entirely down to mySQL configuration. If vBulletin is optmised to make use of persistant connections is another matter and I don't know, but from the config comments I'd say that only the larger/busier boards would see any benefit from $pconnect being on.
      HP DL-380 G6, 2x E5520, 28GB RAM, 4x300GB SAS, VMWare ESXi
      -
      Unreal Tournament : Assault forums - irc://irc.utassault.net:6667 -

      Comment


      • I hear that if you're a vbulletin customer, as we all are that as a free service provided as part of your license they will optimize your MySQL setup. 3 Questions...

        Is that true?
        Has it been done on the Stern site
        How do I get somone to do it to mine?

        I own my server and have full access to it (even physically).

        Comment


        • For help with optimizing your server, please see this thread:

          http://www.vbulletin.com/forum/showt...threadid=70117

          Then post the requested info in a new thread in that forum.
          Steve Machol, former vBulletin Customer Support Manager (and NOT retired!)
          Change CKEditor Colors to Match Style (for 4.1.4 and above)

          Steve Machol Photography


          Mankind is the only creature smart enough to know its own history, and dumb enough to ignore it.


          Comment


          • Having run one of the top 10 largest vbulletin forums for the past couple years (ok so we only got big recently, but whatever), I've done more than my share of research into what can be done to keep high traffic sites afloat.

            One of my most valuable insights into truly high volume sites has been from the guys over at livejournal.com who have documented the problems they've encountered...

            One thing that everyone has been overlooking this whole time is that while nobody is denying that it is expensive to run a high volume site, that not everyone is willing to toss enterprise level quad processor machines at vbulletin sites.

            It's just not needed! By adding some level of database abstraction to the app, you could be using 1u pentium 4 cheapy nodes as database slave servers and not have to spend $15,000+ on a 4-way database server that is going to end up bottlenecking eventually anyway!

            If vbulletin wants to really seal the deal, it NEEDS to abstract read/writes somehow.

            Envision a setup where you have your webservers loadbalanced (we loadbalance our dynamic apache servers (which use mmcache), and our thttpd image servers via a hardware loadbalancer) and then say, 3 mysql servers, 2 of which are "cheap" boxes, say p4s or xeons with 2gb of ram, and a larger master database (maybe a dualie xeon or opteron or something < $5000).

            All writes hit the master, and the slave databases are well, slaves.

            They can do the brunt of the work and free up the writes from blocking (making posting nice and fast), and you can actually scale your SQL backend. The way things are now, you are going to hit a limit either financially (not being able to afford some giant box to handle your sql stuff, or hardware wise as even tossing a 4-way beast at howards minions will bring it to its knees).
            Kyle Christensen
            PbNation.com - one of the biggest and busiest vbulletin forums on the net!

            Comment


            • Originally posted by KrON
              Having run one of the top 10 largest vbulletin forums for the past couple years (ok so we only got big recently, but whatever), I've done more than my share of research into what can be done to keep high traffic sites afloat.

              One of my most valuable insights into truly high volume sites has been from the guys over at livejournal.com who have documented the problems they've encountered...

              One thing that everyone has been overlooking this whole time is that while nobody is denying that it is expensive to run a high volume site, that not everyone is willing to toss enterprise level quad processor machines at vbulletin sites.

              It's just not needed! By adding some level of database abstraction to the app, you could be using 1u pentium 4 cheapy nodes as database slave servers and not have to spend $15,000+ on a 4-way database server that is going to end up bottlenecking eventually anyway!

              If vbulletin wants to really seal the deal, it NEEDS to abstract read/writes somehow.

              Envision a setup where you have your webservers loadbalanced (we loadbalance our dynamic apache servers (which use mmcache), and our thttpd image servers via a hardware loadbalancer) and then say, 3 mysql servers, 2 of which are "cheap" boxes, say p4s or xeons with 2gb of ram, and a larger master database (maybe a dualie xeon or opteron or something < $5000).

              All writes hit the master, and the slave databases are well, slaves.

              They can do the brunt of the work and free up the writes from blocking (making posting nice and fast), and you can actually scale your SQL backend. The way things are now, you are going to hit a limit either financially (not being able to afford some giant box to handle your sql stuff, or hardware wise as even tossing a 4-way beast at howards minions will bring it to its knees).
              i didnt understand a word of that, but it sounded cool

              Comment


              • I like the idea of sending writes to the master and the slave would purely be used for reads and it doesn't look too overly complicated in the way of implementation. Though we'd need to do sit and go through the code making sure it uses the appropriate database class.

                http://www.danga.com/words/2004_mysq...sql-slides.pdf

                I guess it would be something to look at now.

                Originally posted by KrON
                Having run one of the top 10 largest vbulletin forums for the past couple years (ok so we only got big recently, but whatever), I've done more than my share of research into what can be done to keep high traffic sites afloat.

                One of my most valuable insights into truly high volume sites has been from the guys over at livejournal.com who have documented the problems they've encountered...

                One thing that everyone has been overlooking this whole time is that while nobody is denying that it is expensive to run a high volume site, that not everyone is willing to toss enterprise level quad processor machines at vbulletin sites.

                It's just not needed! By adding some level of database abstraction to the app, you could be using 1u pentium 4 cheapy nodes as database slave servers and not have to spend $15,000+ on a 4-way database server that is going to end up bottlenecking eventually anyway!

                If vbulletin wants to really seal the deal, it NEEDS to abstract read/writes somehow.

                Envision a setup where you have your webservers loadbalanced (we loadbalance our dynamic apache servers (which use mmcache), and our thttpd image servers via a hardware loadbalancer) and then say, 3 mysql servers, 2 of which are "cheap" boxes, say p4s or xeons with 2gb of ram, and a larger master database (maybe a dualie xeon or opteron or something < $5000).

                All writes hit the master, and the slave databases are well, slaves.

                They can do the brunt of the work and free up the writes from blocking (making posting nice and fast), and you can actually scale your SQL backend. The way things are now, you are going to hit a limit either financially (not being able to afford some giant box to handle your sql stuff, or hardware wise as even tossing a 4-way beast at howards minions will bring it to its knees).
                Scott MacVicar

                My Blog | Twitter

                Comment


                • Originally posted by Scott MacVicar
                  I like the idea of sending writes to the master and the slave would purely be used for reads and it doesn't look too overly complicated in the way of implementation. Though we'd need to do sit and go through the code making sure it uses the appropriate database class.

                  http://www.danga.com/words/2004_mysq...sql-slides.pdf

                  I guess it would be something to look at now.
                  If you were to use the same database class, and instead use query_write or query_select functions, wouldnt that do a better job than using 2 classes to manage database connections?

                  Especially when most forums will only need one connection, not 2.

                  Comment


                  • Originally posted by Scott MacVicar
                    I like the idea of sending writes to the master and the slave would purely be used for reads and it doesn't look too overly complicated in the way of implementation. Though we'd need to do sit and go through the code making sure it uses the appropriate database class.

                    http://www.danga.com/words/2004_mysq...sql-slides.pdf

                    I guess it would be something to look at now.

                    Scott, I would love you forever if you guy's implemented an abstraction layer.

                    Tony Rieker | Founder of PBNation.com - one of the LARGEST vBulletin forums on the net!

                    Comment


                    • Scott, one of the places I've seen a lot of technical babble about LJ backend and architecture stuff is in these two communities:

                      http://www.livejournal.com/userinfo....lj_maintenance

                      http://www.livejournal.com/userinfo.bml?user=lj_backend

                      Brad from LJ has also given several talks that I've seen slides from (like what you posted) that have some good points in them.

                      I'm not sure much of it is really applicable to vBulletin, but I find it helpful to keep some of the points they bring up in mind when doing development/design (or at least thinking about ways to improve stuff I work with).

                      One of the things I was thinking about also was trying to figure out the best way to scale with multiple slave servers. Since I have a hardware load balancer, I could use that to loadbalance connections to the slave servers (and run them on the webservers themselves), or to remote servers.

                      For those w/o the benefit of a hardware load balancer, i wonder if it would be better to have an option in the vbcontrol panel to specify multiple slave server ip addresses, or to just do round robin dns.
                      Kyle Christensen
                      PbNation.com - one of the biggest and busiest vbulletin forums on the net!

                      Comment


                      • The overhead of two connections in minimum and we have a class as long as two different objects were created. $DB_site and $DB_wsite. There might be a bit of lag between master and slave so some reads should be done at master. Flood checking for one.
                        Scott MacVicar

                        My Blog | Twitter

                        Comment


                        • Originally posted by KrON
                          Having run one of the top 10 largest vbulletin forums for the past couple years (ok so we only got big recently, but whatever), I've done more than my share of research into what can be done to keep high traffic sites afloat.

                          One of my most valuable insights into truly high volume sites has been from the guys over at livejournal.com who have documented the problems they've encountered...

                          One thing that everyone has been overlooking this whole time is that while nobody is denying that it is expensive to run a high volume site, that not everyone is willing to toss enterprise level quad processor machines at vbulletin sites.

                          It's just not needed! By adding some level of database abstraction to the app, you could be using 1u pentium 4 cheapy nodes as database slave servers and not have to spend $15,000+ on a 4-way database server that is going to end up bottlenecking eventually anyway!

                          If vbulletin wants to really seal the deal, it NEEDS to abstract read/writes somehow.

                          Envision a setup where you have your webservers loadbalanced (we loadbalance our dynamic apache servers (which use mmcache), and our thttpd image servers via a hardware loadbalancer) and then say, 3 mysql servers, 2 of which are "cheap" boxes, say p4s or xeons with 2gb of ram, and a larger master database (maybe a dualie xeon or opteron or something < $5000).

                          All writes hit the master, and the slave databases are well, slaves.

                          They can do the brunt of the work and free up the writes from blocking (making posting nice and fast), and you can actually scale your SQL backend. The way things are now, you are going to hit a limit either financially (not being able to afford some giant box to handle your sql stuff, or hardware wise as even tossing a 4-way beast at howards minions will bring it to its knees).
                          That is a great idea for a setup.
                          Avatar Chat

                          Comment


                          • Originally posted by Scott MacVicar
                            Well they could try 3.0.3 with the fulltext search and if they could get a little mail server they could use the SMTP code.

                            This would reduce locking of postindex / word and get rid of mail causing load spikes.

                            A few added indexes should also reduce the amount of data scanned. The problem is generally not SQL now we need to try and to make improvements in the code, also regarding Invision using Oracle / MSSQL they just use standard queries and none of the advanced features. Foreign Keys and Stored Procedures so they might as well use MySQL with InnoDB.
                            Which other indexes should be added that are not added by default? The ipaddress one in post is one I did which helped.
                            Avatar Chat

                            Comment


                            • Howard Stern Bulletin Board 116,996 members with 924,705 posts in 59,764 topics

                              I was able to register today. Talk about one crazy site. People are posting like crazy.

                              Comment


                              • Heya Scott,


                                Any news or information regarding the abstraction layer? I think it would sincerely help vbulletin intallations that are under heavy usage.

                                Thanks
                                Last edited by TonyPBN; Thu 30th Sep '04, 4:09pm. Reason: test
                                Tony Rieker | Founder of PBNation.com - one of the LARGEST vBulletin forums on the net!

                                Comment

                                widgetinstance 262 (Related Topics) skipped due to lack of content & hide_module_if_empty option.
                                Working...
                                X