vBulletin 3.0, 3.5, 3.6, 3.7 benchmarking

[mrlister]

Hey guys, so I was really curious as to what the trend in vBulletin speed was. I run a big board so it's important the script run fast. I ran a few benchmarks and here are the stats.

Downloaded 10,000 times with 50 connections per second. Same server. (Server has no external traffic). I ran the first test twice to make sure that it's running fine. All other pages were ran just once. The test was done right after the install, with no tweaking, caching, plug-ins or mods of any kind.

Home page:

3 26.535514 26.255133
3.5 34.39289 34.25656
3.6 35.250994 35.382716
3.7 36.71319 36.602198

Thread list:

3 30.336566
3.5 37.176372
3.6 39.115502
3.7 42.662701

Show post:

3 37.977961
3.5 47.88809
3.6 48.498011
3.7 52.642907

[Dilly]

With what features enabled?

[mrlister]

Out of the box, right after install.

[Dilly]

OK, but doesn't that make it an unfair test?

If you wanted to benchmark the actual software, you need to set it so that the optional features added by the later versions are turned off, otherwise you aren't really testing the speed of the software, you are testing the speed of the software + extra features.

[mrlister]

Maybe, I'm interested in the pure speed, rather than the features considering there are barely any differences on those pages (extra ajax as versions progress) but no extreme features.

Personally I don't see any of these extra features on the different versions. (other than ajax and HTML changes)

[Redseal]

I wonder if it is the change in html that is slowing things down. I dont know much about benchmarking, but would rendering the page and then styling the page via css be a reason for the load times?

Did 3.0 even have css or was it all hard coded? Oh and are those times in miliseconds?

[mrlister]

Those times are in seconds. HTML I can guarantee does not account for any of it since all pages are very close in size.

What those results mean is 10,000 pages were downloaded in __ seconds using 50 concurrent connections.

[Joe Gronlund]

Tests look ok to me, i would expect a slight increase....
Nice job!!

[Redseal]

Cool, it would be weird if I ever had a board that big. I would be famous!

[ManagerJosh]

It's a partially fair examination. However it would be nice to see a common set of features performed to do the test.