Benchmark Comparison of 1.3 RC3 and 1.2.6
This article is also available in the following languages:
CakePHP 1.3 has lots of new features.
But we may worry about the cost for addition of them.
This article reports result of benchmarks that we can compare the performance of 1.3 RC3 and 1.2.6.
But we may worry about the cost for addition of them.
This article reports result of benchmarks that we can compare the performance of 1.3 RC3 and 1.2.6.
Environment
- OS Windows Vista 32bit Home Edition
- PHP5.2.6
- No APC
- mysql Ver 14.14 Distrib 5.1.42, for Win32 (ia32)
Setup
- Download archives from http://github.com/cakephp/cakephp1x/downloads
- Extract them into following directory.
- WEBROOT/cakebench/12 <= 1.2.6
- WEBROOT/cakebench/13 <= 1.3 RC3
- Rewriting config/core.php
- Configure::write('debug', 0); // production mode
- Rewrite Security.salt and Security.cipherSeed to proper value
- Set Security.level to medium for both version.
Route:
<?php
Router::connect('/', array('controller' => 'bench', 'action' => 'index'));
Router::connect('/:action', array('controller' => 'bench'));
?>
Controller Class:
<?php
// controllers/bench_controller.php
class BenchController extends AppController{
var $name = "Bench";
var $uses = array();
function index(){
}
function model(){
$this->loadModel('Event');
$this->helpers = array('Form');
}
}
?>
Model Class:
<?php
// models/event.php
class Event extends AppModel{
var $name = "Event";
}
?>
View templates:
layouts/default.ctp
<?php echo $content_for_layout ?>
bench/index.ctp
Hello World!!
bench/model.ctp
<?php
echo $form->create('Event');
echo $form->input('name');
echo $form->end('submit');
?>
Comparison method
Use Apache Bench for the test with "-c 100 -n 1000" option.Test the command for 10 times, then take the stuff that Request per second(#/sec) is the largest for comparison.
Simple Hello World Application
This tests "index" action of the controller.1.2.6
C:\apache\bin>ab -c 100 -n 1000 http://localhost/cakebench/12/
...(omitted)
Requests per second: 13.99 [#/sec] (mean)
Time per request: 7147.920 [ms] (mean)
Time per request: 71.479 [ms] (mean, across all concurrent requests)
...
1.3 RC3
C:\apache\bin>ab -c 100 -n 1000 http://localhost/cakebench/13/There was no significant difference.
...
Requests per second: 13.97 [#/sec] (mean)
Time per request: 7160.400 [ms] (mean)
Time per request: 71.604 [ms] (mean, across all concurrent requests)
...
Loading Model
This tests "model" action of the controller.Loading just one model and output very simple form with form helper.
1.2.6
C:\apache\bin>ab -c 100 -n 1000 http://localhost/cakebench/12/model
...
Requests per second: 8.35 [#/sec] (mean)
Time per request: 11977.680 [ms] (mean)
Time per request: 119.777 [ms] (mean, across all concurrent requests)
...
1.3 RC3
C:\apache\bin>ab -c 100 -n 1000 http://localhost/cakebench/13/model1.3 RC3 is about 1.8% faster than 1.2.6.
...
Requests per second: 8.50 [#/sec] (mean)
Time per request: 11767.080 [ms] (mean)
Time per request: 117.671 [ms] (mean, across all concurrent requests)
...
This difference cannot be ignored.
Finally
This benchmarks were done on very simple cases.Also, CakePHP1.3 has lots of performance improvement that this benchmarks does not test.
Fortunately, my friend reported his application is over 20% faster on 1.3 RC3 than on 1.2.6.
Could you do benchmarks when your application migrate 1.2.x to 1.3 ?
If the database architecture is setup properly, your site/app will run better with CakePHP 1.1 than CakePHP 1.3 with poor database architecture. Fractions of a second between Cake versions and even other frameworks out there. You may even read about Cake compared to other frameworks and all the benchmarks out there.
So I'm calling major BS on all of it...(not you, or your tests, more data is always useful to someone) You know what the "benchmarks" (most done incorrectly) and blogs about which CMS or framework is faster do? They make my day a living hell because some "CTO" (who doesn't jack about web development) sees information like that and it's false...But they don't know any better, and they say that I need to use X, Y and Z to get the job done. That's exactly the effect benchmarks have on this industry. Someone "in charge" loads it up like it's ammunition and then declares unholy programmer war on my sorry rear end. So, it's not that I don't believe in them, it's that I believe in every site being a very unique case. Unique traffic, unique code base, and unique database schema. Everyone's benchmarks will vary.
...But while we're on the subject...Take a look at Pylot too, it has a nearly as simple procedure as ab but is much better, more comprehensive, and it gives you some pretty graphs.
"This benchmarks were done on very simple cases.
Real applications is more complicated than such a simple application.
It means the more complicated your application, the faster your application if you use 1.3."
A simple FormHelper test is faster. Therefore the entire thing is faster. Rather illogical. Get rid of that statement and the quality of the article will be greatly improved.
I will remove following 2 lines.
"Real applications is more complicated than such a simple application.
It means the more complicated your application, the faster your application if you use 1.3."
Thank you for your review :)
Comments are closed for articles over a year old