# Benchmarks of cherokee vs nginx **URL:** \ **Category:** NGINX\ **Created:** [May 24, 2011, 2:09pm UTC](https://www.ruby-forum.com/t/benchmarks-of-cherokee-vs-nginx/207345 "2011-05-24T14:09:29Z")\ **Posts on this page:** 14\ **Page:** 1
**Author:** ![luislavena](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/l/0ea827/32.png) [@luislavena](https://www.ruby-forum.com/u/luislavena)\ **Post date:** [May 24, 2011, 2:09pm UTC](https://www.ruby-forum.com/t/benchmarks-of-cherokee-vs-nginx/207345/1 "2011-05-24T14:09:29Z")
This is static content (only http, and no php enabled), also using gzip-static.in nginx (cache-io doesn’t quite cut it in cherokee) I found the cache too aggressive in cherokee, if I upload a newer file I’d still keep serving the the cached file for a while (I wasn’t actually sure when it actually expired)… so I manually lowered the expiry time for the cache (900secs), performance dives 😕 Okay a quick break down of the stats… Nginx-generated traffic is cut in half (thanks to gzip-static) vs Cherokee Nginx: 118Mb Ram, Cherokee: 260Mb CPU: nginx is by-far using less, that race isn’t even close. A break down, [http://i.imgur.com/JVO1w.png](http://i.imgur.com/JVO1w.png) Both web servers packages are compiled on ubuntu 11.04/openvz hosting. ---
**Author:** ![casper\_the\_ghost](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/58956e/32.png) [@casper\_the\_ghost](https://www.ruby-forum.com/u/casper_the_ghost)\ **Post date:** [May 24, 2011, 6:30pm UTC](https://www.ruby-forum.com/t/benchmarks-of-cherokee-vs-nginx/207345/2 "2011-05-24T18:30:31Z")
Hmm You can easy compress things in cherokee too. Just enable it in cherokee-admin 😉 For “gzip static” like behaviour you need to enable flcache (with PURGE support for your config). IOCache in cherokee is caching only plain files when you don’t use gzip… That’s why. Greetings, Jędrzej Nowak ---
**Author:** ![casper\_the\_ghost](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/58956e/32.png) [@casper\_the\_ghost](https://www.ruby-forum.com/u/casper_the_ghost)\ **Post date:** [May 24, 2011, 6:31pm UTC](https://www.ruby-forum.com/t/benchmarks-of-cherokee-vs-nginx/207345/3 "2011-05-24T18:31:08Z")
> Both web servers packages are compiled on ubuntu 11.04/openvz hosting. Using openvz hosting? That is not even a real test. There are so many variables, tweaks and adjustments when using OpenVZ that you could not possibly count on that bench mark even if you controlled the server. If you are using a hosting providers server, it makes it even more nebulous because they will be using their own controls. The memory and processing units are not even for real memory. With OpenVZ you can tweak those at will. The only way to get real benchmarks is on a real machine. Tony 2011/5/24 Jędrzej Nowak [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid): ---
**Author:** ![casper\_the\_ghost](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/58956e/32.png) [@casper\_the\_ghost](https://www.ruby-forum.com/u/casper_the_ghost)\ **Post date:** [May 24, 2011, 7:17pm UTC](https://www.ruby-forum.com/t/benchmarks-of-cherokee-vs-nginx/207345/4 "2011-05-24T19:17:30Z")
I’m guessing any form of vps setup’s would have issues with benchmarking… I was hardly going to run this on my home connection or pay for a uber dedi. I’m not trying to play favourites, I’m just comparing the difference between the 2 web servers running on my vps, and its pretty obvious looking a long/short-term graphs, using the “oh you’re on a openvz/vps, therefore this doesn’t count” card doesn’t hold. The graphs don’t deviate much. The node isn’t under any great load, and it would show in the graphs (I could post longer 2-3-4 days but they’re pretty much the same). 2011/5/25 Tony Z. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid): ---
**Author:** ![casper\_the\_ghost](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/58956e/32.png) [@casper\_the\_ghost](https://www.ruby-forum.com/u/casper_the_ghost)\ **Post date:** [May 24, 2011, 7:36pm UTC](https://www.ruby-forum.com/t/benchmarks-of-cherokee-vs-nginx/207345/5 "2011-05-24T19:36:02Z")
Right, as with any benchmark, it is better to say up front what conditions you tested with, how you tested, and what variables were involved. So [myhost.com](http://myhost.com), with this VPS package, with Cherokee configured this way, and Nginx configured this way, etc. It is not a “this doesn’t count card” to you, but it is for most people. I am happy to see people share, after all, the conclusion would be, migrate to a host that better supports Cherokee if you love using it, or ask your host to tweak their VPS if you cannot get the benchmarks acceptable. The conclusion is not that Nginx is better than Cherokee. Even those memory and cpu numbers are totally fake. They are software numbers fed by OpenVZ to the container. Tony 2011/5/24 Jędrzej Nowak [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid): ---
**Author:** ![casper\_the\_ghost](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/58956e/32.png) [@casper\_the\_ghost](https://www.ruby-forum.com/u/casper_the_ghost)\ **Post date:** [May 24, 2011, 8:29pm UTC](https://www.ruby-forum.com/t/benchmarks-of-cherokee-vs-nginx/207345/6 "2011-05-24T20:29:14Z")
On Wed, 2011-05-25 at 01:02 +1200, Ryan B wrote: > I’m guessing any form of vps setup’s would have issues with > benchmarking.. I was hardly going to run this on my home connection or > pay for a uber dedi. Actually, it’s a fine benchmark so long as both HTTP servers are running in the same environment. Can you extrapolate and draw conclusions about how they might run on different hardware or in a different environment? Maybe with a grain of salt, but that’s the case with any benchmark, so complaining about OpenVZ tuning is pointless. Putting it on dedicated hardware isn’t terribly useful either, since fewer people do that now. Even dedicated servers are virtualized for management reasons. The only issue I’d have is if the node has other VE’s on it that you don’t have control over since those could clearly affect the outcome. Still I’d expect deviations to show from multiple runs that would reveal that type of issue. Cliff ---
**Author:** ![casper\_the\_ghost](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/58956e/32.png) [@casper\_the\_ghost](https://www.ruby-forum.com/u/casper_the_ghost)\ **Post date:** [May 24, 2011, 8:41pm UTC](https://www.ruby-forum.com/t/benchmarks-of-cherokee-vs-nginx/207345/7 "2011-05-24T20:41:36Z")
I am not complaining about OpenVZ. I deploy servers on a regular basis using OpenVZ. I also deploy using KVM, and others. There is a massive difference between OpenVZ and KVM in implementation, system stability, etc. They are entirely different animals. You cannot discount the hypervisor. The hypervisor can make or break your application depending on how you’re application is structured. For instance, Java apps run great on KVM hypervisors, but really poor if at all on OpenVZ unless you tune the OpenVZ instance to meet your Java apps needs. The reason for that is the way Java handles memory. Threads is another big issue hypervisors. To say that the hypervisor has no effect on application performance is not accurate at all. Tony ---
**Author:** ![casper\_the\_ghost](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/58956e/32.png) [@casper\_the\_ghost](https://www.ruby-forum.com/u/casper_the_ghost)\ **Post date:** [May 24, 2011, 8:54pm UTC](https://www.ruby-forum.com/t/benchmarks-of-cherokee-vs-nginx/207345/8 "2011-05-24T20:54:35Z")
On Tue, 2011-05-24 at 09:25 -0500, Tony Z. wrote: > has no effect on application performance is not accurate at all. > So are you suggesting no benchmark is ever valid unless it is run on > every hypervisor available as well as on the bare metal? Clearly every > benchmark has defined parameters, and in this case OpenVZ was one of > those parameters. So long as the benchmark is defined as “Nginx vs Cherokee under OpenVZ” then it’s a perfectly valid benchmark. Cliff ---
**Author:** ![casper\_the\_ghost](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/58956e/32.png) [@casper\_the\_ghost](https://www.ruby-forum.com/u/casper_the_ghost)\ **Post date:** [May 24, 2011, 9:02pm UTC](https://www.ruby-forum.com/t/benchmarks-of-cherokee-vs-nginx/207345/9 "2011-05-24T21:02:40Z")
On Tue, 2011-05-24 at 07:39 -0700, Cliff W. wrote: > > Threads is another big issue hypervisors. To say that the hypervisor > > has no effect on application performance is not accurate at all. > > So are you suggesting no benchmark is ever valid unless it is run on > every hypervisor available as well as on the bare metal? Clearly every > benchmark has defined parameters, and in this case OpenVZ was one of > those parameters. > > So long as the benchmark is defined as “Nginx vs Cherokee under OpenVZ” > then it’s a perfectly valid benchmark. Let me clarify that last sentence: there are other issues that might invalidate this benchmark (should be run on a dedicated LAN and on a dedicated node at least), but the virtualization (or lack of) isn’t one of those factors. Cliff ---
**Author:** ![casper\_the\_ghost](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/58956e/32.png) [@casper\_the\_ghost](https://www.ruby-forum.com/u/casper_the_ghost)\ **Post date:** [May 24, 2011, 10:53pm UTC](https://www.ruby-forum.com/t/benchmarks-of-cherokee-vs-nginx/207345/10 "2011-05-24T22:53:24Z")
Was there any info on how OpenVZ was setup or how many other containers were running on the machine, Do we know what the load was on the machine at each stage of the tests? We really do not know anything. The web servers were not even setup the same. The hosting outfit could see a system spike and automatically kick in a limit that we do not know about. I do that all the time. That is a common practice. If you do not control the system, you cannot do a valid benchmark. ---
**Author:** ![casper\_the\_ghost](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/58956e/32.png) [@casper\_the\_ghost](https://www.ruby-forum.com/u/casper_the_ghost)\ **Post date:** [May 24, 2011, 7:35pm UTC](https://www.ruby-forum.com/t/benchmarks-of-cherokee-vs-nginx/207345/11 "2011-05-24T19:35:25Z")
Well… You compared cherokee without GZIP to nginx with GZIP… And you’re comparing a memory used by cache… It’s a miss point. Greetings, Jędrzej Nowak ---
**Author:** ![casper\_the\_ghost](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/58956e/32.png) [@casper\_the\_ghost](https://www.ruby-forum.com/u/casper_the_ghost)\ **Post date:** [May 25, 2011, 1:35am UTC](https://www.ruby-forum.com/t/benchmarks-of-cherokee-vs-nginx/207345/12 "2011-05-25T01:35:27Z")
On Tue, 2011-05-24 at 09:50 -0500, Tony Z. wrote: > Was there any info on how OpenVZ was setup or how many other > containers were running on the machine, Do we know what the load was > on the machine at each stage of the tests? We really do not know > anything. The web servers were not even setup the same. The hosting > outfit could see a system spike and automatically kick in a limit that > we do not know about. I do that all the time. That is a common > practice. If you do not control the system, you cannot do a valid > benchmark. I agree with that. I only disagree with your earlier, broader assessment. Cliff ---
**Author:** ![casper\_the\_ghost](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/58956e/32.png) [@casper\_the\_ghost](https://www.ruby-forum.com/u/casper_the_ghost)\ **Post date:** [May 24, 2011, 10:53pm UTC](https://www.ruby-forum.com/t/benchmarks-of-cherokee-vs-nginx/207345/13 "2011-05-24T22:53:24Z")
And you certainly cannot report on memory consumption relying on OpenVZ fake memory numbers. ---
**Author:** ![casper\_the\_ghost](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/58956e/32.png) [@casper\_the\_ghost](https://www.ruby-forum.com/u/casper_the_ghost)\ **Post date:** [May 25, 2011, 3:09am UTC](https://www.ruby-forum.com/t/benchmarks-of-cherokee-vs-nginx/207345/14 "2011-05-25T03:09:48Z")
On Tue, May 24, 2011 at 10:04 AM, Cliff W. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > I agree with that. I only disagree with your earlier, broader > assessment. > > Cliff I should have been more clear. Point well taken.