Benchmarks of cherokee vs nginx

This is static content (only http, and no php enabled), also using
gzip-static.in nginx (cache-io doesn’t quite cut it in cherokee)

I found the cache too aggressive in cherokee, if I upload a newer file
I’d still keep serving the the cached file for a while (I wasn’t
actually sure when it actually expired)… so I manually lowered the
expiry time for the cache (900secs), performance dives :confused:

Okay a quick break down of the stats…

Nginx-generated traffic is cut in half (thanks to gzip-static) vs
Cherokee
Nginx: 118Mb Ram, Cherokee: 260Mb
CPU: nginx is by-far using less, that race isn’t even close.

A break down, http://i.imgur.com/JVO1w.png

Both web servers packages are compiled on ubuntu 11.04/openvz hosting.

Hmm

You can easy compress things in cherokee too. Just enable it in
cherokee-admin :wink:
For “gzip static” like behaviour you need to enable flcache (with
PURGE support for your config).

IOCache in cherokee is caching only plain files when you don’t use
gzip… That’s why.

Greetings,
Jędrzej Nowak

Both web servers packages are compiled on ubuntu 11.04/openvz hosting.

Using openvz hosting? That is not even a real test. There are so
many variables, tweaks and adjustments when using OpenVZ that you
could not possibly count on that bench mark even if you controlled the
server. If you are using a hosting providers server, it makes it even
more nebulous because they will be using their own controls. The
memory and processing units are not even for real memory. With
OpenVZ you can tweak those at will.

The only way to get real benchmarks is on a real machine.

Tony

2011/5/24 Jędrzej Nowak [email protected]:

I’m guessing any form of vps setup’s would have issues with
benchmarking… I was hardly going to run this on my home connection or
pay for a uber dedi. I’m not trying to play favourites, I’m just
comparing the difference between the 2 web servers running on my vps,
and its pretty obvious looking a long/short-term graphs, using the “oh
you’re on a openvz/vps, therefore this doesn’t count” card doesn’t
hold.

The graphs don’t deviate much. The node isn’t under any great load,
and it would show in the graphs (I could post longer 2-3-4 days but
they’re pretty much the same).

2011/5/25 Tony Z. removed_email_addr[email protected]:

Right, as with any benchmark, it is better to say up front what
conditions you tested with, how you tested, and what variables were
involved.

So myhost.com, with this VPS package, with Cherokee configured this
way, and Nginx configured this way, etc.

It is not a “this doesn’t count card” to you, but it is for most
people. I am happy to see people share, after all, the conclusion
would be, migrate to a host that better supports Cherokee if you love
using it, or ask your host to tweak their VPS if you cannot get the
benchmarks acceptable. The conclusion is not that Nginx is better
than Cherokee.

Even those memory and cpu numbers are totally fake. They are software
numbers fed by OpenVZ to the container.

Tony

2011/5/24 Jędrzej Nowak [email protected]:

On Wed, 2011-05-25 at 01:02 +1200, Ryan B wrote:

I’m guessing any form of vps setup’s would have issues with
benchmarking… I was hardly going to run this on my home connection or
pay for a uber dedi.

Actually, it’s a fine benchmark so long as both HTTP servers are running
in the same environment. Can you extrapolate and draw conclusions about
how they might run on different hardware or in a different environment?
Maybe with a grain of salt, but that’s the case with any benchmark, so
complaining about OpenVZ tuning is pointless.

Putting it on dedicated hardware isn’t terribly useful either, since
fewer people do that now. Even dedicated servers are virtualized for
management reasons.

The only issue I’d have is if the node has other VE’s on it that you
don’t have control over since those could clearly affect the outcome.
Still I’d expect deviations to show from multiple runs that would reveal
that type of issue.

Cliff

I am not complaining about OpenVZ. I deploy servers on a regular
basis using OpenVZ. I also deploy using KVM, and others. There is a
massive difference between OpenVZ and KVM in implementation, system
stability, etc. They are entirely different animals. You cannot
discount the hypervisor. The hypervisor can make or break your
application depending on how you’re application is structured. For
instance, Java apps run great on KVM hypervisors, but really poor if
at all on OpenVZ unless you tune the OpenVZ instance to meet your Java
apps needs. The reason for that is the way Java handles memory.
Threads is another big issue hypervisors. To say that the hypervisor
has no effect on application performance is not accurate at all.

Tony

On Tue, 2011-05-24 at 09:25 -0500, Tony Z. wrote:

has no effect on application performance is not accurate at all.
So are you suggesting no benchmark is ever valid unless it is run on
every hypervisor available as well as on the bare metal? Clearly every
benchmark has defined parameters, and in this case OpenVZ was one of
those parameters.

So long as the benchmark is defined as “Nginx vs Cherokee under OpenVZ”
then it’s a perfectly valid benchmark.

Cliff

On Tue, 2011-05-24 at 07:39 -0700, Cliff W. wrote:

Threads is another big issue hypervisors. To say that the hypervisor
has no effect on application performance is not accurate at all.

So are you suggesting no benchmark is ever valid unless it is run on
every hypervisor available as well as on the bare metal? Clearly every
benchmark has defined parameters, and in this case OpenVZ was one of
those parameters.

So long as the benchmark is defined as “Nginx vs Cherokee under OpenVZ”
then it’s a perfectly valid benchmark.

Let me clarify that last sentence: there are other issues that might
invalidate this benchmark (should be run on a dedicated LAN and on a
dedicated node at least), but the virtualization (or lack of) isn’t one
of those factors.

Cliff

Was there any info on how OpenVZ was setup or how many other
containers were running on the machine, Do we know what the load was
on the machine at each stage of the tests? We really do not know
anything. The web servers were not even setup the same. The hosting
outfit could see a system spike and automatically kick in a limit that
we do not know about. I do that all the time. That is a common
practice. If you do not control the system, you cannot do a valid
benchmark.

Well…

You compared cherokee without GZIP to nginx with GZIP… And you’re
comparing a memory used by cache…
It’s a miss point.

Greetings,
Jędrzej Nowak

On Tue, 2011-05-24 at 09:50 -0500, Tony Z. wrote:

Was there any info on how OpenVZ was setup or how many other
containers were running on the machine, Do we know what the load was
on the machine at each stage of the tests? We really do not know
anything. The web servers were not even setup the same. The hosting
outfit could see a system spike and automatically kick in a limit that
we do not know about. I do that all the time. That is a common
practice. If you do not control the system, you cannot do a valid
benchmark.

I agree with that. I only disagree with your earlier, broader
assessment.

Cliff

And you certainly cannot report on memory consumption relying on
OpenVZ fake memory numbers.

On Tue, May 24, 2011 at 10:04 AM, Cliff W. [email protected] wrote:

I agree with that. I only disagree with your earlier, broader
assessment.

Cliff

I should have been more clear. Point well taken.

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs