On Mon, 07 Apr 2008 23:14:47 -0700
http://www.misuse.org/science/2008/04/07/thin-ruby-on-rails-nginx-fair-proxy-performance-testing/
Yay! You mentioned standard deviations. Bad! You didn’t give any.
That’d help people figure out if the 4% is even worth it.
I did! I did! The raw data (including std dev’s) were linked off the
main post. That link is:
http://www.misuse.org/downloads/thin_mongrel_test_results.html - I
thought that level of info on the main post would be too
distracting/confusing for many readers…
I tried to be meticulous in collecting these data, and Zed, for every
std dev cut and paste out of JMeter I thought off you. 
[snip]
“Thin is the newest incarnation of Zed S.'s (and now community
managed) masterwork Mongrel.”That statement isn’t correct. Please don’t make it since I don’t
think
the author of Thin or the authors of Mongrel would say that. Oh, well
they’d say my stuff is a masterwork, just not that Thin is an
incarnation of Mongrel.
Ok - that’s fixed. I said that b/c I thought Thin is using
your/mongrel’s ragel http states. But you know (way) more than me on
this. So I changed the language.
[snip]
I don’t think it’s unix sockets vs. ip sockets. These days those
aren’t really that much faster than a localhost connection thanks to
advancements in performance for internal IPC.
The Thin author Marc said the same thing. I’ll adjust that in the
article soon.
[snip]
Another comment I’d make is, why not also release the methodology you
gave. I’m actually working on a presentation for RuPy so if you’d
like
to hack on a repeatable study with an automated report let me know.
Could be a good test for what I’m writing for the conference.
Thanks for the encouragement. I just published my methodology, which
was the same for both tests, at the end of the second report:
http://www.misuse.org/science/2008/04/07/thin-vs-mongrel-a-ruby-on-rails-performance-shootout/
Feedback and criticism are welcome on that methodology. I have done a
fair bit of performance testing in the past, but not in such a general
sense as comparing two different application servers, so I may have
done something wrong.
I’d also be open to performing more tests using this methodology and
codebase, but I don’t have access to an unloaded high performance
webserver anymore. I could probably talk Ezra or Wayne into loaning me
one though.
Drop me a line here or personally if that sounds useful.
Steve