On Mon, 07 Apr 2008 23:14:47 -0700
Yay! You mentioned standard deviations. Bad! You didn’t give any.
That’d help people figure out if the 4% is even worth it.
I did! I did! The raw data (including std dev’s) were linked off the
main post. That link is:
http://www.misuse.org/downloads/thin_mongrel_test_results.html - I
thought that level of info on the main post would be too
distracting/confusing for many readers…
I tried to be meticulous in collecting these data, and Zed, for every
std dev cut and paste out of JMeter I thought off you.
“Thin is the newest incarnation of Zed S.'s (and now community
managed) masterwork Mongrel.”
That statement isn’t correct. Please don’t make it since I don’t
the author of Thin or the authors of Mongrel would say that. Oh, well
they’d say my stuff is a masterwork, just not that Thin is an
incarnation of Mongrel.
Ok - that’s fixed. I said that b/c I thought Thin is using
your/mongrel’s ragel http states. But you know (way) more than me on
this. So I changed the language.
I don’t think it’s unix sockets vs. ip sockets. These days those
aren’t really that much faster than a localhost connection thanks to
advancements in performance for internal IPC.
The Thin author Marc said the same thing. I’ll adjust that in the
Another comment I’d make is, why not also release the methodology you
gave. I’m actually working on a presentation for RuPy so if you’d
to hack on a repeatable study with an automated report let me know.
Could be a good test for what I’m writing for the conference.
Thanks for the encouragement. I just published my methodology, which
was the same for both tests, at the end of the second report:
Feedback and criticism are welcome on that methodology. I have done a
fair bit of performance testing in the past, but not in such a general
sense as comparing two different application servers, so I may have
done something wrong.
I’d also be open to performing more tests using this methodology and
codebase, but I don’t have access to an unloaded high performance
webserver anymore. I could probably talk Ezra or Wayne into loaning me
Drop me a line here or personally if that sounds useful.