Mongrel 0.1.0 -- A Fast As Hell Mostly Ruby Web Server

Ruby
1

Hi,

This is an announce for Mongrel, a mostly Ruby webserver. I just
started it as I was trying to work on solutions for people deploying
Ruby web applications, and already it’s pretty damn fast. We’re
talking really really damn fast. I mean like people won’t believe me
that it’s this fast and I don’t really believe it.

INSTALL

First off, you can grab it at http://www.zedshaw.com/downloads/
mongrel-0.1.0.tar.bz2 to play with it.

*** You’ll need Ruby 1.8.4, but it’s tested on Mac OSX and Linux at
least. ***

When you get it do:

rake
ruby -Ilib examples/simpletest.rb

Go to http://localhost:3000/test to play with it. Other URIs give
you a lame “not found”.

HOW?

Feel free to look at the source in lib/mongrel.rb and if you want to
go insane you can try to grok the HTTP parser in ext/http11/*.c (not
that bad really).

CAN IT BAKE CHEDDAR?

So far it’s very light and isn’t nearly capable enough, so I’m
looking for things people want in a simple web app server for
running their Ruby web applications in. Keep in mind that I’m not
going to be Ruby on Rails specific. I love all you weirdos.

Now for the fun (and surely flamewar generating) statistics on how it
compares to WEBrick:

WEBrick/Linux: 175 req/sec
Mongrel/Linux: 1350 req/sec

WEBrick/OSX: 5.3 req/sec
Mongrel/OSX: 304.4 req/sec

– Both tests were a simple handler that sent back “hello!” when /
test was accessed. WEBrick’s logging was redirected to /dev/null
since I can’t figure out how to turn it the hell off.

The OSX results are pathetic and I suspect there’s something
seriously wrong with my setup since OSX does considerably worse than
the comparable Linux machine. But, the important thing is that
Mongrel seems to be doing way better than WEBrick. I’m hoping to
keep the high performance, but don’t trust my word for it, go grab
the source and grab httperf to see if you get similar performance on
your machines. Hell, it might even work on Win32.

DOWNSIDES

It uses some Ruby 1.8.4 stuff and it uses a C extension for the HTTP
parsing. The C is all ANSI so you should be OK, but let me know if
it don’t compile somewhere. It’s also nowhere near complete, mostly
just enough code to prove the concept.

FEEDBACK

Since I just started this, and I’m getting such great performance out
of it already, I’m looking to get it working with Rails this weekend
(along with SCGI fixes to keep people happy), then maybe Og+Nitro.
I’d love for people to send me their wish list and opinions on this.
I’m mostly motivated to write this as the answer to Java’s Tomcat.
What are your thoughts? Too ambitious? Not enough?

I’m especially interested if you can improve the performance or
disprove that it’s actually that fast. I still don’t quite believe
it myself.

Thanks for the time, and enjoy.

Zed A. Shaw

2

On Jan 19, 2006, at 10:45 PM, Zed S. wrote:

Zed A. Shaw
http://www.zedshaw.com/

Zed-

This looks like fun. I tried to install it but got errors. Can you

see whats wrong here when I run rake?

root@grunt:~/mongrel-0.1.0# rake
(in /home/ez/mongrel-0.1.0)
make
make: Nothing to be done for all'. cp ext/http11/http11.so lib /usr/local/bin/ruby -Ilib:test "/usr/local/lib/ruby/gems/1.8/gems/ rake-0.7.0/lib/rake/rake_test_loader.rb" "test/test_http11.rb" "test/ test_trie.rb" "test/test_ws.rb" ./lib/mongrel.rb:54: warning: already initialized constant ERROR_404_RESPONSE ./lib/mongrel.rb:56: warning: already initialized constant CHUNK_SIZE Loaded suite /usr/local/lib/ruby/gems/1.8/gems/rake-0.7.0/lib/rake/ rake_test_loader Started Error result after 6 bytes of 15 .Read 18 string was 18 ...Running server Hitting server /usr/local/lib/ruby/1.8/timeout.rb:54:in open’: execution expired
(Timeout::Error)
from /usr/local/lib/ruby/1.8/net/http.rb:562:in connect' from /usr/local/lib/ruby/1.8/timeout.rb:48:in timeout’
from /usr/local/lib/ruby/1.8/timeout.rb:76:in timeout' from /usr/local/lib/ruby/1.8/net/http.rb:562:in connect’
from /usr/local/lib/ruby/1.8/net/http.rb:555:in do_start' from /usr/local/lib/ruby/1.8/net/http.rb:544:in start’
from /usr/local/lib/ruby/1.8/net/http.rb:404:in post_form' from ./test/test_ws.rb:20:in test_simple_server’
… 11 levels…
from /usr/local/lib/ruby/1.8/test/unit/autorunner.rb:200:in
run' from /usr/local/lib/ruby/1.8/test/unit/autorunner.rb:13:in run’
from /usr/local/lib/ruby/1.8/test/unit.rb:285
from /usr/local/lib/ruby/gems/1.8/gems/rake-0.7.0/lib/rake/
rake_test_loader.rb:5

I’m looking forward to playing with this.

Thanks-
-Ezra Z.
WebMaster
Yakima Herald-Republic Newspaper
http://yakimaherald.com
[email protected]
blog: http://brainspl.at

3

Zed S. wrote:

Hi,

This is an announce for Mongrel, a mostly Ruby webserver. I just
started it as I was trying to work on solutions for people deploying
Ruby web applications, and already it’s pretty damn fast. We’re
talking really really damn fast. I mean like people won’t believe me
that it’s this fast and I don’t really believe it.

Really? I don’t believe you.

INSTALL

First off, you can grab it at http://www.zedshaw.com/downloads/
mongrel-0.1.0.tar.bz2 to play with it.

*** You’ll need Ruby 1.8.4, but it’s tested on Mac OSX and Linux at
least. ***

Ah. The Windows Challenge.

…

CAN IT BAKE CHEDDAR?

So far it’s very light and isn’t nearly capable enough, so I’m looking
for things people want in a simple web app server for running their
Ruby web applications in. Keep in mind that I’m not going to be Ruby
on Rails specific. I love all you weirdos.

I’ve hacked around with WEBrick for various lightweight Web
services-like things (along the lines of Catapult). Of particular
interest are things that sit and spin and spit back data on request.
Not much heavy logic or calculation, but reaching into Madeleine or
SQLite or something, where having a simple, pure-Ruby data-managing
process allows for plain CGI or command-line apps to offload work.

For example, sticking ri/rdoc data into some persistent in-memory
process to speed up search and display.

James

–

http://www.ruby-doc.org - Ruby Help & Documentation
Ruby Code & Style - The Journal By & For Rubyists
http://www.rubystuff.com - The Ruby Store for Ruby Stuff
http://www.jamesbritt.com - Playing with Better Toys
http://www.30secondrule.com - Building Better Tools

4

In message “[ANN] Mongrel 0.1.0 – A Fast As Hell Mostly Ruby Web
Server”
on 20.01.2006, Zed S. [email protected] writes:

ZS> This is an announce for Mongrel, a mostly Ruby webserver. I just
ZS> started it as I was trying to work on solutions for people deploying
ZS> Ruby web applications, and already it’s pretty damn fast. We’re
ZS> talking really really damn fast. I mean like people won’t believe
me
ZS> that it’s this fast and I don’t really believe it.

ZS> When you get it do:

ZS> rake
ZS> ruby -Ilib examples/simpletest.rb

ZS> Go to http://localhost:3000/test to play with it. Other URIs give
ZS> you a lame “not found”.

works without any problem on freebsd 5.4

great!

karl-heinz

5

The already initialized constants are just from how the tests are
loaded (I think). Those constants will go away anyhow so safe to
ignore.

The timeout explosion though is interesting. If you look in test/
test_ws.rb you can see I put a 15 second timeout around a simple test
of the web server. If you’re getting this then loading and hitting
the webserver is taking longer than 15 seconds. Either it really
doesn’t work on your system, or your system is too slow. Can you try
putting a longer timeout just for test? Also, what are you running?

Otherwise, does the examples/simpletest.rb work at least? This test
failure may just be a false positive.

Zed A. Shaw

6

Hi Zed,

this is awesome. I seem to have the same problem under Cygwin as Ezra.
Here
is my output:

$ rake
(in /cygdrive/h/lib/Ruby/serving/mongrel-0.1.0)
/usr/bin/ruby extconf.rb
checking for main() in -lc… yes
creating Makefile
make
gcc -g -O2 -I. -I/usr/lib/ruby/1.8/i386-cygwin
-I/usr/lib/ruby/1.8/i386-cygwin -I. -c http11.c
gcc -g -O2 -I. -I/usr/lib/ruby/1.8/i386-cygwin
-I/usr/lib/ruby/1.8/i386-cygwin -I. -c http11_parser.c
gcc -shared -s -Wl,–enable-auto-import,–export-all -L"/usr/lib" -o
http11.so http11.o http11_parser.o -lruby -lc -lcrypt
cp ext/http11/http11.so lib
/usr/bin/ruby -Ilib:test
“/usr/lib/ruby/gems/1.8/gems/rake-0.6.2/lib/rake/rake_test_loader.rb”
“test/test_http11.rb” “test/test_trie.rb” “te
st/test_ws.rb”
./lib/mongrel.rb:54: warning: already initialized constant
ERROR_404_RESPONSE
./lib/mongrel.rb:56: warning: already initialized constant CHUNK_SIZE
Loaded suite
/usr/lib/ruby/gems/1.8/gems/rake-0.6.2/lib/rake/rake_test_loader
Started
Error result after 6 bytes of 15
.Read 18 string was 18
…Running server
Hitting server
E
Finished in 3.207 seconds.

  1. Error:
    test_simple_server(WSTest):
    Errno::ECONNRESET: Connection reset by peer
    /usr/lib/ruby/1.8/net/protocol.rb:133:in sysread' /usr/lib/ruby/1.8/net/protocol.rb:133:inrbuf_fill’
    /usr/lib/ruby/1.8/timeout.rb:45:in timeout' /usr/lib/ruby/1.8/timeout.rb:62:intimeout’
    /usr/lib/ruby/1.8/net/protocol.rb:132:in rbuf_fill' /usr/lib/ruby/1.8/net/protocol.rb:116:inreaduntil’
    /usr/lib/ruby/1.8/net/protocol.rb:126:in readline' /usr/lib/ruby/1.8/net/http.rb:1988:inread_status_line’
    /usr/lib/ruby/1.8/net/http.rb:1977:in read_new' /usr/lib/ruby/1.8/net/http.rb:1046:inrequest’
    /usr/lib/ruby/1.8/net/http.rb:405:in post_form' /usr/lib/ruby/1.8/net/http.rb:545:instart’
    /usr/lib/ruby/1.8/net/http.rb:404:in post_form' ./test/test_ws.rb:20:intest_simple_server’
    /usr/lib/ruby/1.8/timeout.rb:45:in timeout' ./test/test_ws.rb:11:intest_simple_server’

5 tests, 8 assertions, 0 failures, 1 errors
rake aborted!
Command failed with status (1): [/usr/bin/ruby -Ilib:test
"/usr/lib/ruby/ge…]

-Sascha

7

Thanks Sascha. I actually haven’t tried under cygwin. The problem
may just be that 15 seconds isn’t long enough to start the server and
use net/http to grab it as a test. Try changing the Timeout.timeout
(15) to 30. It really shouldn’t take longer than that. Otherwise I
must debug it.

Otherwise, it seems to at least be sort of working in cygwin.
That’s really cool. Does the examples/simpletest.rb work at all?

Zed A. Shaw

8

Looks interesting ;-D Thanks!

-g.

9

On Thursday 19 January 2006 11:45 pm, Zed S. wrote:

Hi,

This is an announce for Mongrel, a mostly Ruby webserver. I just
started it as I was trying to work on solutions for people deploying
Ruby web applications, and already it’s pretty damn fast. We’re
talking really really damn fast. I mean like people won’t believe me
that it’s this fast and I don’t really believe it.

Ooooh. Nice. I use the heck out of webrick when developing IOWA
apps. I
will definitely have to see how well this can work for the same sort of
thing, because webrick is going to be hard to beat. It’s just so easy
to use
and just works. I can take an entire production app or website, and
usually
with no changes, run it all under webrick quite adequately.

It’d be great if I could do the same thing, with the same ease of use,
with
better performance. One thing some customers like to do is to take
their
production sites/apps and put them onto a laptop or a CD/DVD that they
then
take to tradeshows. The better the performance of the app, the better
they
look to their potential customers.

Thanks,

Kirk H.

10

Ah, figured it out. I’m basically retarded and included two past
version of the same source in different files. I was sabotaged by
rsync I tell ya!

Anyway, I’ll post another announce with all the fixes.

Zed

11

On Jan 20, 2006, at 4:21 AM, Zed S. wrote:

Hi Zed,

this is awesome. I seem to have the same problem under Cygwin as
Ezra. Here is my output:

Zedas-

Yeah I noticed that in the lib dir there are two files mongrel.rb

and ws.rb. They both seems to have some very similar code in them. So
it must just be an issue of conflict between them.

I'm running it on a p4 2.4hz with 512MB ram on Kubuntu latest with a

fresh ruby1.8.4. So it should be fast on this machine. I’m eagerly
looking forward to testing this for you when you get another tarball
out the door.

I have some intranet applications I have written for the newspaper I

work for that we want to bundle to sell as a solution to other
newspapers that use the same newsroom database as us. And I think
that it would be great if I could bundle mongrel to serve the apps
instead of instructions for installing lighttpd. Webrick is a bit to
slow to handle the job.

Thanks-
-Ezra Z.
WebMaster
Yakima Herald-Republic Newspaper
http://yakimaherald.com
[email protected]
blog: http://brainspl.at

12

On 1/20/06, Zed S. [email protected] wrote:

Hi,

This is an announce for Mongrel, a mostly Ruby webserver. I just
started it as I was trying to work on solutions for people deploying
Ruby web applications, and already it’s pretty damn fast. We’re
talking really really damn fast. I mean like people won’t believe me
that it’s this fast and I don’t really believe it.

WEBrick does not implement application roles which are implemented on
both Lighttpd and Apache. It would be nice if any new servers
implemented this feature, it would make it easier to test/deploy
things without involving Apache.

http://www.fastcgi.com/devkit/doc/fastcgi-whitepaper/fastcgi.htm
* Responder. This is the basic FastCGI role, and corresponds to
the simple functionality offered by CGI today.
* Authorizer. The FastCGI program performs an access control
decision for the request (such as performing a username/password
database lookup).

The rubgy fcgi support only implements the responsder and authorizer
roles. Roles are not specific to FCGI for example Apache modules also
support them. Roles are very useful for protecting access to files,
like having to log in before being able to download a file.

Example lighttpd setup:
fastcgi.server = (
“/private” =>
( “localhost-d” =>
( “min-procs” => 1,
“max-procs” => 1,
“socket” => “log/fcgi.socket”,
“bin-path” => “public/dispatch.fcgi”,
“bin-environment” => ( “RAILS_ENV” => “development” ),
“docroot” => “private”,
“mode” => “authorizer”
)
),
“.fcgi” =>
( “localhost-f” =>
(
“min-procs” => 1,
“max-procs” => 1,
“socket” => “log/fcgi.socket”,
“bin-path” => “public/dispatch.fcgi”,
“bin-environment” => ( “RAILS_ENV” => “development” ),
“mode” => “responder”
)
)
)

13

My initial thought on the goal would be to make a system that worked
well for production deployment, but in the past when I’ve done this
it also solved problems for development deployments. I can’t see why
you wouldn’t be able to do what you describe. About the only thing I
can think of is I’d be making decisions based on performance over
ease of use when there’s a conflict there.

Anyway, stay tuned for more.

Zed A. Shaw

14

On Jan 20, 2006, at 12:33 PM, Ezra Z. wrote:

Zedas-

Yeah I noticed that in the lib dir there are two files mongrel.rb
and ws.rb. They both seems to have some very similar code in them.
So it must just be an issue of conflict between them.

Yes, that’s all fixed up now.

I’m running it on a p4 2.4hz with 512MB ram on Kubuntu latest with
a fresh ruby1.8.4. So it should be fast on this machine. I’m
eagerly looking forward to testing this for you when you get
another tarball out the door.

Go ahead and check the other posting I made for Mongrel 0.1.1.
Should work great and there’s already a post about getting it to work
under Cygwin.

I have some intranet applications I have written for the newspaper
I work for that we want to bundle to sell as a solution to other
newspapers that use the same newsroom database as us. And I think
that it would be great if I could bundle mongrel to serve the apps
instead of instructions for installing lighttpd. Webrick is a bit
to slow to handle the job.

This would be a good test case. Feel free to shoot me design
requirements on this. The API and setup is still totally in the air.

Zed A. Shaw

15

On Jan 20, 2006, at 1:00 PM, Jon S. wrote:

On 1/20/06, Zed S. [email protected] wrote:

WEBrick does not implement application roles which are implemented on
both Lighttpd and Apache. It would be nice if any new servers
implemented this feature, it would make it easier to test/deploy
things without involving Apache.

I believe I’ve heard this before, but I’ve never been clear why
it’s needed. Is it just so you can have lighttpd server static files
after checking if the access is allowed?

http://www.fastcgi.com/devkit/doc/fastcgi-whitepaper/fastcgi.htm
* Responder. This is the basic FastCGI role, and corresponds to
the simple functionality offered by CGI today.
* Authorizer. The FastCGI program performs an access control
decision for the request (such as performing a username/password
database lookup).

I’m thinking this wouldn’t be too hard, but like I said previously
I’m not quite sure what’s involved. Is it possible for you to
describe the problem less in terms of what you need and more in terms
of what you want to do? That might help me.

Zed A. Shaw

16

On 1/20/06, Zed S. [email protected] wrote:

I believe I’ve heard this before, but I’ve never been clear why
it’s needed. Is it just so you can have lighttpd server static files
after checking if the access is allowed?

Web server first pings the app on the authorizer hook
App says yes/no for authorization
If yes, web server tries to serve static file
if not found, 404 mechanism will trigger fcgi app and static file will
get created.

The authorization ping is there is allow conditional access to static
(or cached) files.

You could ignore the authorization ping but then you have to serve the
file up from the fcgi app. That will tie the fcgi app up forever if a
dialup user asks for a 100MB file.

In your case of a ruby/C hybrid the goal is to send the file using
sendfile() if possible.