Announcing Rev: a high performance event library for Ruby 1.9

Ruby
1

Hello everyone,

I’ve just made my initial public release of Rev, a high performance
event
library for Ruby 1.9 built on top of libev. Events are monitored with
advanced system calls including epoll on Linux, kqueues on BSD, and
event
ports on Solaris. You can read more about libev here:
http://software.schmorp.de/pkg/libev.html

The release has not yet reached the gem mirrors, so you can grab it here
for
the time being:

http://rev.rubyforge.org/releases/gems/rev-0.1.0.gem

Rev gives you the ability to monitor Ruby IO objects for readability
and/or
writability. It also lets you configure either one-shot or periodic
timers. Future plans include wrappers for monitoring signals as well as
filesystem events.

Rev makes use of a number of new Ruby 1.9 features, namely the ability
to do
blocking calls inside of native threads using the
rb_thread_blocking_region() function. The entire event loop is written
with
scalability in mind, with nearly constant time overhead compared to the
number of events being monitored, at least on Linux, FreeBSD, or
Solaris.

In addition to a C extension which provides Ruby bindings to libev, Rev
also
provides asynchronous observer-like wrappers for Ruby’s core socket
classes
using the new Ruby 1.9 *_nonblock methods. It also provides
asynchronous
DNS resolution for the purpose of making outgoing connections.

You can view the RDoc here:

http://rev.rubyforge.org/

Enjoy!

2

I’ve just made my initial public release of Rev, a high performance
event library for Ruby 1.9 built on top of libev. Events are monitored
with advanced system calls including epoll on Linux, kqueues on BSD,
and event ports on Solaris.

Yum, thanks :slight_smile:

Buffered writing seems to be broken; the buffer is sliced incorrectly,
so it spins writing the last few KB, and the Watcher doesn’t have an
enabled? method so that fails if it ever reaches it. Patch:

http://voi.aagh.net/rev-0.1.0_lib-rev-buffered_io.rb.diff

Few other issues: spurious closed connections - if I do one big write
I’m ok, but splitting a 500k buffer into 4k writes almost always results
in a closed stream IOError on the second write, and a ECONNRESET on the
reading side.

Servers don’t serve much; they serve one client and exit even in the
exception-free case, including the example echo server. Multiple
clients will be served if they manage to connect before the rest close,
but once the listen queue’s empty Rev::Loop#run returns.

This is tested with FreeBSD 6.2, using kqueue, select and poll.

3

On Dec 28, 2007 2:05 PM, Thomas H. [email protected] wrote:

Buffered writing seems to be broken; the buffer is sliced incorrectly,
so it spins writing the last few KB

Heh, I caught that one earlier today and a fix is checked in. Thanks
for
pointing it out.

and the Watcher doesn’t have an enabled? method so that fails if it ever
reaches it. Patch:

http://voi.aagh.net/rev-0.1.0_lib-rev-buffered_io.rb.diff

Thanks, I’ll get that addressed as well. You might notice the painful
lack
of tests… I’d like to use RSpec but that’s unfortunately broken by
1.9changes at the moment. Perhaps I could use Rubinius’s minispec if
that’s
working on 1.9, but my guess is I’ll have to end up using Test::Unit for
the
time being.

Few other issues: spurious closed connections - if I do one big write
I’m ok, but splitting a 500k buffer into 4k writes almost always results
in a closed stream IOError on the second write, and a ECONNRESET on the
reading side.

I may have a fix for that shortly… I’ll ping you off list if I do

Servers don’t serve much; they serve one client and exit even in the
exception-free case, including the example echo server. Multiple
clients will be served if they manage to connect before the rest close,
but once the listen queue’s empty Rev::Loop#run returns.

This is tested with FreeBSD 6.2, using kqueue, select and poll.

Bizzare. I don’t presently have a FreeBSD system, although I plan on
installing FreeBSD later tonight actually, so I can test there.

I’m using kqueue on MacOS X and have not experienced that problem, nor
have
I experienced the problem with Linux/epoll.

I’m curious how that’s even happening as the listen socket should be
open
and the Rev::Listener object should still be attached to the loop.

4

On Dec 28, 2007 6:21 PM, Thomas H. [email protected] wrote:

mspec seems to work. After changing target in mspec/scripts/mspec.rb
from shotgun/rubinius to ruby19, I can do:

-% ruby19 bin/mspec spec/core/fixnum/*
Started
…F… (etc)

Nice, I’ll get that installed…

I just tried on my laptop which is running Ubuntu; behavior’s identical

to FreeBSD with epoll and select.
Further testing shows this doesn’t happen if I use small writes, so it
seems to be related to buffering. I can’t see any immediately obvious
reason why, though.

Running off the svn trunk version of Rev I get:

rev/lib/rev/buffered_io.rb:44:in write_nonblock': Resource temporarily unavailable (Errno::EAGAIN) rev/lib/rev/buffered_io.rb:44:in write’

Which should be relatively trivial to fix. That exception can probably
be
safely swallowed, although that may not be the greatest thing from a
performance perspective.

If you try the version of Rev in trunk do you get the same thing?

5

On Dec 29, 2007 11:13 AM, Tony A. [email protected] wrote:

Running off the svn trunk version of Rev I get:

rev/lib/rev/buffered_io.rb:44:in write_nonblock': Resource temporarily unavailable (Errno::EAGAIN) rev/lib/rev/buffered_io.rb:44:in write’

Which should be relatively trivial to fix. That exception can probably be
safely swallowed, although that may not be the greatest thing from a
performance perspective.

Yeah, that’s a bit of a snag. Do you know how to do a nonblocking write
that swallows that exception and gets the return value from write(2)?

6

Thanks, I’ll get that addressed as well. You might notice the painful
lack of tests… I’d like to use RSpec but that’s unfortunately broken
by 1.9changes at the moment. Perhaps I could use Rubinius’s minispec
if that’s working on 1.9, but my guess is I’ll have to end up using
Test::Unit for the time being.

mspec seems to work. After changing target in mspec/scripts/mspec.rb
from shotgun/rubinius to ruby19, I can do:

-% ruby19 bin/mspec spec/core/fixnum/*
Started
…F… (etc)

Few other issues: spurious closed connections […]
I may have a fix for that shortly… I’ll ping you off list if I do

Thanks.

Servers don’t serve much; they serve one client and exit even in the
exception-free case, including the example echo server. Multiple
clients will be served if they manage to connect before the rest
close, but once the listen queue’s empty Rev::Loop#run returns.

Bizzare. I don’t presently have a FreeBSD system, although I plan on
installing FreeBSD later tonight actually, so I can test there.

I just tried on my laptop which is running Ubuntu; behavior’s identical
to FreeBSD with epoll and select.

Also, :backend => :kqueue makes it crash horribly there :wink:

I’m using kqueue on MacOS X and have not experienced that problem, nor
have I experienced the problem with Linux/epoll.

I’m curious how that’s even happening as the listen socket should be
open and the Rev::Listener object should still be attached to the
loop.

Further testing shows this doesn’t happen if I use small writes, so it
seems to be related to buffering. I can’t see any immediately obvious
reason why, though.

7

On Dec 29, 2007 12:57 PM, Tony A. [email protected] wrote:

Yeah, that’s a bit of a snag. Do you know how to do a nonblocking write
that swallows that exception and gets the return value from write(2)?

I released Rev 0.1.1 (should be available through the gem servers now)
which
resolves these problems, and also bundles an asynchronous HTTP client.

Give it a try and let me know if it fixes your problems.