Forum: Ruby Announcing Rev: a high performance event library for Ruby 1.9

Hello everyone,

I've just made my initial public release of Rev, a high performance 
event
library for Ruby 1.9 built on top of libev. Events are monitored with
advanced system calls including epoll on Linux, kqueues on BSD, and 
event
ports on Solaris.  You can read more about libev here:
http://software.schmorp.de/pkg/libev.html

The release has not yet reached the gem mirrors, so you can grab it here 
for
the time being:

http://rev.rubyforge.org/releases/gems/rev-0.1.0.gem

Rev gives you the ability to monitor Ruby IO objects for readability 
and/or
writability.  It also lets you configure either one-shot or periodic
timers.  Future plans include wrappers for monitoring signals as well as
filesystem events.

Rev makes use of a number of new Ruby 1.9 features, namely the ability 
to do
blocking calls inside of native threads using the
rb_thread_blocking_region() function.  The entire event loop is written 
with
scalability in mind, with nearly constant time overhead compared to the
number of events being monitored, at least on Linux, FreeBSD, or 
Solaris.

In addition to a C extension which provides Ruby bindings to libev, Rev 
also
provides asynchronous observer-like wrappers for Ruby's core socket 
classes
using the new Ruby 1.9 *_nonblock methods.  It also provides 
asynchronous
DNS resolution for the purpose of making outgoing connections.

You can view the RDoc here:

http://rev.rubyforge.org/

Enjoy!

* Tony Arcieri (tony@clickcaster.com) wrote:

> I've just made my initial public release of Rev, a high performance
> event library for Ruby 1.9 built on top of libev. Events are monitored
> with advanced system calls including epoll on Linux, kqueues on BSD,
> and event ports on Solaris.

Yum, thanks :)

Buffered writing seems to be broken; the buffer is sliced incorrectly,
so it spins writing the last few KB, and the Watcher doesn't have an
enabled? method so that fails if it ever reaches it.  Patch:

  http://voi.aagh.net/rev-0.1.0_lib-rev-buffered_io.rb.diff

Few other issues: spurious closed connections - if I do one big write
I'm ok, but splitting a 500k buffer into 4k writes almost always results
in a closed stream IOError on the second write, and a ECONNRESET on the
reading side.

Servers don't serve much; they serve one client and exit even in the
exception-free case, including the example echo server.  Multiple
clients will be served if they manage to connect before the rest close,
but once the listen queue's empty Rev::Loop#run returns.

This is tested with FreeBSD 6.2, using kqueue, select and poll.

On Dec 28, 2007 2:05 PM, Thomas Hurst <tom.hurst@clara.net> wrote:

> Buffered writing seems to be broken; the buffer is sliced incorrectly,
> so it spins writing the last few KB


Heh, I caught that one earlier today and a fix is checked in.  Thanks 
for
pointing it out.


> and the Watcher doesn't have an enabled? method so that fails if it ever
> reaches it.  Patch:
>
>  http://voi.aagh.net/rev-0.1.0_lib-rev-buffered_io.rb.diff
>

Thanks, I'll get that addressed as well.  You might notice the painful 
lack
of tests... I'd like to use RSpec but that's unfortunately broken by
1.9changes at the moment.  Perhaps I could use Rubinius's minispec if
that's
working on 1.9, but my guess is I'll have to end up using Test::Unit for 
the
time being.


> Few other issues: spurious closed connections - if I do one big write
> I'm ok, but splitting a 500k buffer into 4k writes almost always results
> in a closed stream IOError on the second write, and a ECONNRESET on the
> reading side.
>

I may have a fix for that shortly... I'll ping you off list if I do


> Servers don't serve much; they serve one client and exit even in the
> exception-free case, including the example echo server.  Multiple
> clients will be served if they manage to connect before the rest close,
> but once the listen queue's empty Rev::Loop#run returns.


> This is tested with FreeBSD 6.2, using kqueue, select and poll.
>

Bizzare.  I don't presently have a FreeBSD system, although I plan on
installing FreeBSD later tonight actually, so I can test there.

I'm using kqueue on MacOS X and have not experienced that problem, nor 
have
I experienced the problem with Linux/epoll.

I'm curious how that's even happening as the listen socket should be 
open
and the Rev::Listener object should still be attached to the loop.

* Tony Arcieri (tony@clickcaster.com) wrote:

> Thanks, I'll get that addressed as well.  You might notice the painful
> lack of tests... I'd like to use RSpec but that's unfortunately broken
> by 1.9changes at the moment.  Perhaps I could use Rubinius's minispec
> if that's working on 1.9, but my guess is I'll have to end up using
> Test::Unit for the time being.

mspec seems to work.  After changing target in mspec/scripts/mspec.rb
from shotgun/rubinius to ruby19, I can do:

 -% ruby19 bin/mspec spec/core/fixnum/*
 Started
 .......................F....................... (etc)

> > Few other issues: spurious closed connections [..]
> I may have a fix for that shortly... I'll ping you off list if I do

Thanks.

> > Servers don't serve much; they serve one client and exit even in the
> > exception-free case, including the example echo server.  Multiple
> > clients will be served if they manage to connect before the rest
> > close, but once the listen queue's empty Rev::Loop#run returns.
> 
> Bizzare.  I don't presently have a FreeBSD system, although I plan on
> installing FreeBSD later tonight actually, so I can test there.

I just tried on my laptop which is running Ubuntu; behavior's identical
to FreeBSD with epoll and select.

Also, :backend => :kqueue makes it crash horribly there ;)

> I'm using kqueue on MacOS X and have not experienced that problem, nor
> have I experienced the problem with Linux/epoll.
> 
> I'm curious how that's even happening as the listen socket should be
> open and the Rev::Listener object should still be attached to the
> loop.

Further testing shows this doesn't happen if I use small writes, so it
seems to be related to buffering.  I can't see any immediately obvious
reason why, though.

On Dec 28, 2007 6:21 PM, Thomas Hurst <tom.hurst@clara.net> wrote:

>
> mspec seems to work.  After changing target in mspec/scripts/mspec.rb
> from shotgun/rubinius to ruby19, I can do:
>
>  -% ruby19 bin/mspec spec/core/fixnum/*
>  Started
>  .......................F....................... (etc)
>

Nice, I'll get that installed...

I just tried on my laptop which is running Ubuntu; behavior's identical
> to FreeBSD with epoll and select.
> Further testing shows this doesn't happen if I use small writes, so it
> seems to be related to buffering.  I can't see any immediately obvious
> reason why, though.
>

Running off the svn trunk version of Rev I get:

rev/lib/rev/buffered_io.rb:44:in `write_nonblock': Resource temporarily
unavailable (Errno::EAGAIN)
rev/lib/rev/buffered_io.rb:44:in `write'

Which should be relatively trivial to fix.  That exception can probably 
be
safely swallowed, although that may not be the greatest thing from a
performance perspective.

If you try the version of Rev in trunk do you get the same thing?

On Dec 29, 2007 11:13 AM, Tony Arcieri <tony@clickcaster.com> wrote:

> Running off the svn trunk version of Rev I get:
>
> rev/lib/rev/buffered_io.rb:44:in `write_nonblock': Resource temporarily
> unavailable (Errno::EAGAIN)
> rev/lib/rev/buffered_io.rb:44:in `write'
>
> Which should be relatively trivial to fix.  That exception can probably be
> safely swallowed, although that may not be the greatest thing from a
> performance perspective.
>

Yeah, that's a bit of a snag.  Do you know how to do a nonblocking write
that swallows that exception and gets the return value from write(2)?

On Dec 29, 2007 12:57 PM, Tony Arcieri <tony@clickcaster.com> wrote:

> Yeah, that's a bit of a snag.  Do you know how to do a nonblocking write
> that swallows that exception and gets the return value from write(2)?
>

I released Rev 0.1.1 (should be available through the gem servers now) 
which
resolves these problems, and also bundles an asynchronous HTTP client.

Give it a try and let me know if it fixes your problems.