Forum: Ruby Announcing Rev: a high performance event library for Ruby 1.9

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
58a68672882676f6bdf817b3c8ff3880?d=identicon&s=25 Tony Arcieri (Guest)
on 2007-12-26 05:47
(Received via mailing list)
Hello everyone,

I've just made my initial public release of Rev, a high performance
event
library for Ruby 1.9 built on top of libev. Events are monitored with
advanced system calls including epoll on Linux, kqueues on BSD, and
event
ports on Solaris.  You can read more about libev here:
http://software.schmorp.de/pkg/libev.html

The release has not yet reached the gem mirrors, so you can grab it here
for
the time being:

http://rev.rubyforge.org/releases/gems/rev-0.1.0.gem

Rev gives you the ability to monitor Ruby IO objects for readability
and/or
writability.  It also lets you configure either one-shot or periodic
timers.  Future plans include wrappers for monitoring signals as well as
filesystem events.

Rev makes use of a number of new Ruby 1.9 features, namely the ability
to do
blocking calls inside of native threads using the
rb_thread_blocking_region() function.  The entire event loop is written
with
scalability in mind, with nearly constant time overhead compared to the
number of events being monitored, at least on Linux, FreeBSD, or
Solaris.

In addition to a C extension which provides Ruby bindings to libev, Rev
also
provides asynchronous observer-like wrappers for Ruby's core socket
classes
using the new Ruby 1.9 *_nonblock methods.  It also provides
asynchronous
DNS resolution for the purpose of making outgoing connections.

You can view the RDoc here:

http://rev.rubyforge.org/

Enjoy!
Aa8b5f9730c6a8a934959d62cd064619?d=identicon&s=25 Thomas Hurst (Guest)
on 2007-12-28 22:06
(Received via mailing list)
* Tony Arcieri (tony@clickcaster.com) wrote:

> I've just made my initial public release of Rev, a high performance
> event library for Ruby 1.9 built on top of libev. Events are monitored
> with advanced system calls including epoll on Linux, kqueues on BSD,
> and event ports on Solaris.

Yum, thanks :)

Buffered writing seems to be broken; the buffer is sliced incorrectly,
so it spins writing the last few KB, and the Watcher doesn't have an
enabled? method so that fails if it ever reaches it.  Patch:

  http://voi.aagh.net/rev-0.1.0_lib-rev-buffered_io.rb.diff

Few other issues: spurious closed connections - if I do one big write
I'm ok, but splitting a 500k buffer into 4k writes almost always results
in a closed stream IOError on the second write, and a ECONNRESET on the
reading side.

Servers don't serve much; they serve one client and exit even in the
exception-free case, including the example echo server.  Multiple
clients will be served if they manage to connect before the rest close,
but once the listen queue's empty Rev::Loop#run returns.

This is tested with FreeBSD 6.2, using kqueue, select and poll.
58a68672882676f6bdf817b3c8ff3880?d=identicon&s=25 Tony Arcieri (Guest)
on 2007-12-29 00:19
(Received via mailing list)
On Dec 28, 2007 2:05 PM, Thomas Hurst <tom.hurst@clara.net> wrote:

> Buffered writing seems to be broken; the buffer is sliced incorrectly,
> so it spins writing the last few KB


Heh, I caught that one earlier today and a fix is checked in.  Thanks
for
pointing it out.


> and the Watcher doesn't have an enabled? method so that fails if it ever
> reaches it.  Patch:
>
>  http://voi.aagh.net/rev-0.1.0_lib-rev-buffered_io.rb.diff
>

Thanks, I'll get that addressed as well.  You might notice the painful
lack
of tests... I'd like to use RSpec but that's unfortunately broken by
1.9changes at the moment.  Perhaps I could use Rubinius's minispec if
that's
working on 1.9, but my guess is I'll have to end up using Test::Unit for
the
time being.


> Few other issues: spurious closed connections - if I do one big write
> I'm ok, but splitting a 500k buffer into 4k writes almost always results
> in a closed stream IOError on the second write, and a ECONNRESET on the
> reading side.
>

I may have a fix for that shortly... I'll ping you off list if I do


> Servers don't serve much; they serve one client and exit even in the
> exception-free case, including the example echo server.  Multiple
> clients will be served if they manage to connect before the rest close,
> but once the listen queue's empty Rev::Loop#run returns.


> This is tested with FreeBSD 6.2, using kqueue, select and poll.
>

Bizzare.  I don't presently have a FreeBSD system, although I plan on
installing FreeBSD later tonight actually, so I can test there.

I'm using kqueue on MacOS X and have not experienced that problem, nor
have
I experienced the problem with Linux/epoll.

I'm curious how that's even happening as the listen socket should be
open
and the Rev::Listener object should still be attached to the loop.
Aa8b5f9730c6a8a934959d62cd064619?d=identicon&s=25 Thomas Hurst (Guest)
on 2007-12-29 02:22
(Received via mailing list)
* Tony Arcieri (tony@clickcaster.com) wrote:

> Thanks, I'll get that addressed as well.  You might notice the painful
> lack of tests... I'd like to use RSpec but that's unfortunately broken
> by 1.9changes at the moment.  Perhaps I could use Rubinius's minispec
> if that's working on 1.9, but my guess is I'll have to end up using
> Test::Unit for the time being.

mspec seems to work.  After changing target in mspec/scripts/mspec.rb
from shotgun/rubinius to ruby19, I can do:

 -% ruby19 bin/mspec spec/core/fixnum/*
 Started
 .......................F....................... (etc)

> > Few other issues: spurious closed connections [..]
> I may have a fix for that shortly... I'll ping you off list if I do

Thanks.

> > Servers don't serve much; they serve one client and exit even in the
> > exception-free case, including the example echo server.  Multiple
> > clients will be served if they manage to connect before the rest
> > close, but once the listen queue's empty Rev::Loop#run returns.
>
> Bizzare.  I don't presently have a FreeBSD system, although I plan on
> installing FreeBSD later tonight actually, so I can test there.

I just tried on my laptop which is running Ubuntu; behavior's identical
to FreeBSD with epoll and select.

Also, :backend => :kqueue makes it crash horribly there ;)

> I'm using kqueue on MacOS X and have not experienced that problem, nor
> have I experienced the problem with Linux/epoll.
>
> I'm curious how that's even happening as the listen socket should be
> open and the Rev::Listener object should still be attached to the
> loop.

Further testing shows this doesn't happen if I use small writes, so it
seems to be related to buffering.  I can't see any immediately obvious
reason why, though.
58a68672882676f6bdf817b3c8ff3880?d=identicon&s=25 Tony Arcieri (Guest)
on 2007-12-29 19:13
(Received via mailing list)
On Dec 28, 2007 6:21 PM, Thomas Hurst <tom.hurst@clara.net> wrote:

>
> mspec seems to work.  After changing target in mspec/scripts/mspec.rb
> from shotgun/rubinius to ruby19, I can do:
>
>  -% ruby19 bin/mspec spec/core/fixnum/*
>  Started
>  .......................F....................... (etc)
>

Nice, I'll get that installed...

I just tried on my laptop which is running Ubuntu; behavior's identical
> to FreeBSD with epoll and select.
> Further testing shows this doesn't happen if I use small writes, so it
> seems to be related to buffering.  I can't see any immediately obvious
> reason why, though.
>

Running off the svn trunk version of Rev I get:

rev/lib/rev/buffered_io.rb:44:in `write_nonblock': Resource temporarily
unavailable (Errno::EAGAIN)
rev/lib/rev/buffered_io.rb:44:in `write'

Which should be relatively trivial to fix.  That exception can probably
be
safely swallowed, although that may not be the greatest thing from a
performance perspective.

If you try the version of Rev in trunk do you get the same thing?
58a68672882676f6bdf817b3c8ff3880?d=identicon&s=25 Tony Arcieri (Guest)
on 2007-12-29 20:58
(Received via mailing list)
On Dec 29, 2007 11:13 AM, Tony Arcieri <tony@clickcaster.com> wrote:

> Running off the svn trunk version of Rev I get:
>
> rev/lib/rev/buffered_io.rb:44:in `write_nonblock': Resource temporarily
> unavailable (Errno::EAGAIN)
> rev/lib/rev/buffered_io.rb:44:in `write'
>
> Which should be relatively trivial to fix.  That exception can probably be
> safely swallowed, although that may not be the greatest thing from a
> performance perspective.
>

Yeah, that's a bit of a snag.  Do you know how to do a nonblocking write
that swallows that exception and gets the return value from write(2)?
58a68672882676f6bdf817b3c8ff3880?d=identicon&s=25 Tony Arcieri (Guest)
on 2008-01-01 21:01
(Received via mailing list)
On Dec 29, 2007 12:57 PM, Tony Arcieri <tony@clickcaster.com> wrote:

> Yeah, that's a bit of a snag.  Do you know how to do a nonblocking write
> that swallows that exception and gets the return value from write(2)?
>

I released Rev 0.1.1 (should be available through the gem servers now)
which
resolves these problems, and also bundles an asynchronous HTTP client.

Give it a try and let me know if it fixes your problems.
This topic is locked and can not be replied to.