Considering Ruby For a Networking Application


#1

I’m going to be working on a fairly basic networking application
shortly. This is low level socket work. (I cannot use a nicety like
DRB sadly.)

I’m very familiar with this kind of stuff in Perl, but have done very
little socket work in Ruby. I need something that can reliably work
with around 50 users sending light activity. There is the
possibility of a large write to them here and there and that cannot
block the process.

I know a little from past questions here:

http://rubyurl.com/miq

However, I think I remember someone saying Ruby 1.8.4 is better with
nonblocking sockets. So, can some networking guru take a stab at
these questions:

  1. I need to use threads and nonblocking sockets to be safe, right?
  2. Can I use gets() and puts() now in Ruby 1.8.4 or do I still need
    to do low level reads and writes?
  3. Do I need to rescue Errno::EWOULDBLOCK?
  4. This isn’t going to work on Windows, right? (That’s okay in this
    case. Just making sure.)

Thanks for the help.

James Edward G. II


#2

On 5/5/06, James Edward G. II removed_email_address@domain.invalid wrote:

I’m going to be working on a fairly basic networking application
shortly. This is low level socket work. (I cannot use a nicety like
DRB sadly.)

James, you might want to look for “EventMachine”.

It seems interesting.

-austin


#3

James Edward G. II wrote:

However, I think I remember someone saying Ruby 1.8.4 is better with
nonblocking sockets. So, can some networking guru take a stab at
these questions:

I haven’t noticed any difference between 1.8.3 and 1.8.4.

  1. I need to use threads and nonblocking sockets to be safe, right?

No. The whole point of select() and non-blocking sockets is to
de-multiplex
I/O.

  1. Can I use gets() and puts() now in Ruby 1.8.4 or do I still need
    to do low level reads and writes?

I would guess these would be “unsafe” as would any network I/O function
that
depends on specific separators. Would gets() effectively block on a
non-blocking socket if the EOL separator never arrives?
shrug I don’t know.

  1. Do I need to rescue Errno::EWOULDBLOCK?

Yes.

  1. This isn’t going to work on Windows, right? (That’s okay in this
    case. Just making sure.)

Sure it work on Windows.

Here’s a pretty good example. No threads. Single select client/server
code
that uses non-blocking sockets.

http://sourcery.dyndns.org/svn/teensymud/release/tmud-2.9.0/lib/network/


#4

From: “James Edward G. II” removed_email_address@domain.invalid

to do low level reads and writes?
3. Do I need to rescue Errno::EWOULDBLOCK?
4. This isn’t going to work on Windows, right? (That’s okay in this
case. Just making sure.)

I’m sorry I don’t have definitive answers. I’m going to
need to find out for sure pretty soon, myself. But . . . .

From looking at the changes in the ruby sources between 1.8.2
and 1.8.4, it looks as though nonblocking sockets are pretty
well supported now. (Even on Windows!)

It would appear nonblocking is handled properly internally
whether you use ruby threads or not. (I.e. from the perspective
of the underlying mechanisms, the main ruby thread is just as
much a thread as any other ruby thread.)

In 1.8.4, you can:

require ‘io/nonblock’

This adds methods to the IO class, allowing:

some_socket.nonblock = true

In the past, I’ve done all my ruby socket programming using the
lower level send() and recv() methods. However, again from
reading the source (io.c and ext/openssl/lib/openssl/buffering.rb)
it looks like IO#readpartial or IO#sysread (but not IO#read)
may be the new favored approach to dealing with nonblocking
descriptors in general.

(In Windows, I’m pretty sure only sockets have any chance of
working with nonblocking semantics. Whereas in Unix, I’d
presume support for any file descriptor. So using IO#sysread
and IO#syswrite, now that these methods apparently are
nonblocking-savvy, should be more general than using send()
and recv().)

If you look at ext/openssl/lib/openssl/buffering.rb, you’ll
see what looks like a fairly generic module for building
a buffered read and readpartial, as well as write, on top of
sysread and syswrite. I note that the lowest level read and
write methods here both rescue Errno::EAGAIN, and retry. I’m
a little surprised by that - because it appears that EAGAIN
is already being handled along with EWOULDBLOCK in io.c
anyway. But I haven’t really studied the code in io.c, just
perused it.

Hmm…

My sense of it–at least, this is what I plan to try in my
first test–is:

  1. require ‘io/nonblock’ and set ‘nonblock = true’ on all
    my sockets.

  2. use a separate ruby thread to handle I/O for each socket.

  3. use IO#readpartial to read… (note that the comments in
    io.c say that IO#readpartial can block regardless of
    the nonblocking flag… but I’m pretty sure this means
    block the ruby thread, NOT the native thread…!)

  4. use IO#syswrite to write… (I think, with 1.8.4 and a
    nonblocking socket, ruby appears to be handling
    EWOULDBLOCK internally, so that we can do a large write
    but not block the whole process.)

    • But maybe we need to rescue Errno::EAGAIN for some
      reason, as seen in openssl/buffering.rb

With luck, this will work on Unix and Windows (sockets only
on windows, any file descriptor on unix), and never block
the whole native thread ruby runs in.

If so, we’ve got a reasonable solution for TCP. UDP had its
own unique blocking issues (related to linux kernels breaking
POSIX semantics for select/recv for packets with bad UDP
checksums… or something like that… I dunno if this has
changed since 1.8.2 or not…)

Or, yeah, maybe that EventMachine library…

Regards,

Bill


#5

From: “Jon A. Lambert” removed_email_address@domain.invalid

James Edward G. II wrote:

  1. This isn’t going to work on Windows, right? (That’s okay in this
    case. Just making sure.)

Sure it work on Windows.

If nonblocking works on Windows, it’s a post-1.8.2 development…
(But I’m stoked if it does, now.)

Regards,

Bill


#6

On May 5, 2006, at 11:42 PM, Bill K. wrote:

I’m sorry I don’t have definitive answers. I’m going to
need to find out for sure pretty soon, myself. But . . . .

Thank you for the very detailed breakdown. It was quite helpful.

James Edward G. II


#7

On May 5, 2006, at 11:13 PM, Jon A. Lambert wrote:

Here’s a pretty good example. No threads. Single select client/
server code that uses non-blocking sockets.

http://sourcery.dyndns.org/svn/teensymud/release/tmud-2.9.0/lib/
network/

Yeah, this is pretty much how I did it in Perl. Thank you from the
example though.

James Edward G. II

P.S. I need to play with your MUD sometime. Looks like a blast!


#8

On May 5, 2006, at 10:28 PM, Austin Z. wrote:

On 5/5/06, James Edward G. II removed_email_address@domain.invalid wrote:

I’m going to be working on a fairly basic networking application
shortly. This is low level socket work. (I cannot use a nicety like
DRB sadly.)

James, you might want to look for “EventMachine”.

This is a terrific find. Thank you so much!

James Edward G. II


#9

A great deal depends on what you really need to accomplish. Are you
doing a
plain-vanilla protocol handler for some standard server protocol? If
your
performance and scalability requirements are low, then the easiest thing
is
probably to use blocking i/o with a thread per socket. If you use
blocking
i/o, you’re better off multiplexing the i/o with a select loop. (My
preference in these cases is ALMOST ALWAYS to avoid a thread pool for a
range of reasons, but most people disagree with me ;-).) If you use
nonblocking i/o, then you really want to avoid multithreading because
you’ll
need to poll in each one. Ruby can handle it because its implementation
of
select is integrated with its internal thread scheduler. (If you try to
run
select on a native thread, you’ll have major problems with Ruby
threads.)

If your process has to do work on other threads, meaning this isn’t a
pure
server application, then
the calculations change. UDP is really quite a lot easier than TCP, but
it
always seems to take some getting used to. So the major questions are:
tell
us more about the nature of the application, and tell us what kind of
performance and scalability you need. And additionally, do you need
encryption?

If you decide to look at EventMachine, I can support you if you like.
Just
email me or ask questions here. It runs on Windows, works correctly with
Ruby threads, and supports encryption.


#10

On May 8, 2006, at 12:12 PM, Francis C. wrote:

If your performance and scalability requirements are low, then the
easiest thing is
probably to use blocking i/o with a thread per socket. If you use
blocking
i/o, you’re better off multiplexing the i/o with a select loop.

But then a large write could still block the entire process, since
Ruby’s threads aren’t native. Right?

If you decide to look at EventMachine, I can support you if you like.

I believe this is going to be my first choice. I haven’t had time to
play with it yet, but it looks very nice.

Are there API docs available anywhere for EventMachine?

James Edward G. II


#11

On May 8, 2006, at 2:09 PM, Francis C. wrote:

If you download the eventmachine gem and install the rdocs, the API
docs
will all be there, and there is sample code too.

Forgive me if this is answered in the documentation, I have had time
to look: Is EventMachine multi-threaded? I’m mainly wondering if I
need to synchronize the methods that socket data arriving.

James Edward G. II


#12

Writes can block your process whether they are large or small- it all
depends on what is happening in your kernel at that particular point in
time. However, if a socket has just selected writable, then it generally
will be able to take a write that is the size of an ethernet packet
(1400
bytes or so). (Of course the sharper readers will point out that there
are
also good reasons to avoid such small writes, especially if you don’t
turn
off the Nagle algorithm.) But you’re asking a good question. This is one
of
the reasons why blocking i/o on threads can make a lot of sense if your
app
has relatively light requirements. You may find this odd but I worry
more
about accepts blocking than writes. An accepting socket can select
readable
(meaning a new connection is available), but be empty by the time you
get
around to reading it (like if there was a network error that caused a
pending connection to reset). This can be a real nasty.

If you download the eventmachine gem and install the rdocs, the API docs
will all be there, and there is sample code too.


#13

James, your application needs to support about 50 users with the
occasional
large write. Is the application a client app or a server app. It’s funny
to
re-read your message, because I just started designing a
presence-dissemination system, which will have very similar requirements
to
yours! (The network protocol will work a lot like BGP4, except that the
“routing tables” are actually the IM-presence status of users.) And this
will be both a client and a server.


#14

I’m glad you asked. Eventmachine is single-threaded. That’s an essential
part of its design. (More for performance and scalability than to remove
any
need for synchronization, which is very difficult between native threads
and
Ruby threads. (Well, it’s actually impossible. Eventmachine does it
through
a mechanism that is unrelated to threads.) There’s an EventMachine wiki
that
has a page on the “Reactor model” which explains this in more detail.
http://eventmachine.rubyforge.org/wiki/wiki.pl

However, if you want to implement a thread-pool in Ruby threads to
handle
your workload, that’s perfectly compatible with Eventmachine, and your
threads will run concurrently with Eventmachine. This might make sense
if
the processing you have to do on behalf of your clients involves
something
like writing to a database, where there is a system latency that you
can’t
hand off to Eventmachine (because DBMS client libraries are usually not
asynchronous).


#15

On May 8, 2006, at 2:32 PM, Francis C. wrote:

I’m glad you asked. Eventmachine is single-threaded. That’s an
essential
part of its design.

So my callbacks need to return reasonably quickly right? So
EventMachine can get back to processing input from other sockets. If
I need to do a long-running calculation, I should spawn a Ruby Thread
for that. Sound right?

I’m very excited about working with you’re library. Thanks so much
for answering my questions. I’m sure I will have more at some point…

James Edward G. II


#16

On May 8, 2006, at 2:21 PM, Francis C. wrote:

James, your application needs to support about 50 users with the
occasional
large write. Is the application a client app or a server app.

I’m writing a server.

James Edward G. II


#17

On May 8, 2006, at 4:24 PM, Bill K. wrote:

Judging from the results of my test program on Linux and
Windows, it appears nonblocking writes are supported in
1.8.4.

This is some awesome information. Thanks Bill!

I believe I am going to try EventMachine first though as it looks
quite nice to work with.

James Edward G. II


#18

From: “James Edward G. II” removed_email_address@domain.invalid

Ruby’s threads aren’t native. Right?
Judging from the results of my test program on Linux and
Windows, it appears nonblocking writes are supported in
1.8.4.

ruby 1.8.4 (2005-12-24) [i686-linux]
ruby 1.8.4 (2005-12-24) [i386-mswin32]

Interestingly, while the Linux version behaved as expected,
displaying blocking characteristics until I explicitly set
nonblock=true, the windows version appears to sport some
sort of always-on nonblocking buffered implementation behind
the scenes now.

On Windows, even if I never set nonblock=true, I could syswrite
40 megs to the socket and #syswrite would return immediately!
(Subsequent syswrites would then appear to block until the
data from the previous huge syswrite was consumed; but just the
ruby thread performing the syswrite was blocked, never the whole
process. So anyway there’s some kind of dynamic buffering going
on behind the scenes in the windows implementation–but in any
case, it never blocked the whole process. Excellent.)

In my test program (attached) I’m catching just about every
exception under the sun, including EAGAIN, which I saw being
done in lib/openssl/buffering.rb. However, the only
exception I actually saw raised was EOFError. (I never tried
to force an EPIPE condition on the write.)

Anyway, for what it’s worth, it appears ruby 1.8.4 now
definitely supports nonblocking socket I/O on both Linux
and Windows!! Yay!

I’ll still be looking at EventMachine though. :slight_smile:

Regards,

Bill


#19

Yes, you want your callbacks to return quickly. Spin a Ruby thread
otherwise. If it turns out that many users of Eventmachine face that
issue
(such as doing a database call while processing a callback) then we’ll
look
at incorporating a thread-pool into Eventmachine. That would fit with
the
goals of Eventmachine, which are performance/scalability, ease-of-use,
and
feature-completeness for server implementations.


#20

From what you said, this code doesn’t do what you think it should do:

hmm, io/nonblock refuses to work on windows because

of missing GETFL :frowning: Just fake it…

if RUBY_PLATFORM =~ /mswin32/
class IO
def nonblock=(nb)
fcntl(Fcntl::F_SETFL, nb ? Fcntl::O_NONBLOCK : 0)
end
end
end

That’s because on Windows, this isn’t the way to set sockets
nonblocking.
Microsoft reinvented a lot of wheels in Windows, and this is one of 'em.
Try
this C code instead (I’ll leave it to you to turn it into Ruby):

unsigned long one = 1;
ioctlsocket (the_socket_descriptor, FIONBIO, &one);

This will return non-zero in case of error (I’ve never seen it return an
error).
Best
-francis