Write_noblock is blocking?

Anyone know what’s up with TCPSocket#write_noblock actually blocking
when sending a packet that exceeds the buffer size of the socket in
1.8.6 on Mac.

The behaviour seems slightly counter-productive.

/C

Christoffer Lernö wrote:

Anyone know what’s up with TCPSocket#write_noblock

The method hasn’t been invented yet?

On Oct 13, 2007, at 4:20 PM, 7stud – wrote:

The method hasn’t been invented yet?

cfp:~ > ruby -r io/wait -r yaml -e’ y IO.instance_methods.grep(/
non/) ’

  • read_nonblock
  • write_nonblock

but it works for me on osx

can the OP post a non-working code example?

cheers.

a @ http://codeforpeople.com/

On Oct 13, 2007, at 15:20 , 7stud – wrote:

Christoffer Lernö wrote:

Anyone know what’s up with TCPSocket#write_noblock

The method hasn’t been invented yet?

That’s rather harsh for a one letter typo.

On Oct 13, 2007, at 14:55 , Christoffer Lernö wrote:

Anyone know what’s up with TCPSocket#write_noblock actually
blocking when sending a packet that exceeds the buffer size of the
socket in 1.8.6 on Mac.

The behaviour seems slightly counter-productive.

How long does it block? I see ruby occasionally being delayed 10ms
when using write_nonblock, but not repeatably for the same packet
sizes. Probably my benchmark script is wrong.

Could you supply a benchmark script that illustrates your problem?

#write_nonblock first does rb_io_check_closed() which might call down
to fseek(3) then write(2) before calling rb_io_set_nonblock(). I’m
not even close to being a sockets expert though, I just read the man
pages.

Eric H. wrote:

On Oct 13, 2007, at 15:20 , 7stud – wrote:

Christoffer Lern� wrote:

Anyone know what’s up with TCPSocket#write_noblock

The method hasn’t been invented yet?

That’s rather harsh for a one letter typo.

Harsh? In what way?

I wasn’t aware there was a similarly named method. I looked in the
index of pickaxe2, and there was no write_noblock method listed, and now
that I look again there is no write_nonblock method listed either. I
also checked the standard library documentation for BasicSocket, Socket,
and TCPSocket, and there was no write_noblock method listed, and now
that I look again, there is no write_nonblock method either. In fact,
there is no method that starts with ‘write’. I also searched google and
came up with 0 hits for write_noblock, which has to be a first for me.

Of course, because the ruby documentation is so deplorable, maybe I
should have just assumed it was a commonly used method.

On Oct 13, 2007, at 18:01 , ara.t.howard wrote:

Harsh? In what way?

responding that way when you didn’t know

Exactly.

Of course, because the ruby documentation is so deplorable, maybe I
should have just assumed it was a commonly used method.

now you are being harsh again.

cfp:~ > PAGER=cat ri IO|grep non
cfp:~ > PAGER=cat ri IO

Also,

$ ri -l | egrep “write.*block”
IO#write_nonblock

i know ruby-core is always looking for people to contribute for
docs. you certainly get what you pay for with bloody open source eh?

Absolutely. The main reason that ruby documentation is “so
deplorable” is because the people complaining about it are rarely the
ones doing anything to improve it.

On Oct 13, 2007, at 6:31 PM, 7stud – wrote:

Harsh? In what way?
responding that way when you didn’t know

google and
came up with 0 hits for write_noblock, which has to be a first for me.

searching != knowing

besides, google pulls ‘ruby write_noblock’ up on the first hit

http://www.google.com/search?q=ruby
+noblock&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-
US:official&client=firefox-a

Of course, because the ruby documentation is so deplorable, maybe I
should have just assumed it was a commonly used method.

now you are being harsh again.

cfp:~ > PAGER=cat ri IO|grep non
pos, pos=, print, printf, putc, puts, read, read_nonblock,
write_nonblock

cfp:~ > PAGER=cat ri IO
-------------------------------------------------------------- Class: IO
Class +IO+ is the basis for all input and output in Ruby. An I/O
stream may be duplexed (that is, bidirectional), and so may use
more than one native operating system stream.

  Many of the examples in this section use class +File+, the only
  standard subclass of +IO+. The two classes are closely associated.

  As used in this section, _portname_ may take any of the following
  forms.

  *   A plain string represents a filename suitable for the
      underlying operating system.

  *   A string starting with ``+|+'' indicates a subprocess. The
      remainder of the string following the ``+|+'' is invoked as a
      process with appropriate input/output channels connected to

it.

  *   A string equal to ``+|-+'' will create another Ruby

instance as
a subprocess.

  Ruby will convert pathnames between different operating system
  conventions if possible. For instance, on a Windows system the
  filename ``+/gumby/ruby/test.rb+'' will be opened as
  ``+\gumby\ruby\test.rb+''. When specifying a Windows-style

filename
in a Ruby string, remember to escape the backslashes:

     "c:\gumby\ruby\test.rb"

  Our examples here will use the Unix-style forward slashes;
  +File::SEPARATOR+ can be used to get the platform-specific
  separator character.

  I/O ports may be opened in any one of several different modes,
  which are shown in this section as _mode_. The mode may either

be a
Fixnum or a String. If numeric, it should be one of the operating
system specific constants (O_RDONLY, O_WRONLY, O_RDWR, O_APPEND
and
so on). See man open(2) for more information.

  If the mode is given as a String, it must be one of the values
  listed in the following table.

    Mode |  Meaning
    -----+--------------------------------------------------------
    "r"  |  Read-only, starts at beginning of file  (default mode).
    -----+--------------------------------------------------------
    "r+" |  Read-write, starts at beginning of file.
    -----+--------------------------------------------------------
    "w"  |  Write-only, truncates existing file
         |  to zero length or creates a new file for writing.
    -----+--------------------------------------------------------
    "w+" |  Read-write, truncates existing file to zero length
         |  or creates a new file for reading and writing.
    -----+--------------------------------------------------------
    "a"  |  Write-only, starts at end of file if file exists,
         |  otherwise creates a new file for writing.
    -----+--------------------------------------------------------
    "a+" |  Read-write, starts at end of file if file exists,
         |  otherwise creates a new file for reading and
         |  writing.
    -----+--------------------------------------------------------
     "b" |  (DOS/Windows only) Binary file mode (may appear with
         |  any of the key letters listed above).

  The global constant ARGF (also accessible as $<) provides an
  IO-like stream which allows access to all files mentioned on the
  command line (or STDIN if no files are mentioned). ARGF provides
  the methods +#path+ and +#filename+ to access the name of the file
  currently being read.

Includes:

  Enumerable(all?, any?, collect, detect, each_cons, each_slice,
  each_with_index, entries, enum_cons, enum_slice, enum_with_index,
  find, find_all, grep, group_by, include?, include_any?, index_by,
  inject, injecting, map, max, member?, min, partition, reject,
  select, sort, sort_by, sum, to_a, to_json, to_set, zip),
  File::Constants()

Constants:

  SEEK_CUR: INT2FIX(SEEK_CUR)
  SEEK_END: INT2FIX(SEEK_END)
  SEEK_SET: INT2FIX(SEEK_SET)

Class methods:

  for_fd, foreach, new, open, pipe, popen, read, readlines, select,
  sysopen

Instance methods:

  <<, binmode, block_scanf, close, close_read, close_write, closed?,
  each, each_byte, each_line, eof, eof?, fcntl, fileno, flush,

fsync,
getc, gets, inspect, ioctl, isatty, lineno, lineno=, open, pid,
pos, pos=, print, printf, putc, puts, read, read_nonblock,
readbytes, readchar, readline, readlines, readpartial, reopen,
rewind, scanf, seek, soak_up_spaces, stat, sync, sync=, sysread,
sysseek, syswrite, tell, to_i, to_io, tty?, ungetc, write,
write_nonblock

i know ruby-core is always looking for people to contribute for
docs. you certainly get what you pay for with bloody open source eh?

a @ http://codeforpeople.com/

ara.t.howard wrote:

On Oct 13, 2007, at 6:31 PM, 7stud – wrote:

searching != knowing

Maybe in your native language they don’t have question marks for
punctuation. If you don’t know what they mean, perhaps you shouldn’t
criticize?

besides, google pulls ‘ruby write_noblock’ up on the first hit

http://www.google.com/search?q=ruby
+noblock&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-
US:official&client=firefox-a

Now that google has had time to index this page, the hits you claim to
be the Holy Grail for “write no_block” are actually links back to this
thread. I wonder if you post any information on this forum that isn’t
erroneous.

Of course, because the ruby documentation is so deplorable, maybe I
should have just assumed it was a commonly used method.

now you are being harsh again.

It sounds like you are suffering a case of sour grapes because I
corrected you in that other thread. If you are going to post poorly
written examples, which are also not responsive to a poster’s question,
you will get corrected. If that hurts your feelings, maybe you should
construct better examples–or just not post at all.

Absolutely. The main reason that ruby documentation is “so
deplorable” is because the people complaining about it are rarely the
ones doing anything to improve it.

Yes, when a language is poorly designed and/or documented, blame the
people who are trying to learn it.

On Sun, 14 Oct 2007 11:55:56 +0900, 7stud (who has been learning Ruby
for
seven months) wrote to ara (who has been posting here since 2002, and
who
contributed a core bug fix in his first two weeks):

It sounds like you are suffering a case of sour grapes because I
corrected you in that other thread. If you are going to post poorly
written examples, which are also not responsive to a poster’s question,
you will get corrected. If that hurts your feelings, maybe you should
construct better examples–or just not post at all.

::Cough::

Uh. 7stud?

So, yeah.

You’ve come a little late to the party, you’re speaking a little too
loudly, and you keep referring to the caviar as “bean dip”.

It’s awkward.

On 14 Oct 2007, at 01:23, Eric H. wrote:

Could you supply a benchmark script that illustrates your problem?

#write_nonblock first does rb_io_check_closed() which might call
down to fseek(3) then write(2) before calling rb_io_set_nonblock
(). I’m not even close to being a sockets expert though, I just
read the man pages.

Thanks. Know it “should work” made me try to analyze the situation.
Here is an example code:

require ‘socket’
a = TCPServer.new(5000)
socket = TCPSocket.new(“localhost”, 5000)

Using a.accept will give a socket that does not exhibit problems,

Only TCPServer#accept_nonblock seems to create this

odd situation.

other = a.accept_nonblock
puts “Start Write”

Lower values, say “X” * 2000, writes all in one sweep and does

not cause it block

other.write_nonblock(“X” * 300000)
puts “End Write”

It looks like write_nonblock together with accept_nonblock causes
some issues.
Now I could be using these wrong as both are new (in 1.8.6?)

/Christoffer

On Oct 14, 2007, at 24:13 , Christoffer Lernö wrote:

packet sizes. Probably my benchmark script is wrong.

[…]

It looks like write_nonblock together with accept_nonblock causes
some issues.

Now I could be using these wrong as both are new (in 1.8.6?)

I couldn’t reproduce with your this test using:

$ ruby -v
ruby 1.8.6 (2007-09-23 patchlevel 5000) [powerpc-darwin8.10.0]

I’ll to try again tomorrow though, since I’m tired.

I experimented a bit with your benchmark, and found that writing a
large string would only write as much as the socket buffer would
accept (81660 bytes, in my case).

Maybe you need to set the TCP_NODELAY socket option?

socket.setsockopt Socket::IPPROTO_TCP, Socket::TCP_NODELAY, 1

I once made write_nonblock raise Errno::EAGAIN, but I accidentally
deleted that change. I’ll play some more tomorrow when I’m not so
tired.

On 14 Oct 2007, at 09:58, Eric H. wrote:

10ms when using write_nonblock, but not repeatably for the same
situation. Here is an example code:
$ ruby -v
socket.setsockopt Socket::IPPROTO_TCP, Socket::TCP_NODELAY, 1

I once made write_nonblock raise Errno::EAGAIN, but I accidentally
deleted that change. I’ll play some more tomorrow when I’m not so
tired.

Could this be due to me running

ruby 1.8.6 (2007-03-13 patchlevel 0) [i686-darwin8.10.1] ?

The funny thing was that only the socket returned by accept_nonblock
had this behaviour, that is

require ‘socket’
a = TCPServer.new(5000)
socket = TCPSocket.new(“localhost”, 5000)
other = a.accept
puts “Start Write”
other.write_nonblock(“X” * 300000)
puts “End Write”

Works.

require ‘socket’
a = TCPServer.new(5000)
socket = TCPSocket.new(“localhost”, 5000)
other = a.accept_nonblock
puts “Start Write”
other.write_nonblock(“X” * 300000)
puts “End Write”

Doesn’t work.

What’s so special about a socket returned by accept_nonblock?

/Christoffer

In article [email protected],
Christoffer Lernö [email protected] writes:

Could this be due to me running

ruby 1.8.6 (2007-03-13 patchlevel 0) [i686-darwin8.10.1] ?

The funny thing was that only the socket returned by accept_nonblock
had this behaviour, that is

I think it is a Mac OS X problem.

Your example works without problem on GNU/Linux and FreeBSD
box but not on Mac OS X.

% cat z1.rb
require ‘socket’
require ‘fcntl’
a = TCPServer.new(5000)
socket = TCPSocket.new(“localhost”, 5000)
other = a.accept
p other.fcntl(Fcntl::F_GETFL) & Fcntl::O_NONBLOCK
puts “Start Write”
p other.write_nonblock(“X” * 300000)
puts “End Write”
% ./ruby -v z1.rb
ruby 1.8.6 (2007-03-13 patchlevel 0) [powerpc-darwin8.3.0]
0
Start Write
81560
End Write

The IO returned from accept is not nonblocking mode.
So write_nonblock makes it nonblocking mode at first.

% cat z2.rb
require ‘socket’
require ‘fcntl’
a = TCPServer.new(5000)
socket = TCPSocket.new(“localhost”, 5000)
other = a.accept_nonblock
p other.fcntl(Fcntl::F_GETFL) & Fcntl::O_NONBLOCK
puts “Start Write”
p other.write_nonblock(“X” * 300000)
puts “End Write”
% ./ruby -v z2.rb
ruby 1.8.6 (2007-03-13 patchlevel 0) [powerpc-darwin8.3.0]
4
Start Write
^Cz2.rb:8: Interrupt

The IO returned from accept_nonblock seems nonblocking mode.
So write_nonblock doesn’t make it nonblocking mode.
But actually it is not nonblocking mode, I think.

Workaround:

Index: ext/socket/socket.c

— ext/socket/socket.c (revision 13670)
+++ ext/socket/socket.c (working copy)
@@ -1464,6 +1464,24 @@
return init_inetsock(sock, Qnil, arg1, Qnil, Qnil, INET_SERVER);
}

+static void
+make_fd_nonblock(int fd)
+{

  • int flags;
    +#ifdef F_GETFL
  • flags = fcntl(fd, F_GETFL);
  • if (flags == -1) {
  •    rb_sys_fail(0);
    
  • }
    +#else
  • flags = 0;
    +#endif
  • flags |= O_NONBLOCK;
  • if (fcntl(fd, F_SETFL, flags) == -1) {
  •    rb_sys_fail(0);
    
  • }
    +}

static VALUE
s_accept_nonblock(VALUE klass, OpenFile *fptr, struct sockaddr
*sockaddr, socklen_t *len)
{
@@ -1475,6 +1493,7 @@
if (fd2 < 0) {
rb_sys_fail(“accept(2)”);
}

  • make_fd_nonblock(fd2);
    return init_sock(rb_obj_alloc(klass), fd2);
    }

Anyway nonblocking flag of accepted fd is not portable.
http://cr.yp.to/docs/unixport.html

So setting it unconditionally is good thing for portability.

On 14 Oct 2007, at 18:14, Tanaka A. wrote:

I think it is a Mac OS X problem.
p other.fcntl(Fcntl::F_GETFL) & Fcntl::O_NONBLOCK
The IO returned from accept is not nonblocking mode.
p other.write_nonblock(“X” * 300000)
puts “End Write”
% ./ruby -v z2.rb
ruby 1.8.6 (2007-03-13 patchlevel 0) [powerpc-darwin8.3.0]
4
Start Write
^Cz2.rb:8: Interrupt

The IO returned from accept_nonblock seems nonblocking mode.
So write_nonblock doesn’t make it nonblocking mode.
But actually it is not nonblocking mode, I think.

Yeah, that seems to be it.

This workaround in the ruby code gets around the problem:

require ‘socket’
require ‘fcntl’
a = TCPServer.new(5000)
socket = TCPSocket.new(“localhost”, 5000)
other = a.accept_nonblock
other.fcntl(Fcntl::F_SETFL, other.fcntl(Fcntl::F_GETFL) & ~
(Fcntl::O_NONBLOCK))
puts “Start Write”
other.write_nonblock(“X” * 300000)
puts “End Write”

/C

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs