IO and non-strings


#1

How can I write non-string data using the IO class?
In particular, I’d like to write an integer across a socket connection.
All the methods I see in IO for writing either only write strings or
only write the to_s version of an object, which is of course a string.


#2

On Wed, 1 Feb 2006, Mark V. wrote:

How can I write non-string data using the IO class?
In particular, I’d like to write an integer across a socket connection.
All the methods I see in IO for writing either only write strings or
only write the to_s version of an object, which is of course a string.

see String#unpack and Array#pack.

you probably want something like

socket.write [42].pack(“N”)

cheers.

-a


#3

On 1/31/06, removed_email_address@domain.invalid removed_email_address@domain.invalid wrote:

socket.write [42].pack(“N”)

Thanks! I wasn’t aware of pack/unpack. I still can’t get this simple
example to work though. I’m wondering if I need to use puts and gets.
If I use write, I’m not sure if the other end will know how many bytes
to read. Can you spot anything I’m doing wrong? The client sends
integers to the server. The server computes the average and returns
it. Currently the server only gets two of the three ints and the
client says “‘gets’: Invalid argument”.


client.rb

require ‘socket’

Create socket.

host, port = ‘localhost’, 1919
socket = TCPSocket.new(host, port)

Send request message.

binary_string = [19, 10, 8].pack(‘N*’)
socket.puts binary_string

Get response message.

binary_string = socket.gets
avg = binary_string.unpack(‘g’) # float
puts “avg = #{avg}”

socket.close


server.rb

require ‘socket’

Create server.

host, port = ‘localhost’, 1919
server = TCPServer.new(host, port)

Process connection requests.

loop do
session = server.accept
break if server == nil

Process session requests.

binary_string = session.gets
values = binary_string.unpack(‘N*’)
sum = 0
values.each { |value| sum += value }
puts “sum = #{sum}”
puts “size = #{values.size}”

avg = sum.to_f / values.size

binary_string = [avg].pack(‘g’) # float
session.puts(binary_string)
session.close
end


#4

Mark V. wrote:

require ‘socket’

Create socket.

host, port = ‘localhost’, 1919
socket = TCPSocket.new(host, port)

Send request message.

binary_string = [19, 10, 8].pack(‘N*’)
socket.puts binary_string

You want to avoid using gets/puts with binary strings. Those functions
are for text, and will detect/insert line ends. In fact, 10 is the same
as a \n character, so that’s probably where it’s choking.


#5

On 2/1/06, Joel VanderWerf removed_email_address@domain.invalid wrote:

socket.puts binary_string

You want to avoid using gets/puts with binary strings. Those functions
are for text, and will detect/insert line ends. In fact, 10 is the same
as a \n character, so that’s probably where it’s choking.

What do you recommend I use instead of gets and puts?
If I use write, will I need to also call flush?
Will I have to first send a number to the server that indicates the
number of bytes I’m going to send next so it will know how many to
read?


#6

Mark V. wrote:

binary_string = [19, 10, 8].pack(‘N*’)


R. Mark V.
Partner, Object Computing, Inc.

I am in the habit of using send/recv, which are unbuffered, and also
sending a length field before the data. If you’re interested, I’ve got a
couple of classes (subclasses of TCPSocket and TCPServer) that
encapsulate this and are pretty well tested. (Also a C version of the
same thing.)


#7

On Thu, 2 Feb 2006, Mark V. wrote:

binary_string = [19, 10, 8].pack(‘N*’)
socket.puts binary_string

You want to avoid using gets/puts with binary strings. Those functions
are for text, and will detect/insert line ends. In fact, 10 is the same
as a \n character, so that’s probably where it’s choking.

What do you recommend I use instead of gets and puts?
If I use write, will I need to also call flush?

yes. yes.

Will I have to first send a number to the server that indicates the number
of bytes I’m going to send next so it will know how many to read?

that certainly makes it easier in some cases. be careful about the
length of
the lenght though; for instance, if you decide to send something like
‘length:data’ to the client then

42:foobar # length length == 2
4242:foobar # length length == 4

might want the client to read chars, based on return value of select,
into a
re-sizeable buf until ‘:’ is seen and then convert to int to do a
read(n).

of course, if this were ruby to ruby you would be using drb right?

cheers.

-a


#8

On Thu, 2 Feb 2006, Joel VanderWerf wrote:

I am in the habit of using send/recv, which are unbuffered, and also
sending a length field before the data. If you’re interested, I’ve got a
couple of classes (subclasses of TCPSocket and TCPServer) that
encapsulate this and are pretty well tested. (Also a C version of the
same thing.)

please share! i just wrote that, for like the 10th time, last month :wink:

-a


#9

On Feb 1, 2006, at 10:31 AM, removed_email_address@domain.invalid wrote:

the lenght though; for instance, if you decide to send something like
‘length:data’ to the client then

42:foobar # length length == 2
4242:foobar # length length == 4

might want the client to read chars, based on return value of
select, into a
re-sizeable buf until ‘:’ is seen and then convert to int to do a
read(n).

If you want a general mechanism for doing this type of work between
Ruby and Other Languages, use NetStrings [1]. If it’s Ruby to Ruby,
drb is a great choice. (Oh, there is also a Ruby NetStrings class [2]
but I can’t vouch for its correctness.)

cr

[1] http://cr.yp.to/proto/netstrings.txt
[2] http://raa.ruby-lang.org/project/djb-netstrings/


#10

On Feb 1, 2006, at 11:11 AM, Mark V. wrote:

read?
I’d like to put a plug in for the book: Advanced Programming in the Unix
Environment (APUE) by Rich Stevens. (Disclaimer: I worked with and
co-authored
a book with Rich). If you are doing lots of network programming then
you
want to read Unix Network Programming also.

Many of the IO and networking questions that pop up on this list are
not really Ruby questions but questions about the Unix IO model, socket
programming and/or Unix OS concepts (or Posix if you prefer). If you
are
doing lots of programming of that type it will really help to understand
the underlying OS abstractions. Understanding the Posix OS model will
also help you write more portable code that works correctly on Linux,
Mac OS X,
Windows, and so on.

Gary W.


#11

On Thu, 2 Feb 2006 removed_email_address@domain.invalid wrote:

that certainly makes it easier in some cases. be careful about the length

If you want a general mechanism for doing this type of work between Ruby and
Other Languages, use NetStrings [1]. If it’s Ruby to Ruby, drb is a great
choice. (Oh, there is also a Ruby NetStrings class [2] but I can’t vouch for
its correctness.)

cr

[1] http://cr.yp.to/proto/netstrings.txt
[2] http://raa.ruby-lang.org/project/djb-netstrings/

thanks. i actually read that and used something very close to
netstrings for
my acgi package - but i’d forgotten where i read about it. now i
remember!
:wink:

-a


#12

On Thu, 2 Feb 2006, Joel VanderWerf wrote:

with both blocking and non-blocking sockets. The c lib code works on
windows, but the test doesn’t (it forks). It’s fine on linux and qnx.
The ruby version works on all platforms I’ve tried (windows and linux).


vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407

sweet! thanks joel.

-a


#13

removed_email_address@domain.invalid wrote:

-a

MTCP – Message TCP – Wrapper around TCPSocket and TCPServer that
provides a message (datagram) abstraction implemented by data stream
with added length fields.

Here’s the ruby version, attached. There’s a tarball at
http://redshift.sourceforge.net/mtcp/
with the full deal: c version, tests for both, etc. The c version works
with both blocking and non-blocking sockets. The c lib code works on
windows, but the test doesn’t (it forks). It’s fine on linux and qnx.
The ruby version works on all platforms I’ve tried (windows and linux).