Hi everyone,
I’m currently developing a Ruby TCP client and have a problem with
socket.recv().
After establishing a connection with my server, I exchange a couple of
messages until the server begins to do some calculations based on the
client’s request that take about 10 seconds. After the server finishes,
it
sends back the result. Now my problem is that “data = socket.recv(size)”
seems to behave non-blocking for some reason. It does not wait for the
10
seconds but rather executes immediately and “data” remains empty. When I
add
“sleep(10)” before, it works…
Any hints would be great.
Thanks in advance,
Frank
Frank P. wrote:
“sleep(10)” before, it works…
#recv gives you data as soon as there is data available, not necessarily
waiting for size bytes.
You may want to buffer it in your ruby code.
On 10/30/07, Frank P. [email protected] wrote:
Yes, I understand that socket.recv() reads UP TO size bytes, but doesn’t
necessarily wait.
How would you buffer the data?
Maybe my server indeed sends some data when I think it doesn’t, so my
client
receives a bit of data and thinks that’s it. Buffering should do the trick
here. But how?
If the server closes the connection after sending the data, you can
simply
use socket#read. If you need to wait for the data and then process them
as
they comes in, use #select in a loop until all the data have been
received.
Yes, I understand that socket.recv() reads UP TO size bytes, but doesn’t
necessarily wait.
How would you buffer the data?
Maybe my server indeed sends some data when I think it doesn’t, so my
client
receives a bit of data and thinks that’s it. Buffering should do the
trick
here. But how?
Thanks!
2007/10/30, Francis C. [email protected]:
If the server closes the connection after sending the data, you can simply
use socket#read. If you need to wait for the data and then process them as
they comes in, use #select in a loop until all the data have been received.
There is another option - even if the server does not close after
sending: use #read with a given size limit. #read will block and
return as soon data is available. Using #read without limit is a bad
idea on a socket anyway because it has to read until EOS, as you
indicated.
DRb would be another option if both sides are written in Ruby.
Kind regards
robert
Frank P. wrote:
Now my problem is that “data = socket.recv(size)”
seems to behave non-blocking for some reason. It does not wait…
I just wanted to add that as far as I know, you can’t count on
recv(1012) to read all the bytes sent when, say, only 500 bytes are
sent. recv() may only read 1 byte the first time it is called or it may
read 200 bytes, or it may read all 500 bytes. As a consequence, in
order to get all the data you want out of the socket, you have to loop
over recv() and either:
-
count the number of bytes that recv() read using String#length, and
stop after a certain number of bytes, or
-
keep looping until recv encounters eof(i.e. a blank string is sent
when the server closes the socket) and you have read all the data.
In your case, you say you want to block until all the data is read, so
you could use read() and specify a large limit that is bigger than the
length of the data that will be sent, and simply write:
all_data = read(8196)
#do something with all_data
My tests show read() will block and wait until it either reads all 8196
bytes or the server closes the socket.
Robert K. wrote:
2007/10/30, Francis C. [email protected]:
If the server closes the connection after sending the data, you can simply
use socket#read. If you need to wait for the data and then process them as
they comes in, use #select in a loop until all the data have been received.
There is another option - even if the server does not close after
sending: use #read with a given size limit. #read will block and
return as soon data is available.
That’s not what I’m seeing. What I see is: read() blocks until it
either receives the limit number of bytes or eof is encountered(when the
server closes the socket). In other words, read() does not return as
soon as data is available if the amount of bytes read is less than the
limit number of bytes. On the other hand, recv() returns whatever it
reads immediately. Here is the code:
require ‘socket’
port = 3030
server = TCPServer.new(‘’, port)
while(conn = server.accept) #serves forever
conn.print ‘h’
sleep(5)
conn.print ‘ello world’
conn.close
end
#client using recv():
require ‘socket’
port = 3030
socket = TCPSocket.new(‘localhost’, port)
all_data = []
while true
partial_data = socket.recv(1012)
puts partial_data
if partial_data.length == 0
break
end
all_data << partial_data
end
socket.close
puts all_data.join()
–output:–
h
ello world
#blank string returned when server closed socket
hello world
#client using read():
require ‘socket’
port = 3030
socket = TCPSocket.new(‘localhost’, port)
all_data = []
while partial_data = socket.read(1012)
puts partial_data
all_data << partial_data
end
socket.close
puts all_data.join()
–output: –
hello world
hello world
So the way I see it:
-
If the amount of data sent by the server is less than the size limit
specified in read(), read() will block until the server closes the
socket. Therefore, the server must close the socket.
-
If the amount of data sent by the server is more than the size limit
specified in read(), and you just write:
all_data = read(1012)
puts data
then you aren’t reading all the data. You just get the first 1012 bytes.
2007/10/30, 7stud – [email protected]:
That’s not what I’m seeing. What I see is: read() blocks until it
either receives the limit number of bytes or eof is encountered(when the
server closes the socket). In other words, read() does not return as
soon as data is available if the amount of bytes read is less than the
limit number of bytes. On the other hand, recv() returns whatever it
reads immediately. Here is the code:
Yes, I was imprecise: when using #read(limit) the method will block
until either limit bytes were sent or the server closes the
connection; up to limit bytes will be returned. When the server closed
the connection then the next #read will return nil indicating EOS.
Thanks for catching that!
Kind regards
robert
On 10/30/07, 7stud – [email protected] wrote:
That’s not what I’m seeing. What I see is: read() blocks until it
either receives the limit number of bytes or eof is encountered(when the
server closes the socket). In other words, read() does not return as
soon as data is available if the amount of bytes read is less than the
limit number of bytes. On the other hand, recv() returns whatever it
reads immediately. Here is the code:
…snip…
So the way I see it:
then you aren’t reading all the data. You just get the first 1012 bytes.
We know nothing about the protocol that the OP is using (because he
hasn’t
told us yet). It may not be realistic for the client to have an
expectation
about how much data the server will send. #select may turn out to be the
best choice.
#read with no parameters works if the server closes the connection after
sending. But as a general rule, that’s a bad server design (ideally, the
client should close the connection, for reasons relating to the
TIME_WAIT
state).
2007/10/30, Francis C. [email protected]:
We know nothing about the protocol that the OP is using (because he hasn’t
told us yet). It may not be realistic for the client to have an expectation
about how much data the server will send. #select may turn out to be the
best choice.
Quoting from the original posting:
Now my problem is that “data = socket.recv(size)”
seems to behave non-blocking for some reason. It does not wait for the 10
seconds but rather executes immediately and “data” remains empty. When I add
“sleep(10)” before, it works…
So it seems pretty clear that he knows how many bytes to expect. In
which case I’d use #read because the code will be simpler than with
#select.
#read with no parameters works if the server closes the connection after
sending. But as a general rule, that’s a bad server design (ideally, the
client should close the connection, for reasons relating to the TIME_WAIT
state).
Agreed: the client should establish and close a connection.
Kind regards
robert
Make sure you flush on the sending side, as it might not send it all
immediately (uses the nagle algorithm).
Frank P. wrote:
Hi everyone,
I’m currently developing a Ruby TCP client and have a problem with
socket.recv().
After establishing a connection with my server, I exchange a couple of
messages until the server begins to do some calculations based on the
client’s request that take about 10 seconds. After the server finishes,
it
sends back the result. Now my problem is that “data = socket.recv(size)”
seems to behave non-blocking for some reason. It does not wait for the
10
seconds but rather executes immediately and “data” remains empty. When I
add
“sleep(10)” before, it works…
Any hints would be great.
Thanks in advance,
Frank
Frank P. wrote:
Yes, I understand that socket.recv() reads UP TO size bytes, but doesn’t
necessarily wait.
How would you buffer the data?
Here’s an idea that’s a little different from what you’re asking for (it
assumes the data stream is chunked using 4 byte length fields, rather
than knowing the length in advance), but it shows a working buffer
implementation. (See also
http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/177704)
module Messageable
LEN_LEN = 4
def recv_message
if (data = recv(LEN_LEN))
if data.empty?
nil
else
len = Message::Length.new(data).value
for extra safety:
if len > MAXLEN
raise MessageLengthError, "MAXLEN exceeded: #{len} >
#{MAXLEN}"
end
msg = ""
part = nil
while (delta = len - msg.length) > 0 and
(part = recv(delta)) and
not part.empty?
yield part if block_given?
msg << part
end
msg.empty? ? nil : msg
end
end
end
end
First of all thanks a lot for all your inputs!
Using read instead of recv indeed leads to blocking (like I want it to),
at
least when sending a bigger amount of data. But only receiving say 2
bytes
using socket.read(2) somehow does not work, probably because I don’t
flush
on the sender side. Compared to socket.revc, it does not skip
immediately,
but blocks until it times out, although the sender has sent the 2 bytes.
It
receives nothing in the end. I’ll have to investigate a bit more. Seems
like
I did not do my homework in understanding how sockets work in detail
(especially on the server side which I wrote in C++/QT).
Thanks for the fruitful discussion,
Frank
On 10/30/07, Roger P. [email protected] wrote:
messages until the server begins to do some calculations based on the
Thanks in advance,
Frank
–
Posted via http://www.ruby-forum.com/.
–
Frank P.
Breitestrasse 50
4132 Muttenz
phone: +41 61 463 80 30
mobile: +41 76 390 60 29