Net::HTTPResponse#read_body usable for timing?

Ruby
1

I have some simple network performance analysis I need to do; easiest
and most flexible thing to do would be to use Net::HTTP in Ruby to do
this, but I want to make sure I’m getting somewhat realistic results.
(Note since I’m mainly using this for comparisons, not specific timings,
the numbers don’t need to be exact, but would be nice if they were
close.)

In any event, I’m using Net::HTTPResponse#read_body since I want to be
able to get intra-download timings, so I’ve got something like:

Net::HTTP.start(url.host, url.port) do |http|
http.request_get(’/test.txt’) do |res|
# start timer
res.read_body do |fragment|
# stop timer, calculate stats for this interval & fragment size
# restart timer
end
end
end

The numbers I’m getting seem somewhat right but vary a lot,
particularly because each fragment is pretty small (doc says it’s as it
comes from the socket).

Does anyone know if read_body is synchronous wrt to the network reads?
E.g. is it realistic to use this for timing this way? I’ll do some
comparisons with wget, but any help would be appreciated.

Thanks,

dwh

2

Denis H. wrote:

Does anyone know if read_body is synchronous wrt to the network reads?

The source will be installed somewhere like
/usr/lib/ruby/1.8/net/http.rb - it’s all in Ruby and fairly easy to
understand. Look for read_body and read_body_0.

However, you can see it relies on some Ruby internals, such as
Socket#read calling << on the dest buffer object; see ReadAdapter in
net/protocol.rb (*)

Basically, you’ll get a separate Socket#read call for each chunk if
you’re getting HTTP/1.1 chunked encoding for your result body, and a
single Socket#read call for non-chunked with Content-Length. tcpdump
should show what you’re actually getting.

What I don’t know is, if you do Socket#read(1_000_000, dest) is whether
Ruby breaks the read up into smaller ones and calls << onto dest for
each chunk, and if so, what buffer size it’s using. That would involve
looking at the C source code.

HTH,

Brian.

(*) In fact Net::HTTP violates Ruby’s own documentation, which says
explicitly that the result buffer may only be a String object.

---------------------------------------------------------------- IO#read
ios.read([length [, buffer]]) => string, buffer, or nil

 Reads at most _length_ bytes from the I/O stream, or to the end of
 file if _length_ is omitted or is +nil+. _length_ must be a
 non-negative integer or nil. If the optional _buffer_ argument is
 present, it must reference a String, which will receive the data.
3

Thanks, Brian – I was figuring I would probably have to dive into the
source to look at this, thanks for the roadmap. I’ll probably look at
it over the holidays for fun :wink:

dwh