Forum: Ruby Net::HTTPResponse#read_body usable for timing?

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
2a39aed820c125e6db9826fa8180043f?d=identicon&s=25 Denis Haskin (Guest)
on 2008-12-23 19:12
(Received via mailing list)
I have some simple network performance analysis I need to do; easiest
and most flexible thing to do would be to use Net::HTTP in Ruby to do
this, but I want to make sure I'm getting somewhat realistic results.
(Note since I'm mainly using this for comparisons, not specific timings,
the numbers don't need to be *exact*, but would be nice if they were
close.)

In any event, I'm using Net::HTTPResponse#read_body since I want to be
able to get intra-download timings, so I've got something like:

Net::HTTP.start(url.host, url.port) do |http|
  http.request_get('/test.txt') do |res|
    # start timer
    res.read_body do |fragment|
      # stop timer, calculate stats for this interval & fragment size
      # restart timer
    end
  end
end

The numbers I'm getting seem *somewhat* right but vary a lot,
particularly because each fragment is pretty small (doc says it's as it
comes from the socket).

Does anyone know if read_body is synchronous wrt to the network reads?
E.g. is it realistic to use this for timing this way?  I'll do some
comparisons with wget, but any help would be appreciated.

Thanks,

dwh
753dcb78b3a3651127665da4bed3c782?d=identicon&s=25 Brian Candler (candlerb)
on 2008-12-24 10:56
Denis Haskin wrote:
> Does anyone know if read_body is synchronous wrt to the network reads?

The source will be installed somewhere like
/usr/lib/ruby/1.8/net/http.rb - it's all in Ruby and fairly easy to
understand. Look for read_body and read_body_0.

However, you can see it relies on some Ruby internals, such as
Socket#read calling << on the dest buffer object; see ReadAdapter in
net/protocol.rb (*)

Basically, you'll get a separate Socket#read call for each chunk if
you're getting HTTP/1.1 chunked encoding for your result body, and a
single Socket#read call for non-chunked with Content-Length. tcpdump
should show what you're actually getting.

What I don't know is, if you do Socket#read(1_000_000, dest) is whether
Ruby breaks the read up into smaller ones and calls << onto dest for
each chunk, and if so, what buffer size it's using. That would involve
looking at the C source code.

HTH,

Brian.

(*) In fact Net::HTTP violates Ruby's own documentation, which says
explicitly that the result buffer may only be a String object.

---------------------------------------------------------------- IO#read
     ios.read([length [, buffer]])    => string, buffer, or nil
------------------------------------------------------------------------
     Reads at most _length_ bytes from the I/O stream, or to the end of
     file if _length_ is omitted or is +nil+. _length_ must be a
     non-negative integer or nil. If the optional _buffer_ argument is
     present, it must reference a String, which will receive the data.
2a39aed820c125e6db9826fa8180043f?d=identicon&s=25 Denis Haskin (Guest)
on 2008-12-24 15:20
(Received via mailing list)
Thanks, Brian -- I was figuring I would probably have to dive into the
source to look at this, thanks for the roadmap.  I'll probably look at
it over the holidays for fun ;-)

dwh
This topic is locked and can not be replied to.