I am building a nice little system using Ruby (on Rails), one part of
which uses Net::HTTP to retrieve some data over HTTP. Everything seems
to work fine, but on some requests, I get an EOFError.
As I found out, this problem has already been reported, but without any
I think I may have traced the problem back to a bug in Net::HTTP. Here
is a two-liner to reproduce the error:
EOFError: end of file reached
sysread' from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/net/protocol.rb:133:inrbuf_fill’
timeout' from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/timeout.rb:76:intimeout’
rbuf_fill' from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/net/protocol.rb:116:inreaduntil’
readline' from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/net/http.rb:2236:inread_chunked’
read_body_0' from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/net/http.rb:2182:inread_body’
body' from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/net/http.rb:2146:inreading_body’
request' from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/net/http.rb:957:inrequest_get’
get_response' from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/net/http.rb:547:instart’
My quick analysis:
In http.rb, line 2236 the “read_chunked” function calls
@socket.readline. This “readline” function, in protocol.rb line 126,
calls readuntil("\n"). This works fine if the data chunk is
“\n-terminated”, but throws an EOFError if it is not.
I am not sure about the underlying standards of chunked http, maybe the
data chunk is supposed to always been \n-terminated, and it may be a
mis-behaving server, but the fact is: I can get the example image fine
with any browser, but not with Net::HTTP.
As a tentative fix, I wrote a patch that catches the EOFError in
read_chunked. You will the patch file attached. With the patch, things
=> #<Net::HTTPOK 200 OK readbody=true>
With the patch, the above gives me a perfectly fine JPEG file. However,
I am afraid my current patch, with a big begin…rescue around most of
the body of the read_chunked function, catches the EOFError at level
higher than necessary, which is not good practice…
Anyway, before continuing any further, could someone involved in the
development of Ruby take a look at this, confirm the existence of the
bug, and maybe even come up with a better fix?
PS: please let me know if I posted this in the wrong list, or if I
should open a bug report on some bug tracking system.
Thank you for your help.