Forum: Ruby on Rails Rails 3.2 and Streaming using response_body

C2a9b1cbe4fe9db10a34fc672cff2cf6?d=identicon&s=25 Matthew Fordham (Guest)
on 2012-03-29 22:09
(Received via mailing list)
I am working on a streaming download (CSV) from Rails 3.2 and am coming
up
against an issue of the initial page request taking a long time. The
following controller code illustrates my issue:

      self.response_body = Enumerator.new do |y|
        10_000_000.times do
          y << "Hello World"
        end
      end

With the above, the response does seem like its streaming (from a server
than can support it... Unicorn, in my case). That said, before it starts
streaming it hangs for a much longer time than I'd like. If I change it
to
the following, it starts much faster:

      self.response_body = Enumerator.new do |y|
        1000.times do
          y << "Hello World"
        end
      end

My understanding is that the response should begin with the first
iteration
of the loop, but it seems the larger loops are causing that initial load
time to lengthen. If each iteration is output as it happens, shouldn't
it
take the same amount of time to kick off the streaming process,
regardless
of how many total iterations there will be???
Here is an explanation of the technique I am attempting. Maybe I am
misinterpreting or missing a step?:
http://facebook.stackoverflow.com/questions/350759...

Thanks for any insight you may have!
Cb146d04eaf373d3946d75fd1c4ed17e?d=identicon&s=25 Cam Allen (Guest)
on 2013-08-15 21:19
(Received via mailing list)
Hi Matthew,

I have built a CSV streaming response that suffers from the same problem
that you've described. It seems the larger content amount will delay the
response headers being sent, thus signifying the start of the stream,
i.e.
save dialog in browser. It seems this has to do with they web server
being
used and possibly buffering responses before sending the stream but
haven't
found a directive to fix it.

I've tested with webrick, puma, thin and passenger+nginx and dug into
the
configurations to no avail. Note: only passenger and puma stream
properly
and they both do so with the delayed response for large streams.

Did you every find out what was causing the delay on larger streams?
Cam
Cb146d04eaf373d3946d75fd1c4ed17e?d=identicon&s=25 Cam Allen (Guest)
on 2013-08-22 17:12
(Received via mailing list)
For future reference, it turns out that Rack is calling each twice on
the
response_body, see

https://groups.google.com/forum/#!topic/rack-devel...
Once i added some checks to only stream once on the second call (rack
closing the connection) it worked as expected! Note: if you only respond
on
the first call then the response will be empty.

Hopefully this saves someone some time in the future!
Cam
Please log in before posting. Registration is free and takes only a minute.
Existing account

NEW: Do you have a Google/GoogleMail, Yahoo or Facebook account? No registration required!
Log in with Google account | Log in with Yahoo account | Log in with Facebook account
No account? Register here.