# Non-blocking "send\_data \`foo\`"

**URL:** <https://www.ruby-forum.com/t/non-blocking-send-data-foo/139021>\
**Category:** Rails\
**Created:** [May 30, 2008, 12:28am UTC](https://www.ruby-forum.com/t/non-blocking-send-data-foo/139021 "2008-05-30T00:28:00Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![mjcharl](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/ac91a4/32.png) [@mjcharl](https://www.ruby-forum.com/u/mjcharl)\
**Post date:** [May 30, 2008, 12:28am UTC](https://www.ruby-forum.com/t/non-blocking-send-data-foo/139021/1 "2008-05-30T00:28:00Z")

</div>

Hello list,

The subject pretty much says it all: I’m looking for a way to send the  
output from a long-running, slowly-outputting system command to the  
browser without the app hanging until that system command completes.  
I’ve tried the obvious idioms (see below), but there doesn’t seem to be  
a way to prevent the application blocking on the system command. When  
(eventually) foo completes, the data is sent to the browser, but the  
poor user has to sit watching the egg-timer until all of the data has  
been slurped into memory by rails before it starts the send. I’d like  
the data to start streaming to the browser as soon as it becomes  
available from the system command.

I have tried:  
backticks  
IO.popen  
Proc.new { … File.popen … }

Perhaps I need something other than send\_data? Is there a way to jump  
out of rails to use exec from the controller (“application failed to  
start”)?

I am considering POSTing to a small bespoke CGI. Or maybe I could do  
something cunning with Net::SSH channels? The first of these will  
certainly work, but neither seem very satisfactory. What am I missing?

Thanks for any enlightment,

Martin.

---

<div class="post-metadata">

**Author:** ![mjcharl](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/ac91a4/32.png) [@mjcharl](https://www.ruby-forum.com/u/mjcharl)\
**Post date:** [May 30, 2008, 12:40am UTC](https://www.ruby-forum.com/t/non-blocking-send-data-foo/139021/2 "2008-05-30T00:40:49Z")

</div>

Martin C. wrote:

> Hello list,
> 
> The subject pretty much says it all: I’m looking for a way to send the  
> output from a long-running, slowly-outputting system command to the  
> browser without the app hanging until that system command completes.  
> I’ve tried the obvious idioms (see below), but there doesn’t seem to be  
> a way to prevent the application blocking on the system command. When  
> (eventually) foo completes, the data is sent to the browser, but the  
> poor user has to sit watching the egg-timer until all of the data has  
> been slurped into memory by rails before it starts the send. I’d like  
> the data to start streaming to the browser as soon as it becomes  
> available from the system command.
> 
> I have tried:  
> backticks  
> IO.popen  
> Proc.new { … File.popen … }
> 
> Perhaps I need something other than send\_data? Is there a way to jump  
> out of rails to use exec from the controller (“application failed to  
> start”)?
> 
> I am considering POSTing to a small bespoke CGI. Or maybe I could do  
> something cunning with Net::SSH channels? The first of these will  
> certainly work, but neither seem very satisfactory. What am I missing?
> 
> Thanks for any enlightment,
> 
> Martin.

Pretty sure that rails doesn’t do streaming of any sort.

You could spin up a BackgroundDRB process, and send the user to a page  
with a big happy animated gif that says loading. And refresh, or make a  
little AJAX call, every few seconds. If the job is done, get its data,  
show it to the user.

Or do some sort of polling to get the computed values from your  
background process as it chugs along.

Someone may know of a way, but I don’t think rails is capable of  
streaming much of anything.

---

<div class="post-metadata">

**Author:** ![mjcharl](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/ac91a4/32.png) [@mjcharl](https://www.ruby-forum.com/u/mjcharl)\
**Post date:** [May 30, 2008, 1:45am UTC](https://www.ruby-forum.com/t/non-blocking-send-data-foo/139021/3 "2008-05-30T01:45:58Z")

</div>

On 29 May 2008, at 21:26, Martin C. wrote:

> (eventually) foo completes, the data is sent to the browser, but the  
> Perhaps I need something other than send\_data? Is there a way to jump  
> out of rails to use exec from the controller (“application failed to  
> start”)?

Use a mongrel handler? That won’t block your app (but ties you to  
mongrel)

Fred

---

<div class="post-metadata">

**Author:** ![mjcharl](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/ac91a4/32.png) [@mjcharl](https://www.ruby-forum.com/u/mjcharl)\
**Post date:** [May 30, 2008, 8:05pm UTC](https://www.ruby-forum.com/t/non-blocking-send-data-foo/139021/4 "2008-05-30T20:05:38Z")

</div>

On May 30, 6:48 pm, Martin C. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> [http://blog.rubyenrails.nl/articles/2008/02/23/breaking-out-of-rails-](http://blog.rubyenrails.nl/articles/2008/02/23/breaking-out-of-rails-)…  
> The mongrel documentation is a little… sparse :-/  
> The mongrel handler is just ruby code in the same memory space as the  
> rest of your app. Put the shared code in a module and include it in  
> both handler & controller  
> Your mongrel handler isn’t a controller so all the niceties like the  
> flash and redirect\_to are gone. If you want to redirect then you’ll  
> need to generate the 302 response yourself. Be wary about calling  
> code from your app, there are bits of rails that are not thread safe.

Fred

---

<div class="post-metadata">

**Author:** ![mjcharl](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/ac91a4/32.png) [@mjcharl](https://www.ruby-forum.com/u/mjcharl)\
**Post date:** [June 1, 2008, 9:24pm UTC](https://www.ruby-forum.com/t/non-blocking-send-data-foo/139021/5 "2008-06-01T21:24:51Z")

</div>

On Fri, 2008-05-30 at 11:05 -0700, Frederick C. wrote:

> The mongrel handler is just ruby code in the same memory space as the  
> rest of your app. Put the shared code in a module and include it in  
> both handler & controller

All fixed - thanks for the assistance!

Martin.

---

<div class="post-metadata">

**Author:** ![mjcharl](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/ac91a4/32.png) [@mjcharl](https://www.ruby-forum.com/u/mjcharl)\
**Post date:** [May 30, 2008, 7:50pm UTC](https://www.ruby-forum.com/t/non-blocking-send-data-foo/139021/6 "2008-05-30T19:50:19Z")

</div>

On Thu, 2008-05-29 at 23:57 +0100, Frederick C. wrote:

> Use a mongrel handler? That won’t block your app (but ties you to  
> mongrel)

Thanks Fred, good suggestion (although I also don’t particularly like  
being tied to mongrel, it does happen to be what we’re using, and it  
gets me over the hump for now). The other big help was Alex’s  
re-emphasis of the word “streaming” which got me thinking about audio  
and video streaming, and turned up this nice blog entry:

[http://blog.rubyenrails.nl/articles/2008/02/23/breaking-out-of-rails-mvc-with-mongrel-handlers](http://blog.rubyenrails.nl/articles/2008/02/23/breaking-out-of-rails-mvc-with-mongrel-handlers)

Problem solved!

I have a couple of smaller issues if there’s anyone with any deeper  
mongrel handler knowledge here - for instance, I would like to  
redirect\_to a controller (possibly setting flash[:error] on the way) if  
there’s a problem, but redirect\_to is undefined. I’d also like to DRY-up  
my handler since I’ve reimplemented (copy/paste) a couple of small  
private methods from my controller - they could be moved to the model,  
but then I can’t see how to call those from the mongrel handler either?  
The mongrel documentation is a little… sparse :-/

Thanks again,

Martin.