Forum: Ruby Capturing incremental output from STDOUT

Hi all,

If I run an external process, e.g.

IO.popen('cucumber')

Then Ruby blocks while waiting for the whole output of the subprocess. 
Is
there a way to read the subprocess' output incrementally as it is being
written?

James Coglan wrote:
> Hi all,
> 
> If I run an external process, e.g.
> 
> IO.popen('cucumber')
> 
> Then Ruby blocks while waiting for the whole output of the subprocess.

really?

irb(main):001:0> IO.popen("yes yes") do |fp|
irb(main):002:1*     loop do
irb(main):003:2*         puts Time.now
irb(main):004:2>         fp.gets
irb(main):005:2>         sleep 1
irb(main):006:2>     end
irb(main):007:1> end
Wed Mar 10 06:30:25 +0900 2010
Wed Mar 10 06:30:26 +0900 2010
Wed Mar 10 06:30:27 +0900 2010
Wed Mar 10 06:30:28 +0900 2010
Wed Mar 10 06:30:29 +0900 2010
Wed Mar 10 06:30:30 +0900 2010
...

Seems it do not wait the EOF.

James Coglan wrote:
> Hi all,
> 
> If I run an external process, e.g.
> 
> IO.popen('cucumber')
> 
> Then Ruby blocks while waiting for the whole output of the subprocess. 
> Is
> there a way to read the subprocess' output incrementally as it is being
> written?

What's almost certainly happening is that the process which is sending 
the output is buffering it, and not flushing the buffer.

If the subprocess is written in ruby, try adding "$stdout.sync = true" 
at the beginning.

Some apps will switch into unbuffered mode if they think they are 
talking to a human on a terminal (a tty or pty). There is an 
almost-undocumented 'pty' module in the standard library which you can 
use to run a program under a pty: see

http://www.ruby-forum.com/topic/133560

so give that a try if the process you're spawning is not one that you 
can modify.

2010/3/8 James Coglan <jcoglan@googlemail.com>:

> If I run an external process, e.g.
>
> IO.popen('cucumber')
>
> Then Ruby blocks while waiting for the whole output of the subprocess.

IO.popen does not block anything.  You would at least have to read
from or write to the stream in order to get a chance of blocking.

> Is
> there a way to read the subprocess' output incrementally as it is being
> written?

You can use IO#read(int), i.e. with a size limit, or you can read line
based.  What code do you have and what do you want to accomplish?

Kind regards

robert

Robert Klemme wrote:
> 2010/3/8 James Coglan <jcoglan@googlemail.com>:
> 
>> If I run an external process, e.g.
>>
>> IO.popen('cucumber')
>>
>> Then Ruby blocks while waiting for the whole output of the subprocess.
> 
> IO.popen does not block anything.  You would at least have to read
> from or write to the stream in order to get a chance of blocking.

Although the OP didn't mention the platform, as Windows can be broken 
with regards to forking.

But I just tried it with the one-click installer under XP, and it seems 
to be fine:

ruby 1.8.6 (2009-08-04 patchlevel 383) [i386-mingw32]

>> c = IO.popen("pause","w+")
=> #<IO:0x2bea818>
>> c.readpartial(1024)
=> "Press any key to continue . . . "
>> c.puts
=> nil
>> c.readpartial(1024)
=> "\n"
>> c.readpartial(1024)
EOFError: end of file reached

(readpartial reads between 1 and the given number of bytes, depending on 
how many are available at the time)