Forum: Ruby Asynchronous IO

I have found quite a bit on the web regarding asynchronous IO, but
nothing really works the way I expected it to in my case.

I'm new to this but herewith my question anyway:

Currently I'm using OI.popen to open a new pipe to another process, and
I feed commands to it; but sometimes the other process could run for
multiple minutes before closing the pipe; and only at that stage can I
then see the output. Now, what I really need is to see the output of the
process as it writes to stdout already and not only when closing the
pipe?

Ta.

 This is far from a clean solution, mind you:

    pipe = IO.popen("command", "options")
    Thread.new do
      while(true)
        p pipe.gets        
      end
    end
    pipe.puts("I'm writing this on the pipe")

You should see the process's output in your STDOUT.

Andrea
 
--
 
http://usingimho.wordpress.com
http://github.com/bolthar
Da: "F1 dreamer" lists@ruby-forum.com
A: ruby-talk@ruby-lang.org
Cc:
Data: Thu, 28 Jun 2012 23:12:28 +0900
Oggetto: Asynchronous IO

On 06/28/2012 09:12 AM, F1 dreamer wrote:
> Currently I'm using OI.popen to open a new pipe to another process, and
> I feed commands to it; but sometimes the other process could run for
> multiple minutes before closing the pipe; and only at that stage can I
> then see the output. Now, what I really need is to see the output of the
> process as it writes to stdout already and not only when closing the
> pipe?

The problem you're probably facing is due to the program being run via
popen buffering its output.  This is pretty normal when the program is
outputting to a pipe, as it does when run via popen.

This page has a reasonable summary of things:

http://blog.habets.pp.se/2008/06/Buffering-in-pipes

Unless the program you're trying to run has some specific options
available to force it to disable buffering, you're probably going to be
forced to write your own sort of popen method that allocates a pty
rather than a pipe for interaction with the program.

-Jeremy

Hi guys,

Thanks a lot, I'll go through and try everything you guys suggested and 
let you know.

Thank you!

On Thu, Jun 28, 2012 at 4:36 PM, Andrea Dallera
<andrea@andreadallera.com> wrote:
> This is far from a clean solution, mind you:

Things to improve:

- use the block form of IO.popen
- close writing when finished
- join on the reader thread

>   pipe =IO.popen("command", "options")
>   Thread.new do
>    while(true)
>     p pipe.gets
>    end
>   end
>   pipe.puts("I'm writing this on the pipe")
>
> You should see the process's output in your STDOUT.

$ ruby19 popen.rb
  0.000 main      : "started"
  0.040 writer    : "about to write first line"
  0.041 reader    : "     1\tfirst line\n"
  1.041 writer    : "about to write second line"
  1.041 reader    : "     2\tsecond line\n"
  6.041 writer    : "about to write last line"
  6.041 reader    : "     3\tlast line\n"
  6.041 writer    : "closed"
  6.041 writer    : "joined"
  6.042 main      : "finished"

Kind regards

robert

Hi,

> Currently I'm using OI.popen to open a new pipe to another process, and
> I feed commands to it; but sometimes the other process could run for
> multiple minutes before closing the pipe; and only at that stage can I
> then see the output. Now, what I really need is to see the output of the
> process as it writes to stdout already and not only when closing the
> pipe?

I just so happen to have been mucking about with the PTY
(pseudoterminal) library under ruby 1.9.3.
A pseudoterminal might be what you need if you can't change the
buffering of the programs you're trying to manage.

I've put my current test script here: https://gist.github.com/3020873

You'll need to play with the options to get the result you want -
there's no really generic way to automatically manage interactive
programs that expect to interact with a person.

I've found it useful to run the test script in one terminal with the
--debug option and in another use

  tail -F dbg.log

to see exactly what is passing between the parent and the child process.

As an example you could look at the differences between:

  ./pty-test.rb --debug irb --simple-prompt
  ./pty-test.rb --debug --raw irb --simple-prompt
  ./pty-test.rb --debug --errpipe --raw irb --simple-prompt

Another one to try is

  ./pty-test.rb --debug --errpipe --raw vi

Regards,
Sean