# Fatal flaw in popen4 on windows? Re: Nonblocking IO read

**URL:** <https://www.ruby-forum.com/t/fatal-flaw-in-popen4-on-windows-re-nonblocking-io-read/74691>\
**Category:** Ruby\
**Created:** [November 7, 2006, 10:52am UTC](https://www.ruby-forum.com/t/fatal-flaw-in-popen4-on-windows-re-nonblocking-io-read/74691 "2006-11-07T10:52:47Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![unknown](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/u/4bbf92/32.png) [@unknown](https://www.ruby-forum.com/u/unknown)\
**Post date:** [November 7, 2006, 10:52am UTC](https://www.ruby-forum.com/t/fatal-flaw-in-popen4-on-windows-re-nonblocking-io-read/74691/1 "2006-11-07T10:52:47Z")

</div>

On Thu, 2 Nov 2006, Robert K. wrote:

> Tom P. wrote:
> 
> > Anyway, you would only need nonblocking IO if you wanted to read bits of  
> > the stderr stream before the command exited, but that doesn’t sound like  
> > what you’re want.
> 
> Actually this is not correct: if there is a lot written to stderr then you  
> need to read that concurrently. If you do not do that then the process will  
> block on some stderr write operation that fills up the pipe and you get a  
> deadlock because your code waits for process termination.

not only is that true but, afaik, it’s why popen4 cannot even work on  
windows!  
this program will eventually hang on either windows or unix

```
 harp:~ > cat a.rb
 require 'rubygems'
 require 'popen4'

 n = (ARGV.shift || 4242).to_i
 ruby = ARGV.shift || 'ruby'

 system "ruby -e 42" or abort "ruby not in your path!"

 STDOUT.sync = STDERR.sync = true

 program = <<-program
   #{ n }.times do
     t = Time.now.to_f
     STDOUT.puts t
     STDERR.puts t
   end
 program

 POpen4.popen4(ruby) do |stdout, stderr, stdin, pid|
   STDOUT.puts pid

   stdin.puts program
   stdin.close

   Thread.new{ stdout.each{|line| STDOUT.puts line} }
   #
   # uncomment and it won't hang!!!
   #
   #Thread.new{ stderr.each{|line| STDERR.puts line} }
 end

 puts 'done'

```

however, on windows it will always hang - even if the line above is  
uncommented. this is because if one popen4s a process it’s  
**essential** , as  
robert correctly points out, to continually consume any stdout or stderr  
produced - otherwise the program will eventually get stuck in EPIPE and  
you’ll  
be waiting for this stuck program.

and here’s the rub: you cannot reliably consume both stdout and stderr  
under  
windows using threads or, afaik, nonblocking io. perhaps the new  
nonblock\_\*  
methods could help with this? it’d no doubt be a major reworking…

\*\*\* i’m really hoping someone will chime in here and prove me wrong \*\*\*

for reference i’m including the code for my open4 lib’s spawn method:  
which  
illustrates the logical concept of what must be done to avoid a  
subprocess  
blocked writing to it’s parent’s pipes…

# 

# for complete code see

# [http://codeforpeople.com/lib/ruby/open4/open4-0.9.1/lib/open4.rb](http://codeforpeople.com/lib/ruby/open4/open4-0.9.1/lib/open4.rb)

# [http://rubyforge.org/frs/?group\_id=1024&release\_id=7556](http://rubyforge.org/frs/?group_id=1024&release_id=7556)

# 

def spawn arg, \*argv  
#–{{{  
argv.unshift(arg)  
opts = ((argv.size \> 1 and Hash === argv.last) ? argv.pop : {})  
argv.flatten!  
cmd = argv.join(’ ')

```
 getopt = getopts opts

 ignore_exit_failure = getopt[ 'ignore_exit_failure', 

```

getopt[‘quiet’, false] ]  
ignore\_exec\_failure = getopt[ ‘ignore\_exec\_failure’,  
!getopt[‘raise’, true] ]  
exitstatus = getopt[%w( exitstatus exit\_status status )]  
stdin = getopt[%w( stdin in i 0 ) \<\< 0]  
stdout = getopt[%w( stdout out o 1 ) \<\< 1]  
stderr = getopt[%w( stderr err e 2 ) \<\< 2]  
pid = getopt[‘pid’]  
timeout = getopt[%w( timeout spawn\_timeout )]  
stdin\_timeout = getopt[%w( stdin\_timeout )]  
stdout\_timeout = getopt[%w( stdout\_timeout io\_timeout )]  
stderr\_timeout = getopt[%w( stderr\_timeout )]  
status = getopt[%w( status )]  
cwd = getopt[%w( cwd dir ), Dir.pwd]

```
 exitstatus =
   case exitstatus
     when TrueClass, FalseClass
       ignore_exit_failure = true if exitstatus
       [0]
     else
       [*(exitstatus || 0)].map{|i| Integer i}
   end

 stdin ||= '' if stdin_timeout
 stdout ||= '' if stdout_timeout
 stderr ||= '' if stderr_timeout

 started = false

 status =
   begin
     Dir.chdir(cwd) do
       Timeout::timeout(timeout) do
         popen4(*argv) do |c, i, o, e|
           started = true

           %w( replace pid= << push update ).each do |msg|
             break(pid.send(msg, c)) if pid.respond_to? msg
           end

```

# 

# this is the critical bit!!!

# 

```
           te = ThreadEnsemble.new c

           te.add_thread(i, stdin) do |i, stdin|
             relay stdin, i, stdin_timeout
             i.close rescue nil
           end

           te.add_thread(o, stdout) do |o, stdout|
             relay o, stdout, stdout_timeout
           end

           te.add_thread(e, stderr) do |o, stderr|
             relay e, stderr, stderr_timeout
           end

           te.run
         end
       end
     end
   rescue
     raise unless(not started and ignore_exec_failure)
   end

 raise SpawnError.new(cmd, status) unless
   (ignore_exit_failure or (status.nil? and ignore_exec_failure) or 

```

exitstatus.include?(status.exitstatus))

```
 status

```

#–}}}  
end

kind regards.

## -a

my religion is very simple. my religion is kindness. – the dalai lama

-a