On Thu, 2 Nov 2006, Robert K. wrote:
Tom P. wrote:
Anyway, you would only need nonblocking IO if you wanted to read bits of
the stderr stream before the command exited, but that doesn’t sound like
what you’re want.Actually this is not correct: if there is a lot written to stderr then you
need to read that concurrently. If you do not do that then the process will
block on some stderr write operation that fills up the pipe and you get a
deadlock because your code waits for process termination.
not only is that true but, afaik, it’s why popen4 cannot even work on
windows!
this program will eventually hang on either windows or unix
harp:~ > cat a.rb
require 'rubygems'
require 'popen4'
n = (ARGV.shift || 4242).to_i
ruby = ARGV.shift || 'ruby'
system "ruby -e 42" or abort "ruby not in your path!"
STDOUT.sync = STDERR.sync = true
program = <<-program
#{ n }.times do
t = Time.now.to_f
STDOUT.puts t
STDERR.puts t
end
program
POpen4.popen4(ruby) do |stdout, stderr, stdin, pid|
STDOUT.puts pid
stdin.puts program
stdin.close
Thread.new{ stdout.each{|line| STDOUT.puts line} }
#
# uncomment and it won't hang!!!
#
#Thread.new{ stderr.each{|line| STDERR.puts line} }
end
puts 'done'
however, on windows it will always hang - even if the line above is
uncommented. this is because if one popen4s a process it’s
essential, as
robert correctly points out, to continually consume any stdout or stderr
produced - otherwise the program will eventually get stuck in EPIPE and
you’ll
be waiting for this stuck program.
and here’s the rub: you cannot reliably consume both stdout and stderr
under
windows using threads or, afaik, nonblocking io. perhaps the new
nonblock_*
methods could help with this? it’d no doubt be a major reworking…
*** i’m really hoping someone will chime in here and prove me wrong ***
for reference i’m including the code for my open4 lib’s spawn method:
which
illustrates the logical concept of what must be done to avoid a
subprocess
blocked writing to it’s parent’s pipes…
for complete code see
http://codeforpeople.com/lib/ruby/open4/open4-0.9.1/lib/open4.rb
http://rubyforge.org/frs/?group_id=1024&release_id=7556
def spawn arg, *argv
#–{{{
argv.unshift(arg)
opts = ((argv.size > 1 and Hash === argv.last) ? argv.pop : {})
argv.flatten!
cmd = argv.join(’ ')
getopt = getopts opts
ignore_exit_failure = getopt[ 'ignore_exit_failure',
getopt[‘quiet’, false] ]
ignore_exec_failure = getopt[ ‘ignore_exec_failure’,
!getopt[‘raise’, true] ]
exitstatus = getopt[ %w( exitstatus exit_status status ) ]
stdin = getopt[ %w( stdin in i 0 ) << 0 ]
stdout = getopt[ %w( stdout out o 1 ) << 1 ]
stderr = getopt[ %w( stderr err e 2 ) << 2 ]
pid = getopt[ ‘pid’ ]
timeout = getopt[ %w( timeout spawn_timeout ) ]
stdin_timeout = getopt[ %w( stdin_timeout ) ]
stdout_timeout = getopt[ %w( stdout_timeout io_timeout ) ]
stderr_timeout = getopt[ %w( stderr_timeout ) ]
status = getopt[ %w( status ) ]
cwd = getopt[ %w( cwd dir ), Dir.pwd ]
exitstatus =
case exitstatus
when TrueClass, FalseClass
ignore_exit_failure = true if exitstatus
[0]
else
[*(exitstatus || 0)].map{|i| Integer i}
end
stdin ||= '' if stdin_timeout
stdout ||= '' if stdout_timeout
stderr ||= '' if stderr_timeout
started = false
status =
begin
Dir.chdir(cwd) do
Timeout::timeout(timeout) do
popen4(*argv) do |c, i, o, e|
started = true
%w( replace pid= << push update ).each do |msg|
break(pid.send(msg, c)) if pid.respond_to? msg
end
this is the critical bit!!!
te = ThreadEnsemble.new c
te.add_thread(i, stdin) do |i, stdin|
relay stdin, i, stdin_timeout
i.close rescue nil
end
te.add_thread(o, stdout) do |o, stdout|
relay o, stdout, stdout_timeout
end
te.add_thread(e, stderr) do |o, stderr|
relay e, stderr, stderr_timeout
end
te.run
end
end
end
rescue
raise unless(not started and ignore_exec_failure)
end
raise SpawnError.new(cmd, status) unless
(ignore_exit_failure or (status.nil? and ignore_exec_failure) or
exitstatus.include?(status.exitstatus))
status
#–}}}
end
kind regards.
-a
my religion is very simple. my religion is kindness. – the dalai lama
-a