Variants on this theme have been posted time and again, but I can’t find
a solution for this specific problem:
Say I have a program I need to use that performs a task for a given
amount of time then finishes, all the while providing a ‘percent
complete’ style output on STDOUT. eg:
./counting.rb
[RUBY]
#!/usr/bin/env ruby
n = 0
while n <= 10
$stdout.puts “Completed: #{n*10}%”
n = n + 1
end
[/RUBY]
Now, I’d like to build a ruby script that will call this script, wait
until completion and then continue with its tasks BUT all the while
updating a variable with the percentage complete. eg:
./script.rb
[RUBY]
def updatePercentage(pc)
p “The percentage is now #{pc}”
end
Thread.new {
– Problem area
Attempt 1
./counting.rb
This will wait until after counting.rb is complete before
“All Done!” is printed, but I can’t get to its stdout stream
before the program has finished running
Attempt 2
Open3.popen3(’./counting.rb’) { |stdin, stdout, stderr|
p stdout.readpartial(1)
# With a view to doing something like:
updatePercentage(stdout.readline.gsub(/^.*([0-9]+)%.*$/,"$1"))
}
This doesn’t work, counting.rb never begins executing
(tested by adding a line: open(‘output.txt’,‘w’) {|file| file.puts
“Percentage: #{n*10}%” }
to counting.rb and watching the file externally)
p “All done!”
}
p “Irrelevant things”
sleep
[/RUBY]
Any ideas anyone? Essentially I’m looking for ways to wrap C programs
into ruby.
Cheers, JP
I won’t go into details but you should look into pipes in this case. By
creating a process which does the work your wrapper code can just wait
on
the pipe.
On Thu, Mar 19, 2009 at 10:13 AM, Jp Hastings-spital
Dylan E. wrote:
I won’t go into details but you should look into pipes in this case.
Thanks for the prompt response - I’m assuming you mean Ruby pipes not
BASH pipes or equivalent. (class IO - RDoc Documentation)
I’ll post here if I find a solution.
On 19.03.2009 10:02, Jp Hastings-spital wrote:
Dylan E. wrote:
I won’t go into details but you should look into pipes in this case.
Thanks for the prompt response - I’m assuming you mean Ruby pipes not
BASH pipes or equivalent. (class IO - RDoc Documentation)
I’ll post here if I find a solution.
I have no idea what you mean by “bash pipes” but the underlying
mechanism is the same. It’s a feature of the operating system. One
solution is to use IO.popen. Your code does not read all the lines,
that might be one reason for your issue.
robert@fussel /c/Temp
$ ./run-counter.rb
Got 0
Got 10
Got 20
Got 30
Got 40
Got 50
Got 60
Got 70
Got 80
Got 90
Got 100
robert@fussel /c/Temp
$ cat run-counter.rb
#!/usr/bin/env ruby19
IO.popen “./counting.rb” do |io|
io.each do |line|
pct = line[%r{:\s*(\d+)%}, 1].to_i
puts “Got #{pct}”
end
end
robert@fussel /c/Temp
$ cat counting.rb
#!/usr/bin/env ruby19
$stdout.sync = true
11.times do |n|
puts “Completed: #{n*10}%”
end
robert@fussel /c/Temp
$
Similar with Open3, only that you then get multiple pipes.
Cheers
robert
Robert,
Thanks very much for your reply - its helped no end! However I’m still
having problems:
Firstly (though really this isn’t vitally important, as I only need
stdout):
Open3.popen3 doesn’t appear to work when I drop it in place of IO.popen
(where |stdout| is replaced with |stdin,stdout,stderr|).
The called program is definitely running, however even the following
code outputs nothing:
Open3.popen3 “./counting.rb” do |stdin,stdout,stderr|
p stdout.read
end
But as I say, I only really need the stdout pipe. The problem here is
that the program I’m calling doesn’t appear to send the \n character at
all (its the output from HandBrake’s CLI), and as such the io block only
gets called once, at the end of the program’s execution. Here’s some
code:
$ HandBrakeCLI {options} 2> /dev/null
\rEncoding: task 1 of 1, 0.09 %\rEncoding: task 1 of 1, 0.13
%\rEncoding: task 1 of 1, 0.18 %\rEncoding: task 1 of 1, 0.23 %
$ cat wrapper.rb
input = “input.avi”
output = “output.mp4”
IO.popen("/usr/local/bin/HandBrakeCLI -i “#{input}” -o “#{output}”")
{ |stdout|
p stdout.read
}
This prints nothing! Am I right in thinking that the popen block gets
called every time a \n character is captured by the pipe? (Your
‘counting.rb’ uses \n rather than \r - and works) Any ideas (other than
say, passing the output through sed or something similar)?
Thanks again, JP
Few more remarks:
On 19.03.2009 22:00, Jp Hastings-spital wrote:
Open3.popen3 “./counting.rb” do |stdin,stdout,stderr|
p stdout.read
This will read the whole stream to the end! Which means, it will return
only after the process died.
%\rEncoding: task 1 of 1, 0.18 %\rEncoding: task 1 of 1, 0.23 %
If you have “\r” as line terminator, you can do the reading like this
io.each “\r” do |line|
do whatever with line
end
e.g.
robert@fussel ~
$ ruby -e ‘3.times {|i| printf “%03d\r”, i}’ | ruby -e
‘$stdin.each("\r") {|l| p l}’
“000\r”
“001\r”
“002\r”
robert@fussel ~
$
$ cat wrapper.rb
input = “input.avi”
output = “output.mp4”
IO.popen("/usr/local/bin/HandBrakeCLI -i “#{input}” -o “#{output}”")
{ |stdout|
p stdout.read
See above.
}
This prints nothing! Am I right in thinking that the popen block gets
called every time a \n character is captured by the pipe? (Your
‘counting.rb’ uses \n rather than \r - and works) Any ideas (other than
say, passing the output through sed or something similar)?
Thanks again, JP
Note that the proper line reading (as suggested above) still does not
help if the other program internally buffers IO.
Kind regards
robert
On 19.03.2009 22:00, Jp Hastings-spital wrote:
Open3.popen3 “./counting.rb” do |stdin,stdout,stderr|
p stdout.read
end
No idea what you’re doing, but it works for me
robert@fussel /c/Temp
$ ./run-counter.rb
Got 0
Got 10
Got 20
Got 30
Got 40
Got 50
Got 60
Got 70
Got 80
Got 90
Got 100
Got3 0
Got3 10
Got3 20
Got3 30
Got3 40
Got3 50
Got3 60
Got3 70
Got3 80
Got3 90
Got3 100
robert@fussel /c/Temp
$ cat run-counter.rb
#!/usr/bin/env ruby19
require ‘open3’
IO.popen “./counting.rb” do |io|
io.each do |line|
pct = line[%r{:\s*(\d+)%}, 1].to_i
puts “Got #{pct}”
end
end
Open3.popen3 “./counting.rb” do |sin,sout,serr|
sin.close
th = Thread.new { serr.read }
sout.each do |line|
pct = line[%r{:\s*(\d+)%}, 1].to_i
puts “Got3 #{pct}”
end
th.join
end
robert@fussel /c/Temp
$
$ cat wrapper.rb
input = “input.avi”
output = “output.mp4”
IO.popen("/usr/local/bin/HandBrakeCLI -i “#{input}” -o “#{output}”")
{ |stdout|
p stdout.read
}
This prints nothing! Am I right in thinking that the popen block gets
called every time a \n character is captured by the pipe?
No. The block is called once per execution of the other process.
(Your
‘counting.rb’ uses \n rather than \r - and works) Any ideas (other than
say, passing the output through sed or something similar)?
Could be that your problem is IO buffering on the sender side (i.e. the
program you invoke). Alternatively, if you do not close stdin of the
subprogram that might be waiting for some input.
Cheers
robert
On 20.03.2009 19:05, Jp Hastings-spital wrote:
IO.popen method with io.each “\r” works spot on! Thanks so much!
Haven’t tried working with Open3.popen3 yet, I may try reinstalling the
gem.
Thanks for your replies, you’ve been patient and very helpful!
You’re welcome. Good to hear that your issue is fixed now!
Kind regards
robert
IO.popen method with io.each “\r” works spot on! Thanks so much!
Haven’t tried working with Open3.popen3 yet, I may try reinstalling the
gem.
Thanks for your replies, you’ve been patient and very helpful!
Robert K. wrote:
%\rEncoding: task 1 of 1, 0.18 %\rEncoding: task 1 of 1, 0.23 %
If you have “\r” as line terminator, you can do the reading like this
io.each “\r” do |line|
do whatever with line
end
This makes it work beautifully!
e.g.
robert@fussel ~
$ ruby -e ‘3.times {|i| printf “%03d\r”, i}’ | ruby -e
‘$stdin.each("\r") {|l| p l}’
“000\r”
“001\r”
“002\r”
robert@fussel ~
$
$ cat wrapper.rb
input = “input.avi”
output = “output.mp4”
IO.popen("/usr/local/bin/HandBrakeCLI -i “#{input}” -o “#{output}”")
{ |stdout|
p stdout.read
See above.
}
This prints nothing! Am I right in thinking that the popen block gets
called every time a \n character is captured by the pipe? (Your
‘counting.rb’ uses \n rather than \r - and works) Any ideas (other than
say, passing the output through sed or something similar)?
Thanks again, JP
Note that the proper line reading (as suggested above) still does not
help if the other program internally buffers IO.
Kind regards
robert