Forum: Ruby Wrapping applications in Ruby

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
8f196eedfa0d23f02e6d92159e84e63d?d=identicon&s=25 Jp Hastings-spital (jphastings)
on 2009-03-19 01:16
Variants on this theme have been posted time and again, but I can't find
a solution for this specific problem:

Say I have a program I need to use that performs a task for a given
amount of time then finishes, all the while providing a 'percent
complete' style output on STDOUT. eg:

./counting.rb
[RUBY]
#!/usr/bin/env ruby
n = 0

while n <= 10
  $stdout.puts "Completed: #{n*10}%"
  n = n + 1
end
[/RUBY]

Now, I'd like to build a ruby script that will call this script, wait
until completion and then continue with its tasks BUT all the while
updating a variable with the percentage complete. eg:

./script.rb
[RUBY]
def updatePercentage(pc)
  p "The percentage is now #{pc}"
end

Thread.new {
  # -- Problem area

  # Attempt 1
  `./counting.rb`
  # This will wait until after counting.rb is complete before
  # "All Done!" is printed, but I can't get to its stdout stream
  # before the program has finished running

  # Attempt 2
  Open3.popen3('./counting.rb') { |stdin, stdout, stderr|
    p stdout.readpartial(1)

    # With a view to doing something like:
    updatePercentage(stdout.readline.gsub(/^.*([0-9]+)%.*$/,"$1"))
  }
  # This doesn't work, counting.rb never begins executing
  # (tested by adding a line: open('output.txt','w') {|file| file.puts
"Percentage: #{n*10}%" }
  # to counting.rb and watching the file externally)

  p "All done!"
}

p "Irrelevant things"
sleep
[/RUBY]

Any ideas anyone? Essentially I'm looking for ways to wrap C programs
into ruby.

Cheers, JP
42773b24e0c3fb506a8a875c058a9dd7?d=identicon&s=25 Dylan Evans (Guest)
on 2009-03-19 03:40
(Received via mailing list)
I won't go into details but you should look into pipes in this case. By
creating a process  which does the work your wrapper code can just wait
on
the pipe.


On Thu, Mar 19, 2009 at 10:13 AM, Jp Hastings-spital
8f196eedfa0d23f02e6d92159e84e63d?d=identicon&s=25 Jp Hastings-spital (jphastings)
on 2009-03-19 10:05
Dylan Evans wrote:
> I won't go into details but you should look into pipes in this case.

Thanks for the prompt response - I'm assuming you mean Ruby pipes not
BASH pipes or equivalent. (http://www.ruby-doc.org/core/classes/IO.html)

I'll post here if I find a solution.
E0d864d9677f3c1482a20152b7cac0e2?d=identicon&s=25 Robert Klemme (Guest)
on 2009-03-19 10:40
(Received via mailing list)
On 19.03.2009 10:02, Jp Hastings-spital wrote:
> Dylan Evans wrote:
>> I won't go into details but you should look into pipes in this case.
>
> Thanks for the prompt response - I'm assuming you mean Ruby pipes not
> BASH pipes or equivalent. (http://www.ruby-doc.org/core/classes/IO.html)
>
> I'll post here if I find a solution.

I have no idea what you mean by "bash pipes" but the underlying
mechanism is the same.  It's a feature of the operating system.  One
solution is to use IO.popen.  Your code does not read all the lines,
that might be one reason for your issue.

robert@fussel /c/Temp
$ ./run-counter.rb
Got 0
Got 10
Got 20
Got 30
Got 40
Got 50
Got 60
Got 70
Got 80
Got 90
Got 100

robert@fussel /c/Temp
$ cat run-counter.rb
#!/usr/bin/env ruby19

IO.popen "./counting.rb" do |io|
   io.each do |line|
     pct = line[%r{:\s*(\d+)%}, 1].to_i
     puts "Got #{pct}"
   end
end

robert@fussel /c/Temp
$ cat counting.rb
#!/usr/bin/env ruby19

$stdout.sync = true

11.times do |n|
   puts "Completed: #{n*10}%"
end

robert@fussel /c/Temp
$

Similar with Open3, only that you then get multiple pipes.

Cheers

  robert
8f196eedfa0d23f02e6d92159e84e63d?d=identicon&s=25 Jp Hastings-spital (jphastings)
on 2009-03-19 22:03
Robert,
Thanks very much for your reply - its helped no end! However I'm still
having problems:

Firstly (though really this isn't vitally important, as I only *need*
stdout):
Open3.popen3 doesn't appear to work when I drop it in place of IO.popen
(where |stdout| is replaced with |stdin,stdout,stderr|).
The called program is definitely running, however even the following
code outputs nothing:

Open3.popen3 "./counting.rb" do |stdin,stdout,stderr|
  p stdout.read
end

But as I say, I only really need the stdout pipe. The problem here is
that the program I'm calling doesn't appear to send the \n character at
all (its the output from HandBrake's CLI), and as such the io block only
gets called once, at the end of the program's execution. Here's some
code:

$ HandBrakeCLI {options} 2> /dev/null
\rEncoding: task 1 of 1, 0.09 %\rEncoding: task 1 of 1, 0.13
%\rEncoding: task 1 of 1, 0.18 %\rEncoding: task 1 of 1, 0.23 %

$ cat wrapper.rb
input = "input.avi"
output = "output.mp4"
IO.popen("/usr/local/bin/HandBrakeCLI -i \"#{input}\" -o \"#{output}\"")
{ |stdout|
  p stdout.read
}

This prints nothing! Am I right in thinking that the popen block gets
called every time a \n character is captured by the pipe? (Your
'counting.rb' uses \n rather than \r - and works) Any ideas (other than
say, passing the output through sed or something similar)?

Thanks again, JP
E0d864d9677f3c1482a20152b7cac0e2?d=identicon&s=25 Robert Klemme (Guest)
on 2009-03-20 15:20
(Received via mailing list)
On 19.03.2009 22:00, Jp Hastings-spital wrote:
>
> Open3.popen3 "./counting.rb" do |stdin,stdout,stderr|
>   p stdout.read
> end

No idea what you're doing, but it works for me

robert@fussel /c/Temp
$ ./run-counter.rb
Got 0
Got 10
Got 20
Got 30
Got 40
Got 50
Got 60
Got 70
Got 80
Got 90
Got 100
Got3 0
Got3 10
Got3 20
Got3 30
Got3 40
Got3 50
Got3 60
Got3 70
Got3 80
Got3 90
Got3 100

robert@fussel /c/Temp
$ cat run-counter.rb
#!/usr/bin/env ruby19

require 'open3'

IO.popen "./counting.rb" do |io|
   io.each do |line|
     pct = line[%r{:\s*(\d+)%}, 1].to_i
     puts "Got #{pct}"
   end
end

Open3.popen3 "./counting.rb" do |sin,sout,serr|
   sin.close
   th = Thread.new { serr.read }

   sout.each do |line|
     pct = line[%r{:\s*(\d+)%}, 1].to_i
     puts "Got3 #{pct}"
   end

   th.join
end

robert@fussel /c/Temp
$

> $ cat wrapper.rb
> input = "input.avi"
> output = "output.mp4"
> IO.popen("/usr/local/bin/HandBrakeCLI -i \"#{input}\" -o \"#{output}\"")
> { |stdout|
>   p stdout.read
> }
>
> This prints nothing! Am I right in thinking that the popen block gets
> called every time a \n character is captured by the pipe?

No.  The block is called once per execution of the other process.

> (Your
> 'counting.rb' uses \n rather than \r - and works) Any ideas (other than
> say, passing the output through sed or something similar)?

Could be that your problem is IO buffering on the sender side (i.e. the
program you invoke).  Alternatively, if you do not close stdin of the
subprogram that might be waiting for some input.

Cheers

  robert
E0d864d9677f3c1482a20152b7cac0e2?d=identicon&s=25 Robert Klemme (Guest)
on 2009-03-20 16:00
(Received via mailing list)
Few more remarks:

On 19.03.2009 22:00, Jp Hastings-spital wrote:
>
> Open3.popen3 "./counting.rb" do |stdin,stdout,stderr|
>   p stdout.read

This will read the whole stream to the end!  Which means, it will return
only after the process died.

> %\rEncoding: task 1 of 1, 0.18 %\rEncoding: task 1 of 1, 0.23 %
If you have "\r" as line terminator, you can do the reading like this

io.each "\r" do |line|
   # do whatever with line
end

e.g.

robert@fussel ~
$ ruby -e '3.times {|i| printf "%03d\r", i}' | ruby -e
'$stdin.each("\r") {|l| p l}'
"000\r"
"001\r"
"002\r"

robert@fussel ~
$

> $ cat wrapper.rb
> input = "input.avi"
> output = "output.mp4"
> IO.popen("/usr/local/bin/HandBrakeCLI -i \"#{input}\" -o \"#{output}\"")
> { |stdout|
>   p stdout.read

See above.

> }
>
> This prints nothing! Am I right in thinking that the popen block gets
> called every time a \n character is captured by the pipe? (Your
> 'counting.rb' uses \n rather than \r - and works) Any ideas (other than
> say, passing the output through sed or something similar)?
>
> Thanks again, JP

Note that the proper line reading (as suggested above) still does not
help if the other program internally buffers IO.

Kind regards

  robert
8f196eedfa0d23f02e6d92159e84e63d?d=identicon&s=25 Jp Hastings-spital (jphastings)
on 2009-03-20 19:08
IO.popen method with io.each "\r" works spot on! Thanks so much!

Haven't tried working with Open3.popen3 yet, I may try reinstalling the
gem.

Thanks for your replies, you've been patient and very helpful!

Robert Klemme wrote:
>> %\rEncoding: task 1 of 1, 0.18 %\rEncoding: task 1 of 1, 0.23 %
> If you have "\r" as line terminator, you can do the reading like this
>
> io.each "\r" do |line|
>    # do whatever with line
> end

This makes it work beautifully!
>
> e.g.
>
> robert@fussel ~
> $ ruby -e '3.times {|i| printf "%03d\r", i}' | ruby -e
> '$stdin.each("\r") {|l| p l}'
> "000\r"
> "001\r"
> "002\r"
>
> robert@fussel ~
> $
>
>> $ cat wrapper.rb
>> input = "input.avi"
>> output = "output.mp4"
>> IO.popen("/usr/local/bin/HandBrakeCLI -i \"#{input}\" -o \"#{output}\"")
>> { |stdout|
>>   p stdout.read
>
> See above.
>
>> }
>>
>> This prints nothing! Am I right in thinking that the popen block gets
>> called every time a \n character is captured by the pipe? (Your
>> 'counting.rb' uses \n rather than \r - and works) Any ideas (other than
>> say, passing the output through sed or something similar)?
>>
>> Thanks again, JP
>
> Note that the proper line reading (as suggested above) still does not
> help if the other program internally buffers IO.
>
> Kind regards
>
>   robert
E0d864d9677f3c1482a20152b7cac0e2?d=identicon&s=25 Robert Klemme (Guest)
on 2009-03-21 11:23
(Received via mailing list)
On 20.03.2009 19:05, Jp Hastings-spital wrote:
> IO.popen method with io.each "\r" works spot on! Thanks so much!
>
> Haven't tried working with Open3.popen3 yet, I may try reinstalling the
> gem.
>
> Thanks for your replies, you've been patient and very helpful!

You're welcome.  Good to hear that your issue is fixed now!

Kind regards

  robert
This topic is locked and can not be replied to.