Processes and Portability

dmatrix00d · July 11, 2006, 5:57pm

I would like to run a process concurrently (thus if the process goes to
sleep or runs forever, I can still run other code from my main process)
and have all standard out of that process be put into a text file. I
would also like this code to be portable. And finally, I would like to
be able to kill this process at any time (So I will need the pid)

This thus eliminates the use of fork. Also, I have tried:

a=IO.popen(“program > logfile”), but on Windows this creates two
processes, cmd.exe and program. a.pid only gives the pid of cmd.exe.
And the behavior on *nix seems to be just the program (although I can be
wrong).

So I require portability in that sense.

Is this possible?

thanks

dmatrix00d · July 11, 2006, 6:34pm

On Wed, 12 Jul 2006, Name N. wrote:

And the behavior on *nix seems to be just the program (although I can be
wrong).

So I require portability in that sense.

Is this possible?

thanks

try this - i haven’t tested on windows but it can certainly be made to
work:

harp:~ > cat a.rb
#!/usr/bin/env ruby

require ‘rbconfig’
require ‘thread’

class Spawn
CONFIG = Config::CONFIG

RUBY =
if system(‘ruby -e 42’) # ruby is in our path
‘ruby’
else # guess from rbconfig
File.join CONFIG[‘bindir’], CONFIG[‘ruby_install_name’]
end

PROGRAM = “p $$;STDOUT.flush;STDERR.reopen(STDOUT);exec(*ARGV)”

ATTRIBUTES = %w[
mutex
out
exitstatus
exitstatus_q
cmd
pipe
pid
thread
]

ATTRIBUTES.each{|a| attr_accessor a}

def initialize *argv
@mutex = Mutex.new
@out = ‘’
@exitstatus = nil
@exitstatus_q = Queue.new
@cmd = “#{ RUBY } -e ‘#{ PROGRAM }’ #{ argv.join(’ ') }”
@pipe = IO.popen @cmd
@pid = @pipe.gets.to_i
at_exit{ Process.kill -9, @pid rescue nil }
@thread = Thread.new{
begin
while((line = @pipe.gets))
@mutex.synchronize{ @out << line }
end
ensure
@pipe.close rescue nil
exitstatus = $?.exitstatus
@exitstatus_q.push exitstatus
end
}
end

def out
@mutex.synchronize{ @out }
end

def exitstatus non_block = false
@exitstatus ||= @exitstatus_q.pop(non_block)
end

class << self; alias [] new; end
end

require ‘yaml’

cmd = ARGV.join ’ ’

spawn = Spawn[ cmd ]

y ‘spawn.cmd’ => spawn.cmd
y ‘spawn.pid’ => spawn.pid
y ‘spawn.exitstatus’ => spawn.exitstatus
y ‘spawn.out’ => spawn.out

harp:~ > ruby a.rb date
spawn.cmd: ruby -e ‘p $$;STDOUT.flush;STDERR.reopen(STDOUT);exec(*ARGV)’
date
spawn.pid: 27232
spawn.exitstatus: 0
spawn.out: |
Tue Jul 11 10:31:22 MDT 2006

harp:~ > ruby a.rb no-exist
spawn.cmd: ruby -e ‘p $$;STDOUT.flush;STDERR.reopen(STDOUT);exec(*ARGV)’
no-exist
spawn.pid: 27235
spawn.exitstatus: 1
spawn.out: |
-e:1:in `exec’: No such file or directory - no-exist (Errno::ENOENT)
from -e:1

regards.

-a

dmatrix00d · July 11, 2006, 8:20pm

I know this sounds nieve, but is there a way to do what I want without
requiring an external gem, and without that extremely long code on the
secnod post. I would really prefer just to use native ruby.

Be able to concurrently run a process (that could potentially block
forever or run forever) written in another language and having the
ability to stop it at anytime.

a=IO.popen (program_name)
Process.kill(9,a.pid)

works, but it won’t log to a file

a=IO.popen(program_name > logfile)
Process.kill(9, a.pid)

spawns two processes and does not kill both.

I need something to be able to do the latter.

thanks

dmatrix00d · July 11, 2006, 6:51pm

I would like to run a process concurrently (thus if the process goes to
sleep or runs forever, I can still run other code from my main process)
and have all standard out of that process be put into a text file. I
would also like this code to be portable. And finally, I would like to
be able to kill this process at any time (So I will need the pid)

This thus eliminates the use of fork. Also, I have tried:

If you are doing the equivalent of fork/exec, here’s some code that is
in
my IOWA package that will help. It depends on Daniel B.'s
win32-process library, which can be found at RubyForge:

http://rubyforge.org/projects/win32utils

It, in turn, is dependent on window-pr, found at the same URL.

This code has been tested on Linux and WinXP. You should be able to
adapt
it to your purposes.

module IWATestSupport
def self.create_process(args)
@fork_ok = true unless @fork_ok == false
pid = nil
begin
raise NotImplementedError unless @fork_ok
unless pid = fork
Dir.chdir args[:dir]
exec(*args[:cmd])
end
rescue NotImplementedError
@fork_ok = false

   begin
     require 'rubygems'
   rescue LoadError
   end

   begin
     require 'win32/process'
   rescue LoadError
     raise "Please install win32-process."
   end

   cwd = Dir.pwd
   Dir.chdir args[:dir]
   pid = Process.create(:app_name => args[:cmd].join(' '))
   Dir.chdir cwd
 end
 pid

end
end

Kirk H.

dmatrix00d · July 11, 2006, 9:19pm

or is there is a way to directly write to a file using IO.popen.

dmatrix00d · July 11, 2006, 9:22pm

Name N. wrote:

…

a=IO.popen(program_name > logfile)
Process.kill(9, a.pid)

spawns two processes and does not kill both.

I need something to be able to do the latter.

thanks

What about:

a=IO.popen("#{program_name} > #{logfile}")
Process.kill(9, a.pid)

Don’t know if that works on MSWindows.

Harmen

dmatrix00d · July 11, 2006, 9:27pm

I don’t understand how that is different than:
a=IO.popen(program_name > logfile)
Process.kill(9, a.pid)

Harmen Schut wrote:

Name N. wrote:

…

a=IO.popen(program_name > logfile)
Process.kill(9, a.pid)

spawns two processes and does not kill both.

I need something to be able to do the latter.

thanks

What about:

a=IO.popen("#{program_name} > #{logfile}")
Process.kill(9, a.pid)

Don’t know if that works on MSWindows.

Harmen

dmatrix00d · July 11, 2006, 9:33pm

It is one String argument (a command line) instead of an expression
(compairing two String variables?)

Harmen

Name N. wrote:

I don’t understand how that is different than:
a=IO.popen(program_name > logfile)
Process.kill(9, a.pid)

Harmen Schut wrote:

Name N. wrote:

…

a=IO.popen(program_name > logfile)
Process.kill(9, a.pid)

spawns two processes and does not kill both.

I need something to be able to do the latter.

thanks

What about:

a=IO.popen("#{program_name} > #{logfile}")
Process.kill(9, a.pid)

Don’t know if that works on MSWindows.

Harmen

dmatrix00d · July 11, 2006, 9:36pm

I’m sorry if my previous thing was confusing, but it was actually:

IO.popen(“program_name > logfile”).

Harmen Schut wrote:

It is one String argument (a command line) instead of an expression
(compairing two String variables?)

Harmen

Name N. wrote:

I don’t understand how that is different than:
a=IO.popen(program_name > logfile)
Process.kill(9, a.pid)

Harmen Schut wrote:

Name N. wrote:

…

a=IO.popen(program_name > logfile)
Process.kill(9, a.pid)

spawns two processes and does not kill both.

I need something to be able to do the latter.

thanks

What about:

a=IO.popen("#{program_name} > #{logfile}")
Process.kill(9, a.pid)

Don’t know if that works on MSWindows.

Harmen

dmatrix00d · July 11, 2006, 9:54pm

Name N. [email protected] writes:

a=IO.popen(“program > logfile”), but on Windows this creates two
processes, cmd.exe and program. a.pid only gives the pid of cmd.exe.
And the behavior on *nix seems to be just the program (although I can be
wrong).

The behaviour in *nix depends on your /bin/sh. Some shells gives the
child’s pid, other gives the pid of itself.

pdksh gives the pid of itself.
bash gives the pid of the child (the behaviour you wanted).
Don’t know about other shells.

The lesson here, you can’t get the pid of the child consistently
unless you do the forking yourself.

YS

dmatrix00d · July 11, 2006, 9:58pm

On Linux that should work if program ‘progam_name’ produces output and
file ‘logfile’ is writable. One of the two processes you mentioned is an
intermediate shell by the way.

Harmen

Name N. wrote:

I’m sorry if my previous thing was confusing, but it was actually:

IO.popen(“program_name > logfile”).

…

dmatrix00d · July 11, 2006, 10:13pm

I want this to be portable, ie not just *nix, so I can’t fork. However,
I am doing something similar to forking in terms of getting the pid
which popen will do. I’m sure if I used fork (which I can’t) on program

logfile, it won’t be the pid of program (or it might be, i’m not sure,
but the point is, the logic is messed up since I am doign the extra
redirecting)

Yohanes S. wrote:

Name N. [email protected] writes:

a=IO.popen(“program > logfile”), but on Windows this creates two
processes, cmd.exe and program. a.pid only gives the pid of cmd.exe.
And the behavior on *nix seems to be just the program (although I can be
wrong).

The behaviour in *nix depends on your /bin/sh. Some shells gives the
child’s pid, other gives the pid of itself.

pdksh gives the pid of itself.
bash gives the pid of the child (the behaviour you wanted).
Don’t know about other shells.

The lesson here, you can’t get the pid of the child consistently
unless you do the forking yourself.

YS

I know that Harmen. I want the pid of not the intermediate shell, which
is the one I am getting back with popen. I want the pid of the program
itself that I opened. Also, program does produce output and logfile is
writeable, and I need this to run on any system.

On Linux that should work if program ‘progam_name’ produces output and
file ‘logfile’ is writable. One of the two processes you mentioned is an
intermediate shell by the way.

Harmen

dmatrix00d · July 11, 2006, 10:24pm

so i was just testing right now using process explorer that:

Process.kill(9, a.pid) only kills the parent and detaches the child,
letting the child persist.

I also noticed that if i right click on the parent and click kill tree,
it will kill both on process explorer.

how do i kill the parent and the children, not jsut that parent as in
the case above.

dmatrix00d · July 11, 2006, 10:30pm

Name N. [email protected] writes:

I want this to be portable, ie not just *nix, so I can’t fork.

It is because of portability issue that I wrote my reply.
Being portable does not mean staying of OS-specific features. You can
do that, if there is another option for you to take. In this case,
however, there is no other.

So, being portable for your needs means providing different workaround
for different environment.

I don’t know much about win32, but I know for *nix, you need to do the
forking and setup IO redirection yourself (not using IO.popen at all).

The other people here have replied about doing this in win32.

YS.

dmatrix00d · July 11, 2006, 11:01pm

Is there a command in ruby that when given the pid , it will terminate
all children processes of that pid including itself.

thanks

Yohanes S. wrote:

Name N. [email protected] writes:

I want this to be portable, ie not just *nix, so I can’t fork.

It is because of portability issue that I wrote my reply.
Being portable does not mean staying of OS-specific features. You can
do that, if there is another option for you to take. In this case,
however, there is no other.

So, being portable for your needs means providing different workaround
for different environment.

I don’t know much about win32, but I know for *nix, you need to do the
forking and setup IO redirection yourself (not using IO.popen at all).

The other people here have replied about doing this in win32.

YS.

dmatrix00d · July 12, 2006, 5:36am

Joel, if you do t.status after you thread off the system, you’ll notice
that it says sleeping instead of running. Pretty much the thread will
make the system call and then wait on that system call to return. So
that thread is owning that process in some respects, but killing that
thread will in no means kill the process, since that process is on its
own now.

On an unrelated note, does anyone know how to kill all children pids
including itself when one is given a pid? If that question is answered,
I think my post will be answered.

Joel VanderWerf wrote:

Name N. wrote:

I would like to run a process concurrently (thus if the process goes to
sleep or runs forever, I can still run other code from my main process)
and have all standard out of that process be put into a text file. I
would also like this code to be portable. And finally, I would like to
be able to kill this process at any time (So I will need the pid)

I was half expecting the following to work, but it doesn’t:

t = Thread.new do
system “sleep 100000” # or IO.popen or fork…
end

puts “press return to stop child thread”
gets

t.kill

puts ps -Af | grep sleep
Process.wait # still waiting for child until ^C

Killing a thread that is waiting on system() doesn’t stop the child
process.

Should this be part of the Thread abstraction? Should threads own
processes? Maybe not. Just wondering what people think…

dmatrix00d · July 12, 2006, 3:52am

Name N. wrote:

I would like to run a process concurrently (thus if the process goes to
sleep or runs forever, I can still run other code from my main process)
and have all standard out of that process be put into a text file. I
would also like this code to be portable. And finally, I would like to
be able to kill this process at any time (So I will need the pid)

I was half expecting the following to work, but it doesn’t:

t = Thread.new do
system “sleep 100000” # or IO.popen or fork…
end

puts “press return to stop child thread”
gets

t.kill

puts ps -Af | grep sleep
Process.wait # still waiting for child until ^C

Killing a thread that is waiting on system() doesn’t stop the child
process.

Should this be part of the Thread abstraction? Should threads own
processes? Maybe not. Just wondering what people think…

dmatrix00d · July 12, 2006, 7:05am

Name N. wrote:

Joel, if you do t.status after you thread off the system, you’ll notice
that it says sleeping instead of running. Pretty much the thread will
make the system call and then wait on that system call to return. So
that thread is owning that process in some respects, but killing that
thread will in no means kill the process, since that process is on its
own now.

Sure, that’s why the Process.wait call hangs the program until you
interrupt it.

The thread owns the process in one sense: the thread continues (and in
this case finishes) when the process exits. I’m just wondering whether
it would make sense for the thread to be more closely associated with
the process. Maybe killing the thread should not kill the process, but
instead there should be an additional status result that tells you
whether the thread is sleeping waiting for an external process and if so
the command line and the pid.

dmatrix00d · July 12, 2006, 5:51am

On Wed, 12 Jul 2006, Name N. wrote:

Joel, if you do t.status after you thread off the system, you’ll notice
that it says sleeping instead of running. Pretty much the thread will
make the system call and then wait on that system call to return. So
that thread is owning that process in some respects, but killing that
thread will in no means kill the process, since that process is on its
own now.

On an unrelated note, does anyone know how to kill all children pids
including itself when one is given a pid? If that question is answered,
I think my post will be answered.

harp:~ > cat a.rb
Thread.new{ system ‘sleep 60’ }
sleep 1
system ‘ps’
Process.kill -9, Process.pid

harp:~ > ruby a.rb
PID TTY TIME CMD
15149 pts/2 00:00:00 bash
6308 pts/2 00:00:00 ruby
6309 pts/2 00:00:00 sleep
6310 pts/2 00:00:00 ps
Killed

harp:~ > ps
PID TTY TIME CMD
15149 pts/2 00:00:00 bash
6311 pts/2 00:00:00 ps

-a