Multithreading in Different Directories

Rails
1

Hello Everyone,

First message out here on the group, and I hope you can help me out.

I am running a bunch of commands that can take a long time (10 seconds
to 15
minutes and beyond). Naturally, I decided to multithread them, which
makes
them run a lot faster since I have a multi-core machine. The problem is
they
all access a log file in the current directory, and they collide.

So, my next step was to use Dir.chdir to move to another directory for
each
thread and run there. Now it works.

Now, my problem is that I have been using backticks (`) to run commands,
and
I would like to switch to popen in order to stifle the output (it spits
out
a lot of info that I don’t want all over my tests, and it generates odd
characters that mess up my terminal).

The problem is that popen doesn’t respect Dir.chdir. I have also tried
things like:

popen(“cd otherdir; command”)
and
popen(“cd otherdir && command”)

but it doesn’t seem to care at all, and always runs in rails root.

So I need one of two things:

  1. Change directory for popen
  2. Stifle stdout and stderr for command

Thanks,
Nick

2

Thanks for your help.

Clever N. wrote:

On May 14, 4:27 pm, “Nick Gauthier” [email protected] wrote:

So, my next step was to use Dir.chdir to move to another directory for each
thread and run there. Now it works.

This is kind of hackish. Are you sure the binary does not have a
switch or option to change where the log file goes? Almost all the
standard Unix utils that have logfiles let you modify this. If you
have access to the source, adding such a flag is trivial. If the flag
exists, simply appending a “random number” (I use the unix timestamp
plus another counter (just in case I start two thread in the same
second) myself quite often) to a common log name will usually
suffice. This is also handy because you can then pull the date and
time it was executed from the log file’s name, if you don’t need a
running log with detailed timestamps.

I know it’s hackish :frowning:

The reason I can’t use a switch for the log file is that the program I
am calling (which does have a log file switch) calls another program
(which has a log file switch). But, the first program does not have a
log file switch for the second program’s log file.

So I need one of two things:

  1. Change directory for popen
  2. Stifle stdout and stderr for command

2 is easy. The backtick style of calling commands actually spawns a
shell, so shell-based redirects to /dev/null (as well as file
globbing, scripting, conditional execution, etc.) work as if you were
actually at your command line.

I tried piping the output using >, but I didn’t realize my program was
actually outputting to stderr, so I just changed it to be 2> /dev/null
and now its nice and quiet.

Thanks!

3

On May 14, 4:27 pm, “Nick Gauthier” [email protected] wrote:

So, my next step was to use Dir.chdir to move to another directory for each
thread and run there. Now it works.

This is kind of hackish. Are you sure the binary does not have a
switch or option to change where the log file goes? Almost all the
standard Unix utils that have logfiles let you modify this. If you
have access to the source, adding such a flag is trivial. If the flag
exists, simply appending a “random number” (I use the unix timestamp
plus another counter (just in case I start two thread in the same
second) myself quite often) to a common log name will usually
suffice. This is also handy because you can then pull the date and
time it was executed from the log file’s name, if you don’t need a
running log with detailed timestamps.

So I need one of two things:

  1. Change directory for popen
  2. Stifle stdout and stderr for command

2 is easy. The backtick style of calling commands actually spawns a
shell, so shell-based redirects to /dev/null (as well as file
globbing, scripting, conditional execution, etc.) work as if you were
actually at your command line.