Help on I/O pipe


#1

One process of my analysis program uses an external non-linear
optimizer.
That optimizer needs a model file as the argument. The current approach
I
employ to accomplish this is to create a model file on disk everytime
running the optimizer. The code is simply like

@model = File.open(“model.mod”, “w”)

@model.close
result = optimizer.exe model.mod

The problem is that I have to repeat running optimizer on different
models
many many times during the analysis. I wonder if such heavy writing on a
single file will do any harm to my hard disk. My concern of improving it
is
to use I/O pipe to feed the optimizer without the needs to create real
model
files. Can anyone give me some tips or examples on using the shell pipe?
The
rdoc isn’t so clear.

Thanks


#2

Sky Y. removed_email_address@domain.invalid writes:

The problem is that I have to repeat running optimizer on different models
many many times during the analysis.

I wonder if such heavy writing on a single file will do any harm to
my hard disk.

Unlikely. HD does not have a limitation on number of writes like CD-RW
and flash drives (USB key chains, SD, memory stick, etc.)

My concern of improving it is to use I/O pipe to feed the optimizer
without the needs to create real model files.

Before you can use pipe, you have to know if optimizer.exe read
model.mod sequentially or not.

If it reads model.mod sequentially, then you can use pipe. Otherwise,
you’d need to write it to a file (basically do what you are doing
above).

There are two kind of pipe: anonymous pipe and named pipe.

Anonymous pipe is setup automatically by your environment (your shell
in unix or cmd.exe in win32) when you do:

foo | bar

For this to work, your model generator just need to write to its
standard output ($stdout in ruby) and your optimizer.exe reads from
its standard input. This may require changes to optimizer.exe as I
don’t think it is common for a win32 program to read from stdin (some
kind of aversion to CLI?)

Named pipe involves creating a special file in the filesystem. Your
model generator opens this file in write mode and start writing. At
about the same time, you need to invoke optimizer.exe to read from
this file: optimizer.exe specialfile.mod. The data transfer will not
go through disk I/O. However, I’m not sure if win32 supports named
pipe.

Can anyone give me some tips or examples on using the shell pipe?

As I explained above, shell pipe needs to be setup by your
environment. You can, of course, set up your own pipe programatically,
but as I’m not a win32 programmer, I don’t know how or even if that is
possible since the process model of Unix and Win32 are different.

YS.


#3

On Tue, 3 Jan 2006, Sky Y. wrote:

The problem is that I have to repeat running optimizer on different models
many many times during the analysis. I wonder if such heavy writing on a
single file will do any harm to my hard disk. My concern of improving it is
to use I/O pipe to feed the optimizer without the needs to create real model
files. Can anyone give me some tips or examples on using the shell pipe? The
rdoc isn’t so clear.

Thanks

assuming that your analysis code can read it’s input sequentially you
might
want to try session:

require ‘session’

sh = Session::new

model_input = …

stdout, stderr = sh.execute ‘optimizer.exe’, ‘stdin’ => model_input

p sh.exitstatus

regards.

-a


#4

On Wed, 4 Jan 2006, Sky Y. wrote:

As for the session gem, I installed, but require ‘session’ simply returns a
false under Win32.

Thanks anyway.

session is not going to work under win32. you’ll need to do something
like

io = IO.popen(“optimizer.exe”, “r+”)

io.puts “…” # stdin<model.mod

io.close_write

stdout = io.read

io.close

to avoid hanging on a gets with line buffered output.

regards.

-a


#5

Thank Yohanes for the detailed explanation. I’ve just tried using

io = IO.popen(“optimizer.exe”, “r+”)
io.puts “…” #stdin<model.mod
io.gets #stdout

io.close

It’s weird that first two io.gets worked OK to retrieve the first two
lines
of the result, but then io.gets halted on the third io.gets. No clue.

As for the session gem, I installed, but require ‘session’ simply
returns a
false under Win32.

Thanks anyway.


#6

It works very smoothly now. Thanks a lot for the timely help.
Everyday I love ruby a bit more :^)