Forum: Ruby Help on I/O pipe

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
Cee38055ae36590c654c04c2d5cc2778?d=identicon&s=25 Sky Yin (cookoo)
on 2006-01-03 13:35
(Received via mailing list)
One process of my analysis program uses an external non-linear
optimizer.
That optimizer needs a model file as the argument. The current approach
I
employ to accomplish this is to create a model file on disk everytime
running the optimizer. The code is simply like

@model = File.open("model.mod", "w")
...
@model.close
result  = `optimizer.exe model.mod`

The problem is that I have to repeat running optimizer on different
models
many many times during the analysis. I wonder if such heavy writing on a
single file will do any harm to my hard disk. My concern of improving it
is
to use I/O pipe to feed the optimizer without the needs to create real
model
files. Can anyone give me some tips or examples on using the shell pipe?
The
rdoc isn't so clear.

Thanks
B000982a23d5c6a34292902caf225dd7?d=identicon&s=25 Yohanes Santoso (Guest)
on 2006-01-03 15:20
(Received via mailing list)
Sky Yin <sky.yin@gmail.com> writes:

> The problem is that I have to repeat running optimizer on different models
> many many times during the analysis.

> I wonder if such heavy writing on a single file will do any harm to
> my hard disk.

Unlikely. HD does not have a limitation on number of writes like CD-RW
and flash drives (USB key chains, SD, memory stick, etc.)

> My concern of improving it is to use I/O pipe to feed the optimizer
> without the needs to create real model files.

Before you can use pipe, you have to know if optimizer.exe read
model.mod sequentially or not.

If it reads model.mod sequentially, then you can use pipe. Otherwise,
you'd need to write it to a file (basically do what you are doing
above).

There are two kind of pipe: anonymous pipe and named pipe.

Anonymous pipe is setup automatically by your environment (your shell
in unix or cmd.exe in win32) when you do:

foo | bar

For this to work, your model generator just need to write to its
standard output ($stdout in ruby) and your optimizer.exe reads from
its standard input. This may require changes to optimizer.exe as I
don't think it is common for a win32 program to read from stdin (some
kind of aversion to CLI?)


Named pipe involves creating a special file in the filesystem. Your
model generator opens this file in write mode and start writing. At
about the same time, you need to invoke optimizer.exe to read from
this file: optimizer.exe specialfile.mod. The data transfer will not
go through disk I/O. However, I'm not sure if win32 supports named
pipe.

> Can anyone give me some tips or examples on using the shell pipe?

As I explained above, shell pipe needs to be setup by your
environment. You can, of course, set up your own pipe programatically,
but as I'm not a win32 programmer, I don't know how or even if that is
possible since the process model of Unix and Win32 are different.


YS.
Cb48ca5059faf7409a5ab3745a964696?d=identicon&s=25 unknown (Guest)
on 2006-01-03 15:33
(Received via mailing list)
On Tue, 3 Jan 2006, Sky Yin wrote:

> The problem is that I have to repeat running optimizer on different models
> many many times during the analysis. I wonder if such heavy writing on a
> single file will do any harm to my hard disk. My concern of improving it is
> to use I/O pipe to feed the optimizer without the needs to create real model
> files. Can anyone give me some tips or examples on using the shell pipe? The
> rdoc isn't so clear.
>
> Thanks

assuming that your analysis code can read it's input sequentially you
might
want to try session:

   require 'session'

   sh = Session::new

   model_input = ...

   stdout, stderr = sh.execute 'optimizer.exe', 'stdin' => model_input

   p sh.exitstatus

regards.

-a
Cee38055ae36590c654c04c2d5cc2778?d=identicon&s=25 Sky Yin (cookoo)
on 2006-01-03 23:58
(Received via mailing list)
Thank Yohanes for the detailed explanation. I've just tried using

io = IO.popen("optimizer.exe", "r+")
io.puts "..." #stdin<model.mod
io.gets #stdout
...
io.close

It's weird that first two io.gets worked OK to retrieve the first two
lines
of the result, but then io.gets halted on the third io.gets. No clue.

As for the session gem, I installed, but require 'session' simply
returns a
false under Win32.

Thanks anyway.
Cb48ca5059faf7409a5ab3745a964696?d=identicon&s=25 unknown (Guest)
on 2006-01-04 00:04
(Received via mailing list)
On Wed, 4 Jan 2006, Sky Yin wrote:

>
> As for the session gem, I installed, but require 'session' simply returns a
> false under Win32.
>
> Thanks anyway.

session is not going to work under win32.  you'll need to do something
like

   io = IO.popen("optimizer.exe", "r+")

   io.puts "..." # stdin<model.mod

   io.close_write

   stdout = io.read

   io.close

to avoid hanging on a gets with line buffered output.

regards.

-a
Cee38055ae36590c654c04c2d5cc2778?d=identicon&s=25 Sky Yin (cookoo)
on 2006-01-04 03:00
(Received via mailing list)
It works very smoothly now. Thanks a lot for the timely help.
Everyday I love ruby a bit more :^)
This topic is locked and can not be replied to.