Resolving pipe deadlock

On Sat, Feb 1, 2014 at 3:18 PM, Martin H. [email protected]
wrote:

Also, as I’ve said before I’d rather stack the msg handling pipeline
class on top of Pipe. That way you keep concerns nicely separated and
can use class Pipe for other purposes as well. The way you did it you
created a class that can only handle MessagePack messages.

Again, I get it working one way and then there is some other superior
way :o) - I’ll give it a shot.

Actually I mentioned that already - maybe it was not explicit enough.
:slight_smile:

Now, I want two additional features.

First, I want a to_s method in the Pipe class that return a string with
the commands and options run like: “Pipe.new.add(:cat. input:
“foo”).add(:save, output: “bar”).add(:dump).run”. Strings like this I
would like to log in a history file for documentation of commands run
and easy rerunning (in irb). I think there might be a problem with this,
with the lambdas giving the separation of the Pipe and commands (from
the Pipe class there is no way to see what is in the lambdas)?

There is generally no way to get at the contents of the lambdas.
However, if you generate the lambdas yourself (like in those CAT
methods we have discussed earlier) you can do something like:

def CAT(*args)
l = lambda do … end

def l.to_s
“CAT(#{args.inspect})” # or whatever
end

l
end

You could devise other schemes that would allow to execute the code
again. You could even write a file and create the lambda from there.
Lots of options.

Second, I would like to keep track of some basic statistics from each
command: number of records in and out, runtime, and keys seen (I will be
using records consisting of simple hashes). Each command could write a
temporary file and then the stats could be compiled after execution.
Alternative, the stats could be passed along the IO stream as the last
record, or each command should also have separate pipes inter process
communication of stats.

Without giving too much thought I would prefer the approach with the
statistics sent down the pipe at the end because it avoids fiddling
with temporary files.

Kind regards

robert

So after a busy week, I have at last some time to look more into this.
So far it looks very promising, but then I wanted to use the
(de-)serializer method of msgpack and it hangs once again?

line 49-50

The File.pipe? test of cause needs to be changed, but there is more to
it …

Cheers,

Martin