# Command Pipeline (pipes & filters) in Ruby

**URL:** <https://www.ruby-forum.com/t/command-pipeline-pipes-filters-in-ruby/66417>\
**Category:** Ruby\
**Created:** [February 13, 2006, 6:33am UTC](https://www.ruby-forum.com/t/command-pipeline-pipes-filters-in-ruby/66417 "2006-02-13T06:33:49Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![unknown](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/u/4bbf92/32.png) [@unknown](https://www.ruby-forum.com/u/unknown)\
**Post date:** [February 13, 2006, 6:33am UTC](https://www.ruby-forum.com/t/command-pipeline-pipes-filters-in-ruby/66417/1 "2006-02-13T06:33:49Z")

</div>

Anyone know of any generic implementation of a pipeline in ruby? That  
is, one which takes a bunch of GoF Command objects, and strings them  
together in the classical pipeline configuration.

Any advice on that?

I have a process with a lot of sequential processing, and think that a  
pipeline of Command objects (not just Proc’s) might be the way to do  
it.

The downsides of pipelines:

1. You can’t do selections (if) or sequences (loops). If you can  
handle this, though, this becomes a virtue, as it keeps everything  
crystal clear and simple. Witness the ever useful Unix command line  
pipelines.

2. They don’t encapsulate very well, as each stage totally overwrites  
the previous ones. I’m not sure if I should just accept this, or try  
to use some modification (append only pipeline, etc.).

Any experience or ideas on this appreciated.

---

<div class="post-metadata">

**Author:** ![unknown](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/u/4bbf92/32.png) [@unknown](https://www.ruby-forum.com/u/unknown)\
**Post date:** [February 13, 2006, 6:48am UTC](https://www.ruby-forum.com/t/command-pipeline-pipes-filters-in-ruby/66417/2 "2006-02-13T06:48:14Z")

</div>

removed\_email\_address@domain.invalid wrote:

> Anyone know of any generic implementation of a pipeline in ruby? That  
> is, one which takes a bunch of GoF Command objects, and strings them  
> together in the classical pipeline configuration.
> 
> Any advice on that?

Does Rake approach this?

Define each command as a task. Let Rake handle the chaining and  
dependency logic.

–  
James B.

?Design depends largely on constraints.?  
? Charles Eames

---

<div class="post-metadata">

**Author:** ![unknown](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/u/4bbf92/32.png) [@unknown](https://www.ruby-forum.com/u/unknown)\
**Post date:** [February 13, 2006, 10:33am UTC](https://www.ruby-forum.com/t/command-pipeline-pipes-filters-in-ruby/66417/3 "2006-02-13T10:33:35Z")

</div>

removed\_email\_address@domain.invalid wrote:

> The downsides of pipelines:
> 
> 1. You can’t do selections (if) or sequences (loops). If you can  
> handle this, though, this becomes a virtue, as it keeps everything  
> crystal clear and simple. Witness the ever useful Unix command line  
> pipelines.
> 
> 2. They don’t encapsulate very well, as each stage totally overwrites  
> the previous ones. I’m not sure if I should just accept this, or try  
> to use some modification (append only pipeline, etc.).
> 
> Any experience or ideas on this appreciated.

There are several ways you can do pipelining. Do you want concurrency  
with that? Does every stage have a single output the next stage wants  
to  
operate on? etc.

Probably the simplest thing you can do is to use #inject:

commands = [  
lambda {|x| x \* 2},  
lambda {|x| 0 - x},  
lambda {|x| x + 10},  
]

> > commands.inject(0) {|x,cmd| cmd[x]}  
> > =\> 10
> 
> > commands.inject(1) {|x,cmd| cmd[x]}  
> > =\> 8

Kind regards

```
robert
```

---

<div class="post-metadata">

**Author:** ![unknown](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/u/4bbf92/32.png) [@unknown](https://www.ruby-forum.com/u/unknown)\
**Post date:** [February 13, 2006, 5:15pm UTC](https://www.ruby-forum.com/t/command-pipeline-pipes-filters-in-ruby/66417/4 "2006-02-13T17:15:21Z")

</div>

On Mon, 13 Feb 2006 removed\_email\_address@domain.invalid wrote:

> The downsides of pipelines:
> 
> 1. You can’t do selections (if) or sequences (loops). If you can  
> handle this, though, this becomes a virtue, as it keeps everything  
> crystal clear and simple. Witness the ever useful Unix command line  
> pipelines.
> 
> 2. They don’t encapsulate very well, as each stage totally overwrites  
> the previous ones. I’m not sure if I should just accept this, or try  
> to use some modification (append only pipeline, etc.).
> 
> Any experience or ideas on this appreciated.

this handles any dag. i wrote it a long time ago

[http://codeforpeople.com/lib/ruby/flow/flow-2.0.0/](http://codeforpeople.com/lib/ruby/flow/flow-2.0.0/)

hth.

-a

---

<div class="post-metadata">

**Author:** ![unknown](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/u/4bbf92/32.png) [@unknown](https://www.ruby-forum.com/u/unknown)\
**Post date:** [February 13, 2006, 5:43pm UTC](https://www.ruby-forum.com/t/command-pipeline-pipes-filters-in-ruby/66417/5 "2006-02-13T17:43:40Z")

</div>

That’s neat, this might be handy too:

class Proc  
def |(other)  
proc{|\*a| other.call(self.call(\*a)) }  
end  
end

I think I got the idea from something on Why’s site, though I can’t  
find it now. Anyways if you could now do:

f = lambda {|x| x \* 2} | lambda {|x| 0 - x} | lambda {|x| x + 10}

or more verbosely:

a = lambda {|x| x \* 2}  
b = lambda {|x| 0 - x}  
c = lambda {|x| x + 10}

f = (a|b|c)

Think of it as unix pipes 🙂  
.adam

---

<div class="post-metadata">

**Author:** ![unknown](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/u/4bbf92/32.png) [@unknown](https://www.ruby-forum.com/u/unknown)\
**Post date:** [February 13, 2006, 6:03pm UTC](https://www.ruby-forum.com/t/command-pipeline-pipes-filters-in-ruby/66417/6 "2006-02-13T18:03:34Z")

</div>

Adam S. wrote:

> Think of it as unix pipes 🙂  
> .adam

## Depending on the number of processors you might even run into stack size limits which won’t happen with an #inject based solution (see my other posting). (Ok, I know that I’m being obsessive about Enumerable#inject

it was love at first sight. :-)) )

Although this looks nice, when associating Unix pipes you might be  
tempted  
to expect those to execute concurrently which they don’t.

We can combine both solutions:

class ProcChain  
def initialize(\*procs) @chain = procs end  
def |(proc)  
@chain \<\< proc  
self  
end  
def call(a)  
@chain.inject(a) {|x,pr| pr[x]}  
end  
alias [] call  
end

class Proc  
def |(proc)  
ProcChain.new(self, proc)  
end  
end

irb(main):039:0\> f = lambda {|x| x \* 2} | lambda {|x| 0 - x} | lambda  
{|x|  
x + 10}  
=\> #\<ProcChain:0x4a6fed8 @chain=[#Proc:0x04a70238@:39(irb),  
#Proc:0x04a70148@:39(irb), #Proc:0x04a6ffc8@:39(irb)]\>  
irb(main):040:0\> f[0]  
=\> 10  
irb(main):041:0\> f.call 1  
=\> 8

:-))

Kind regards

```
robert
```

---

<div class="post-metadata">

**Author:** ![unknown](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/u/4bbf92/32.png) [@unknown](https://www.ruby-forum.com/u/unknown)\
**Post date:** [February 13, 2006, 6:38pm UTC](https://www.ruby-forum.com/t/command-pipeline-pipes-filters-in-ruby/66417/7 "2006-02-13T18:38:39Z")

</div>

removed\_email\_address@domain.invalid writes:

> The downsides of pipelines:
> 
> 1. You can’t do selections (if) or sequences (loops). If you can  
> handle this, though, this becomes a virtue, as it keeps everything  
> crystal clear and simple. Witness the ever useful Unix command line  
> pipelines.
> 
> 2. They don’t encapsulate very well, as each stage totally overwrites  
> the previous ones. I’m not sure if I should just accept this, or try  
> to use some modification (append only pipeline, etc.).
> 
> Any experience or ideas on this appreciated.

A long, long, time ago, I can still remember:

[http://kronavita.de/chris/data/pipeline\_rdoc.html](http://kronavita.de/chris/data/pipeline_rdoc.html)