# Pipe binary data

**URL:** <https://www.ruby-forum.com/t/pipe-binary-data/70568>\
**Category:** Ruby\
**Created:** [September 30, 2006, 6:01am UTC](https://www.ruby-forum.com/t/pipe-binary-data/70568 "2006-09-30T06:01:40Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![unknown](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/u/4bbf92/32.png) [@unknown](https://www.ruby-forum.com/u/unknown)\
**Post date:** [September 30, 2006, 6:01am UTC](https://www.ruby-forum.com/t/pipe-binary-data/70568/1 "2006-09-30T06:01:40Z")

</div>

Hi there,  
I have a little program that mines the web. It consists of two ruby  
programs. The first is responsible to get the data from the web; the  
second is responsible of mining that data. I connect the two via  
command line redirection/ pipe (the first executes stdin.puts data; the  
second calls data = stdin.gets)

This works just fine when I access html pages. However, when I access  
binary data, at times, the pipe misbehaves. I encountered one jpg file  
that causes a crash of the consuming data mining program on linux; and  
a infinite loop (stdin.gets just returns nil after a while without  
having read the entire data) on windows.

I suppose what I have to do is encode the data before I place it on the  
pipe and decode it after I consume it from the pipe. Any suggestions?

Cheers

Christian

---

<div class="post-metadata">

**Author:** ![unknown](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/u/4bbf92/32.png) [@unknown](https://www.ruby-forum.com/u/unknown)\
**Post date:** [September 30, 2006, 9:25am UTC](https://www.ruby-forum.com/t/pipe-binary-data/70568/2 "2006-09-30T09:25:55Z")

</div>

Hi Christian,

I’m no expert by any means, but this sounds odd to me – more like  
something is broken with your scripts rather than the piping. The  
reason I say so is because I can pipe /dev/random and /dev/dsp to a  
file without any problem; you’d think that I’d hit the same problem at  
some point given the (pseudo-)random nature of those pipes; but I never  
have.

Regards,  
Jordan

---

<div class="post-metadata">

**Author:** ![unknown](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/u/4bbf92/32.png) [@unknown](https://www.ruby-forum.com/u/unknown)\
**Post date:** [September 30, 2006, 10:47am UTC](https://www.ruby-forum.com/t/pipe-binary-data/70568/3 "2006-09-30T10:47:59Z")

</div>

Jordan, some more digging revealed this was related to a parsing bug in  
REXML. It somehow didnt recognize a closing tag…so it wasnt related  
to the pipe afterall.  
thanks-  
christian