# Opening a stream to multiple (possibly zipped) files

**URL:** <https://www.ruby-forum.com/t/opening-a-stream-to-multiple-possibly-zipped-files/188751>\
**Category:** Ruby\
**Created:** [May 17, 2010, 8:48pm UTC](https://www.ruby-forum.com/t/opening-a-stream-to-multiple-possibly-zipped-files/188751 "2010-05-17T20:48:51Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![himdel](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/h/c67d28/32.png) [@himdel](https://www.ruby-forum.com/u/himdel)\
**Post date:** [May 17, 2010, 8:48pm UTC](https://www.ruby-forum.com/t/opening-a-stream-to-multiple-possibly-zipped-files/188751/1 "2010-05-17T20:48:51Z")

</div>

Hello,

I am interested in opening a single data stream to multiple (possibly  
gzipped) files, and was wondering if you could do that elegantly in  
Ruby. I have some dodgy old Perl code that works, but it isn’t really  
elegant/robust/portable:

[http://pastie.org/963613](http://pastie.org/963613)

Cheers,

Martin

---

<div class="post-metadata">

**Author:** ![himdel](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/h/c67d28/32.png) [@himdel](https://www.ruby-forum.com/u/himdel)\
**Post date:** [May 18, 2010, 1:03am UTC](https://www.ruby-forum.com/t/opening-a-stream-to-multiple-possibly-zipped-files/188751/2 "2010-05-18T01:03:03Z")

</div>

Hi

On Mon, May 17, 2010 at 3:49 PM, Martin H. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> Hello,
> 
> I am interested in opening a single data stream to multiple (possibly  
> gzipped) files, and was wondering if you could do that elegantly in  
> Ruby. I have some dodgy old Perl code that works, but it isn’t really  
> elegant/robust/portable:
> 
> [http://pastie.org/963613](http://pastie.org/963613)

There are bindings in ruby for zlib:  
[http://ruby-doc.org/core/classes/Zlib/GzipReader.html](http://ruby-doc.org/core/classes/Zlib/GzipReader.html)  
And facets has a wrapper around it:  
[http://facets.rubyforge.org/apidoc/api/more/classes/Zlib.html](http://facets.rubyforge.org/apidoc/api/more/classes/Zlib.html)  
Maybe you could reimplement the Perl code trying to decompress always  
and trapping the exception when the file is not compressed? Something  
like this:

irb1.9 -rfacets/zlib -rzlib -rstringio  
irb(main):001:0\> Zlib.decompress(“plain text”)  
Zlib::GzipFile::Error: not in gzip format

---

<div class="post-metadata">

**Author:** ![himdel](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/h/c67d28/32.png) [@himdel](https://www.ruby-forum.com/u/himdel)\
**Post date:** [May 18, 2010, 8:11am UTC](https://www.ruby-forum.com/t/opening-a-stream-to-multiple-possibly-zipped-files/188751/3 "2010-05-18T08:11:42Z")

</div>

> Maybe you could reimplement the Perl code trying to decompress always  
> and trapping the exception when the file is not compressed? Something  
> like this:
> 
> irb1.9 -rfacets/zlib -rzlib -rstringio  
> irb(main):001:0\> Zlib.decompress(“plain text”)  
> Zlib::GzipFile::Error: not in gzip format

That is a good idea (I don’t think you can rescue exceptions in Perl).  
However, I am still uncertain if wise/possible and how to get a single  
file handle on a stream from multiple files. I need to do  
zipping/unzipping on the fly since I want to work on huge files.

Martin

---

<div class="post-metadata">

**Author:** ![himdel](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/h/c67d28/32.png) [@himdel](https://www.ruby-forum.com/u/himdel)\
**Post date:** [May 18, 2010, 1:48pm UTC](https://www.ruby-forum.com/t/opening-a-stream-to-multiple-possibly-zipped-files/188751/4 "2010-05-18T13:48:04Z")

</div>

Le 18 mai à 08:11, Martin H. a écrit :

> file handle on a stream from multiple files. I need to do  
> zipping/unzipping on the fly since I want to work on huge files.

This should work :

require “zlib”

def openf(fn)  
File.open(fn) do |f|  
sig = f.read(3)  
f.rewind  
if sig.bytes.to\_a == [31, 139, 8] # GZip file signature  
gz = Zlib::GzipReader.new(f)  
yield gz  
gz.close rescue nil  
else  
yield f  
end  
end  
end

> > openf(“toto.gz”) { |f| puts f.read }  
> > aljezalkezajlkezaj  
> > =\> nil
> 
> > openf(“toto2.txt”) { |f| puts f.read }  
> > iappoeipozaipza  
> > =\> nil

(You’d want to work on the exception handling etc, of course.)

Fred