Opening a stream to multiple (possibly zipped) files

Ruby
#1

Hello,

I am interested in opening a single data stream to multiple (possibly
gzipped) files, and was wondering if you could do that elegantly in
Ruby. I have some dodgy old Perl code that works, but it isn’t really
elegant/robust/portable:

http://pastie.org/963613

Cheers,

Martin

#2

Hi

On Mon, May 17, 2010 at 3:49 PM, Martin H. [email protected] wrote:

Hello,

I am interested in opening a single data stream to multiple (possibly
gzipped) files, and was wondering if you could do that elegantly in
Ruby. I have some dodgy old Perl code that works, but it isn’t really
elegant/robust/portable:

http://pastie.org/963613

There are bindings in ruby for zlib:
http://ruby-doc.org/core/classes/Zlib/GzipReader.html
And facets has a wrapper around it:
http://facets.rubyforge.org/apidoc/api/more/classes/Zlib.html
Maybe you could reimplement the Perl code trying to decompress always
and trapping the exception when the file is not compressed? Something
like this:

irb1.9 -rfacets/zlib -rzlib -rstringio
irb(main):001:0> Zlib.decompress(“plain text”)
Zlib::GzipFile::Error: not in gzip format

#3

Maybe you could reimplement the Perl code trying to decompress always
and trapping the exception when the file is not compressed? Something
like this:

irb1.9 -rfacets/zlib -rzlib -rstringio
irb(main):001:0> Zlib.decompress(“plain text”)
Zlib::GzipFile::Error: not in gzip format

That is a good idea (I don’t think you can rescue exceptions in Perl).
However, I am still uncertain if wise/possible and how to get a single
file handle on a stream from multiple files. I need to do
zipping/unzipping on the fly since I want to work on huge files.

Martin

#4

Le 18 mai à 08:11, Martin H. a écrit :

file handle on a stream from multiple files. I need to do
zipping/unzipping on the fly since I want to work on huge files.

This should work :

require “zlib”

def openf(fn)
File.open(fn) do |f|
sig = f.read(3)
f.rewind
if sig.bytes.to_a == [31, 139, 8] # GZip file signature
gz = Zlib::GzipReader.new(f)
yield gz
gz.close rescue nil
else
yield f
end
end
end

openf(“toto.gz”) { |f| puts f.read }
aljezalkezajlkezaj
=> nil

openf(“toto2.txt”) { |f| puts f.read }
iappoeipozaipza
=> nil

(You’d want to work on the exception handling etc, of course.)

Fred