Unix zcat with ruby?


#1

I have to read in many files.
I prefer to concat those files and reading only one large file.
There is a way like unix zcat?


#2

On 4/23/07, music removed_email_address@domain.invalid wrote:

I have to read in many files.
I prefer to concat those files and reading only one large file.
There is a way like unix zcat?

Sure is
ARGF
HTH
R.


#3

Robert D. wrote:

On 4/23/07, music removed_email_address@domain.invalid wrote:

I have to read in many files.
I prefer to concat those files and reading only one large file.
There is a way like unix zcat?

Sure is
ARGF

ARGF ok, but how can I pass ARGF to a method?


#4

On 4/24/07, music removed_email_address@domain.invalid wrote:

ARGF ok, but how can I pass ARGF to a method?

As a global constant it is visible anywhere, you can thus simply do
things like this

def my_method
ARGF.each_line do
| line |
whatever
end
end

R.


#5

Robert D. wrote:

end
end

Great!!! and if input files are in gzip format? how can I read them?


#6

On 24.04.2007 09:04, music wrote:

ARGF ok, but how can I pass ARGF to a method?
What???

some_method(ARGF)

robert


#7

On 4/24/07, music removed_email_address@domain.invalid wrote:

ARGF
whatever
end
end

Great!!! and if input files are in gzip format? how can I read them?

http://www.ruby-doc.org/stdlib/libdoc/zlib/rdoc/index.html


#8

Robert D. wrote:

Sure is
| line |
whatever
end
end

Great!!! and if input files are in gzip format? how can I read them?

http://www.ruby-doc.org/stdlib/libdoc/zlib/rdoc/index.html

Yes but can I use ARGF with zlib?


#9

music wrote:

ARGF.each_line do

Yes but can I use ARGF with zlib?

Any help? I’m searching in Zlib::GzipReader class but I can’t undestand
how I can use with ARGF.


#10

On 4/24/07, music removed_email_address@domain.invalid wrote:

def my_method

Yes but can I use ARGF with zlib?

Any help? I’m searching in Zlib::GzipReader class but I can’t undestand
how I can use with ARGF.

Something like this
ruby -rzlib -e ‘reader = Zlib::GzipReader.new(ARGF);reader.each_line
do |x| puts x end’ xxx.gz
I discovered however that
Zlib::GzipReader.new(ARGF)
seems to ignore all but the first file in the argument list - unless I
did something stupid - that complicates matters slightly, you still
can do things like

ARGV.each do
| filename |
Zlib::GeipReader.open( filename) do
|gz|
gz.each_line …

end
end

of course but this behavior surprises me a little bit, maybe you
should mine the Zlib doc for this.

Robert


#11

Robert D. wrote:

of course but this behavior surprises me a little bit,
forget what I said, I just switched my brain on
the concatenation of two gzipped streams is of course not a gzipped stream,
Zlib does well to inflate the first one, maybe a warning would be nice.
R.

Sorry for my newbie condition but I don’t understand.


#12

of course but this behavior surprises me a little bit,
forget what I said, I just switched my brain on
the concatenation of two gzipped streams is of course not a gzipped
stream,
Zlib does well to inflate the first one, maybe a warning would be nice.
R.


#13

On Tue, Apr 24, 2007 at 07:52:41PM +0900, Robert D. wrote:

of course but this behavior surprises me a little bit,
forget what I said, I just switched my brain on
the concatenation of two gzipped streams is of course not a gzipped stream,

But the standalone gzip program handles this:

$ (echo “foo” | gzip -c; echo “bar” | gzip -c) | gzip -dc
foo
bar
$

It enables you to grow a gzip file without having to entirely decompress
and
recompress what’s already there.


#14

On 4/24/07, Brian C. removed_email_address@domain.invalid wrote:

On Tue, Apr 24, 2007 at 07:52:41PM +0900, Robert D. wrote:

of course but this behavior surprises me a little bit,
forget what I said, I just switched my brain on
the concatenation of two gzipped streams is of course not a gzipped stream,

But the standalone gzip program handles this:
zcat does too, which is nice of course.
But this is an extra feature and I did not want to criticize zlib for
not implementing this extra feature, maybe someone will provide a
patch to ruby-core :wink:

$ (echo “foo” | gzip -c; echo “bar” | gzip -c) | gzip -dc
foo
bar
$

It enables you to grow a gzip file without having to entirely decompress and
recompress what’s already there.
Yeah I just tested this, as I use gzip only with tar.
Hmm does that mean that I was wrong that the concatenation of two
gzipped streams is not a gzipped stream?
Nahh, that surely is just an underlying feature of gzip (and bzip2 for
that matter).
Gotta look that after…

Cheers
Robert