Forum: Ruby Ruby Newbie Problems with deflate, base64...

Announcement (2017-05-07): is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see and for other Rails- und Ruby-related community platforms.
6fcaca6c91a5719628a5c1b566e32d4a?d=identicon&s=25 Pat Patterson (Guest)
on 2007-03-13 05:23
(Received via mailing list)
I'm implementing a spec that calls for messages to be deflated, then
base64 encoded, then URL encoded, so they can be passed as parameters to
an HTTP GET. In PHP, this is achieved with:

$encodedMsg = urlencode( base64_encode( gzdeflate( $msg ) ) )

Looking at the docs, the equivalent in Ruby would be something like:

require "cgi"
require "base64"
require "zlib"

encodedMsg = CGI::escape( Base64.encode64( Zlib::Deflate.deflate( msg )
) )

But this gives me completely different output to my (working) PHP model.

Some googling later, it appears that Zlib::Deflate.deflate prepends and
appends 'stuff' to the deflated data. Investigating, the PHP

$deflated = gzdeflate( "Hello world" );
echo bin2hex( $deflated );

Gives me


While the Ruby

deflated = Zlib::Deflate.deflate( "Hello world" )
myhex = ""
1.upto(deflated.length) { |i| myhex << "%02x" % deflated[i] }
puts myhex



(BTW - if anyone knows a more succinct way to hex encode a string in
Ruby, that would be useful)

Notice the additional '9c' at the start of the string and '18ab043d00'
at the end. OK - there seems to be a consistent amount of data prepended
and appended, so I can slice those off.

But now Base 64 is acting weird. In PHP:

$base64Encoded = base64_encode( $deflated );
echo $base64Encoded;



While, in Ruby

# Remove extra stuff from deflated string
deflated = deflated[1,deflated.length-6]
base64encoded = Base64.encode64( deflated )
puts base64encoded



Completely different! Now, if I feed that back through PHP's
base64_unencode, I get


Notice, again '9c' prepended, but this time the trailing '00' has been

What is going on here? Any ideas??? Am I missing something really
obvious about string handling in Ruby?

(BTW - this is on ruby 1.8.4 (2005-12-24) [i486-linux])


753dcb78b3a3651127665da4bed3c782?d=identicon&s=25 Brian Candler (Guest)
on 2007-03-13 21:48
(Received via mailing list)
On Tue, Mar 13, 2007 at 01:21:52PM +0900, Pat Patterson wrote:
> (BTW - if anyone knows a more succinct way to hex encode a string in
> Ruby, that would be useful)

str = "\001\377"
puts str.unpack("H*")
6fcaca6c91a5719628a5c1b566e32d4a?d=identicon&s=25 Pat Patterson (Guest)
on 2007-03-14 00:44
(Received via mailing list)
Thanks, Brian! That revealed an obvious bug in the code I was using to
examine the deflated data (should have been 0.upto(deflated.length-1)).

So Base64 /is/ working correctly. Deflate prepends 2 bytes (seems to be
constant 0x789c for default deflate level) and appends 4 bytes (rather
than 1 and 5 as I thought) to the deflated data. When I cut those off, I
can get Ruby to work the same as PHP.

Still - it would be nice if deflate worked the same as on Java, PHP, ...


6fcaca6c91a5719628a5c1b566e32d4a?d=identicon&s=25 Pat Patterson (Guest)
on 2007-03-14 01:17
(Received via mailing list)
OK - so now I know what is happening...

Zlib::Deflate.deflate implements ZLIB compression according to RFC 1950.
ZLIB defines a 2 byte header containing a variety of flags and a 4 byte
trailer containing an Adler-32 checksum. Just out of interest, the
header that I'm seeing translates as 'compression method = deflate,
windows size = 32k, no preset dictionary, default compression level',
which makes perfect sense. Deflate compression itself is defined by RFC

So - if you want 'raw' deflated data (which is called for in many
situations), cutting off the leading 2 and trailing 4 bytes is exactly
what you need to do.


8bc543795b502900b5333aea73ad5533?d=identicon&s=25 Eden Li (edenli)
on 2007-03-14 02:36
(Received via mailing list)
You can prevent deflate from generating the header by passing in -
MAX_WBITS to the options for  The following method
emulates gzdeflate from php:

def gzdeflate(s), -Zlib::MAX_WBITS).deflate(s, Zlib::FINISH)

puts gzdeflate("Hello World").unpack('H*').first
# => f348cdc9c95708cf2fca490100
6fcaca6c91a5719628a5c1b566e32d4a?d=identicon&s=25 Pat Patterson (Guest)
on 2007-03-14 18:15
(Received via mailing list)
Thanks, Eden - that works great and is much cleaner than cutting the
header and checksum off of the deflated data.


This topic is locked and can not be replied to.