# Pure Ruby Zlib::GzipWriter

**URL:** <https://www.ruby-forum.com/t/pure-ruby-zlib-gzipwriter/155694>\
**Category:** Ruby\
**Created:** [December 15, 2008, 5:40pm UTC](https://www.ruby-forum.com/t/pure-ruby-zlib-gzipwriter/155694 "2008-12-15T17:40:08Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![danielb](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/c57346/32.png) [@danielb](https://www.ruby-forum.com/u/danielb)\
**Post date:** [December 15, 2008, 5:40pm UTC](https://www.ruby-forum.com/t/pure-ruby-zlib-gzipwriter/155694/1 "2008-12-15T17:40:08Z")

</div>

Hi,

There’s a project out there called “zliby” [1] which partially  
implements the zlib library. Unfortunately, only the Zlib::GzipReader  
class has been implemented so far. I’d like to have a pure Ruby  
implementation of the Zlib::GzipWriter class.

I’ve put in a feature request [2] but I’d like to see it expedited. In  
order to expedite this, I’m offering $100 to anyone who can implement  
it. In order to qualify for the bounty, your Zlib::GzipWriter class  
must meet the following criteria:

- It must be pure Ruby. No extensions. No FFI.
- It must have tests (and pass them)
- It must match the current Zlib::GzipWriter interface
- It must be cross platform
- It must be able to generate a proper gem file from a gemspec (the  
acid test)
- It must be contributed back to the zliby project
- It must be done within 60 days of this post

If these conditions are met, you will receive your bounty via PayPal.

What’s my motivation? Mainly MS Windows. I run a hand built version of  
Ruby, and getting the zlib wrapper to build with MS VC++ is such a  
colossal PITA that I’d like to avoid it completely. Beyond that,  
however, is that it eliminates a third party dependency (zlib) and  
gives me the ability to create gems. At the moment I can only read  
them.

Any takers? Any matchers?

Regards,

Dan

[1] http:/www.rubyforge.org/projects/zliby  
[2]  
[http://rubyforge.org/tracker/index.php?func=detail&aid=23239&group\_id=5769&atid=22371](http://rubyforge.org/tracker/index.php?func=detail&aid=23239&group_id=5769&atid=22371)

---

<div class="post-metadata">

**Author:** ![danielb](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/c57346/32.png) [@danielb](https://www.ruby-forum.com/u/danielb)\
**Post date:** [December 21, 2008, 12:30am UTC](https://www.ruby-forum.com/t/pure-ruby-zlib-gzipwriter/155694/2 "2008-12-21T00:30:37Z")

</div>

> I’ve put in a feature request [2] but I’d like to see it expedited. In  
> order to expedite this, I’m offering $100 to anyone who can implement  
> it. In order to qualify for the bounty, your Zlib::GzipWriter class  
> must meet the following criteria:  
> That would indeed be convenient.  
> Would be nice to someday no longer have the dependency of rubygems on  
> zlib.  
> [http://groups.google.com/group/comp.lang.ruby/browse\_thread/thread/a6966f6fc1424f61](http://groups.google.com/group/comp.lang.ruby/browse_thread/thread/a6966f6fc1424f61)  
> Then there’d be one less setup hurdle for ruby.  
> I’m a watcher for now though 🙂  
> -=R

---

<div class="post-metadata">

**Author:** ![danielb](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/c57346/32.png) [@danielb](https://www.ruby-forum.com/u/danielb)\
**Post date:** [December 21, 2008, 7:17pm UTC](https://www.ruby-forum.com/t/pure-ruby-zlib-gzipwriter/155694/3 "2008-12-21T19:17:18Z")

</div>

rogerdpack wrote:

> > I’ve put in a feature request [2] but I’d like to see it expedited. In  
> > order to expedite this, I’m offering $100 to anyone who can implement  
> > it. In order to qualify for the bounty, your Zlib::GzipWriter class  
> > must meet the following criteria:  
> > That would indeed be convenient.  
> > Would be nice to someday no longer have the dependency of rubygems on  
> > zlib.  
> > [http://groups.google.com/group/comp.lang.ruby/browse\_thread/thread/a6966f6fc1424f61](http://groups.google.com/group/comp.lang.ruby/browse_thread/thread/a6966f6fc1424f61)  
> > Then there’d be one less setup hurdle for ruby.  
> > I’m a watcher for now though 🙂

Of course you could use JRuby, which doesn’t have such issues with zlib  
and Windows…

- Charlie

---

<div class="post-metadata">

**Author:** ![danielb](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/c57346/32.png) [@danielb](https://www.ruby-forum.com/u/danielb)\
**Post date:** [December 21, 2008, 9:49pm UTC](https://www.ruby-forum.com/t/pure-ruby-zlib-gzipwriter/155694/4 "2008-12-21T21:49:53Z")

</div>

On Dec 21, 1:08 pm, Charles Oliver N. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
wrote:

> > I’m a watcher for now though 🙂
> 
> Of course you could use JRuby, which doesn’t have such issues with zlib  
> and Windows…

I thought JRuby used FFI. No? If not, it should just be matter of  
porting the code. But if so, you still have two third party  
dependencies - FFI and zlib.dll.

Regards,

Dan

---

<div class="post-metadata">

**Author:** ![danielb](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/c57346/32.png) [@danielb](https://www.ruby-forum.com/u/danielb)\
**Post date:** [December 21, 2008, 10:09pm UTC](https://www.ruby-forum.com/t/pure-ruby-zlib-gzipwriter/155694/5 "2008-12-21T22:09:49Z")

</div>

Daniel B. wrote:

> I thought JRuby used FFI. No? If not, it should just be matter of  
> porting the code. But if so, you still have two third party  
> dependencies - FFI and zlib.dll.

The JVM includes zlib and obviously has a very nice version for Windows.  
We just wrap the JVM’s zlib-based compression classes. But that doesn’t  
really solve your original problem since you want to run a handbuilt C  
Ruby…

- Charlie

---

<div class="post-metadata">

**Author:** ![danielb](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/c57346/32.png) [@danielb](https://www.ruby-forum.com/u/danielb)\
**Post date:** [December 22, 2008, 4:04am UTC](https://www.ruby-forum.com/t/pure-ruby-zlib-gzipwriter/155694/6 "2008-12-22T04:04:16Z")

</div>

On Dec 21, 5:16 pm, The Higgs bozo [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> file, decode it, then call it with DL or FFI?  
> I’m not really sure how to do that, and it’s not really what I’m  
> looking for I’m afraid.

Regards,

Dan

---

<div class="post-metadata">

**Author:** ![danielb](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/c57346/32.png) [@danielb](https://www.ruby-forum.com/u/danielb)\
**Post date:** [December 22, 2008, 5:07am UTC](https://www.ruby-forum.com/t/pure-ruby-zlib-gzipwriter/155694/7 "2008-12-22T05:07:43Z")

</div>

Daniel B. wrote:

> On Dec 21, 5:16ï¿½pm, The Higgs bozo [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:
> 
> > file, decode it, then call it with DL or FFI?  
> > I’m not really sure how to do that, and it’s not really what I’m  
> > looking for I’m afraid.

You didn’t know how, or didn’t understand? DL is part of the standard  
library. You could carry around the zlib dll right inside an .rb file,  
and use DL to call the zlib function(s) you needed.

I understand you were looking for a general-purpose solution, but in the  
meantime this does what you need, albeit gymnastically. Namely, zlib on  
Windows without the requirement of building a ruby extension.

---

<div class="post-metadata">

**Author:** ![danielb](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/c57346/32.png) [@danielb](https://www.ruby-forum.com/u/danielb)\
**Post date:** [December 21, 2008, 11:25pm UTC](https://www.ruby-forum.com/t/pure-ruby-zlib-gzipwriter/155694/8 "2008-12-21T23:25:02Z")

</div>

Daniel B. wrote:

> What’s my motivation? Mainly MS Windows. I run a hand built version of  
> Ruby, and getting the zlib wrapper to build with MS VC++ is such a  
> colossal PITA that I’d like to avoid it completely. Beyond that,  
> however, is that it eliminates a third party dependency (zlib) and  
> gives me the ability to create gems. At the moment I can only read  
> them.

As a cheap shortcut, could you store a base64-encoded zlib dll in a .rb  
file, decode it, then call it with DL or FFI?

---

<div class="post-metadata">

**Author:** ![danielb](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/c57346/32.png) [@danielb](https://www.ruby-forum.com/u/danielb)\
**Post date:** [December 22, 2008, 10:59pm UTC](https://www.ruby-forum.com/t/pure-ruby-zlib-gzipwriter/155694/9 "2008-12-22T22:59:39Z")

</div>

The Higgs bozo wrote:

> library. You could carry around the zlib dll right inside an .rb file,  
> and use DL to call the zlib function(s) you needed.

Out of curiosity, how do you suggest doing this? Put the dll in the DATA  
section, write to a tmp file, and load? Or is there some magic?

---

<div class="post-metadata">

**Author:** ![danielb](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/c57346/32.png) [@danielb](https://www.ruby-forum.com/u/danielb)\
**Post date:** [December 22, 2008, 8:16am UTC](https://www.ruby-forum.com/t/pure-ruby-zlib-gzipwriter/155694/10 "2008-12-22T08:16:51Z")

</div>

On Dec 21, 10:59 pm, The Higgs bozo [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> Daniel B. wrote:
> 
> > On Dec 21, 5:16 pm, The Higgs bozo [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:
> > 
> > > file, decode it, then call it with DL or FFI?  
> > > I’m not really sure how to do that, and it’s not really what I’m  
> > > looking for I’m afraid.
> 
> You didn’t know how, or didn’t understand? DL is part of the standard  
> library. You could carry around the zlib dll right inside an .rb file,  
> and use DL to call the zlib function(s) you needed.

Just to clarify for purposes of the bounty, no DL, and no distributed  
dll’s.

> I understand you were looking for a general-purpose solution, but in the  
> meantime this does what you need, albeit gymnastically. Namely, zlib on  
> Windows without the requirement of building a ruby extension.

I really do appreciate the suggestion but it’s not what I’m looking  
for.

Regards,

Dan

---

<div class="post-metadata">

**Author:** ![danielb](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/c57346/32.png) [@danielb](https://www.ruby-forum.com/u/danielb)\
**Post date:** [December 27, 2008, 2:05am UTC](https://www.ruby-forum.com/t/pure-ruby-zlib-gzipwriter/155694/11 "2008-12-27T02:05:45Z")

</div>

> What’s my motivation? Mainly MS Windows. I run a hand built version of  
> Ruby, and getting the zlib wrapper to build with MS VC++ is such a  
> colossal PITA that I’d like to avoid it completely. Beyond that,  
> however, is that it eliminates a third party dependency (zlib) and  
> gives me the ability to create gems. At the moment I can only read  
> them.

One thing I use to help me compile on doze is Lavena’s rubyinstaller  
[I’m sure you’ve seen it..].  
Anyway I’ll match $50 🙂  
-=r

---

<div class="post-metadata">

**Author:** ![danielb](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/c57346/32.png) [@danielb](https://www.ruby-forum.com/u/danielb)\
**Post date:** [December 27, 2008, 3:38am UTC](https://www.ruby-forum.com/t/pure-ruby-zlib-gzipwriter/155694/12 "2008-12-27T03:38:44Z")

</div>

On Dec 26, 8:05 pm, Roger P. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> > What’s my motivation? Mainly MS Windows. I run a hand built version of  
> > Ruby, and getting the zlib wrapper to build with MS VC++ is such a  
> > colossal PITA that I’d like to avoid it completely. Beyond that,  
> > however, is that it eliminates a third party dependency (zlib) and  
> > gives me the ability to create gems. At the moment I can only read  
> > them.
> 
> One thing I use to help me compile on doze is Lavena’s rubyinstaller  
> [I’m sure you’ve seen it..].  
> Anyway I’ll match $50 🙂

Excellent Roger, thank you.

By the way, MS Windows is not the only reason removing the external  
dependency on zlib would be a good thing:

[http://rubyforge.org/tracker/index.php?func=detail&aid=23354&group\_id=126&atid=575](http://rubyforge.org/tracker/index.php?func=detail&aid=23354&group_id=126&atid=575)

Regards,

Dan

---

<div class="post-metadata">

**Author:** ![danielb](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/c57346/32.png) [@danielb](https://www.ruby-forum.com/u/danielb)\
**Post date:** [January 3, 2009, 3:06am UTC](https://www.ruby-forum.com/t/pure-ruby-zlib-gzipwriter/155694/13 "2009-01-03T03:06:35Z")

</div>

On Dec 15 2008, 9:32 am, Daniel B. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> must meet the following criteria:  
> If these conditions are met, you will receive your bounty via PayPal.
> 
> What’s my motivation? Mainly MS Windows. I run a hand built version of  
> Ruby, and getting the zlib wrapper to build with MS VC++ is such a  
> colossal PITA that I’d like to avoid it completely. Beyond that,  
> however, is that it eliminates a third party dependency (zlib) and  
> gives me the ability to create gems. At the moment I can only read  
> them.

Yet more ammunition:

[http://www.misuse.org/science/2007/09/07/ruby-zlib-buffer-error-on-windows/](http://www.misuse.org/science/2007/09/07/ruby-zlib-buffer-error-on-windows/)

Regards,

Dan

---

<div class="post-metadata">

**Author:** ![danielb](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/c57346/32.png) [@danielb](https://www.ruby-forum.com/u/danielb)\
**Post date:** [January 4, 2009, 4:22am UTC](https://www.ruby-forum.com/t/pure-ruby-zlib-gzipwriter/155694/14 "2009-01-04T04:22:39Z")

</div>

> Yet more ammunition:
> 
> [http://www.misuse.org/science/2007/09/07/ruby-zlib-buffer-error-on-windows/](http://www.misuse.org/science/2007/09/07/ruby-zlib-buffer-error-on-windows/)

Interesting. I wonder if it’s somehow related to  
[http://redmine.ruby-lang.org/issues/show/776](http://redmine.ruby-lang.org/issues/show/776)  
-=r

---

<div class="post-metadata">

**Author:** ![danielb](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/c57346/32.png) [@danielb](https://www.ruby-forum.com/u/danielb)\
**Post date:** [January 5, 2009, 6:50pm UTC](https://www.ruby-forum.com/t/pure-ruby-zlib-gzipwriter/155694/15 "2009-01-05T18:50:26Z")

</div>

On Jan 3, 8:21 pm, Roger P. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> > Yet more ammunition:
> 
> > [http://www.misuse.org/science/2007/09/07/ruby-zlib-buffer-error-on-wi](http://www.misuse.org/science/2007/09/07/ruby-zlib-buffer-error-on-wi)…
> 
> Interesting. I wonder if it’s somehow related to [http://redmine.ruby-lang.org/issues/show/776](http://redmine.ruby-lang.org/issues/show/776)

It probably is, but there’s nothing we can do about it since zlib  
isn’t developed separately as a gem, and it would still require that  
someone build a binary.

If we had a pure Ruby version available as a gem on the other hand…

🙂

Regards,

Dan

---

<div class="post-metadata">

**Author:** ![danielb](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/c57346/32.png) [@danielb](https://www.ruby-forum.com/u/danielb)\
**Post date:** [January 8, 2009, 4:31am UTC](https://www.ruby-forum.com/t/pure-ruby-zlib-gzipwriter/155694/16 "2009-01-08T04:31:16Z")

</div>

Hi,

2009/1/6 Daniel B. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid):

> If we had a pure Ruby version available as a gem on the other hand…
> 
> 🙂

I finished translation and testing of the pure ruby version.  
The ruby version is exact same api interface and functionality with  
the zlib c source code.  
It is also compatible with ruby 1.8.x and 1.9.x and includes  
examble.rb and minizip.rb for testing.

You can download at [http://121.78.227.9/rbzlib/](http://121.78.227.9/rbzlib/)

I am working on translation of ruby library zlib.c to pure ruby code.

Regards,

Park H.

---

<div class="post-metadata">

**Author:** ![danielb](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/c57346/32.png) [@danielb](https://www.ruby-forum.com/u/danielb)\
**Post date:** [January 8, 2009, 3:34am UTC](https://www.ruby-forum.com/t/pure-ruby-zlib-gzipwriter/155694/17 "2009-01-08T03:34:05Z")

</div>

Daniel B. wrote:

> If we had a pure Ruby version available as a gem on the other hand…
> 
> 🙂
> 
> Regards,
> 
> Dan

I’m most of the way done, just missing dynamic huffman trees, and more  
interface similarity with zlib. Need to change the specs from zliby  
though as the algorithm I’m using is not identical to zlib and gives  
different (but still valid) results.

For example, zlib’s lz77 phase doesn’t seem to behave optimally wrt  
repeats:

> > Zlib::Deflate.deflate(0.chr \* 10, 9).length  
> > =\> 10  
> > CZlib::Deflate.deflate(0.chr \* 10, 9).length  
> > =\> 11  
> > CZlib::Inflate.inflate(Zlib::Deflate.deflate(0.chr \* 10, 9)) == 0.chr \* 10  
> > =\> true  
> > CZlib::Inflate.inflate(CZlib::Deflate.deflate(0.chr \* 10, 9)) == 0.chr \* 10  
> > =\> true

(using CZlib for inflation there as I’ve not integrated it with the  
zliby code yet).

---

<div class="post-metadata">

**Author:** ![danielb](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/c57346/32.png) [@danielb](https://www.ruby-forum.com/u/danielb)\
**Post date:** [January 8, 2009, 5:45am UTC](https://www.ruby-forum.com/t/pure-ruby-zlib-gzipwriter/155694/18 "2009-01-08T05:45:30Z")

</div>

Heesob P. wrote:

> It is also compatible with ruby 1.8.x and 1.9.x and includes  
> examble.rb and minizip.rb for testing.
> 
> You can download at [http://121.78.227.9/rbzlib/](http://121.78.227.9/rbzlib/)

Awesome!

> I am working on translation of ruby library zlib.c to pure ruby code.

Looks like you and Charles L. are in a race. 🙂

Regards,

Dan

---

<div class="post-metadata">

**Author:** ![danielb](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/c57346/32.png) [@danielb](https://www.ruby-forum.com/u/danielb)\
**Post date:** [January 8, 2009, 6:01am UTC](https://www.ruby-forum.com/t/pure-ruby-zlib-gzipwriter/155694/19 "2009-01-08T06:01:03Z")

</div>

Daniel B. wrote:

> Sounds like you’re on track. But, you’re in a race with Park H.. 🙂
> 
> Dan

Heh. 🙂 Somehow I think he’s got it covered. That reimplementation of  
zlib in ruby is actually pretty cool. I’m curious Park, as to how much  
you did by hand and how much you were able to automate it…?

I’ve put the deflate code i wrote on pastie - [http://pastie.org/355367](http://pastie.org/355367)

It’s about 600 lines, but needs cleaning up. Will probably be ~500 when  
I’ve added in dynamic trees and cleaned it up a bit.

Prelim benchmarks show the current bottleneck to be the LZ77  
implementation which is interesting. I think for it to perform  
acceptably, a lot more of the code needs to become more idiomatic,  
higher-level ruby.

---

<div class="post-metadata">

**Author:** ![danielb](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/c57346/32.png) [@danielb](https://www.ruby-forum.com/u/danielb)\
**Post date:** [January 8, 2009, 5:46am UTC](https://www.ruby-forum.com/t/pure-ruby-zlib-gzipwriter/155694/20 "2009-01-08T05:46:41Z")

</div>

Charles L. wrote:

> interface similarity with zlib. Need to change the specs from zliby
> 
> > > CZlib::Inflate.inflate(Zlib::Deflate.deflate(0.chr \* 10, 9)) == 0.chr \* 10  
> > > =\> true  
> > > CZlib::Inflate.inflate(CZlib::Deflate.deflate(0.chr \* 10, 9)) == 0.chr \* 10  
> > > =\> true
> 
> (using CZlib for inflation there as I’ve not integrated it with the  
> zliby code yet).

Sounds like you’re on track. But, you’re in a race with Park H.. 🙂

Dan

[Next page](https://www.ruby-forum.com/t/pure-ruby-zlib-gzipwriter/155694.md?page=2)
