I am working on translation of ruby library zlib.c to pure ruby code.
Looks like you and Charles L. are in a race.
Maybe you can collaborate? 
Let us know when somebody think it’s about done
-=r
On Jan 7, 11:08 pm, “Heesob P.” [email protected] wrote:
you did by hand and how much you were able to automate it…?
I dit it all by hand. There is no royal road to translating C to Ruby:)
I’ve put the deflate code i wrote on pastie -http://pastie.org/355367
Excellent, thanks!
bad performance in Ruby 1.8.x.
As long as it’s usable I’m happy. We can benchmark, profile and
optimize later.
regards,
Dan
Hi,
2009/1/8 Charles L. [email protected]:
Daniel B. wrote:
Sounds like you’re on track. But, you’re in a race with Park H…
Dan
Heh.
Somehow I think he’s got it covered. That reimplementation of
zlib in ruby is actually pretty cool. I’m curious Park, as to how much
you did by hand and how much you were able to automate it…?
I dit it all by hand. There is no royal road to translating C to Ruby:)
I’ve put the deflate code i wrote on pastie - http://pastie.org/355367
It’s about 600 lines, but needs cleaning up. Will probably be ~500 when
I’ve added in dynamic trees and cleaned it up a bit.Prelim benchmarks show the current bottleneck to be the LZ77
implementation which is interesting. I think for it to perform
acceptably, a lot more of the code needs to become more idiomatic,
higher-level ruby.
In my test, the deflation is very slow but the inflation is endurable.
Moreover, for the compatibility of the various Ruby version, It shows
bad performance in Ruby 1.8.x.
Regards,
Park H.
2009/1/8 Daniel B. [email protected]:
It’s about 600 lines, but needs cleaning up. Will probably be ~500 when
As long as it’s usable I’m happy. We can benchmark, profile and
optimize later.
Finally I translated the whole zlib.c to ruby code.
You can download at http://121.78.227.9/rbzlib/pzlib.rb
Regards,
Park H.
On Jan 8, 6:56 pm, “Charles L.” [email protected] wrote:
The big thing is just good specs, and the existing rubyspec ones could
use some work which will benefit all of us.
Agreed.
I’ve added most of the remaining missing functionality to my version (ie
dynamic huffman tree’s etc), and added back in the inflate functionality
from zliby, and put the result up on github -GitHub - aquasync/ruby-zlib: Pure ruby zlib implementation
Thanks. Unfortunately, I cannot download it. IE7 and FF3 simply do
nothing when I click on the download link. FF2 (before I upgraded)
gave me a “file not found” error.
This actually happens to me a lot with github, on not just on Windows.
I’ve since added enough of the GzipWriter & GzipFile functionality that
the existing specs all pass (see below. note the GzipFile specs use
GzipWriter). Unable to push this last commit to github presently, but
will try later.
Ok, I’ll look for it and just copy/paste files individually when
you’re ready.
Regards,
Dan
Roger P. wrote:
Maybe you can collaborate?
Let us know when somebody think it’s about done-=r
I’m not sure there’s much room for collaboration at the Zlib wrapper &
below level, but perhaps the GzipFile and upwards (ie Reader/Writer)
implementations can be shared.
The big thing is just good specs, and the existing rubyspec ones could
use some work which will benefit all of us.
I’ve added most of the remaining missing functionality to my version (ie
dynamic huffman tree’s etc), and added back in the inflate functionality
from zliby, and put the result up on github -
I’ve since added enough of the GzipWriter & GzipFile functionality that
the existing specs all pass (see below. note the GzipFile specs use
GzipWriter). Unable to push this last commit to github presently, but
will try later.
It is likely complete enough for the purposes of creating rubygems, but
is still missing functionality, which needs specing out (Eg wtf is
Zlib::GzipWriter.wrap, Zlib::GzipWriter reimplements a lot of IO
functions like puts, etc).
Zlib::Deflate#set_dictionary
- sets the dictionary
Zlib::Deflate#deflate
- deflates some data
Zlib::Deflate::deflate
- deflates some data
Zlib::GzipFile#closed?
- returns the closed status
Zlib::GzipFile#close
- finishes the stream and closes the io
Zlib::GzipFile#comment
- returns the name
- raises an error on a closed stream
Zlib::GzipFile#orig_name
- returns the name
- raises an error on a closed stream
Zlib::GzipWriter#mtime=
- sets mtime using Integer
- sets mtime using Time
- raises if the header was written
GzipWriter#write
- writes some compressed data
Hi,
On Jan 8, 8:23 pm, “Heesob P.” [email protected] wrote:
It’s about 600 lines, but needs cleaning up. Will probably be ~500 when
As long as it’s usable I’m happy. We can benchmark, profile and
optimize later.Finally I translated the whole zlib.c to ruby code.
You can download athttp://121.78.227.9/rbzlib/pzlib.rb
Looks very promising but doesn’t quite work yet:
C:>gem install ptools
ERROR: While executing gem … (NoMethodError)
undefined method `ZSTREAM_IS_FINISHED’ for #<Zlib::GzipReader:
0x2f8e8bc>
I think you need some tests. 
BTW, I’m assuming this is basically a wholesale replacement of zliby?
If so, the module should be renamed from PZlib to just Zlib.
Otherwise, Rubygems will complain about the lack of a Gem::Zlib
definition.
Keep up the good work!
Regards,
Dan
Daniel B. wrote:
Ok, I was able to download the project and I ran the specs:
C:\ruby\src\ruby-zlib>spec specs
…FF…
‘Zlib::Deflate#params changes the deflate parameters’ FAILED
expected: “x\001\000\v\000\364\377abcdefghijk\002,’\027\000#
\364\005<”,
got: “x\234KLJNIMK\317\310\314\312\316\311\005\000#
\364\005<” (using ==)
C:/ruby/src/ruby-zlib/specs/deflate/params_spec.rb:16
NameError in ‘Zlib::Deflate#set_dictionary sets the dictionary’
uninitialized constant Zlib::Deflate
C:/ruby/src/ruby-zlib/specs/deflate/set_dictionary_spec.rb:6
C:/ruby/src/ruby-zlib/specs/deflate/set_dictionary_spec.rb:5:in `each’
C:/ruby/src/ruby-zlib/specs/deflate/set_dictionary_spec.rb:5Finished in 1.688 seconds
38 examples, 2 failures
Ruby 1.8.6-p114 built with VC++ 8
Windows XP Pro
RSpec 1.1.11Regards,
Dan
Yeah the github download functionality seems broken for me most of the
time also… 
Anyway, the params failure is the only failure I get. Thats because I’ve
not implemented the NO_COMPRESSION mode yet, and is an easy fix. The
other error you’re getting is odd, as Zlib::Deflate is defined, and if
it weren’t none of the other deflate specs would have passed!
On Jan 8, 6:56 pm, “Charles L.” [email protected] wrote:
The big thing is just good specs, and the existing rubyspec ones could
use some work which will benefit all of us.I’ve added most of the remaining missing functionality to my version (ie
dynamic huffman tree’s etc), and added back in the inflate functionality
from zliby, and put the result up on github -GitHub - aquasync/ruby-zlib: Pure ruby zlib implementation
Ok, I was able to download the project and I ran the specs:
C:\ruby\src\ruby-zlib>spec specs
…FF…
‘Zlib::Deflate#params changes the deflate parameters’ FAILED
expected: “x\001\000\v\000\364\377abcdefghijk\002,'\027\000#
\364\005<”,
got: “x\234KLJNIMK\317\310\314\312\316\311\005\000#
\364\005<” (using ==)
C:/ruby/src/ruby-zlib/specs/deflate/params_spec.rb:16
NameError in ‘Zlib::Deflate#set_dictionary sets the dictionary’
uninitialized constant Zlib::Deflate
C:/ruby/src/ruby-zlib/specs/deflate/set_dictionary_spec.rb:6
C:/ruby/src/ruby-zlib/specs/deflate/set_dictionary_spec.rb:5:in `each’
C:/ruby/src/ruby-zlib/specs/deflate/set_dictionary_spec.rb:5
Finished in 1.688 seconds
38 examples, 2 failures
Ruby 1.8.6-p114 built with VC++ 8
Windows XP Pro
RSpec 1.1.11
Regards,
Dan
Hi,
2009/1/10 Daniel B. [email protected]:
Sounds like you’re on track. But, you’re in a race with Park H…
In my test, the deflation is very slow but the inflation is endurable.
definition.
Keep up the good work!
I fixed some typo and renamed PZlib to Zlib and pzlib.rb to zlib.rb
I succeeded “gem install ptools” with zlib.rb
You can download it at http://121.78.227.9/rbzlib/zlib.rb
Regards,
Park H.
Anyone knows any web application for bounties? I’m interested in
contribute to this one and also one for Readline!!!
Yeah I’m in for Readline
Note that highline has a getc for windows, and cmd is also readline
like, so I believe it is possible
-=r
As a note I get the same error message Luis does.
2009/1/10 Roger P. [email protected]:
I guess you are looking for ruby_readline [1] .
You can download ruby18-rreadline-0.5.4.tbz at [2].
Regards,
Park H.
[1] http://raa.ruby-lang.org/project/ruby_readline
[2]
http://www.filewatcher.com/m/ruby18-rreadline-0.5.4.tbz.13944.0.0.html
On Jan 10, 8:15 am, Heesob P. [email protected] wrote:
I guess you are looking for ruby_readline [1] .
Hello Park,
Actually I’m looking to an API-compatible replacement, which should
work with IRB and auto complete capabilities.
Not only that, but a solution that can work across platforms,
including Windows
There is a solution called RawLine [1] but is not API-compliant with
ReadLine.
Thank you again for your pointer.
Hi,
2009/1/10 Luis L. [email protected]:
There is a solution called RawLine [1] but is not API-compliant with
ReadLine.Thank you again for your pointer.
I found the pure Perl implementation of the readline libraries called
libterm-readline-perl-perl [1]
I hope some one could port it into Ruby.
If no one port it until the end of this month, I will port it.
BTW, I succeeded “gem build” as well as “gem install” with my zlib.rb
and rbzlib.rb.
You should test it with the latest zlib.rb and rbzlib.rb at [2]
[1]
http://www.filewatcher.com/b/ftp/ftp.linux.it/debian/pool/main/libt/libterm-readline-perl-perl.0.0.html
[2] http://121.78.227.9/rbzlib/
Regards,
Park H.
On Jan 9, 9:26 pm, “Charles L.” [email protected] wrote:
got: "x\234KLJNIMK\317\310\314\312\316\311\005\000#Finished in 1.688 seconds
Yeah the github download functionality seems broken for me most of the
time also…Anyway, the params failure is the only failure I get. Thats because I’ve
not implemented the NO_COMPRESSION mode yet, and is an easy fix. The
other error you’re getting is odd, as Zlib::Deflate is defined, and if
it weren’t none of the other deflate specs would have passed!Posted viahttp://www.ruby-forum.com/.
The work-in-progress of One-Click Installer:
Luis@KEORE (D:\Users\Luis\projects\oss\ruby-zlib.git)
$ spec specs
…F…
‘Zlib::Deflate#params changes the deflate parameters’ FAILED
expected: “x\001\000\v\000\364\377abcdefghijk\002,'\027\000#
\364\005<”,
got: “x\234KLJNIMK\317\310\314\312\316\311\005\000#
\364\005<” (using ==)
D:/Users/Luis/projects/oss/ruby-zlib.git/specs/deflate/params_spec.rb:
16
Finished in 1.219 seconds
38 examples, 1 failure
Luis@KEORE (D:\Users\Luis\projects\oss\ruby-zlib.git)
$ ruby -v
ruby 1.8.6 (2008-03-03 patchlevel 114) [i386-mingw32]
Luis@KEORE (D:\Users\Luis\projects\oss\ruby-zlib.git)
$ spec --version
rspec 1.1.11.6
Not bad, really not bad!
Anyone knows any web application for bounties? I’m interested in
contribute to this one and also one for Readline!!!
Regards,
On Jan 10, 2:47 pm, Heesob P. [email protected] wrote:
like, so I believe it is possible
I found the pure Perl implementation of the readline libraries called
libterm-readline-perl-perl [1]I hope some one could port it into Ruby.
If no one port it until the end of this month, I will port it.
Well, I don’t have the time, but I can give some contribution for it,
so 
BTW, I succeeded “gem build” as well as “gem install” with my zlib.rb
and rbzlib.rb.
You should test it with the latest zlib.rb and rbzlib.rb at [2]
I’ve run the specs of Zlib for your code:
Finished in 0.328 seconds
38 examples, 25 failures
On 10/01/2009, Heesob P. [email protected] wrote:
like, so I believe it is possible
work with IRB and auto complete capabilities.I found the pure Perl implementation of the readline libraries called
libterm-readline-perl-perl [1]I hope some one could port it into Ruby.
If no one port it until the end of this month, I will port it.
I think the rreadline implementation is compatible. I once tried it
because the readline on Windows was broken, and aside for renaming
something to make irb find it it worked for me.
Thanks
Michal
On 10/01/2009, Luis L. [email protected] wrote:
so
Finished in 0.328 seconds
38 examples, 25 failures
I guess it’s expected. Different implementations give different
compression results.
Of course, the spec might be more high-level but then an
implementation that does not compress at all would likely work as
well.
Thanks
Michal
2009/1/11 Luis L. [email protected]:
contribute to this one and also one for Readline!!!
Hello Park,
Thank you again for your pointer.
soIt should implement the exact same compression strategies. If not, is
not a “drop-in” replacement.Of course, the spec might be more high-level but then an
implementation that does not compress at all would likely work as
well.The spec is being used across IronRuby, Rubinius, JRuby and the Ruby C
(MRI). So the pure-ruby replacement must comply.
At last, my zlib.rb and rbzlib.rb passed rubyspec zlib test
As before, you can download at http://121.78.227.9/rbzlib/
Finished in 1.323951 seconds
77 files, 38 examples, 88 expectations, 0 failures, 0 errors
Regards,
Park H.
On Jan 9, 4:26 pm, “Charles L.” [email protected] wrote:
got: "x\234KLJNIMK\317\310\314\312\316\311\005\000#Finished in 1.688 seconds
Yeah the github download functionality seems broken for me most of the
time also…Anyway, the params failure is the only failure I get. Thats because I’ve
not implemented the NO_COMPRESSION mode yet, and is an easy fix. The
other error you’re getting is odd, as Zlib::Deflate is defined, and if
it weren’t none of the other deflate specs would have passed!
Ok, I tried again. Reading seemed to work fine. For writing it worked
fine for small libs, but when I tried it against my largest project,
windows-pr, I got this:
C:\Documents and Settings\djberge\workspace\windows-pr>ruby windows-
pr.gemspec
C:/ruby/lib/ruby/site_ruby/1.8/zlib/deflate.rb:859:in log': Result too large - log (Errno::ERANGE) from C:/ruby/lib/ruby/site_ruby/1.8/zlib/deflate.rb:859:in log2’
from C:/ruby/lib/ruby/site_ruby/1.8/zlib/deflate.rb:901:in
chooseblock' from C:/ruby/lib/ruby/site_ruby/1.8/zlib/deflate.rb:882:in times’
from C:/ruby/lib/ruby/site_ruby/1.8/zlib/deflate.rb:882:in
chooseblock' from C:/ruby/lib/ruby/site_ruby/1.8/zlib/deflate.rb:613:in outsym’
from C:/ruby/lib/ruby/site_ruby/1.8/zlib/deflate.rb:575:in
match' from C:/ruby/lib/ruby/site_ruby/1.8/zlib/deflate.rb:430:in lz77_compress’
from C:/ruby/lib/ruby/site_ruby/1.8/zlib/deflate.rb:490:in
deflate' ... 10 levels... from C:/ruby/lib/ruby/site_ruby/1.8/rubygems/builder.rb:68:in open’
from C:/ruby/lib/ruby/site_ruby/1.8/rubygems/builder.rb:68:in
write_package' from C:/ruby/lib/ruby/site_ruby/1.8/rubygems/builder.rb:37:in build’
from windows-pr.gemspec:25
It’s about 60 files. If you’d like to try windows-pr yourself you can
grab it via:
cvs -d :pserver:[email protected]:/var/cvs/win32utils login
cvs -d :pserver:[email protected]:/var/cvs/win32utils checkout
Regards,
Dan