Thinking of dropping YAML from 1.8

Hello all.

As a branch manager I think I cannot maintain YAML lib any longer. No
one had touched lib/yaml for more than 20 months, while bugs are
reported continuously during that. At this point I cannot but say YAML
has already been abandoned. And I cannot maintain this large library by
myself. It’s a real sadness, but it’s time for us to drop it from
stdlib, unless someone put in for the maintenance.

On 11/3/07, Urabe S. [email protected] wrote:

As a branch manager I think I cannot maintain YAML lib any longer.

The whole point of a maintenance branch is “something that should not
have breaking changes”, isn’t it? If you remove YAML from 1.8 at this
point, you will create a lot of grief for a lot of people, because
most production Ruby apps have dependency on YAML (Rails uses it).

If nobody steps up as a maintainer, please, please, please just leave
it alone, with bugs and all.

On Nov 3, 2007, at 3:47 PM, Alexey V. wrote:

On 11/3/07, Urabe S. [email protected] wrote:

As a branch manager I think I cannot maintain YAML lib any longer.

The whole point of a maintenance branch is “something that should not
have breaking changes”, isn’t it? If you remove YAML from 1.8 at this
point, you will create a lot of grief for a lot of people, because
most production Ruby apps have dependency on YAML (Rails uses it).

I have to agree that this is an extremely popular library. This is a
very large change that will affect a lot of software.

James Edward G. II

where to start … to fix the YAML code bugs

Joel VanderWerf wrote:

[#9518] YAML: Derived String and binary data. [verified]

Here’s a quick fix (also posted on the bug page):

— rubytypes.rb.bck 2007-11-03 15:53:56.000000000 -0700
+++ rubytypes.rb 2007-11-03 15:53:56.000000000 -0700
@@ -163,7 +163,7 @@
def to_yaml( opts = {} )
YAML::quick_emit( is_complex_yaml? ? object_id : nil,
opts ) do |out|
if is_binary_data?

  •            out.scalar( "tag:yaml.org,2002:binary", 
    

[self].pack(“m”), :literal )

  •            out.scalar( taguri.sub(/:str:/, ":binary:"), 
    

[self].pack(“m”), :literal )
elsif to_yaml_properties.empty?
out.scalar( taguri, self, self =~ /^:/ ? :quote2 :
to_yaml_style )
else

Btw, there is also a bug with binary strings that have instance
variables, but one at a time…

Ujwal Reddy Malipeddi wrote:

where to start … to fix the YAML code bugs

Pick one!

Open yaml bugs, and [status in 1.8.6p111]:

[#3698] In a highly dynamic ObjectSpace (loads of created and collected
objects) YAML creates erroneous anchors/references [? related to #8548]

[#6819] YAML loses data [verified]

[#7787] YAML.parse fails to recognize document terminator when it
follows a multiline scalar [verified]

[#8026] YAML fails to parse its own input [verified]

[#8548] YAML::Omap loses entries, replacing them with random references
[verified]

[#8886] YAML fails to load a Hash containing a Pathname [verified]

[#9053] YAML.load(file) cannot be used to load documents in sequence.
[verified]

[#9518] YAML: Derived String and binary data. [verified]

[#12761] Yaml doesn’t load large DateTimes [verified]

[#12847] class Module in lib/yaml/tag.rb removes Module from RDoc HTML
documentation output [verified, but solution provided, and this looks
like a slam dumk]

For details, go to

http://rubyforge.org/search/advanced_search.php?group_id=426

and search for YAML.

Maybe it would be better to move YAML out, so that it will give
someone the impetus to do a new and better lib (like one that’s 1.1
compliant for instance!). It’s too bad _why has abandoned his Syck
project, it really is one of the most useful standard libs in Ruby.

On the other hand, I personally think we should consider YAML as an
essential part of Ruby. As _why originally put it, YAML and Rubygo
together like bread and butter. I require ‘yaml’ in most of my
programs, even if it’s just to use #y instead of #p. I’d go so far as
to say that YAML would do well even further integrated into the
language. For instance, how cool would it be to do:

class X
attr_accessor :msg
def sayit; puts msg; end
end

foo = — !X
msg: “Hello, World!”

foo.sayit

Whether you like that idea or not. I think it remains true that YAML
is so important to Ruby, that it’s support should be treated with the
same diligence as the rest of the language.

T.

Joel VanderWerf wrote:

            YAML::quick_emit( is_complex_yaml? ? object_id : nil, 

Btw, there is also a bug with binary strings that have instance
variables, but one at a time…

Sorry, this is the corrected patch.

— rubytypes.rb.bck 2007-11-03 16:22:38.000000000 -0700
+++ rubytypes.rb 2007-11-03 16:22:38.000000000 -0700
@@ -146,7 +146,7 @@
( self.count( “^ -~”, “^\r\n” ) / self.size > 0.3 ||
self.count( “\x00” ) > 0 ) unless empty?
end
def String.yaml_new( klass, tag, val )

  •    val = val.unpack("m")[0] if tag == "tag:yaml.org,2002:binary"
    
  •    val = val.unpack("m")[0] if /\Atag:yaml.org,2002:binary/ =~ tag
        val = { 'str' => val } if String === val
        if Hash === val
            s = klass.allocate
    

@@ -163,7 +163,7 @@
def to_yaml( opts = {} )
YAML::quick_emit( is_complex_yaml? ? object_id : nil,
opts ) do |out|
if is_binary_data?

  •            out.scalar( "tag:yaml.org,2002:binary", 
    

[self].pack(“m”), :literal )

  •            out.scalar( taguri.sub(/:str/, ":binary"), 
    

[self].pack(“m”), :literal )
elsif to_yaml_properties.empty?
out.scalar( taguri, self, self =~ /^:/ ? :quote2 :
to_yaml_style )
else

On 11/3/07, Urabe S. [email protected] wrote:

As a branch manager I think I cannot maintain YAML lib any longer. No
one had touched lib/yaml for more than 20 months, while bugs are
reported continuously during that. At this point I cannot but say YAML
has already been abandoned. And I cannot maintain this large library by
myself. It’s a real sadness, but it’s time for us to drop it from
stdlib, unless someone put in for the maintenance.

Removal of YAML will break RubyGems, at least 0.9.4.

./rubygems/builder.rb
./rubygems/commands/build_command.rb
./rubygems/commands/specification_command.rb
./rubygems/config_file.rb
./rubygems/incremental_fetcher.rb
./rubygems/old_format.rb
./rubygems/package.rb
./rubygems/remote_fetcher.rb
./rubygems/remote_installer.rb
./rubygems/server.rb
./rubygems/source_index.rb
./rubygems/source_info_cache_entry.rb
./rubygems/specification.rb

More important that we find someone to maintain it if _why is
unwilling to do so.

-austin

I’am reviewing the code … YAML 1.1 spec

Joel VanderWerf wrote:

[#6819] YAML loses data [verified]

For details, go to

http://rubyforge.org/search/advanced_search.php?group_id=426

and search for YAML.

There are also two YAML related patches in the queue:

http://rubyforge.org/search/?type_of_search=artifact&group_id=426&atid=1700&words=yaml&Search=Search

Regards,

Dan

On Nov 4, 2007 11:47 AM, Daniel B. [email protected] wrote:

There are also two YAML related patches in the queue:

http://rubyforge.org/search/?type_of_search=artifact&group_id=426&atid=1700&words=yaml&Search=Search

I personally think YAML needs a rewrite, not a removal. The existing
implementation uses a huge amount of memory while parsing, and doesn’t
support the 1.1 spec.

Anyone interested in setting up a bounty? I am too busy with other
projects to write it myself, at the moment.

Wilson B. wrote:

support the 1.1 spec.

Anyone interested in setting up a bounty? I am too busy with other
projects to write it myself, at the moment.

I’m not going to take that bounty. A full YAML implementation is
definitely a very large project (I know…). JRuby used to have 1.1
compatibility but that was crippled to make it more correct from a
1.8-standard.

At this point, the best way would probably be to integrate Kirill
Simonovs libyaml project, but I’m not sure how far that has come.

And sadly, the structure of YAML as a language more or less requires
quite alot of memory usage. (You should see the JvYAML memory usage…
=(

Cheers


Ola B. (http://ola-bini.blogspot.com)
JRuby Core Developer
Developer, ThoughtWorks Studios (http://studios.thoughtworks.com)
Practical JRuby on Rails (http://apress.com/book/view/9781590598818)

“Yields falsehood when quined” yields falsehood when quined.

On Sun, Nov 04, 2007 at 05:29:42AM +0900, Urabe S. wrote:

As a branch manager I think I cannot maintain YAML lib any longer. No
one had touched lib/yaml for more than 20 months, while bugs are
reported continuously during that.

I want to say thankyou to shyouhei (and to matz, nobu, akr, many
others) who have helped patch YAML and improve it. I am certainly
okay if YAML is discontinued due to my unreliability over the past
year. And it’s just not very good code any more.

I would like another shot at improving it, if I can. I will spend
some days this week, gladly. Would that be okay?

_why

On Nov 4, 11:48 pm, _why [email protected] wrote:

I would like another shot at improving it, if I can. I will spend
some days this week, gladly. Would that be okay?

+1

Hopefully you’ve talked to matz and friends via other channels. I
can’t imagine they wouldn’t be more than happy to see you update this
library. I am.

T.