Forum: Ruby-core merging nokogiri to ext/

I would like to merge nokogiri to ext for the 1.9.3 release.  I spoke to
various people about this (including other ruby-core people) and
everyone seems to think it's a good idea.

May I go forward with committing nokogiri to trunk under ext?

After merging it to trunk, can we give Mike Dalession commit access?  He 
helps
me maintain nokogiri and I couldn't continue without him.

Hello,

In message "[ruby-core:32009] merging nokogiri to ext/"
    on Sep.02,2010 17:58:24, <aaron@tenderlovemaking.com> wrote:
> I would like to merge nokogiri to ext for the 1.9.3 release.  I spoke to
> various people about this (including other ruby-core people) and
> everyone seems to think it's a good idea.

!?


> May I go forward with committing nokogiri to trunk under ext?

What is the reason to bundle nokogiri?

I think that we should not increase bundled libraries, because
we already have rubygems as standard library installer.


Regards,

On 2010-09-02 18:07:31 +0900, U.Nakamura wrote:
> > May I go forward with committing nokogiri to trunk under ext?
> 
> What is the reason to bundle nokogiri?
> 
> I think that we should not increase bundled libraries, because
> we already have rubygems as standard library installer.

same thought. nokogiri as gem is fine.

    darix

On Sep 2, 2010, at 02:07 , U.Nakamura wrote:

>> May I go forward with committing nokogiri to trunk under ext?
> 
> What is the reason to bundle nokogiri?
> 
> I think that we should not increase bundled libraries, because
> we already have rubygems as standard library installer.

Then can we please unbundle rexml?

Hello,

In message "[ruby-core:32012] Re: merging nokogiri to ext/"
    on Sep.02,2010 18:34:34, <ryand-ruby@zenspider.com> wrote:
> > I think that we should not increase bundled libraries, because
> > we already have rubygems as standard library installer.
> 
> Then can we please unbundle rexml?

I do not oppose to unbundle rexml.
However, we cannot unbundle already bundled library in 1.9
series because of compatibility.


Regards,

On Thu, Sep 2, 2010 at 6:48 AM, U.Nakamura <usa@garbagecollect.jp> 
wrote:
> However, we cannot unbundle already bundled library in 1.9
> series because of compatibility.
>

As we use 1.9.1 as version compatibility, we can make it 1.9.3 and
break it, right?

That way gems installed will not be considered the same.

Is worth mentioning that lot of folks had issues when upgraded to
1.9.2 from 1.9.1 as the library compatibility was the same.

> Then can we please unbundle rexml?

+1

On Thu, Sep 2, 2010 at 5:34 AM, Ryan Davis <ryand-ruby@zenspider.com> 
wrote:
> On Sep 2, 2010, at 02:07 , U.Nakamura wrote:
>>> May I go forward with committing nokogiri to trunk under ext?
>>
>> What is the reason to bundle nokogiri?
>>
>> I think that we should not increase bundled libraries, because
>> we already have rubygems as standard library installer.
>
> Then can we please unbundle rexml?

What is the reason to unbundle rexml?

> What is the reason to unbundle rexml?

The main reason is that it has no maintainer.

On Thu, Sep 2, 2010 at 6:29 PM, Vincent Isambart
<vincent.isambart@gmail.com> wrote:
>> What is the reason to unbundle rexml?
>
> The main reason is that it has no maintainer.

Is this page out of date then?
http://redmine.ruby-lang.org/wiki/8/Maintainers

On Sep 2, 2010, at 13:47 , brabuhr@gmail.com wrote:

> 
> What is the reason to unbundle rexml?

primarily: usability, speed, and maintenance.

secondarily: nokogiri is a much better tool and more people will use it 
if the playing field is level to begin with... (think of all the hours 
saved from porting rexml code over to nokogiri).

We spend a lot of time on IRC fielding ("unnecessary") questions about 
rexml. I'd like to see that go away in the future.

2010/9/3 Ryan Davis <ryand-ruby@zenspider.com>:
>>>> we already have rubygems as standard library installer.
>>>
>>> Then can we please unbundle rexml?
>>
>> What is the reason to unbundle rexml?
>
> primarily: usability, speed, and maintenance.
>
> secondarily: nokogiri is a much better tool and more people will use it if the playing field is level to begin with... (think of all the hours saved from porting rexml code over to nokogiri).
>
> We spend a lot of time on IRC fielding ("unnecessary") questions about rexml. I'd like to see that go away in the future.

What is the question for example?

Anyway you can simply answer to use nokogiri, can't you?

Hi,

2010/9/2 U.Nakamura <usa@garbagecollect.jp>:
> In message "[ruby-core:32009] merging nokogiri to ext/"
>    on Sep.02,2010 17:58:24, <aaron@tenderlovemaking.com> wrote:
>> I would like to merge nokogiri to ext for the 1.9.3 release.  I spoke to
>> various people about this (including other ruby-core people) and
>> everyone seems to think it's a good idea.
>
> !?

I answered nokogiri's dependency with libxml is a problem on some 
environments.
And if Ruby bundles nokogiri, its update is restricted by Ruby's 
schedule.

>> May I go forward with committing nokogiri to trunk under ext?
>
> What is the reason to bundle nokogiri?
>
> I think that we should not increase bundled libraries, because
> we already have rubygems as standard library installer.

I agree with this.

On Sep 2, 2010, at 22:34 , NARUSE, Yui wrote:

> I answered nokogiri's dependency with libxml is a problem on some environments.
> And if Ruby bundles nokogiri, its update is restricted by Ruby's schedule.

Isn't that also true of ffi's dependency on libffi?

2010/9/3 Ryan Davis <ryand-ruby@zenspider.com>:
>
> On Sep 2, 2010, at 22:34 , NARUSE, Yui wrote:
>
>> I answered nokogiri's dependency with libxml is a problem on some environments.
>> And if Ruby bundles nokogiri, its update is restricted by Ruby's schedule.
>
> Isn't that also true of ffi's dependency on libffi?

Yes, it is open problem on Windows.
Why fiddle is accepted is dl2 can't work well on x86_64 and it can't be 
fixed.

The problem is described following in Japanese but most quotation is in 
English.
http://mkosaki.blog46.fc2.com/blog-entry-1069.html

So we needed another impl.


Well, we could obsolete dl and recommend ffi gem.
It is discussed just before 1.9.2 release but it's too late.

2010/9/3 NARUSE, Yui <naruse@airemix.jp>:
> 2010/9/2 U.Nakamura <usa@garbagecollect.jp>:
>> I think that we should not increase bundled libraries, because
>> we already have rubygems as standard library installer.
>
> I agree with this.

We may have to do more explanation, it is Big Ruby-ism vs. Small 
Ruby-ism.
Big Ruby-ism thinks that Ruby package should be all-in-one package for
common use.
Small Ruby-ism thinks that Ruby package should be as small as it can
(bundle libs depended by RubyGems).

2010/9/3 Vincent Isambart <vincent.isambart@gmail.com>:
>> What is the reason to unbundle rexml?
>
> The main reason is that it has no maintainer.

Kouhei Sutou is the maintainer of REXML.

On 2010/09/03 16:27, NARUSE, Yui wrote:

> We may have to do more explanation, it is Big Ruby-ism vs. Small Ruby-ism.
> Big Ruby-ism thinks that Ruby package should be all-in-one package for
> common use.
> Small Ruby-ism thinks that Ruby package should be as small as it can
> (bundle libs depended by RubyGems).

As far as I understand, to some extent the purpose of integrating
rubygems into Ruby itself (which made Ruby considerably bigger) was
actually to make it easier to have less bundled libraries (i.e. small 
Ruby).

Regards,   Martin.

On Sep 2, 2010, at 23:43 , NARUSE, Yui wrote:

> Well, we could obsolete dl and recommend ffi gem.
> It is discussed just before 1.9.2 release but it's too late.

Sorry. That's not really the direction I was trying to steer us... I 
figured if libffi was allowable, then libxml (much more available) would 
be too.

But I was really mostly focusing on the fact that we get a lot more 
people needing help with rexml than we do nokogiri (helped in part by 
the #nokogiri channel and mailing lists).

Far more people need help with rexml than they do with dl :)

(surprising, really)

On Thu, Sep 02, 2010 at 06:07:31PM +0900, U.Nakamura wrote:
> Hello,
> 
> In message "[ruby-core:32009] merging nokogiri to ext/"
>     on Sep.02,2010 17:58:24, <aaron@tenderlovemaking.com> wrote:
> > I would like to merge nokogiri to ext for the 1.9.3 release.  I spoke to
> > various people about this (including other ruby-core people) and
> > everyone seems to think it's a good idea.
> 
> !?

eh?  Did I make a mistake?  Nobody wants it in stdlib?

> > May I go forward with committing nokogiri to trunk under ext?
> 
> What is the reason to bundle nokogiri?

REXML has many faults.  I can go in to detail with each, but I *think*
most people (who are not new to Ruby) know about them.

I think it is important that Ruby ships with a well maintained,
performant, easy to use, standard abiding XML parsing library.  I 
believe
nokogiri surpasses REXML in all of these fields.

It seems to me that many new users of Ruby start with REXML, then when
they find the problems they must pay the price of porting code from
REXML to nokogiri.  I think it would be a nicer experience for
new-comers to ruby.

> I think that we should not increase bundled libraries, because
> we already have rubygems as standard library installer.

I agree.  We should convert all stdlib to gems.  Now that we have gem
prelude, what is the point of putting things in stdlib?

On Fri, Sep 03, 2010 at 04:27:07PM +0900, NARUSE, Yui wrote:
> Small Ruby-ism thinks that Ruby package should be as small as it can
> (bundle libs depended by RubyGems).

I think there is a third option: Medium Ruby-ism.  Convert all of stdlib
to be gems, and have Ruby ship with a default set of gems.

After that, we can talk about which should be default.  But the
advantage is that we can remove ones we don't want, or upgrade old one
without upgrading Ruby.

On Fri, Sep 03, 2010 at 02:34:09PM +0900, NARUSE, Yui wrote:
> 
> I answered nokogiri's dependency with libxml is a problem on some environments.
> And if Ruby bundles nokogiri, its update is restricted by Ruby's schedule.

Is this true?  rdoc, rubygems, (in past years soap4r) all have regular
release schedules outside of Ruby's release schedule.  Please note the
release dates are not the same as Ruby's:

  http://rubygems.org/gems/rubygems-update
  http://rubygems.org/gems/rdoc
  http://rubygems.org/gems/soap4r

> Well, we could obsolete dl and recommend ffi gem.
> It is discussed just before 1.9.2 release but it's too late.

+1

you could deprecate it in 1.9.3 :)

(2010/09/03 21:58), Aaron Patterson wrote:
> On Fri, Sep 03, 2010 at 04:27:07PM +0900, NARUSE, Yui wrote:
>> 2010/9/3 NARUSE, Yui<naruse@airemix.jp>:
>>> 2010/9/2 U.Nakamura<usa@garbagecollect.jp>:
>>>> I think that we should not increase bundled libraries, because
>>>> we already have rubygems as standard library installer.
>>>
>>> I agree with this.
>>
>> We may have to do more explanation, it is Big Ruby-ism vs. Small Ruby-ism.
>> Big Ruby-ism thinks that Ruby package should be all-in-one package for
>> common use.
>> Small Ruby-ism thinks that Ruby package should be as small as it can
>> (bundle libs depended by RubyGems).
> 
> I think there is a third option: Medium Ruby-ism.  Convert all of stdlib
> to be gems, and have Ruby ship with a default set of gems.

Libraries depended by gem, test or rdoc must be bundled.
Most of current libraries are such things.

> What is the reason to unbundle rexml?

The complaint I have heard once is if people had rexml code from 1.8,
and then upgrade to 1.9, they were no longer able to build the same
XML output as they used to be able to under 1.8.  Since there was no
gem versioning, it was harder to go back than it would have been had
it been a gem.  I for one have no idea how to change the version of a
bundled library.

>> What is the reason to bundle nokogiri?
>
> REXML has many faults.  I can go in to detail with each, but I *think*
> most people (who are not new to Ruby) know about them.
>
> I think it is important that Ruby ships with a well maintained,
> performant, easy to use, standard abiding XML parsing library.  I believe
> nokogiri surpasses REXML in all of these fields.

That is true.  So I guess it makes sense to bundle nokogiri (as a gem)
instead of including rexml, as it would make the "std lib" more
robust.  Dunno if the "extra dependencies" issue is too bit though...

-r

On Fri, Sep 3, 2010 at 3:43 AM, NARUSE, Yui <naruse@airemix.jp> wrote:
>>
>> Isn't that also true of ffi's dependency on libffi?
>
> Yes, it is open problem on Windows.
> Why fiddle is accepted is dl2 can't work well on x86_64 and it can't be fixed.
>
> The problem is described following in Japanese but most quotation is in English.
> http://mkosaki.blog46.fc2.com/blog-entry-1069.html
>
> So we needed another impl.
>

Hello,

As part of RubyInstaller we compile libffi statically and successfully
build Fiddle for 1.9.2 release.

I don't see a problem with that.

>
> Well, we could obsolete dl and recommend ffi gem.
> It is discussed just before 1.9.2 release but it's too late.
>

That will be very problematic at least for WIn32API, as usage of it is
discouraged and usage of DL is recommended, then without DL users will
no longer have choices.

(2010/09/03 0:05), Luis Lavena wrote:
> On Thu, Sep 2, 2010 at 6:48 AM, U.Nakamura<usa@garbagecollect.jp>  wrote:
>> In message "[ruby-core:32012] Re: merging nokogiri to ext/"
>>     on Sep.02,2010 18:34:34,<ryand-ruby@zenspider.com>  wrote:
>>>> I think that we should not increase bundled libraries, because
>>>> we already have rubygems as standard library installer.
>>>
>>> Then can we please unbundle rexml?
>>
>> I do not oppose to unbundle rexml.
>> However, we cannot unbundle already bundled library in 1.9
>> series because of compatibility.
>>
>
> As we use 1.9.1 as version compatibility, we can make it 1.9.3 and
> break it, right?

We can't break it in API version 1.9.1.

Moreover I think, we can't removing library in 1.9 series.

Hi,

2010/9/3 Aaron Patterson <aaron@tenderlovemaking.com>:
>> I answered nokogiri's dependency with libxml is a problem on some environments.
>> And if Ruby bundles nokogiri, its update is restricted by Ruby's schedule.
>
> Is this true?  rdoc, rubygems, (in past years soap4r) all have regular
> release schedules outside of Ruby's release schedule.  Please note the
> release dates are not the same as Ruby's:


I really, really hate separate releases.  It leads to synchronization
problem.  See dismal states of Rdoc and Rubygems:

  - [ruby-core:26679]
  - [ruby-core:31499]
  - [ruby-core:31503]

Even so, I can't suggest to unbundle them, because Rdoc and RubyGems
are needed and actually used for *ruby itself*.

Nokogiri is not.  If separate releases is preserved, I'll vote "no"
against Nokogiri.

Hi,

2010/9/3 Aaron Patterson <aaron@tenderlovemaking.com>:
> I think there is a third option: Medium Ruby-ism. ?Convert all of stdlib
> to be gems, and have Ruby ship with a default set of gems.


I like Medium Ruby-ism too, but it will cause dual gem repositories
in effect (blessed gems and others).
I suggested official gem repository [ruby-core:26388], but I received
many objections.

If we had the blessed repository, Nokogiri would be imported there
without dissent ;-(

You want to talk about REXML?

Hi,
On 2 September 2010 11:34, Ryan Davis <ryand-ruby@zenspider.com> wrote:
> Then can we please unbundle rexml?

I agree we should unbundle rexml (unless it suddenly becomes quite 
active),
 and I think we should even do it if Nokogiri is not merged to ext/.

The reasons were mentioned by others.
It is some kind of a "black hole" in the stdlib, which newcomers
easily fall into because there are no guards.
 (I myself was one of those and tried to make my own parser)

If it is unbundle, they will at least have to search for a decent one,
and probably will find something better.
Letting it there is giving some advice like "use this, it must be
good, it's stdlib".
If Nokogiri was not merged, then the least would be to put a note
about HTML/XML parsing libraries.

It would of course be even easier if Nokogiri was made the default.

I agree with Medium Ruby-ism (but maybe let stdlib in peace for the 
moment).
Probably the best for now is to just automatically install Nokogiri as 
gem.
That way it can keep update and newcomers have a good HTML/XML parsing 
library.

Regards,
B.D.

P.S.: I am not very aware of the details, but AFAIK linking Nokogiri
with a old buggy version of libxml2 is rather bad, and it could then
build a Ruby with a buggy library, which only solution would to
install newer libxml2 and reinstall Ruby.
The gem would be a better solution for this reason.

On Sun, Sep 05, 2010 at 12:17:03AM +0900, Yusuke ENDOH wrote:
> Hi,
> 
> 2010/9/3 Aaron Patterson <aaron@tenderlovemaking.com>:
> > I think there is a third option: Medium Ruby-ism. ?Convert all of stdlib
> > to be gems, and have Ruby ship with a default set of gems.
> 
> 
> I like Medium Ruby-ism too, but it will cause dual gem repositories
> in effect (blessed gems and others).

If "blessed" means "comes with ruby by default" then I agree.  But I
don't understand why it would cause dual gem repositories.

On Fri, Sep 3, 2010 at 19:33, Luis Lavena <luislavena@gmail.com> wrote:

> That will be very problematic at least for WIn32API, as usage of it is
> discouraged and usage of DL is recommended, then without DL users will
> no longer have choices.

One thing’s for sure: Win32API must die.

On Sun, Sep 05, 2010 at 02:37:18PM +0900, NARUSE, Yui wrote:
> (2010/09/03 21:58), Aaron Patterson wrote:
> > On Fri, Sep 03, 2010 at 04:27:07PM +0900, NARUSE, Yui wrote:
> >> 2010/9/3 NARUSE, Yui<naruse@airemix.jp>:
> >>> 2010/9/2 U.Nakamura<usa@garbagecollect.jp>:
> >>>> I think that we should not increase bundled libraries, because
> >>>> we already have rubygems as standard library installer.
> >>>
> >>> I agree with this.
> >>
> >> We may have to do more explanation, it is Big Ruby-ism vs. Small Ruby-ism.
> >> Big Ruby-ism thinks that Ruby package should be all-in-one package for
> >> common use.
> >> Small Ruby-ism thinks that Ruby package should be as small as it can
> >> (bundle libs depended by RubyGems).
> > 
> > I think there is a third option: Medium Ruby-ism.  Convert all of stdlib
> > to be gems, and have Ruby ship with a default set of gems.
> 
> Libraries depended by gem, test or rdoc must be bundled.
> Most of current libraries are such things.

I don't understand.  Because of gem prelude, even if each library in
stdlib was shipped as a gem, user code would not change.

We can convert all libraries in stdlib to gems and it would not change
user code.  Then we ship all of those gems with Ruby by default.

On Sep 4, 2010, at 10:11 , Yusuke ENDOH wrote:

> I really, really hate separate releases.  It leads to synchronization
> problem.  See dismal states of Rdoc and Rubygems:
> 
>  - [ruby-core:26679]
>  - [ruby-core:31499]
>  - [ruby-core:31503]

That isn't a problem with separate releases. It is a problem with dual 
VCs. This is why I don't allow changes to minitest in ruby's repo and 
insist on patches being sent instead. If rubygems and rdoc did the same 
thing then we wouldn't have those problems. We'd still have problems, 
they'd just be easier to manage.

On Sep 4, 2010, at 3:19 PM, Benoit Daloze wrote:

> It would of course be even easier if Nokogiri was made the default.

Yeah, I'm surprised to see so little support for Nokogiri.  I would be 
very for adding it.

We have a standard library for working with CSV and I cannot believe 
that's a bigger need than XML.  We added JSON in 1.9, which is awesome, 
but XML is still probably used in more places.  I'm finding it hard to 
accept that we want those and we don't want a good XML tool.

James Edward Gray II

2010/9/4 Nikolai Weibull <now@bitwi.se>:
> On Fri, Sep 3, 2010 at 19:33, Luis Lavena <luislavena@gmail.com> wrote:
>
>> That will be very problematic at least for WIn32API, as usage of it is
>> discouraged and usage of DL is recommended, then without DL users will
>> no longer have choices.
>
> One thing’s for sure: Win32API must die.

Well, that's offtopic ;)

On Sat, Sep 4, 2010 at 11:51 PM, Ryan Davis <ryand-ruby@zenspider.com> 
wrote:
>
> On Sep 4, 2010, at 10:11 , Yusuke ENDOH wrote:
>
>> I really, really hate separate releases.  It leads to synchronization
>> problem.  See dismal states of Rdoc and Rubygems:
>>
>>  - [ruby-core:26679]
>>  - [ruby-core:31499]
>>  - [ruby-core:31503]
>
> That isn't a problem with separate releases. It is a problem with dual VCs. This is why I don't allow changes to minitest in ruby's repo and insist on patches being sent instead. If rubygems and rdoc did the same thing then we wouldn't have those problems. We'd still have problems, they'd just be easier to manage.

This might be a little OT because it's not specific to the nokogiri
in/out of stdlib/ext, but I believe there are actually two problems in
the current situation.

The first problem, as Ryan  mentions here, is the "dual VC": the same
(sub)tree is tracked in two separate repositories, stand-alone in one
case and as part of the main distribution in the other. This means for
example that bugs can get fixed in one repo and not in the other.

The second problem occurs user-side when the user might want to update
a single component of stdlib without installing a new version of ruby
altogether (e.g. because that particular part of stdlib had a minor
update that fixes a bug that is important to him and no new release is
avaiable). Note that this second problem is disjoint from the dual VC
one because it would occur even if you had disjoint repositories for
the interprenter and for each component of the stdlib.

I will start with the second issue by suggesting that a solution would
be packaging every single component of stdlib as a gem: each Ruby
release would then have the interpreter and a given set of packages at
given versions (the stdlib components), and the user would still have
the possibility to update single components individually (via gem)
whenever a new version comes out.

The problem of the dual VC can probably be solved as well, although in
this case the way to do it is obviously dependent on the revision
control system that is being used.

For example, the repository hosting the git scm includes two
components (gitk and git-gui) that are _also_ tracked separately in
stand-alone repositories. Periodically, the git repository imports any
updates from the stand-alone gitk and git-gui projects, and conversely
the gitk and git-gui project can merge changes that were only
submitted to the git project. Of course, all of this criss-cross
updating is possible and easy because git allows these kinds of crazy
stuff, but I gather that something similar can also be achieved in
SVN, possibly with appropriate use of externals, or even just by
exploiting the ease with which single subdirectories can be isolated
from a svn repository.

In a way, this fits also in-between the "big ruby" and "small ruby"
debate: by clearly separating each stdlib component _and_ at the
source level (e.g. by periodically merging the corresponding
stand-alone repository, and conversely having the stand-alone
repository pull in changes from the ruby repo) _and_ at the
installation level (everything is a rubygem so it can be updated
independently from the main ruby package if necessary) we would have
the benefits of being able to include whatever components are deemded
necessary in stdlib without forcing any kind of "version coupling"
between the interpreter and the stdlib components. It would also make
it possible to manage "different sizes of ruby": it would be possible
to define a "minimal ruby" that only has the interpreter and a very
limited set of selected packages (if any at all), a medium ruby with a
decent-size stdlib and a "full ruby" with a bunch of extras, and the
only diffference between them would be the (now modular) components
included in the packaged stdlib.

Do these ideas make sense?

Hi,

2010/9/6 Aaron Patterson <aaron@tenderlovemaking.com>:
>> I like Medium Ruby-ism too, but it will cause dual gem repositories
>> in effect (blessed gems and others).
>
> If "blessed" means "comes with ruby by default" then I agree. ?But I
> don't understand why it would cause dual gem repositories.

Because rubygems.org is not operated by the core team.
The advantage of stdlib is not only "comes with ruby by default" but
also "tested and provided by the core team", I think.

Frankly speaking, I don't think that rubygems.org is so dependable for
the core team to consign stdlib.
Even if it is dependable, the smaller trusted base is, the better.

Hi,

2010/9/5 Ryan Davis <ryand-ruby@zenspider.com>:
>
> On Sep 4, 2010, at 10:11 , Yusuke ENDOH wrote:
>
>> I really, really hate separate releases. ?It leads to synchronization
>> problem. ?See dismal states of Rdoc and Rubygems:
>>
>>   - [ruby-core:26679]
>>   - [ruby-core:31499]
>>   - [ruby-core:31503]
>
> That isn't a problem with separate releases. It is a problem with dual VCs.

Yes.  But separate release requires dual VCs, doesn't it?


> This is why I don't allow changes to minitest in ruby's repo and insist on
> patches being sent instead.  If rubygems and rdoc did the same thing then
> we wouldn't have those problems. We'd still have problems, they'd just be
> easier to manage.

Your solution works only when the maintainer is active.

If a bug ticket is registered for a library, if its maintainer is too 
busy
to fix it, and if any other committers are prohibited to change the 
library,
the ticket cannot be handled entirely, which blocks release management.
In fact, the problem occured in 1.9.2 release.

+1 for making all of the standard library into gems and to unbundling 
some
them (including REXML).  This would make the core easier to maintain, 
allow
for updates/bug fixes separate from the full ruby release schedule, and 
make
it easier for better libraries (nokigiri) to supercede older ones.

On Sat, Sep 4, 2010 at 4:30 PM, James Edward Gray II
<james@graysoftinc.com> wrote:
> On Sep 4, 2010, at 3:19 PM, Benoit Daloze wrote:
>
>> It would of course be even easier if Nokogiri was made the default.
>
> Yeah, I'm surprised to see so little support for Nokogiri.  I would be very for adding it.

I have no particular objection to switching from REXML to Nokogiri,
but I would object to removing REXML without a replacement.  (Though
REXML has pretty much always met my needs just fine.)

The libraries I use the most would probably be for XML, CSV, and YAML;
even for CSV, I stick to the standard library over FasterCSV: in my
environment, I can generally count on the standard library being
there, but cannot expect any sort of gem profile (though, that's a
political problem rather than a technical one).

On Sun, Sep 5, 2010 at 9:19 PM,  <brabuhr@gmail.com> wrote:
> On Sat, Sep 4, 2010 at 4:30 PM, James Edward Gray II
> <james@graysoftinc.com> wrote:
>> On Sep 4, 2010, at 3:19 PM, Benoit Daloze wrote:
>>
>>> It would of course be even easier if Nokogiri was made the default.
>>
>> Yeah, I'm surprised to see so little support for Nokogiri.  I would be very for adding it.
>
> I have no particular objection to switching from REXML to Nokogiri,
> but I would object to removing REXML without a replacement.  (Though
> REXML has pretty much always met my needs just fine.)

I have a question about this REXML vs Nokogiri debate: does Nokogiri
offer a REXML-compaitble interface? If this is the case (or if one
could be easily written), then the Nokogiri / REXML replacement could
happen without breaking anything

(2010/09/05 11:19), Ryan Melton wrote:
> +1 for making all of the standard library into gems and to unbundling
> some them (including REXML).  This would make the core easier to
> maintain, allow for updates/bug fixes separate from the full ruby
> release schedule, and make it easier for better libraries (nokigiri)
> to supercede older ones.

When release a package, we bundle and pack them.
Before releasing, we must test them.
For testing we fetch them each time.

So unbundling is not the solution; it make the problem difficult.

On Sep 5, 2010, at 12:28 PM, Giuseppe Bilotta wrote:

> On Sun, Sep 5, 2010 at 9:19 PM,  <brabuhr@gmail.com> wrote:
>> On Sat, Sep 4, 2010 at 4:30 PM, James Edward Gray II
>> <james@graysoftinc.com> wrote:
>>> On Sep 4, 2010, at 3:19 PM, Benoit Daloze wrote:
>>> 
>>>> It would of course be even easier if Nokogiri was made the default.
>>> 
>>> Yeah, I'm surprised to see so little support for Nokogiri.  I would be very for adding it.
>> 
>> I have no particular objection to switching from REXML to Nokogiri,
>> but I would object to removing REXML without a replacement.  (Though
>> REXML has pretty much always met my needs just fine.)
> 
> I have a question about this REXML vs Nokogiri debate: does Nokogiri
> offer a REXML-compaitble interface? If this is the case (or if one
> could be easily written), then the Nokogiri / REXML replacement could
> happen without breaking anything

I agree. Couldn't we use the Syck/Psych transition as a model?

- Josh

On Mon, Sep 06, 2010 at 05:02:09AM +0900, Joshua Ballanco wrote:
> On Sep 5, 2010, at 12:28 PM, Giuseppe Bilotta wrote:
> 
> > On Sun, Sep 5, 2010 at 9:19 PM,  <brabuhr@gmail.com> wrote:
> >> On Sat, Sep 4, 2010 at 4:30 PM, James Edward Gray II
> >> <james@graysoftinc.com> wrote:
> >>> On Sep 4, 2010, at 3:19 PM, Benoit Daloze wrote:
> >>> 
> >>>> It would of course be even easier if Nokogiri was made the default.
> >>> 
> >>> Yeah, I'm surprised to see so little support for Nokogiri.  I would be very for adding it.
> >> 
> >> I have no particular objection to switching from REXML to Nokogiri,
> >> but I would object to removing REXML without a replacement.  (Though
> >> REXML has pretty much always met my needs just fine.)
> > 
> > I have a question about this REXML vs Nokogiri debate: does Nokogiri
> > offer a REXML-compaitble interface? If this is the case (or if one
> > could be easily written), then the Nokogiri / REXML replacement could
> > happen without breaking anything
> 
> I agree. Couldn't we use the Syck/Psych transition as a model?

Writing an adapter for Nokogiri to quack like REXML is *very* hard (I've
tried).

Off the top of my head, there are a few large problems:  1) libxml2's
dependence on a context document, 2) rexml doesn't always follow XPath 
spec,
3) lack of REXML tests.

Supposedly there are REXML tests that are maintained outside of Ruby,
but I don't know where those are.  :-(

Aaron Patterson wrote:
> On Mon, Sep 06, 2010 at 05:02:09AM +0900, Joshua Ballanco wrote:
>> On Sep 5, 2010, at 12:28 PM, Giuseppe Bilotta wrote:
>>
>>> On Sun, Sep 5, 2010 at 9:19 PM,  <brabuhr@gmail.com> wrote:
>>>> On Sat, Sep 4, 2010 at 4:30 PM, James Edward Gray II <james@graysoftinc.com> wrote:
>>>>> On Sep 4, 2010, at 3:19 PM, Benoit Daloze wrote:
>>>>>
>>>>>> It would of course be even easier if Nokogiri was made the default.
>>>>> Yeah, I'm surprised to see so little support for Nokogiri.  I would be very for adding it.
>>>> ...
> 
> Writing an adapter for Nokogiri to quack like REXML is *very* hard (I've
> tried).
> 
> Off the top of my head, there are a few large problems:  1) libxml2's
> dependence on a context document, 2) rexml doesn't always follow XPath spec,
> 3) lack of REXML tests.

#2 seems to me to be more like bugs that should be reported
and fixed, rather than something to be baked into
a not-quite-XPath-for-ruby-psuedostandard that other libraries
have to clone bug-for-bug; no?

Perhaps #3 as well.

> Supposedly there are REXML tests that are maintained outside of Ruby,
> but I don't know where those are.  :-(

There are tests in the tar.gz available on the official REXML web page:
http://www.germane-software.com/archives/rexml_3.1.7.3.tgz

On Mon, Sep 6, 2010 at 6:33 AM, Vincent Isambart 
<vincent.isambart@gmail.com
> wrote:

> > Supposedly there are REXML tests that are maintained outside of Ruby,
> > but I don't know where those are.  :-(
>
> There are tests in the tar.gz available on the official REXML web page:
> http://www.germane-software.com/archives/rexml_3.1.7.3.tgz
>
>
They are quite bad.

I think removing REXML and other non-essential libraries is a good idea 
and
important to help open up the library ecosystem. Presence of a library 
in
the standard library discourages developers from creating their own
competing libraries.

Chad

Hi,

In <AANLkTimyHxeMW6j6LjPJjz0cPuDns_Q0K-7yEMW9ZX53@mail.gmail.com>
  "[ruby-core:32078] Re: merging nokogiri to ext/" on Mon, 6 Sep 2010 
10:38:15 +0900,
  Chad Fowler <chad@chadfowler.com> wrote:

> They are quite bad.
I don't know why exsiting test is bad.

> I think removing REXML and other non-essential libraries is a good idea and
> important to help open up the library ecosystem. Presence of a library in the
> standard library discourages developers from creating their own competing
> libraries.

I don't know why there is a relation between the standard
library and the ecosystem. There are many counter examples:
Nokogiri itself, miniunit and so on.


Thanks,

On Tue, Sep 07, 2010 at 09:32:42PM +0900, Kouhei Sutou wrote:
> >     > Supposedly there are REXML tests that are maintained outside of Ruby,
> >     > but I don't know where those are.  :-(
> >    
> >     There are tests in the tar.gz available on the official REXML web page:
> >     http://www.germane-software.com/archives/rexml_3.1.7.3.tgz
> > 
> > They are quite bad.
> 
> I don't know why exsiting test is bad.

I haven't read them carefully, but maybe they are not comprehensive?
There seem to be only 363 tests, but it's something to work with.

> > I think removing REXML and other non-essential libraries is a good idea and
> > important to help open up the library ecosystem. Presence of a library in the
> > standard library discourages developers from creating their own competing
> > libraries.
> 
> I don't know why there is a relation between the standard
> library and the ecosystem. There are many counter examples:
> Nokogiri itself, miniunit and so on.

Of course the standard library has an impact on the ecosystem.  Unless
you're saying that nobody uses the standard library?  I could believe
that, but I don't think it's true.

IMHO when someone downloads Ruby and dives in to the standard library,
they should be greeted with some of the best things Ruby has to offer.

On Sun, Sep 5, 2010 at 1:29 AM, NARUSE, Yui <naruse@airemix.jp> wrote:

>>>>>
>> break it, right?
>>
>
> We can't break it in API version 1.9.1.
>
> Moreover I think, we can't removing library in 1.9 series.
>
>
If you feel we cannot remove REXML in 1.9, I am happy to defer to you. 
In
that case, though, can we begin a conversation about planning 
deprecation of
REXML in Ruby 2.0, in favor of Nokogiri?

Can we also begin a conversation about gemifying stdlib for Ruby 2.0? I
would like to see if this is something ruby-core is willing to commit to 
for
the next major release.

Hi there, rubygems.org/gemcutter maintainer here. I'd just like to say 
that
if the core team was open to moving any of the stdlib's into gems
themselves, I'd be more than willing to make them 'blessed' gems in some
way, perhaps showing them in a different list (maybe
http://rubygems.org/stdlib), make the page look different so it's 
apparent
they're approved by the core team, etc. I'm open to discussion on how we 
can
work together besides that too.

Another note, I was planning on starting off the effort myself by
transforming some or all of the stdlib into gems and then showing how 
this
might look and be maintained for future versions of ruby. More on that 
once
I get around to it :)

-Nick

On Sep 7, 2010, at 05:32 , Kouhei Sutou wrote:

> I don't know why there is a relation between the standard
> library and the ecosystem. There are many counter examples:
> Nokogiri itself, miniunit and so on.

miniunit was written because test/unit was so bad that I couldn't and 
wouldn't maintain it. It isn't a case against Chad's point. It is a case 
FOR Chad's point. Nokogiri was written under much of the same pretense.

On Sep 4, 2010, at 22:37 , NARUSE, Yui wrote:

>>> Big Ruby-ism thinks that Ruby package should be all-in-one package for
>>> common use.
>>> Small Ruby-ism thinks that Ruby package should be as small as it can
>>> (bundle libs depended by RubyGems).
>> 
>> I think there is a third option: Medium Ruby-ism.  Convert all of stdlib
>> to be gems, and have Ruby ship with a default set of gems.
> 
> Libraries depended by gem, test or rdoc must be bundled.
> Most of current libraries are such things.

I agree... except for the tests for that library itself.

rexml is only depended upon by rss and xmlrpc. All three of those could 
be happy happy gems. Since rexml, rss, and xmlrpc are not needed by ruby 
itself for the build or for any other tests, they seem a good candidate 
for gemifying.

The only things needed by rubygems, test, or rdoc (base requirements for 
build and install) are rubygems, minitest+test/unit wrapper, and rdoc. 
Everything else can be gemified and we can have Medium Ruby.

On Sep 6, 2010, at 04:31 , Yusuke ENDOH wrote:

> 
> Frankly speaking, I don't think that rubygems.org is so dependable for
> the core team to consign stdlib.

Proof? rubygems.org has served up 53,846,355 gems to date. The amount of 
downtime listed on their status feed looks to be very low. That seems 
pretty dependable to me.

> Even if it is dependable, the smaller trusted base is, the better.

Why not apply the same logic to stdlib? The smaller it is, the better!

The size of the rubygem repository has nothing to do with this 
discussion. Ruby distributions can package and deliver gems just as 
easily as it can package and deliver a static stdlib. We're not asking 
for ruby core to bless the entire rubygems repo, that wouldn't make 
sense. What we're asking is for stdlib to ship in a more modular and 
upgradable fashion.

>> Even if it is dependable, the smaller trusted base is, the better.
>Why not apply the same logic to stdlib? The smaller it is, the better!

I totally agree with Ryan. If the core team has some reservations
about breaking stdlib into gems in the near future (1.9.x) that is
understandable. Perhaps Ruby 2.0 is the time to make that change.
But I think a great proof of concept would be to gem nokogiri and
remove rexml for 1.9.3 and see what happens. Let's be blave!

Jason




This message and any enclosures are intended only for the addressee. 
Please
notify the sender by email if you are not the intended recipient.  If 
you are
not the intended recipient, you may not use, copy, disclose, or 
distribute this
message or its contents or enclosures to any other person and any such 
actions
may be unlawful.  Ball reserves the right to monitor and review all 
messages
and enclosures sent to or from this email address.

On 8 September 2010 16:11, Thomas, Jason M (Software) 
<jmthomas@ball.com> wrote:
>>> Even if it is dependable, the smaller trusted base is, the better.
>>Why not apply the same logic to stdlib? The smaller it is, the better!
>
> I totally agree with Ryan. If the core team has some reservations
> about breaking stdlib into gems in the near future (1.9.x) that is
> understandable. Perhaps Ruby 2.0 is the time to make that change.
> But I think a great proof of concept would be to gem nokogiri and
> remove rexml for 1.9.3 and see what happens. Let's be blave!
>
> Jason

Sure, blave must be the (secret) word for the Ruby 2.0 community :)
More seriously, I agree this would be a good step to see "how it works".
Then the decision for Ruby 2.0 would be easier because we would have a
real example.
(Anyway libraries like RDoc and JSON are already updated as gems, so
it would just install from a gem instead of bundled files)

B.D.

Currently, we're discussing three different topics:

  1) REXML should be unbundled or not
  2) Nokogiri should be bundled or not
  3) all stdlib should be converted to gem, or not

First, we will NOT remove REXML from 1.9 for compatibility reason,
even if Nokogiri provides REXML-compatible layer.  No matter how
anyone says, this is the fact.  Please accept.  Compatiblity is
not problem of "brave".

Next, the point 3 should be discussed in another thread.
You can't have it both ways at once.

This thread started for discussion about merging Nokogiri.
Let's focus on the point 2 in this thread.


We should present advantage to bundle both Nokogiri and REXML.
I showed some pros (and cons) to committers (on Japanese IRC), and
received some rebuttals immediately:


 pros:
  - newbie tends to search library from stdlib first, but REXML
    should not be used.  By deprecating REXML (but not unbundled)
    and providing Nokogiri, we can indicate to newbie the right
    road.

    -> rebuttal: even if it is really needed, it is enough to
       deprecate REXML.

  - we can save time for many Ruby users to type "gem install
    nokogiri"

    -> rebuttal: OTOH, it wastes time and HDD space for people who
       do not use Nokogiri.

  - "gem install nokogiri" cannot be used on environment not
    connected to internet.

    -> question: is there people who uses Nokogiri on such a strict
       environment?

 cons:
  - Ruby distribution becomes enlarged; more maintainance effort is
    needed (but I can believe Aaron will do so responsibly)

  - Nokogiri may not preserve separate releases


IMO, I don't think it is good idea to refute the rebuttals.
It would be good to find another advantage.  Do you have?

On Thu, Sep 09, 2010 at 01:40:34AM +0900, Yusuke ENDOH wrote:
> 
> Next, the point 3 should be discussed in another thread.
> You can't have it both ways at once.
> 
> This thread started for discussion about merging Nokogiri.
> Let's focus on the point 2 in this thread.

Thanks for keeping us on track.  :-)

> 
>     -> rebuttal: even if it is really needed, it is enough to
>        deprecate REXML.

I agree with this rebuttal.  No XML parser is better than a poor one.
OTOH, it seems that people like having an XML parser ship with Ruby.
Why not ship a good one?

>   - we can save time for many Ruby users to type "gem install
>     nokogiri"
> 
>     -> rebuttal: OTOH, it wastes time and HDD space for people who
>        do not use Nokogiri.

The same argument could be made for any library in stdlib.  Why waste
someone's hard drive space with Psych when they never parse YAML?

>   - "gem install nokogiri" cannot be used on environment not
>     connected to internet.
> 
>     -> question: is there people who uses Nokogiri on such a strict
>        environment?

Not that I know of.

>  cons:
>   - Ruby distribution becomes enlarged; more maintainance effort is
>     needed (but I can believe Aaron will do so responsibly)

I am happy maintaining Nokogiri, but I couldn't do it without Mike.  I
think we're both pretty responsible maintainers.  :-)

>   - Nokogiri may not preserve separate releases

I'm not sure about this.  Other stdlib package have had separate
releases (rake, etc).  Though, I hope that stdlib is turned to gems so
this is easier (I'll respond to the gems thread).

> IMO, I don't think it is good idea to refute the rebuttals.
> It would be good to find another advantage.  Do you have?

As I mentioned earlier, I think people like having an XML parser ship
with ruby.  It would be advantageous to Ruby users if a good XML parser
shipped with Ruby.

My goal is to make sure that Ruby users have the best possible
experience when dealing with XML in Ruby.  REXML is a stumbling block.

I think the best options to improve the situation are:

  1. Remove REXML so that users must search for an XML library

  2. Package nokogiri so that users have a better alternative

  3. Remove REXML *and* package nokogiri

I understand we cannot remove REXML for 1.9.x, but maybe we should
consider packaging nokogiri so people have an alternative?  I would like
to see #2 for Ruby 1.9.3+, then #3 for Ruby 2.0.

Hello,

In message "[ruby-core:32189] Re: merging nokogiri to ext/"
    on Sep.09,2010 10:15:07, <aaron@tenderlovemaking.com> wrote:
> The same argument could be made for any library in stdlib.  Why waste
> someone's hard drive space with Psych when they never parse YAML?

Because rubygems uses YAML library, of course.
It is the only and the absolute reason.


Regards,

On Sep 8, 2010, at 18:15, Aaron Patterson wrote:
> 
> On Thu, Sep 09, 2010 at 01:40:34AM +0900, Yusuke ENDOH wrote:
>> - "gem install nokogiri" cannot be used on environment not
>>    connected to internet.
>> 
>>    -> question: is there people who uses Nokogiri on such a strict
>>       environment?
> 
> Not that I know of.

I've helped some people working at the US State Department (according to 
the email address @state.gov) download gems from rubygems.org manually 
since their production environments were not attached to the internet.

I suspect that this group of people is very small though.

> I think the best options to improve the situation are:
> 
>  1. Remove REXML so that users must search for an XML library
> 
>  2. Package nokogiri so that users have a better alternative
> 
>  3. Remove REXML *and* package nokogiri
> 
> I understand we cannot remove REXML for 1.9.x, but maybe we should
> consider packaging nokogiri so people have an alternative?  I would like
> to see #2 for Ruby 1.9.3+, then #3 for Ruby 2.0.

I would like to see this too.

On Sep 8, 2010, at 8:15 PM, Aaron Patterson wrote:

> to see #2 for Ruby 1.9.3+, then #3 for Ruby 2.0.
I fully support this plan.

I'm sad that I was allowed to replace the old CSV library but Aaron is 
struggling to replace REXML with a super superior library.  XML is way 
more important than CSV.

James Edward Gray II

On Thu, Sep 09, 2010 at 10:24:27AM +0900, U.Nakamura wrote:
> Hello,
> 
> In message "[ruby-core:32189] Re: merging nokogiri to ext/"
>     on Sep.09,2010 10:15:07, <aaron@tenderlovemaking.com> wrote:
> > The same argument could be made for any library in stdlib.  Why waste
> > someone's hard drive space with Psych when they never parse YAML?
> 
> Because rubygems uses YAML library, of course.
> It is the only and the absolute reason.

YAML was an example.  How about DL, json, strscan, etc?  Why waste space
with those?

2010/9/9 James Edward Gray II <james@graysoftinc.com>:
>> I understand we cannot remove REXML for 1.9.x, but maybe we should
>> consider packaging nokogiri so people have an alternative?  I would like
>> to see #2 for Ruby 1.9.3+, then #3 for Ruby 2.0.
>
> I fully support this plan.
>
> I'm sad that I was allowed to replace the old CSV library but Aaron is struggling to replace REXML with a super superior library.  XML is way more important than CSV.

Replacing CSV is before 1.9.0, so incompatibility is allowed.
The fact that fastercsv is pure ruby is also the difference.

Hello,

In message "[ruby-core:32203] Re: merging nokogiri to ext/"
    on Sep.09,2010 12:23:26, <aaron@tenderlovemaking.com> wrote:
> > > The same argument could be made for any library in stdlib.  Why waste
> > > someone's hard drive space with Psych when they never parse YAML?
> > 
> > Because rubygems uses YAML library, of course.
> > It is the only and the absolute reason.
> 
> YAML was an example.  How about DL, json, strscan, etc?  Why waste space
> with those?

DL is needed by win32 platforms.

IMO, json, strscan, and others should be unbundled because
they are not needed to run rubygems.


Regards,

2010/9/9 Aaron Patterson <aaron@tenderlovemaking.com>:
> with those?
DL is needed for some features for Windows, for example tmpdir, before
1.9.0, and standalone.
JSON is before 1.9.0, and standalone.
strscan is from 1.8, and standalone.

Hi,

2010/9/9 Aaron Patterson <aaron@tenderlovemaking.com>:
> OTOH, it seems that people like having an XML parser ship with Ruby.
> Why not ship a good one?

I agree that people want Nokogiri.  But if we import a library to ruby
package whenever people want it, ruby package will become too huge.

Thus it is a weak reason, unfortunately.
We need more convincing motivation.  For example, if rubygems depend
on Nokogiri, it would be very convincing :-)


>>   - Nokogiri may not preserve separate releases
>
> I'm not sure about this.  Other stdlib package have had separate
> releases (rake, etc).  Though, I hope that stdlib is turned to gems so
> this is easier (I'll respond to the gems thread).

As I said in [ruby-core:32054] and [ruby-core:32066], rake, etc. are
NEVER going well.  I don't like to add new source of trouble.

Indeed, this problem may be solved by converting stdlibs to gems.
But it has some objections and is still under discussion.  We should
not rely on the assumption at this time.


> to see #2 for Ruby 1.9.3+, then #3 for Ruby 2.0.
The point of issue is, why is this not enough?

  0. Deprecate REXML so that users must search for an XML library

On Thu, Sep 09, 2010 at 12:30:56PM +0900, U.Nakamura wrote:
> > YAML was an example.  How about DL, json, strscan, etc?  Why waste space
> > with those?
> 
> DL is needed by win32 platforms.
> 
> IMO, json, strscan, and others should be unbundled because
> they are not needed to run rubygems.

Why waste space with rubygems for people who don't use rubygems?

My point is not about who uses what.  Many people use Ruby differently.
I cannot account for every possible usage.  My point is that "disk
usage" arguments are not valid because they can be taken to an extreme.

Also, I think that if we package stdlib as gems, we can mitigate any
real world disk usage problems.

2010/9/9 Aaron Patterson <aaron@tenderlovemaking.com>:
>> >
> My point is not about who uses what.  Many people use Ruby differently.
> I cannot account for every possible usage.  My point is that "disk
> usage" arguments are not valid because they can be taken to an extreme.
>
> Also, I think that if we package stdlib as gems, we can mitigate any
> real world disk usage problems.

Ignore bike-sheds.

On Thu, Sep 09, 2010 at 12:33:07PM +0900, Yusuke ENDOH wrote:
> >>        deprecate REXML.
> >
> > I agree with this rebuttal.  No XML parser is better than a poor one.
> > OTOH, it seems that people like having an XML parser ship with Ruby.
> > Why not ship a good one?
> 
> I agree that people want Nokogiri.  But if we import a library to ruby
> package whenever people want it, ruby package will become too huge.

Can you elaborate on the criteria required for importing a library to
stdlib?  I do not understand the requirements.

> Thus it is a weak reason, unfortunately.
> We need more convincing motivation.  For example, if rubygems depend
> on Nokogiri, it would be very convincing :-)

It's difficult for me to fulfill requirements that are unknown.  :-(
If rubygems dependency is all that is needed, I'm happy to make a few
changes to rubygems.  ;-)

> But it has some objections and is still under discussion.  We should
> not rely on the assumption at this time.

We should continue this conversation in the "stdlib as gem" thread.  But
I do not agree with this as a negative point when it is apparent to me
that the problem with stdlib maintenance is not for lack of willingness,
but for lack of process.

I might be wrong, but I can't accept this negative point without
discussing the stdlib / gem problem further.  IMO, the stdlib problem is
a cyclic problem.  Are stdlib packages so bug-free that users are
willing to wait the 1+ year release cycle of Ruby?  The people who are
not willing to wait will give up on stdlib, thus making stdlib rot.

Without users, we do not improve.

> > to see #2 for Ruby 1.9.3+, then #3 for Ruby 2.0.
> 
> The point of issue is, why is this not enough?
> 
>   0. Deprecate REXML so that users must search for an XML library

I think removing REXML would improve our world.  But I also think that
XML parsing is very important to Ruby users.  Forcing Ruby users to 
search
for an XML library because we couldn't bundle a decent one makes me sad.

I think that any one of the solutions I listed would improve things.  I
just don't like the first one.

On Sep 8, 2010, at 09:40 , Yusuke ENDOH wrote:

>  - "gem install nokogiri" cannot be used on environment not
>    connected to internet.

That just isn't true and I think this misunderstanding is actually part 
of the contention. We can ship the nokogiri gem in the tarball and the 
install can install the gem just like it installs everything else.

% gem install gems/nokogiri-1.4.3.1.gem

The same thing could be done for rake, faster-csv, and any other libs 
that aren't used by the build/test phases directly.

There is a big added benefit to this approach. We can add a post-install 
message that tells people that they should run:

% sudo gem update --system
% sudo gem update

(or it can run it itself if it detects a network)

That way, no matter what gets installed by ruby, the user can update and 
pick up additional features, security fixes (think about the number of 
security updates we've had for webrick), etc.

Hi,

2010/9/9 Aaron Patterson <aaron@tenderlovemaking.com>:
> Can you elaborate on the criteria required for importing a library to
> stdlib?  I do not understand the requirements.

IMO, we need either:

  - An unavoidable reason why the core team couldn't help but bundle
    it, or
  - Matz or Yugui's authority ;-)


Ruby 1.8 was an epoch of enhancing stdlib, but unfortunately, many
maintainers left, which results in severe maintenance problems.
In addition, Ruby 1.9 includes RubyGems, which allow users to
install a library easily.
Furthermore, Ruby has been criticized because of disregard for
compatibility.

These three made the 1.9 stdlib set very pedantic, I think.

Hi,

2010/9/9 Ryan Davis <ryand-ruby@zenspider.com>:
>
> On Sep 8, 2010, at 09:40 , Yusuke ENDOH wrote:
>
>>  - "gem install nokogiri" cannot be used on environment not
>>    connected to internet.
>
> That just isn't true and I think this misunderstanding is actually part of the contention. We can ship the nokogiri gem in the tarball and the install can install the gem just like it installs everything else.

I think you misunderstood my context.
I said it in the context of the advantage to bundle Nokogiri, thus:

  Many users need to do "gem install nokogiri."  But,

  "gem install nokogiri" cannot be used on environment not
  connected to internet.

  If nokogiri gem is bundled, such users will be happy.

On Thu, Sep 09, 2010 at 10:13:31PM +0900, Yusuke ENDOH wrote:
> Hi,
> 
> 2010/9/9 Aaron Patterson <aaron@tenderlovemaking.com>:
> > Can you elaborate on the criteria required for importing a library to
> > stdlib?  I do not understand the requirements.
> 
> IMO, we need either:
> 
>   - An unavoidable reason why the core team couldn't help but bundle
>     it, or

We're all good engineers, so it's easy for us to come up with reasons to
avoid bundling.  Unfortunately, the only unavoidable reasons in favor of
bundling anything in stdlib are

  - it's already been bundled, so we must continue to bundle it

  - it's required for ruby build process

Nokogiri has not been bundled yet, and I don't think we need an XML 
parser for
Ruby build process (maybe we should switch to ant?  ;-) ).

>   - Matz or Yugui's authority ;-)

I guess I am forced to appeal to Matz and Yugui!

> Ruby 1.8 was an epoch of enhancing stdlib, but unfortunately, many
> maintainers left, which results in severe maintenance problems.
> In addition, Ruby 1.9 includes RubyGems, which allow users to
> install a library easily.
> Furthermore, Ruby has been criticized because of disregard for
> compatibility.
> 
> These three made the 1.9 stdlib set very pedantic, I think.

I think you're right.  But I also think giving an easy way for stdlib to
have out of band releases would help these problems.

As an alternate approach:

what would the problem be to start shipping a minimal version of ruby,
from the next release, with everything except stdlib, which is
replaced with a gem distribution? Then we can promote it (with an
appropriate fallback to support the fully bundled version for
environments which need it).

We have the wonderful rake, so I can't imagine it would be much more
difficult to extend our build system to handle the minimal version.

At least this might allow us to break the deadlock and see if people
would be willing to switch to a simpler, clean version of ruby which
ships without monkey-patches and hacks to make libraries work -
allowing us to have a nice clean base to start adding libraries to?

Best,
James

PS: i am sure this equally applies to the other thread; the reason for
posting here is as a humble idea to help break the deadlock.

2010/9/10 James Cox <james@imaj.es>:
>
> At least this might allow us to break the deadlock and see if people
> would be willing to switch to a simpler, clean version of ruby which
> ships without monkey-patches and hacks to make libraries work -
> allowing us to have a nice clean base to start adding libraries to?

This breaks compatibility; it's why REXML is needed.

On Sep 9, 2010, at 17:15 , NARUSE, Yui wrote:

>> difficult to extend our build system to handle the minimal version.
>> 
>> At least this might allow us to break the deadlock and see if people
>> would be willing to switch to a simpler, clean version of ruby which
>> ships without monkey-patches and hacks to make libraries work -
>> allowing us to have a nice clean base to start adding libraries to?
> 
> This breaks compatibility; it's why REXML is needed.

How?

sorry, wrong thread.

On Thu, Sep 9, 2010 at 8:15 PM, NARUSE, Yui <naruse@airemix.jp> wrote:
>> difficult to extend our build system to handle the minimal version.
>>
>> At least this might allow us to break the deadlock and see if people
>> would be willing to switch to a simpler, clean version of ruby which
>> ships without monkey-patches and hacks to make libraries work -
>> allowing us to have a nice clean base to start adding libraries to?
>
> This breaks compatibility; it's why REXML is needed.
>
Yui,

The point is that we can have people self-select the package that best
suits their needs. We can stand behind a more lightweight package,
(medium ruby) and still provide the full-service package for rubyists
who aren't sure which they need. I'll even offer to help make sure
this happens: we just need to make sure the site makes it clear what
each package provides, and we adapt the build system to make both ruby
and ruby-full.

This makes it ok to break compatibility: people can self-select a ruby
that suits their needs. We have all pretty much agreed that a lighter
ruby, with an encouragement to use the gem community to support your
development is the right path, so a transitional period supporting
both seems appropriate?

James

(2010/09/10 23:33), James Cox wrote:
> ruby, with an encouragement to use the gem community to support your
> development is the right path, so a transitional period supporting
> both seems appropriate?

Scripts like ruby -rrexml -e'p REXML' works now if for example Ruby 
1.9.2
is installed and without any other libraries.
Your plan breaks this.

We think it is incompatible and it is not OK.


Yeah, we can break such compatibility on major bump: Ruby 2.0.