Forum: Ruby-core Import gem to Ruby 1.9

Hi,

Matz plans to import gem into Ruby 1.9.  Does anyone has any
comment to this?

- possibility/difficulty to import
  (who is the maintainer of gem?)
- compatibility
- and other problems

Regards,

On 7/8/07, SASADA Koichi <ko1@atdot.net> wrote:
> Hi,
>
> Matz plans to import gem into Ruby 1.9.  Does anyone has any
> comment to this?
>
> - possibility/difficulty to import
>   (who is the maintainer of gem?)
> - compatibility
> - and other problems

I think it would be nice to have a way to provide version control for
libraries irregardless of the packaging system used. I developed such
a system with the help of Peter Vanbroekhoven, which worked well
enough. But it had one serious issue --it required a version tier be
added to a project's lib/ folder. Eg.

  mypkg/
    lib/
      1.0.0/
        mypkg/

I wrote a tiering tool to help manage this, but as Austin Ziegler had
warned, it still isn't as straightforward for developers as it needs
to be, primarily b/c it doesn't mesh well with SCM systems. For this
reason I never released it.

Considering this issue further, I realized that the problem largely
resided with the relationship between the conventions of setup.rb and
#require. How much better if #require itself understood the standard
package layouts rather than having to install a special "installation
image". Eg.

  site_ruby/
    mypkg/
      1.0.0/
        lib/

So that the "installation" is essentially just a copy of a project
--having a conventional layout, minus unnecessaries.

This would be a compromise between current require and Gems'. Gems
would still be Ruby's dedicated packaging system, but this would allow
version control to be used with other packaging systems, as well.

In any case, I think it would be nice to see Ruby progress in this
direction, but it doesn't preclude adding Gems to the standard library
--in fact it may ultimately help move Ruby in this direction. So I am
for it.

And I'm sure a number of others are saying "it's about time" ;-)

T.

> Matz plans to import gem into Ruby 1.9.
I'm all for it! it will be another Top 10 reason to switch to 1.9. The
One-Click Installer for Windows included it for years, and as far as I
know, most people are happy with it.

Maybe we can add a --disable-rubygems switch to make everyone happy?

One thing: If I'm not mistaken, rubygems searches RubyForge
(gems.rubyforge.org) per default; if it is included in Ruby, RubyForge
becomes the default ("built-in") Ruby software archive. While I think
RubyForge is the perfect choice, it should be taken into account.

Have a nice day. [murphy]

TRANS <transfire@gmail.com> writes:

>
> So that the "installation" is essentially just a copy of a project
> --having a conventional layout, minus unnecessaries.

+1.

mypkg-1.0.0/lib could be an option too.

On Jul 8, 2007, at 00:49, SASADA Koichi wrote:
> Matz plans to import gem into Ruby 1.9.  Does anyone has any
> comment to this?
>
> - possibility/difficulty to import

RubyGems uses many features of Ruby, so 1.9's libraries need to be
fairly stable in order to continue working.

>   (who is the maintainer of gem?)

Myself, Jim Weirich and Chad Fowler are maintainers of RubyGems (Rich
Kilmer too, but he's been rather inactive lately.)

> - compatibility
> - and other problems

RubyGems is still missing one key feature, the ability to handle
platform-specific gems.  There may be a few other minor features that
are missing, but I don't think RubyGems is ready for inclusion until
then.

Also, RubyGems can make releases faster than Ruby can, so it should
still be possible to upgrade RubyGems to never versions independent
of Ruby.

On Jul 8, 2007, at 06:29, murphy wrote:

>> Matz plans to import gem into Ruby 1.9.
> I'm all for it! it will be another Top 10 reason to switch to 1.9. The
> One-Click Installer for Windows included it for years, and as far as I
> know, most people are happy with it.
>
> Maybe we can add a --disable-rubygems switch to make everyone happy?

What would it do?  If you don't install any gems, nothing happens.

> > Maybe we can add a --disable-rubygems switch to make everyone happy?
>
> What would it do?  If you don't install any gems, nothing happens.

I'm not the one who suggested it, but I think he meant something the 
following:
If the --disable-rubygems option would be given to configure, the
"make install" would install neither the rubygem library nor the "gem"
command.

On 7/9/07, Eric Hodel <drbrain@segment7.net> wrote:
>
> On Jul 8, 2007, at 00:49, SASADA Koichi wrote:
> > Matz plans to import gem into Ruby 1.9.  Does anyone has any
> > comment to this?
>
> RubyGems is still missing one key feature, the ability to handle
> platform-specific gems.  There may be a few other minor features that
> are missing, but I don't think RubyGems is ready for inclusion until
> then.


Is this feature planned for the near term?

Also, RubyGems can make releases faster than Ruby can, so it should
> still be possible to upgrade RubyGems to never versions independent
> of Ruby.


Perhaps RubyGems could be shipped with Ruby core, as a stub.
This would allow a basic functionality to bootstrap new gems as they are
released, including RubyGems itself.
If Gems are such a great solution, perhaps Core Libs could be repackaged 
as
gems as well.
This could achieve a new agility to Ruby itself.

On Jul 10, 2007, at 15:52, Tim Morgan wrote:
> Is this feature planned for the near term?
Its in the bug tracker.  RubyGems also wants to be in -core, but
nobody has bothered to spare the time to write it until recently,
when somebody on the RubyGems mailing list indicated they were
working on a patch.

>> Also, RubyGems can make releases faster than Ruby can, so it should
>> still be possible to upgrade RubyGems to never versions independent
>> of Ruby.
>
> Perhaps RubyGems could be shipped with Ruby core, as a stub.

There's a small change to #require that needs to be made to invoke
the bit of RubyGems that does $LOAD_PATH manipulation.  So long as
that is as small as possible, the default $LOAD_PATH should allow a
RubyGems in site_ruby to override the one in the standard library.

> This would allow a basic functionality to bootstrap new gems as  
> they are released, including RubyGems itself.
> If Gems are such a great solution, perhaps Core Libs could be  
> repackaged as gems as well.
> This could achieve a new agility to Ruby itself.

I believe this idea floated into the heads of one of the other
RubyGems maintainers.  Possibly, its also been discussed, but I forget.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Eric,

Eric Hodel wrote:
>> Matz plans to import gem into Ruby 1.9.  Does anyone has any
>> comment to this?
>>
>> - possibility/difficulty to import
> 
> RubyGems uses many features of Ruby, so 1.9's libraries need to be
> fairly stable in order to continue working.

>> - compatibility
>> - and other problems
> 
> RubyGems is still missing one key feature, the ability to handle
> platform-specific gems.  There may be a few other minor features that
> are missing, but I don't think RubyGems is ready for inclusion until then.

Are you and RubyGems maintainers negative to include RubyGems in
ruby/1.9.1 ?

I have a plan to propose unbundling some libraries such as soap4r, tk,
webrick, json, etc.  Maintainer-less libraries such as yaml should be
unbundled, too (bear in mind that I'm talking about ruby/1.9).  It can
be a gem when RubyGems is bundled in MRI.  It can be a package of
another packaging system(deb, rpm, etc.), too.  Users will use favorite
packaging system then.

> Also, RubyGems can make releases faster than Ruby can, so it should
> still be possible to upgrade RubyGems to never versions independent of
> Ruby.

Do you mean RubyGems can be a gem?  Looks smart.

// NaHi

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Cygwin)

iQEVAwUBRpx86B9L2jg5EEGlAQJyTAgAtwVeH0vWdsVrwJ+6XkaTp6VUnbSMv6ks
xm2oCfPngVA/poNmz+xMtui6m8oiIzuixpO7/ZsezvsvtFjPRqWfijdLQxGQ/xJp
7RC4kaQ4xhZrOTBr0leOkG3gWGW4NzhCEs9OA+xepntFQs+uluSbGigvXMbnkvOn
H6niYssyCHrz3Y+s7Co8ywgPcKMGDQQzBW7g25UkUcMPmnpIaqYqAosGTF84OQwI
ab9+KcggVsrtDP9L0DSUqWNwBIOEGt66a4IYTI9ldQLQEsg7i8D5IwT216nmwhrU
lY/pE6YShGdKWvuI06RTmwUs5NDoJ/GwxF75hpvc98xALr/PiO/tgA==
=vqQL
-----END PGP SIGNATURE-----

On 7/17/07, NAKAMURA, Hiroshi <nakahiro@sarion.co.jp> wrote:
> >
> Are you and RubyGems maintainers negative to include RubyGems in
> ruby/1.9.1 ?
>
> I have a plan to propose unbundling some libraries such as soap4r, tk,
> webrick, json, etc.  Maintainer-less libraries such as yaml should be
> unbundled, too (bear in mind that I'm talking about ruby/1.9).  It can
> be a gem when RubyGems is bundled in MRI.  It can be a package of
> another packaging system(deb, rpm, etc.), too.  Users will use favorite
> packaging system then.

Unbundle Yaml and Webrick? Are you kidding? Those are bread and butter!

T.

On Jul 17, 2007, at 7:31 AM, TRANS wrote:

>> >> - possibility/difficulty to import
>> > are missing, but I don't think RubyGems is ready for inclusion  
>> be a gem when RubyGems is bundled in MRI.  It can be a package of
>> another packaging system(deb, rpm, etc.), too.  Users will use  
>> favorite
>> packaging system then.
>
> Unbundle Yaml and Webrick? Are you kidding? Those are bread and  
> butter!

Yeah, that would be a pretty drastic change.

Also, what is the "json" library that was mentioned?

$ ruby -v -r json -e1
ruby 1.8.6 (2007-06-23 patchlevel 5000) [i686-darwin8.9.1]
ruby: no such file to load -- json (LoadError)

James Edward Gray II

> Also, what is the "json" library that was mentioned?
>
> $ ruby -v -r json -e1
> ruby 1.8.6 (2007-06-23 patchlevel 5000) [i686-darwin8.9.1]
> ruby: no such file to load -- json (LoadError)

It's a library added to Ruby trunk (so only in 1.9) in June:
Mon Jun  4 21:15:45 2007  NARUSE, Yui  <naruse@ruby-lang.org>
        * lib/json.rb, lib/json, ext/json, test/json:
          import JSON library.

On Jul 17, 2007, at 06:24 , James Edward Gray II wrote:

> ruby: no such file to load -- json (LoadError)
505 % head -1 version.h
#define RUBY_VERSION "1.9.0"
506 % ls lib/json*
lib/json.rb

lib/json:
Array.xpm       Key.xpm         String.xpm      editor.rb       pure
FalseClass.xpm  NilClass.xpm    TrueClass.xpm   ext.rb          pure.rb
Hash.xpm        Numeric.xpm     common.rb       json.xpm
version.rb

On Jul 17, 2007, at 05:31 , TRANS wrote:

> Unbundle Yaml and Webrick? Are you kidding? Those are bread and  
> butter!

So? Installing them via gems changes it how? This would allow them to
grow/mature/change/improve independently of a ruby release schedule.

On Jul 17, 2007, at 01:26 , NAKAMURA, Hiroshi wrote:

> Are you and RubyGems maintainers negative to include RubyGems in
> ruby/1.9.1 ?

No, I don't think we are. We just have that one feature that we
really need before rubygems is more or less considered feature
complete. After that there are some bug fixes and other things that
need to go in, but I'm guessing that'd work out fairly well with the
ruby 1.9.1 timeframe.

> I have a plan to propose unbundling some libraries such as soap4r, tk,
> webrick, json, etc.  Maintainer-less libraries such as yaml should be
> unbundled, too (bear in mind that I'm talking about ruby/1.9).  It can
> be a gem when RubyGems is bundled in MRI.  It can be a package of
> another packaging system(deb, rpm, etc.), too.  Users will use  
> favorite
> packaging system then.

I think this is wonderful. Too long has there been a lot of cruft in
ruby. This will let meritocracy take over. Ownerless libraries should
still probably be released in the ruby project on rubyforge tho so
they can still be represented in terms of ticket tracking and the like.

> Do you mean RubyGems can be a gem?  Looks smart.

No, I don't think that was implied, just that by being independent of
ruby, we can release more and faster than ruby can. For example, here
have already been 5 releases of rubygems and only 1 of ruby this
year. Two of those releases were after the ruby release, so they
would have been postponed.

Once rubygems stabilizes it shouldn't be as much of an issue. I
suspect that the timeframes will work out pretty well.

On 7/17/07, Ryan Davis <ryand-ruby@zenspider.com> wrote:
>
> On Jul 17, 2007, at 05:31 , TRANS wrote:
>
> > Unbundle Yaml and Webrick? Are you kidding? Those are bread and
> > butter!
>
> So? Installing them via gems changes it how? This would allow them to
> grow/mature/change/improve independently of a ruby release schedule.

I understand what you are saying but there is an important
consideration here. It's good for Ruby to have out of the box tools
for certain things. YAML is great for quick and easy config files and
distributed serialization. Webrick gives us a qucik and easy
webserver. These things are very useful and help promote a common
"prototyping" environment, which greatly enhance communication and
learning. This is one of the great things about Ruby.

I agree with you about other items though. Tk for instance is just not
useful enough and is tied to some heavy external code.

T.

On Jul 17, 2007, at 01:26, NAKAMURA, Hiroshi wrote:
> >> - compatibility
> >> - and other problems
> >
> > RubyGems is still missing one key feature, the ability to handle
> > platform-specific gems.  There may be a few other minor features  
> that
> > are missing, but I don't think RubyGems is ready for inclusion  
> until then.
>
> Are you and RubyGems maintainers negative to include RubyGems in
> ruby/1.9.1 ?

No, we'd really, really like RubyGems to be in core, but I'd really,
really like to get the platform-specific gem handling working before
it is in core.

(The C stub to make ruby -r work with an in-core RubyGems should be
easy to implement.)

I'll have some free time coming up in August, so I may be able to get
to it then.

> I have a plan to propose unbundling some libraries such as soap4r, tk,
> webrick, json, etc.  Maintainer-less libraries such as yaml should be
> unbundled, too (bear in mind that I'm talking about ruby/1.9).  It can
> be a gem when RubyGems is bundled in MRI.  It can be a package of
> another packaging system(deb, rpm, etc.), too.  Users will use  
> favorite
> packaging system then.

I'd like to see this too.

> > Also, RubyGems can make releases faster than Ruby can, so it should
> > still be possible to upgrade RubyGems to never versions  
> independent of
> > Ruby.
>
> Do you mean RubyGems can be a gem?  Looks smart.

Currently RubyGems updates come as a gem, but they get installed into
site_ruby.

TRANS wrote:
> I agree with you about other items though. Tk for instance is just not
> useful enough and is tied to some heavy external code.

Not useful? I don't think so ... Tk is the only GUI framework that comes 
with Ruby.
Without it you cannot make a single GUI window.

I won't be surprised if Tk guys see WEBrick "just not useful".  Which is 
obviously
wrong, and you know, Tk being not useful is also wrong.

On 7/17/07, Urabe Shyouhei <shyouhei@ruby-lang.org> wrote:
> TRANS wrote:
> > I agree with you about other items though. Tk for instance is just not
> > useful enough and is tied to some heavy external code.
>
> Not useful? I don't think so ... Tk is the only GUI framework that comes with Ruby.
> Without it you cannot make a single GUI window.
>
> I won't be surprised if Tk guys see WEBrick "just not useful".  Which is obviously
> wrong, and you know, Tk being not useful is also wrong.

"enough" is the keyword in what I said. Obviously it is useful to
those who use Tk. But Ruby is not implementing "Tk". It is only
providing a binding. So you have to have Tk installed for it to be of
use anyway. If Ruby had a dedicated GUI, that would be another matter.

T.

TRANS wrote:
> "enough" is the keyword in what I said. Obviously it is useful to
> those who use Tk. But Ruby is not implementing "Tk". It is only
> providing a binding. So you have to have Tk installed for it to be of
> use anyway. If Ruby had a dedicated GUI, that would be another matter.
Dedicated GUI... I've never been happy with Smalltalk's.  There are good 
reasons why
Ruby does not provide such things.

I'm for unbundling Tk (and WEBRick and so on) from 1.9, but not because 
that is not
useful enough.

On 7/17/07, Urabe Shyouhei <shyouhei@ruby-lang.org> wrote:
> TRANS wrote:
> > "enough" is the keyword in what I said. Obviously it is useful to
> > those who use Tk. But Ruby is not implementing "Tk". It is only
> > providing a binding. So you have to have Tk installed for it to be of
> > use anyway. If Ruby had a dedicated GUI, that would be another matter.
> Dedicated GUI... I've never been happy with Smalltalk's.  There are good reasons why
> Ruby does not provide such things.

But it's not likely that you'd use Webrick for a commercial app
either. That's not the reason for its inclusion. Webrick is written in
Ruby and acts as an lcd for webservering. Which makes it great for
prototypes and quick hacks. Think of all the neat tricks we've seen --
like the smallest wiki in the known universe.

T.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

Thanks for your comments, TRANS, Ryan and Eric.  Good to know you all
are positive about including RubyGems.

TRANS and others, what libraries should be unbundled is the next (or the
 next) topic.  Wait for a while.

Matz, Syouhei, there seems to be no barrier to add RubyGems into
ruby/1.9.1 so far.  Don't you disagree we can continue the discussion
about bundling RubyGems with Ruby/1.9.1?

Eric Hodel wrote:
> is in core.
> 
> (The C stub to make ruby -r work with an in-core RubyGems should be easy
> to implement.)
> 
> I'll have some free time coming up in August, so I may be able to get to
> it then.

Thanks for your work always about ruby.

In above which are you planning to implement 'platform-specific gem
handling' or 'C stub to make ruby -r work'?  In the case of the latter
topic, ruby itself may be able to help it with adding common
'require-hook', if someone can design such a feature.  We should do
things step by step so some features like above can be delayed to 1.9.2
though.

>> > Also, RubyGems can make releases faster than Ruby can, so it should
>> > still be possible to upgrade RubyGems to never versions independent of
>> > Ruby.
>>
>> Do you mean RubyGems can be a gem?  Looks smart.
> 
> Currently RubyGems updates come as a gem, but they get installed into
> site_ruby.

Like Ryan said, it's not as much of an issue.  I was just impressed with
it. :-)

  /  /  /

At first, let's list issues about bundling RubyGems to ruby/1.9.1.

1. Is platform-specific gem handling needed?

2. Does RubyGems need some 'require-hook' feature to be added to
   ruby/1.9.1?  What's the requirements?
   - hooks -r options

3. What gem related commands should be install in BINDIR by the standard
   installer?
     gem, gemlock, gemri, gemwhich, gem_mirror, gem_server,
     index_gem_repository.rb, update_rubygems
   Nothing should be installed? (I mean 'ruby -rgem -e update gemname')

4. What $LOAD_PATH order should be?
  4-1. by default?
    [RUBYLIBDIR, SITEDIR]
  4-2. after requiring rubygems?
    [GEMs, RUBYLIBDIR, SITEDIR] or
    [RUBYLIBDIR, GEMs, SITEDIR] or
    [RUBYLIBDIR, SITEDIR, GEMs]

5. Where's the global repository for bundled rubygems?
   Of course RubyForge should be pointed.  Do we need some
   'rather official' repository at www.ruby-lang.org, too?

6. What libraries does RubyGems depend on?
   - existing bundled libraries
   - new libraries
     I forgot to mention that rake should be added to 1.9.1, too.

// NaHi

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2.1 (Cygwin)

iQEVAwUBRp3OjR9L2jg5EEGlAQI95AgAu1KyJSmx6yaEd7hOfOD0ASaN2DWm2xje
REIBQwF7NCRBTXKj0lDLq2eXFFpJttcyaAHWDxP0k4h4JpkBfm/q3oWm01r0qb1n
iNTCCercZAAFzOTyUtM56wFYPhItS4nvNtMVyLXg4Fu9G1YgGs82wb+N6jPWFs2Y
qPI8w+Kq6LAT96jxmNaGu5ivpYSC1TSS7WS91L2ljcB9dw5V+VTPJI6vhQYiDyn2
QGJv3+kkmvYJJXBr00pI7HNcYtEjdK1opmM63RttQM6OoaIi2Co0qSoEC5Vb9+vG
YE7UuQfbnpaYIlZ0dcELG4nFQdXMdWHcgF6YgzMrXYl7LR8veUAdHQ==
=cEpP
-----END PGP SIGNATURE-----

On 7/18/07, NAKAMURA, Hiroshi <nakahiro@sarion.co.jp> wrote:
>
> >> Are you and RubyGems maintainers negative to include RubyGems in
> > it then.
>
> Thanks for your work always about ruby.
>
> In above which are you planning to implement 'platform-specific gem
> handling' or 'C stub to make ruby -r work'?  In the case of the latter
> topic, ruby itself may be able to help it with adding common
> 'require-hook', if someone can design such a feature.  We should do
> things step by step so some features like above can be delayed to 1.9.2
> though.
>

A built-in require hook would be excellent.  I'm sure I'm going to
raise some feathers here but if we're going to include RubyGems in the
distribution, what is the general feeling about just including the
RubyGems LOAD_PATH semantics as part of how #require naturally works?
That would mean either of two things:  1) #require is enhanced in C to
do the RubyGems logic of looking in the installed gems and adding to
the LOAD_PATH + requiring on match or 2) require 'rubygems' by
default.

I'm not necessarily arguing for this.  Just raising the idea.

>         /       /       /
>
> At first, let's list issues about bundling RubyGems to ruby/1.9.1.
>
> 1. Is platform-specific gem handling needed?
>

We need to at least clean up the way it's implemented now.  It's not
so much a few feature as an annoyance that needs to be fixed.

> 2. Does RubyGems need some 'require-hook' feature to be added to
>    ruby/1.9.1?  What's the requirements?
>    - hooks -r options
>

The problem with -r is that it doesn't delegate to #require.  It runs
the underlying C function with #require delegates to.  So it's
impossible in Ruby code to override the behavior of -r.

> 3. What gem related commands should be install in BINDIR by the standard
>    installer?
>      gem, gemlock, gemri, gemwhich, gem_mirror, gem_server,
>      index_gem_repository.rb, update_rubygems
>    Nothing should be installed? (I mean 'ruby -rgem -e update gemname')
>

I think we need gem and gem_server at a minimum.  gemwhich would
actually be a nice general-purpose Ruby utility.  It could be renamed
rubywhich or rbwhich and installed.

>    'rather official' repository at www.ruby-lang.org, too?
>

Personally I think RubyForge plus its mirrors are sufficient.  We
could do a cname for gems.ruby-lang.org perhaps?  RubyForge is run by
Ruby Central, so you could consider it to be 'rather official' already
I suppose.

> 6. What libraries does RubyGems depend on?
>    - existing bundled libraries
>    - new libraries
>      I forgot to mention that rake should be added to 1.9.1, too.
>

Off the top of the head, YAML/Syck, Webrick, the digest libraries,
rbconfig, rdoc, thread, optparse, forwardable, time, openssl,
open-uri, uri, net/http, fileutils, zlib, stringio, socket, tempfile,
pathname


Chad

On 7/17/07, Ryan Davis <ryand-ruby@zenspider.com> wrote:
> ruby 1.9.1 timeframe.
> ruby. This will let meritocracy take over. Ownerless libraries should
> still probably be released in the ruby project on rubyforge tho so
> they can still be represented in terms of ticket tracking and the like.
>

I strongly agree that unbundling a lot of the built in standard libs
from Ruby would be a very positive thing.  One of the major reasons
I'd like to see this happen is that having, for example, a JSON
library in Ruby's core distribution discourages bazaar-like,
evolutionary development of further JSON libraries.  Sure, the code is
Open Source, and patches are (I'm sure) warmly accepted for most of
the standard libraries, but the fact that a library exists and is
"blessed" as being official kills competition to some extent.

A policy to never bundle libraries which aren't necessary for "core
infrastructure" for the Ruby distribution would greatly improve the
sense that the Ruby community hasn't decided that there can/should
never be another TK, SOAP, JSON, RSS, etc. library written from
scratch.

RubyGems is the defacto standard already.  All but a very small few
Ruby developers install it as a separate step.  Including it in the
distribution enables the unbundling of a whole lot of other libraries.

I'm strongly in favor.  We do need to, as Ryan and/or Eric have
already said, make sure we can still upgrade RubyGems externally, but
I don't see any barrier to that being possible.

On Jul 22, 2007, at 10:09 , Chad Fowler wrote:

>> 6. What libraries does RubyGems depend on?
>>    - existing bundled libraries
>>    - new libraries
>>      I forgot to mention that rake should be added to 1.9.1, too.
>>
>
> Off the top of the head, YAML/Syck, Webrick, the digest libraries,
> rbconfig, rdoc, thread, optparse, forwardable, time, openssl,
> open-uri, uri, net/http, fileutils, zlib, stringio, socket, tempfile,
> pathname

YAML is just used for the gem index and could be dropped in favor of
marshal to allow YAML to be externalized from the standard library.

Webrick is only used for gem_server and not critical (maybe rubygems
could get split up into a core functionality package included in ruby
1.9 and a rubygems_extras gem distro?).

On Jul 22, 2007, at 10:09, Chad Fowler wrote:
> rubywhich or rbwhich and installed.
On the RubyGems list (or maybe just with Chad) moving everything
under the 'gem' command was discussed, and parties were in favor.
I'm doing this now.

On Jul 18, 2007, at 01:26, NAKAMURA, Hiroshi wrote:
> 4. What $LOAD_PATH order should be?
>   4-1. by default?
>     [RUBYLIBDIR, SITEDIR]
>   4-2. after requiring rubygems?
>     [GEMs, RUBYLIBDIR, SITEDIR] or
>     [RUBYLIBDIR, GEMs, SITEDIR] or
>     [RUBYLIBDIR, SITEDIR, GEMs]

Right now gems come before SITEDIR and after RUBYLIBDIR, and this
seems to work well.

Nothing should come before RUBYLIBDIR because it prevents -I from
working for developers.

Hi,

At Mon, 23 Jul 2007 09:30:08 +0900,
Eric Hodel wrote in [ruby-core:11783]:
> Right now gems come before SITEDIR and after RUBYLIBDIR, and this  
> seems to work well.
> 
> Nothing should come before RUBYLIBDIR because it prevents -I from  
> working for developers.

You are confused.  SITEDIR comes before RUBYLIBDIR.

$ ruby19 -v -e 'p $:'
ruby 1.9.0 (2007-07-20 revision 12823) [i686-linux]
["/usr/lib/ruby/site_ruby/1.9",
 "/usr/lib/ruby/site_ruby/1.9/i686-linux",
 "/usr/lib/ruby/site_ruby",
 "/usr/lib/ruby/1.9",
 "/usr/lib/ruby/1.9/i686-linux",
 "."]

Hi,

In message "Re: Import gem to Ruby 1.9"
    on Wed, 18 Jul 2007 17:26:41 +0900, "NAKAMURA, Hiroshi" 
<nakahiro@sarion.co.jp> writes:

|Matz, Syouhei, there seems to be no barrier to add RubyGems into
|ruby/1.9.1 so far.  Don't you disagree we can continue the discussion
|about bundling RubyGems with Ruby/1.9.1?

I'm OK for making it bundled if they can make it before the deadline
(vaguely October or so).  We can discuss what libraries to be removed
as a separate issue.

              matz.

On 7/22/07, Chad Fowler <chad@chadfowler.com> wrote:
>
> A built-in require hook would be excellent.  I'm sure I'm going to
> raise some feathers here but if we're going to include RubyGems in the
> distribution, what is the general feeling about just including the
> RubyGems LOAD_PATH semantics as part of how #require naturally works?
> That would mean either of two things:  1) #require is enhanced in C to
> do the RubyGems logic of looking in the installed gems and adding to
> the LOAD_PATH + requiring on match or 2) require 'rubygems' by
> default.

No at all. That's actually what I've been advocating. But rather than
hobble two different require mechanisms, one on top of the other, we
should create an improved unified mechanism -- unifying gems/ and
site_lib/. That's really the way to go, though there are some
considerations.

1) Efficiency. Gems adds a lot of dirs to the load path. While the
normal require mechanism only has a few. When running a small script,
no one wants to wait, even for 1 second, for a lib to be found. Is
there a good way to address this?

2) Organization of the gems/ directory. 'cache' really doesn't belong
under lib/ space at all. And 'doc' and 'specifications' I would like
to see with their respective packages. GoboLinux is good for
comparison of the Gem philosophy of packaging. See
http://www.gobolinux.org/index.php?page=at_a_glance, for starters.
What they do is add a "Resources" dir to each install version which
houses system info about the project. This way we could remove the
meta tier and just have gems/1.9/{package} rather than the current
gems/1.8/gems/{package}. This would facilitate other packages systems
to use the layout.

3) Per the last point. I'd rather see a tiered layout of
'name/version' rather than the single level 'name-version'. I realize
that's a rather subjective and minor, but it helps keep the directory
clean and easier to navigate.

4) The gemspec should not be crucial to using the gems repository. The
only piece of info in there that I think is required is the
load_paths. Most packages don't need it. So a gemspec for a package
should be optional --which is necessary to be able to install a
package manually or via other package systems too.

T.

On Jul 23, 2007, at 00:47, Nobuyoshi Nakada wrote:
>>>     [RUBYLIBDIR, SITEDIR, GEMs]
> ruby 1.9.0 (2007-07-20 revision 12823) [i686-linux]
> ["/usr/lib/ruby/site_ruby/1.9",
>  "/usr/lib/ruby/site_ruby/1.9/i686-linux",
>  "/usr/lib/ruby/site_ruby",
>  "/usr/lib/ruby/1.9",
>  "/usr/lib/ruby/1.9/i686-linux",
>  "."]

Yes, sorry, I meant 'ruby -I' and 'RUBYLIB=... ruby' paths.  These
two need to be before gems.

On Jul 23, 2007, at 03:03, TRANS wrote:
>> the LOAD_PATH + requiring on match or 2) require 'rubygems' by
>> default.
>
> No at all. That's actually what I've been advocating. But rather than
> hobble two different require mechanisms, one on top of the other,

This isn't what Chad is saying.  We're going to add code to call back
through RubyGems' custom_require from C, or some similar solution.

> we should create an improved unified mechanism -- unifying gems/  
> and site_lib/.  That's really the way to go, though there are some  
> considerations.

This isn't going to happen.  RubyGems abandoned playing in site_lib
ages ago for various well-founded reasons, and isn't going back.

> 1) Efficiency. Gems adds a lot of dirs to the load path. While the
> normal require mechanism only has a few. When running a small script,
> no one wants to wait, even for 1 second, for a lib to be found. Is
> there a good way to address this?

Your claim of 1 second is highly implausible:

1000 runs with 77 extra load paths
Rehearsal -------------------------------------------
empty     0.000000   0.000000   0.000000 (  0.000097)
regular   3.160000   0.290000   3.450000 (  3.444110)
many      4.270000   3.920000   8.190000 (  8.210680)
--------------------------------- total: 11.640000sec

               user     system      total        real
empty     0.000000   0.000000   0.000000 (  0.000118)
regular   3.180000   0.290000   3.470000 (  3.484868)
many      4.290000   3.940000   8.230000 (  8.278954)

This shows an average of 1.1 ms more time to search those 77 extra
paths, so you'd need about 77000 load paths active to cause a 1
second delay.  There aren't even 2000 unique gems.

> 2) Organization of the gems/ directory. [...]
> 3) Per the last point. I'd rather see a tiered layout [...]

These aren't changing.

> 4) The gemspec should not be crucial to using the gems repository. The
> only piece of info in there that I think is required is the
> load_paths. Most packages don't need it. So a gemspec for a package
> should be optional --which is necessary to be able to install a
> package manually or via other package systems too.

Try require_paths, dependencies, required_ruby_version, ...

This isn't changing either.

This is not the time to discuss or make major, destabilizing
architectural changes to RubyGems.

This is not the place to be advocating any shopping list for
RubyGems.  The correct place for that is the RubyGems mailing list,
and the RubyGems tracker.

On 7/23/07, Eric Hodel <drbrain@segment7.net> wrote:
> >> do the RubyGems logic of looking in the installed gems and adding to
> >> the LOAD_PATH + requiring on match or 2) require 'rubygems' by
> >> default.
> >
> > No at all. That's actually what I've been advocating. But rather than
> > hobble two different require mechanisms, one on top of the other,
>
> This isn't what Chad is saying.  We're going to add code to call back
> through RubyGems' custom_require from C, or some similar solution.

What isn't he saying? He's talking about having RubyGems requiring 
built-in.

> > we should create an improved unified mechanism -- unifying gems/
> > and site_lib/.  That's really the way to go, though there are some
> > considerations.
>
> This isn't going to happen.  RubyGems abandoned playing in site_lib
> ages ago for various well-founded reasons, and isn't going back.

I'm not talking about playing in site_lib, I've experimented with that
too -- it's a problem. Hover I think that it might be possible to meet
half-way, so to speak.

> regular   3.160000   0.290000   3.450000 (  3.444110)
> second delay.  There aren't even 2000 unique gems.
After bringing this subject up time and time again, you are the fist
to actually give me a concrete reply. I thank you for that. Perhaps
all my concern about it has been for not. Could you should me the code
you used to run the test? I've seen poor performance in loading up
apps before and I was under the impression that had a lot to do with
gems. Take this thread for example:

  http://rubyforge.org/pipermail/rubygems-developers/2004-December/001286.html

But maybe these issues have been addressed since 2004-5? If so great!

>
> Try require_paths, dependencies, required_ruby_version, ...
>
> This isn't changing either.
>
> This is not the time to discuss or make major, destabilizing
> architectural changes to RubyGems.
>
> This is not the place to be advocating any shopping list for
> RubyGems.  The correct place for that is the RubyGems mailing list,
> and the RubyGems tracker.

As I wrote in my first post to this thread, I'm a distinctly NOT
saying this is the moment to maker such changes, and I am all for
RubyGems going into Ruby NOW. But that certainly doesn't precluded
discussion about where that can ultimately lead. Nor do I believe this
is something to be relegated to RubyGems mailing list when it is
clearly about  the more general question of how Ruby loads libraries,
with or without gems.

When RubyGems goes into core/standard where will be the appropriate
place to talk about them then?

T.

On Jul 23, 2007, at 17:32 , TRANS wrote:

>> This isn't what Chad is saying.  We're going to add code to call back
>> through RubyGems' custom_require from C, or some similar solution.
>
> What isn't he saying? He's talking about having RubyGems requiring  
> built-in.

No, he's talking about having the -r flag invoke "require" via
rb_funcall so that require overrides will be available on the command-
line.

On Jul 23, 2007, at 17:32, TRANS wrote:

>> Your claim of 1 second is highly implausible:
>
> Could you should me the code you used to run the test?

I already deleted it.  It took me about two minutes to write, though.

>> This is not the place to be advocating any shopping list for
>> RubyGems.  The correct place for that is the RubyGems mailing list,
>> and the RubyGems tracker.
>
> [...] I am all for RubyGems going into Ruby NOW. But that certainly  
> doesn't precluded discussion about where that can ultimately lead.

Yes, it does.

Discussions of the future of RubyGems beyond importing it into 1.9 is
completely off-topic for both this thread and this mailing list.

> When RubyGems goes into core/standard where will be the appropriate
> place to talk about them then?

The RubyGems mailing list will always be the place for discussions
about RubyGems.

On 7/23/07, Eric Hodel <drbrain@segment7.net> wrote:
> >> and the RubyGems tracker.
> > place to talk about them then?
>
> The RubyGems mailing list will always be the place for discussions
> about RubyGems.

Wow Eric, that was quite an inspective reply. You should thank God
you're a decent programmer b/c one thing is for sure, if you are at
all like this is your day to day affairs, the last thing you have in
this world is anyone with whom to actually converse.

Goodbye and good riddance,
T.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

Thanks all for your discussions.  I updated the list with your comments
and mine.

Note that 'ruby' means MRI in this mail.

1. Is platform-specific gem handling needed in ruby/1.9.1?

  - Is binary gem support needed in ruby/1.9.1?
    (Independent from this, RubyGems will support this useful feature)

2. Does RubyGems need some 'require-hook' feature to be added to
   ruby/1.9.1?  What's the requirements?

  - not needed. integrate RubyGems with ruby's require.
  - just bundle RubyGems with ruby as one of a packaging system.
    so hooking -r options as same as Kernel#require is needed.
  - Nobu, I heard that you have once designed this feature halfly.
    Is there anything you can share with us?

3. What gem related commands should be install in BINDIR by the standard
   installer?
     gem, gemlock, gemri, gemwhich, gem_mirror, gem_server,
     index_gem_repository.rb, update_rubygems

  - nothing should be installed
    ('ruby -rgem -e update gemname' should enough)
  - moving everything under the 'gem' command and install only 'gem'
    (some work needed by RubyGems team)

4. What $LOAD_PATH order should be?

  4-1. by default?
    [-I, ENV_RUBYLIB, SITELIBDIR, RUBYLIBDIR, .] or
    [-I, ENV_RUBYLIB, SITELIBDIR, RUBYLIBDIR, ., GEMs]

   - enable RubyGems by default?
   - opt-in?

  4-2. after requiring rubygems?
    [-I, ENV_RUBYLIB, SITELIBDIR, RUBYLIBDIR, ., GEMs] or
    [-I, ENV_RUBYLIB, SITELIBDIR, GEMs, RUBYLIBDIR, .] or
    [-I, ENV_RUBYLIB, GEMs, SITELIBDIR, RUBYLIBDIR, .]

   - the first one is the current RubyGems behavior and it should not be
     changed as far as RubyGems team do not change it.  the behavior
     must have been polished up in several years.
   - maybe I (NaHi) is the only person who have a very hard time with
     this behavior in soap4r-ML.  it's because soap4r is the only lib
     which is gem-ed and located in RUBYLIBDIR.  that's exactly why I
     run this thread.

5. Where's the global repository for bundled rubygems?
   Of course RubyForge should be pointed.

  - we need some 'rather official' repository at www.ruby-lang.org, too.
  - RubyForge plus its mirrors are sufficient.

6. What libraries does RubyGems depend on?
     YAML/Syck, WEBrick, the digest libraries, rbconfig, rdoc, thread,
     optparse, forwardable, time, openssl, open-uri, uri, net/http,
     fileutils, zlib, stringio, socket, tempfile, pathname

  - YAML is used for the gem index and could be dropped in favor of
    marshal.
  - WEBrick is only used for gem_server and not critical.

7. Discussion deadline?

  - vaguely October or so


There're many other interesting topics discussed.  Note that the above
lists are topics related to ruby/1.9.1.  Feel free to post RubyGems
related topic to this list.  It may not make an appeal to RubyGems team
but it will affect a discussion on ruby's require-hook feature.

'What libraries to be unbundled?' will be posted as the new thread soon.

// NaHi

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2.1 (Cygwin)

iQEVAwUBRqYCOx9L2jg5EEGlAQLekgf+PrvN89MtB8ZkpZHb03RjrT2vkIdCqMPh
6JirtY5qXXHpS4JYX1SUN8bd9E8Z+7NMrPvRoTcfbZ/chsVwH2aKNJRYf/1J2PJ1
CYhzuAtskCvzi2f55PE5EqLDOB3PiTD8alyVxasAdBYs1kLZ9HYGvKvjb589z2Ju
eRdgmaWjeC5YPo0ozeZ7swzYKaQmLPUN0MKpJz2e/hFgqIsxsiyZvZadm++U4xsl
PyqfbhSPf6FgztRgnjdR2CjKUhTuzuWTq8oQ+DKGfaSNbyaFCFLlQYITZwAkxlD7
K7ga7APFbWXU/sAcnBC/hvEmlByeq61VK0GoYJ8Jr5Kdt09LTCFJWw==
=dEVe
-----END PGP SIGNATURE-----

On Jul 24, 2007, at 06:44, NAKAMURA, Hiroshi wrote:
> Thanks all for your discussions.  I updated the list with your  
> comments
> and mine.
>
> Note that 'ruby' means MRI in this mail.
>
> 1. Is platform-specific gem handling needed in ruby/1.9.1?
>
>   - Is binary gem support needed in ruby/1.9.1?
>     (Independent from this, RubyGems will support this useful feature)

binary/platform gem support is not absolutely necessary for 1.9.1.
If the timeframe is inclusion in October, that's plenty of time to
get the feature implemented.

> 2. Does RubyGems need some 'require-hook' feature to be added to
>    ruby/1.9.1?  What's the requirements?
>
>   - not needed. integrate RubyGems with ruby's require.
>   - just bundle RubyGems with ruby as one of a packaging system.
>     so hooking -r options as same as Kernel#require is needed.

Yes.

>     ('ruby -rgem -e update gemname' should enough)
>   - moving everything under the 'gem' command and install only 'gem'
>     (some work needed by RubyGems team)

Right now there are two executables in RubyGems' bin/, gem and
update_rubygems.

I think I can put update_rubygems inside the gem command too, or
remove it outright.

This means just one command 'gem' needs to be installed into BINDIR.

> 4. What $LOAD_PATH order should be?
>
>   4-1. by default?
>     [-I, ENV_RUBYLIB, SITELIBDIR, RUBYLIBDIR, .] or
>     [-I, ENV_RUBYLIB, SITELIBDIR, RUBYLIBDIR, ., GEMs]
>
>    - enable RubyGems by default?
>    - opt-in?

By default, I think:

   [-I, ENV_RUBYLIB, SITELIBDIR, RUBYLIBDIR, .]

>   4-2. after requiring rubygems?
>     [-I, ENV_RUBYLIB, SITELIBDIR, RUBYLIBDIR, ., GEMs] or
>     [-I, ENV_RUBYLIB, SITELIBDIR, GEMs, RUBYLIBDIR, .] or
>     [-I, ENV_RUBYLIB, GEMs, SITELIBDIR, RUBYLIBDIR, .]
>
>    - the first one is the current RubyGems behavior and it should  
> not be
>      changed as far as RubyGems team do not change it.  the behavior
>      must have been polished up in several years.

Actually, the last one is current default:

   [-I, ENV_RUBYLIB, GEMs, SITELIBDIR, RUBYLIBDIR, .]

This works best for developers who want to work on multiple gems at
once, since they can use ruby -I to source the development version
instead of the gem version.

>    - maybe I (NaHi) is the only person who have a very hard time with
>      this behavior in soap4r-ML.  it's because soap4r is the only lib
>      which is gem-ed and located in RUBYLIBDIR.  that's exactly why I
>      run this thread.

If soap4r were unbundled, would this matter anymore?  (I forgot the
details of the problems of soap4r in RUBYLIBDIR and soap4r gem.)

> 5. Where's the global repository for bundled rubygems?
>    Of course RubyForge should be pointed.
>
>   - we need some 'rather official' repository at www.ruby-lang.org,  
> too.
>   - RubyForge plus its mirrors are sufficient.

Could we make gems.ruby-lang.org a CNAME for gems.rubyforge.org?  Or
vice-versa?

gems.rubyforge.org needs to stay around for backwards compatibility,
of course.

RubyGems does have support for changing the repository path
independent of a release built-in.

> 6. What libraries does RubyGems depend on?
>      YAML/Syck, WEBrick, the digest libraries, rbconfig, rdoc, thread,
>      optparse, forwardable, time, openssl, open-uri, uri, net/http,
>      fileutils, zlib, stringio, socket, tempfile, pathname
>
>   - YAML is used for the gem index and could be dropped in favor of
>     marshal.
>   - WEBrick is only used for gem_server and not critical.

Also fileutils, find and test/unit.

> 7. Discussion deadline?
>
>   - vaguely October or so

I should be able to get platform/binary gems feature complete in
August, since I'm taking an extended vacation.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

This mail is for the topic '4. What $LOAD_PATH order should be?'

Eric Hodel wrote:
> 
>   [-I, ENV_RUBYLIB, GEMs, SITELIBDIR, RUBYLIBDIR, .]
> 
> This works best for developers who want to work on multiple gems at
> once, since they can use ruby -I to source the development version
> instead of the gem version.

I may misunderstand custom_require.rb.  I'm seeing custom_require.rb
revision 1285.

  # We replace Ruby's require with our own, which is capable of
  # loading gems on demand.
  #
  # When you call <tt>require 'x'</tt>, this is what happens:
  # * If the file can be loaded from the existing Ruby loadpath, it
  #   is.
  # * Otherwise, installed gems are searched for a file that matches.
  #   If it's found in gem 'y', that gem is activated (added to the
  #   loadpath).
  #
  # The normal <tt>require</tt> functionality of returning false if
  # that file has already been loaded is preserved.

Custom 'require' at first try to load the specified feature name from;
[-I, ENV_RUBYLIB, SITELIBDIR, RUBYLIBDIR, .]
When it raises LoadError then add GEMs at the top of $LOAD_PATH and try
to load the feature from;
[GEMs, -I, ENV_RUBYLIB, SITELIBDIR, RUBYLIBDIR, .]
I concaticated 2 arys and called Array#uniq (I hope Array#uniq keeps its
order in the future, too).  Is this wrong?

And when we call 'gem "soap4r"' explicitly, the $LOAD_PATH is;
[GEMs, -I, ENV_RUBYLIB, SITELIBDIR, RUBYLIBDIR, .]
isn't it?

I should have wrongly summarized the problem.  Can you sort out a
problem about $LOAD_PATH?

>>    - maybe I (NaHi) is the only person who have a very hard time with
>>      this behavior in soap4r-ML.  it's because soap4r is the only lib
>>      which is gem-ed and located in RUBYLIBDIR.  that's exactly why I
>>      run this thread.
> 
> If soap4r were unbundled, would this matter anymore?  (I forgot the
> details of the problems of soap4r in RUBYLIBDIR and soap4r gem.)

No, it won't be the matter anymore.  So the followings are just an
explanation what was happened.

Say an user gets ruby/1.8.6 (based on soap4r-1.5.5).  Then installs
soap4r-1.5.6 from tarball later (files are copied to site_lib dir).
 Finally moves to RubyGems and installs soap4r-1.5.7 as a gem.

soap4r-1.5.5 has a feature 'soap/a'.
soap4r-1.5.6 adds new feature 'soap/b'.
soap4r-1.5.7 adds another new feature 'soap/c'.

soap/c depends on soap/b.  soap/b depends on soap/a.
soap/a and soap/b are updated in each new version.

  /usr/lib/ruby/1.8/soap/a                            # soap4r-1.5.5
  /usr/lib/ruby/1.8/site_ruby/soap/a                  # soap4r-1.5.6
  /usr/lib/ruby/1.8/site_ruby/soap/b                  # soap4r-1.5.6
  /usr/lib/ruby/gems/1.8/gems/soap4r-1.5.7/lib/soap/a # soap4r-1.5.7
  /usr/lib/ruby/gems/1.8/gems/soap4r-1.5.7/lib/soap/b # soap4r-1.5.7
  /usr/lib/ruby/gems/1.8/gems/soap4r-1.5.7/lib/soap/c # soap4r-1.5.7

Imagine what is loaded when the user runs 'require "soap/c"'.
 Thankfully there's an easy answer how to avoid this; to load the
consistent feature versions, add 'gem "soap4r"' somewhere in a program.
 Once an user found this answer, it gets easier to track down a loader
problem in soap4r-ml.  The rest I want to know is where 'gem "soap4r"'
should be added in RoR environment... (I've not yet been a RoR user.)

// NaHi

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Cygwin)

iQEVAwUBRqa7LB9L2jg5EEGlAQIYaQf/Srn1d9JuqD+3z9wX3lscK2SsJjphDHHf
bMrSz73gDkExvcsvk13YGzqiZTzWu1OdL3Kxf8SUkrQYNz0I6nQ+aw4sFqglhLqU
NwFuViUY+KxeVxR8OP62i7Q+luGcCuA2ziXaopAZ/4C/McLsi93KvE1PXiM/vAAP
WTMcfAOcjTnP177kZbVPxFajZOp4PSLw5mh++FwxF9pdyzLxGoWrEiY8RxbuSthp
vnKC/9+q2+zNnxZwvrIed38W03ifL2PmA0DsQRYk8lsxArD757G4L7baVCgGseA9
LdxvU4Jz0XylbCECWxGTiBquJhcDArVnjWFbaV96VHZjvA5ctuF8aw==
=uSVl
-----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

This mail is for the topic
'5. Where's the global repository for bundled rubygems?'

Eric Hodel wrote:
>> 5. Where's the global repository for bundled rubygems?
>>    Of course RubyForge should be pointed.
>>
>>   - we need some 'rather official' repository at www.ruby-lang.org, too.
>>   - RubyForge plus its mirrors are sufficient.
> 
> Could we make gems.ruby-lang.org a CNAME for gems.rubyforge.org?  Or
> vice-versa?

Matz, shugo, what do you think?

And I should write the reason why I add '- we need some 'rather
official' repository at www.ruby-lang.org, too.' line;

When we unbundle some libraries which are bundled with ruby-1.8.X such
as tk, WEBrick or rss, Matz (and core developers) may think they want to
treat these gems as 'rather official' than gems in gems.rubyforge.org.
 Note that it's only my thought.  Matz said nothing about it as far as I
know.

Matz once allowed (perhaps 'just did not forbid' is correct) these libs
to be called 'standard library', 'std-lib' in ruby-1.8.X. (I always
write "bundled" instead of "standard" though)  So there can be an
opinion to create 'rather official' repository I though.  It also will
help other package managers (deb, rpm, etc) to decide what gems should
be packaged with ruby itself.

I just raised this opinion for discussion.  And current RubyGems should
not have such a feature to distinguish two repositories (right?) so it
can't be actualized.  (and unbundled soap4r gem will completely live
with gems.rubyforge.org :-)

// NaHi

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Cygwin)

iQEVAwUBRqbAMB9L2jg5EEGlAQJ8fwgAgiRbUaIrKSQSws6FhT3MubG2VYkgOoWZ
ZD1QEZoijhIEMKI+SQsg3MkBJ8eYwWFPoImphLh/ZSgtO14J4lfqO1f5D/ZcEKO3
2Fq54bKOc+2UpgEblgX3C1vgdTUGcdndtr1ETVHAyEZOp0na/RM85Xo/drSi1QNr
bduOPWZAogTL7Dr3vBidXcDj+Y3g+w6FSdgD4hlclkgYppIC7OUM9BHyM1TIbWbc
M9gOpyVZmv/tknHqLKnUvvl9ZICBObOAyVtu/z4xCqlrBh4hkufWFX1cfzUgm7ne
FvpZ6PCn0XP7v/B+OfaPIziayfhMMDXXqCjamN7KRakVmpyBxTM5jA==
=P5Cw
-----END PGP SIGNATURE-----

On Jul 24, 2007, at 20:15, NAKAMURA, Hiroshi wrote:

> >
> want to
> help other package managers (deb, rpm, etc) to decide what gems should
> be packaged with ruby itself.

Oh! I understand now.  I agree with you.  There is no need to merge a
ruby-lang.org repository with gems.rubyforge.org.  They can and
should stand separate.

> I just raised this opinion for discussion.  And current RubyGems  
> should
> not have such a feature to distinguish two repositories (right?) so it
> can't be actualized.  (and unbundled soap4r gem will completely live
> with gems.rubyforge.org :-)

RubyGems does have the ability to fetch gems from multiple
repositories.  These can be managed with `gem sources`.  An example
third-party repository is http://code.whytheluckystiff.net/ (http://
code.whytheluckystiff.net/yaml has the index of gems).

There is no ordering of repositories, so you can't say which
repository gets picked first.  This should be fixed to prevent
poisoning users with malicious packages.

Hi,

On 07/25/2007 12:15 PM, NAKAMURA, Hiroshi wrote:
>>> 5. Where's the global repository for bundled rubygems?
>>>    Of course RubyForge should be pointed.
>>>
>>>   - we need some 'rather official' repository at www.ruby-lang.org, too.
>>>   - RubyForge plus its mirrors are sufficient.
>> Could we make gems.ruby-lang.org a CNAME for gems.rubyforge.org?  Or
>> vice-versa?
>
> Matz, shugo, what do you think?

It sounds good to provide 'rather official' repository at
www.ruby-lang.org or gems.ruby-lang.org (I prefer the
latter), but we need a way to upload such a gem to
www.ruby-lang.org.  What do you think?

Should only committers be able to upload a gem to
www.ruby-lang.org?

Shugo

On Jul 24, 2007, at 19:53, NAKAMURA, Hiroshi wrote:

> >>      changed as far as RubyGems team do not change it.  the  
>
>   #   If it's found in gem 'y', that gem is activated (added to the
>   #   loadpath).
>   #
>   # The normal <tt>require</tt> functionality of returning false if
>   # that file has already been loaded is preserved.

The work of adding the gem's require_paths to $LOAD_PATH happens in
Gem::activate.

> Custom 'require' at first try to load the specified feature name from;
> [-I, ENV_RUBYLIB, SITELIBDIR, RUBYLIBDIR, .]
> When it raises LoadError then add GEMs at the top of $LOAD_PATH and  
> try
> to load the feature from;
> [GEMs, -I, ENV_RUBYLIB, SITELIBDIR, RUBYLIBDIR, .]

The behavior you describe is how 0.9.2 and previous worked.

> I concaticated 2 arys and called Array#uniq (I hope Array#uniq  
> keeps its
> order in the future, too).  Is this wrong?

I have experienced problems where gems were activated despite the
files being present in the -I path.  This is a developer-only problem.

I believe my problems involved dependent gems.  When RubyGems
activates a gem, it also activates all its dependencies.

With GEMs before -I, the gem version can get loaded instead of the -I
version.

With GEMs after -I, the -I version will always be preferred.

I would need to experiment to reproduce the behavior I was seeing.

> And when we call 'gem "soap4r"' explicitly, the $LOAD_PATH is;
> [GEMs, -I, ENV_RUBYLIB, SITELIBDIR, RUBYLIBDIR, .]
> isn't it?

In 0.9.2, and earlier, this is correct.

Now RubyGems looks for SITELIBDIR and inserts the gem's require_paths
before it:

   $LOAD_PATH.insert($LOAD_PATH.index(sitelibdir, *require_paths))

See line 270-277 of lib/rubygems.rb.  (Hrm, line 268 may be wrong.)

> >
> soap4r-1.5.5 has a feature 'soap/a'.
>   /usr/lib/ruby/gems/1.8/gems/soap4r-1.5.7/lib/soap/b # soap4r-1.5.7
>   /usr/lib/ruby/gems/1.8/gems/soap4r-1.5.7/lib/soap/c # soap4r-1.5.7
>
> Imagine what is loaded when the user runs 'require "soap/c"'.
>  Thankfully there's an easy answer how to avoid this; to load the
> consistent feature versions, add 'gem "soap4r"' somewhere in a  
> program.

On require 'soap/c' you should get 1.5.7 features.

On require 'soap/b' then 'soap/c', problems may result without 'gem
"soap4r"' first.

>  Once an user found this answer, it gets easier to track down a loader
> problem in soap4r-ml.  The rest I want to know is where 'gem "soap4r"'
> should be added in RoR environment... (I've not yet been a RoR user.)

config/environment.rb is the correct place in RoR.

On Jul 24, 2007, at 23:11, Shugo Maeda wrote:
>> Matz, shugo, what do you think?
>
> It sounds good to provide 'rather official' repository at
> www.ruby-lang.org or gems.ruby-lang.org (I prefer the
> latter), but we need a way to upload such a gem to
> www.ruby-lang.org.  What do you think?

Also, an index needs to be built when a gem is uploaded.  (Or a cron
job needs to be run to generate the index.)

On Jul 24, 2007, at 23:11 , Shugo Maeda wrote:

> Should only committers be able to upload a gem to
> www.ruby-lang.org?

YES

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

Eric Hodel wrote:
>> This mail is for the topic '4. What $LOAD_PATH order should be?'

>>   # The normal <tt>require</tt> functionality of returning false if
> 
> The behavior you describe is how 0.9.2 and previous worked.

Doh.  Sorry for bothering you.  I updated rubygems with
'gem update --system' and I confirmed that the behavior change you
explained.

> With GEMs before -I, the gem version can get loaded instead of the -I
> version.
> 
> With GEMs after -I, the -I version will always be preferred.

Agreed.

So the topic 4-2 should be;

  4. What $LOAD_PATH order should be?
    4-2. after requiring rubygems?
      [-I, ENV_RUBYLIB, GEMs, SITELIBDIR, RUBYLIBDIR, .]

As I wrote in the latest summary, this behavior should be up to RubyGems
team.  RubyGems team can control this behavior.

Then why I raised the topic to this thread?  Because I considered that
RubyGems may be enabled by default.

Eric,

  - you are thinking that $LOAD_PATH does not include GEMs dirs by
    default.
  - you think that 'hooking -r option' is all the requirement for
    ruby/1.9.1 to bundle RubyGems.

Right?  So the expected behavior is as follows?

% gem install httpclient
(done)
% ruby -rhttpclient -e 'p HTTPClient.get_content("http://localhost/")'
=> ruby: no such file to load -- httpclient (LoadError)
% ruby -rgem -rhttpclient -e '...'
=> (runs fine)

Of course we can direct users to use RUBYOPT=-rgem trick to hide this
detail though.

I think RubyGems should not be enabled by default but once after using
RubyGems, GEMs dir should be checked without any trick until the time I
run 'gem cleanup'.  Of course any other Ruby based new packaging system
can do the same thing.  Isn't it the behavior we expect to a packaging
system?

As you know, it requires rather complicated 'require-hook' feature to
ruby such as;

 - ruby checks "custom_require.rb" file in SITELIBDIR and load it before
   enabling any feature.

It may be too much for ruby/1.9.1.  'To use gem, run ruby with "-rgem"
always' can be accepted for the first step I think.

>>  Once an user found this answer, it gets easier to track down a loader
>> problem in soap4r-ml.  The rest I want to know is where 'gem "soap4r"'
>> should be added in RoR environment... (I've not yet been a RoR user.)
> 
> config/environment.rb is the correct place in RoR.

Thanks!  It much help soap4r users.

// NaHi

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Cygwin)

iQEVAwUBRqf+6x9L2jg5EEGlAQIkGQf/VypyqXj1yASSuaZSAtBVIzvHUkX+KiYj
z0GJjVztRXtrSlyqECoLNfm08hq9rsnQYR0p4ObTPzZU5GFADbVArnEE+gmFr/Mx
HN5nI7cYynt2ktHiNFuBS3z3uqLdyUBZM9u1nZPV51RQ5lvab7TV+mcgVrPqbI7X
vldMonN36DdB0PWyyZhYurq9RF1rEnYwYpDsljcv8ryo+Ourr4E+evkTkRgLrtOm
Hoee7UxngppEaZo/QegGrYFYwGh6wB/sG2mTrdDLbz2xIJMXMgte9P6+Yhn+rG39
BXAYY4/Qx9WROcUIyE48DT2IA3u/Y/gy5eWRXk8isGi5f09nA/xwQw==
=oWFH
-----END PGP SIGNATURE-----

On Jul 25, 2007, at 18:55, NAKAMURA, Hiroshi wrote:
> matches.
> from;
> explained.
>
>
> (done)
> % ruby -rhttpclient -e 'p HTTPClient.get_content("http://localhost/")'
> => ruby: no such file to load -- httpclient (LoadError)
> % ruby -rgem -rhttpclient -e '...'
> => (runs fine)
>
> Of course we can direct users to use RUBYOPT=-rgem trick to hide this
> detail though.

Yes.  Not everybody will need RubyGems.  For example many Rails
installations run with everything unpacked into a local directory for
easy deployment.

I would prefer users use RUBYOPT instead of enabling RubyGems by
default.

>
>  - ruby checks "custom_require.rb" file in SITELIBDIR and load it  
> before
>    enabling any feature.
>
> It may be too much for ruby/1.9.1.  'To use gem, run ruby with "-rgem"
> always' can be accepted for the first step I think.

I was hoping we could use rb_funcall to invoke a Kernel#require in
require_libraries() rather than calling the C rb_require directly.
Is this possible?

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

This mail is for the topic
'6. What libraries does RubyGems depend on?'

NAKAMURA, Hiroshi wrote:
> 6. What libraries does RubyGems depend on?
>      YAML/Syck, WEBrick, the digest libraries, rbconfig, rdoc, thread,
>      optparse, forwardable, time, openssl, open-uri, uri, net/http,
>      fileutils, zlib, stringio, socket, tempfile, pathname
       + test/unit
> 
>   - YAML is used for the gem index and could be dropped in favor of
>     marshal.
>   - WEBrick is only used for gem_server and not critical.

Dependency to openssl is a must? (I don't think so but asking for sure.)
I think there's no problem other than it.

// NaHi

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Cygwin)

iQEVAwUBRqhUah9L2jg5EEGlAQLNpAgAj82/tV+0pz4kLwGeW/9OqhN77/s67eE6
T0oXn85fLET25R1jg7GB9OfzKSZdmOMidyDXxZvT23unpEj63iGeIctkWy+VW8fv
LtQt+OgeaBdH0zOkBVtORs3Us65wOg/IOopbkJr+prwgsmLV27/A2UGrkYKFMXDe
n2Wx7zK6CeAyw14FahfR/vzJ0MP7CM4bXCThMHeRftTgnLC7P0Xve6AIH1G/KD4H
3e/Y4YdNS7Wv1CZYIkIaBv2KwsLlsZLawwS6DwzHCud1CwoQR5XWNEsYrzHDKFtz
bRKkyiN+qrTPbIJJHI1YQ8CfCaei5kS2Goj8Oh+FjQIMj1uYP4mocA==
=Jv/J
-----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

Updated the list.

1. Is platform-specific gem handling needed in ruby/1.9.1?

  - Is binary gem support needed in ruby/1.9.1?
    If no one has any objection, it's up to RubyGems team.

2. Does RubyGems need some 'require-hook' feature to be added to
   ruby/1.9.1?  What's the requirements?

  - just bundle RubyGems with ruby as one of a packaging system.
    so hooking -r options as same as Kernel#require is needed.
  - Nobu, I heard that you have once designed this feature halfly.
    Is there anything you can share with us?
  - I was hoping we could use rb_funcall to invoke a Kernel#require in
    require_libraries() rather than calling the C rb_require directly.
    Is this possible?

3. What gem related commands should be install in BINDIR by the standard
   installer?
     gem, gemlock, gemri, gemwhich, gem_mirror, gem_server,
     index_gem_repository.rb, update_rubygems

  - just one command 'gem' to be installed into BINDIR.
  - is there anyone who thinks nothing should be installed?

4. What $LOAD_PATH order should be?

  4-1. by default
    [-I, ENV_RUBYLIB, SITELIBDIR, RUBYLIBDIR, .]
   - RubyGems is not enabled without 'rubygems' library required.

  4-2. after requiring rubygems
    [-I, ENV_RUBYLIB, GEMs, SITELIBDIR, RUBYLIBDIR, .]
   - it's up to RubyGems team because it's opt-in feature.

5. Where's the global repository for bundled rubygems?

  - of course RubyForge should be pointed.
  - prepare gems.ruby-lang.org and add it as a default remote source of
    RubyGems?
  - remote sources ordering support of RubyGems (by RubyGems team.)
  - we need some 'rather official' repository at gems.ruby-lang.org.
  - no, RubyForge plus its mirrors are sufficient.

6. What libraries does RubyGems depend on?
     YAML/Syck, WEBrick, the digest libraries, rbconfig, rdoc, thread,
     optparse, forwardable, time, open-uri, uri, net/http, fileutils,
     zlib, stringio, socket, tempfile, pathname, test/unit

  - YAML is used for the gem index and could be dropped in favor of
    marshal.
  - WEBrick is only used for gem_server and not critical.
  - Dependency to openssl is a must?

7. Discussion deadline?

  - vaguely October or so

// NaHi
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Cygwin)

iQEVAwUBRqhgNh9L2jg5EEGlAQILiAf/UiKo74Y4q7/1024hmjzmvrRLH7a5Cnbm
s1CmQkPWgNgrA/EI2ooZ/dvoFfhoa+4dNhUTg2expfTeEtbYOxgWuGEi9DM+LOOy
UGV2d5oIMvs5bDkBqi6komLwrdvI+p1gsqfpg9TLBX8Y9Kfp/b0ZaDRnFIWYn3qJ
Ee5MFRTsYeN7nZxioUliwgH4DHUaxsjw6xZB6DzgrG4oIwDvuR+VaPTI1A9B98gn
PK7n/CxBM1wU6J7S7qE4WWE7roKcp3l/v83UhaiNWwpkBru7NiMv7qecSkM+TNmM
M3D4kGB/5aJVf6alIkjyB2A/kx5+4MEKj4Qv4M6rO16UgfnHBNTJ7w==
=vWHF
-----END PGP SIGNATURE-----

2007/7/26, NAKAMURA, Hiroshi <nakahiro@sarion.co.jp>:
>        + test/unit
>

Here is a small script to check the number of loaded classes before and
after rubygems. The second number only includes top level classes.

Last time I checked, rubygems used more than one top level class to host
it's code.

-----
def count_classes
    x=0; t=0
    ObjectSpace.each_object(Class) do |c|
        x+=1
        t+=1 if c.name !~ /::/
    end
    [x, t]
end

puts "Without rubygems : #{count_classes.inspect}"
require 'rubygems'
puts "With rubygems : #{count_classes.inspect}"

-----

Without rubygems : [183, 59]
With rubygems : [469, 76]

On Aug 3, 2007, at 12:13 PM, Jonas Pfenniger wrote:

> >      fileutils, zlib, stringio, socket, tempfile, pathname
>     x=0; t=0
>
> -----
>
> Without rubygems : [183, 59]
> With rubygems : [469, 76]

To this effect, what about structuring rubygems so that not all of
it's classes are loaded all the time.

There are 2 use scenarios for "require 'rubygems'". One is typically
one, to be able to use the code contained in gems, the other is for
the 'gem' command to be able to do all it's manipulation and fetching
of gems.

Currently, "require 'rubygems'" is built for the 2nd scenario, but
thats probably incorrect, since that scenario is used vastly less
than scenario one.

If "require 'rubygems'" only included enough code be able to properly
alter the load path to allow a gem's code to be used, it would make
almost no impact on performance, where as now, it includes a huge
amount of unused code into every runtime.

Scenario 2 could easily covered by adding "require 'rubygems/dev'"
which would include the current bevy of classes and code.

After all, what generic rails app uses the TarReader class that
rubygems imports? For that that matter, what part of the rubygems
runtime (scenario 1) uses TarReader?

  - Evan Phoenix

On Aug 3, 2007, at 12:31, Evan Phoenix wrote:
>> >      optparse, forwardable, time, openssl, open-uri, uri, net/http,
> If "require 'rubygems'" only included enough code be able to  
> properly alter the load path to allow a gem's code to be used, it  
> would make almost no impact on performance, where as now, it  
> includes a huge amount of unused code into every runtime.

I'm two days ahead of you:

$ grep -A4 2007-08-01 ChangeLog
2007-08-01  Eric Hodel  <drbrain@segment7.net>

         * lib/*, test/*: Pushed down dependencies as far as reasonable.
         Removed Gem::manage_gems.  Now only 14 files are loaded with
"require
         'rubygems'".  Added tests for `gem build` and `gem cert`.

There might be a bit more trimming to do, but I think that's best
discussed on the RubyGems mailing list.

On 7/25/07, Ryan Davis <ryand-ruby@zenspider.com> wrote:
> On Jul 24, 2007, at 23:11 , Shugo Maeda wrote:
> > Should only committers be able to upload a gem to
> > www.ruby-lang.org?
> YES

Also, all gems on www.ruby-lang.org should be signed. IMO.

-austin

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

It's October, the deadline.  We need to go ahead.

NAKAMURA, Hiroshi wrote:
> 1. Is platform-specific gem handling needed in ruby/1.9.1?
> 
>   - Is binary gem support needed in ruby/1.9.1?
>     If no one has any objection, it's up to RubyGems team.

drbrain, would you please tell me how it goes on?  Incorporating
'platform-specific gem handling' feature to ruby/1.9 is up to RubyGems 
team.

> 2. Does RubyGems need some 'require-hook' feature to be added to
>    ruby/1.9.1?  What's the requirements?
> 
>   - just bundle RubyGems with ruby as one of a packaging system.
>     so hooking -r options as same as Kernel#require is needed.
>   - Nobu, I heard that you have once designed this feature halfly.
>     Is there anything you can share with us?
>   - I was hoping we could use rb_funcall to invoke a Kernel#require in
>     require_libraries() rather than calling the C rb_require directly.
>     Is this possible?


As the result of long (and sparse :-) discussion on ruby-dev, r13580 did
the change.  The last one of the above list.  Nobu and I are still
talking about generic 'require-hook' feature to avoid ad-hoc
Kernel#require rewriting so any comments are welcome about this.
Anyway, go ahead and let's bundle RubyGems into ruby/1.9.

drbrain, would you please check if r13580 is enough for RubyGems and
tell us the result?

> 3. What gem related commands should be install in BINDIR by the standard
>    installer?
>      gem, gemlock, gemri, gemwhich, gem_mirror, gem_server,
>      index_gem_repository.rb, update_rubygems
> 
>   - just one command 'gem' to be installed into BINDIR.
>   - is there anyone who thinks nothing should be installed?

Let's bundle just one command 'gem'.

> 4. What $LOAD_PATH order should be?
> 
>   4-1. by default
>     [-I, ENV_RUBYLIB, SITELIBDIR, RUBYLIBDIR, .]
>    - RubyGems is not enabled without 'rubygems' library required.
> 
>   4-2. after requiring rubygems
>     [-I, ENV_RUBYLIB, GEMs, SITELIBDIR, RUBYLIBDIR, .]
>    - it's up to RubyGems team because it's opt-in feature.

No objections raised.

> 5. Where's the global repository for bundled rubygems?
> 
>   - of course RubyForge should be pointed.
>   - prepare gems.ruby-lang.org and add it as a default remote source of
>     RubyGems?
>   - remote sources ordering support of RubyGems (by RubyGems team.)
>   - we need some 'rather official' repository at gems.ruby-lang.org.
>   - no, RubyForge plus its mirrors are sufficient.

I think we are getting a consensus that we need some official repository
at gems.ruby-lang.org.  Hmm.  What is different from general gems?

I think that an official gem should be;
  * of course the gem author maintains bugfixes etc. but,
  * ruby maintainer (matz, ko1 for ruby/1.9) should have final approval
    for updating gems.

Right?  Tell me what did you think.

And we need to discuss about gem signing and trustness control of
official gem.  Needed?  Secure?  Operation workable?  etc.

> 6. What libraries does RubyGems depend on?
>      YAML/Syck, WEBrick, the digest libraries, rbconfig, rdoc, thread,
>      optparse, forwardable, time, open-uri, uri, net/http, fileutils,
>      zlib, stringio, socket, tempfile, pathname, test/unit
> 
>   - YAML is used for the gem index and could be dropped in favor of
>     marshal.
>   - WEBrick is only used for gem_server and not critical.
>   - Dependency to openssl is a must?

Matz said that he can be the chief maintainer of yaml.rb and syck for
ruby/1.9.  So the dependency to YAML is not matter now.  Now it's up to
RubyGems team.

> 7. Discussion deadline?
> 
>   - vaguely October or so

End of October, I hope.

Regards,
// NaHi

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Cygwin)

iQEVAwUBRwCMAh9L2jg5EEGlAQIotgf/SPBL7HTiJ2zBMkazNghPVxkMpFG9xXUM
B5glIJ83HcrHVZZ6Eb9Pqrq/QCFD48UbuMZtGnSbTR9zmPHvY7lBPskgUhitKkIh
k3gG2apKYbEOtuDGTGGivgDrhHr+uWfupuNURKAjfuWo69TXDgzshGXOPXVXgSTI
Z5GXaX5/IuHHlt6tSWIwSDrNTd+6RzKjMrwaUflpdBtJtSMq6oBmd05SJ75WBuNG
xGVHd8swrl/m/HbQOv8TRG5xGjtqvfGrR+3CwX/eCTalrii+UGO2XKgZ04yB0ro/
IESv7VhymS8xnzxUQNQKWc8Xg/CTwg6CGCyUzHLvqNQHl0A9fawyHQ==
=BtD4
-----END PGP SIGNATURE-----

On Sep 30, 2007, at 22:56 , NAKAMURA, Hiroshi wrote:
> It's October, the deadline.  We need to go ahead.
> NAKAMURA, Hiroshi wrote:
> > 1. Is platform-specific gem handling needed in ruby/1.9.1?
> >
> >   - Is binary gem support needed in ruby/1.9.1?
> >     If no one has any objection, it's up to RubyGems team.
>
> drbrain, would you please tell me how it goes on?  Incorporating
> 'platform-specific gem handling' feature to ruby/1.9 is up to  
> RubyGems team.

The platform-specific gem handling feature is done, barring potential
minor changes in the upcoming week or two.  I expect to release a
beta of RubyGems this week, and an official release next week.

This would not hold up including RubyGems in 1.9, as I could roll any
additional changes in as necessary.

All tests pass on 1.9 except for one, which is caused by a minor 1.9
bug.  I will email separately about it.

> directly.
> drbrain, would you please check if r13580 is enough for RubyGems and
> tell us the result?

Ok.  I will do this tonight (Pacific Time).

> > 3. What gem related commands should be install in BINDIR by the  
> standard
> >    installer?
> >      gem, gemlock, gemri, gemwhich, gem_mirror, gem_server,
> >      index_gem_repository.rb, update_rubygems
> >
> >   - just one command 'gem' to be installed into BINDIR.
> >   - is there anyone who thinks nothing should be installed?
>
> Let's bundle just one command 'gem'.

Yes.  This is the only command needed.

> No objections raised.
Great.

> I think we are getting a consensus that we need some official  
> repository
> at gems.ruby-lang.org.  Hmm.  What is different from general gems?
>
> I think that an official gem should be;
>   * of course the gem author maintains bugfixes etc. but,
>   * ruby maintainer (matz, ko1 for ruby/1.9) should have final  
> approval
>     for updating gems.
>
> Right?  Tell me what did you think.

I think this is a good policy.

> And we need to discuss about gem signing and trustness control of
> official gem.  Needed?  Secure?  Operation workable?  etc.

RubyGems has the ability to sign and verify downloaded gems against
their signatures, and to generate keys for signing gems.  I only know
the basics of how to use these features, though.

Currently it is up to RubyGems users to enable verification of remote
gems.  By default it does not verify the integrity of the gem.

> >   - Dependency to openssl is a must?
>
> Matz said that he can be the chief maintainer of yaml.rb and syck for
> ruby/1.9.  So the dependency to YAML is not matter now.  Now it's  
> up to
> RubyGems team.

RubyGems now has both a Marshal and YAML index, but will be
publishing both versions in case the Marshal format changes.  Also,
YAML will be needed for backwards compatibility with alternate
repositories that don't yet build a Marshal index.

Currently 1.8 and 1.9 use the same Marshal version, but I don't know
if that is guaranteed, and there doesn't seem to be a way to build
multiple Marshal indexes without multiple ruby interpreters.

(This addition of the Marshal index was made primarily for memory
usage reasons.  A full update of the Marshal index uses about 1/3 the
memory of a full update of the YAML index.)

There may be a few gems in YAML archive format, but I believe that
nobody will install those, they are very old.

The ~/.gemrc configuration file is in YAML, but it would be easy to
parse without YAML.

> > 7. Discussion deadline?
> >
> >   - vaguely October or so
>
> End of October, I hope.

By second week of October, I hope.

On Oct 1, 2007, at 09:57 , Eric Hodel wrote:
>> RubyGems team.
>
> The platform-specific gem handling feature is done, barring  
> potential minor changes in the upcoming week or two.  I expect to  
> release a beta of RubyGems this week, and an official release next  
> week.
>
> This would not hold up including RubyGems in 1.9, as I could roll  
> any additional changes in as necessary.

I am ready to import RubyGems into trunk.

Here is what I propose:

rubygems/trunk/lib      => ruby/trunk/lib
rubygems/trunk/bin      => ruby/trunk/bin
rubygems/trunk/test     => ruby/trunk/test/rubygems
RubyGems documentation  => ruby/trunk/doc/rubygems

Are there any other outstanding issues before I go ahead?

I see a thread in ruby-dev about importing RubyGems, but I cannot
read Japanese.

>> directly.
>> drbrain, would you please check if r13580 is enough for RubyGems and
>> tell us the result?
>
> Ok.  I will do this tonight (Pacific Time).

Sorry for the delay.  It works perfectly.

At 15:01 07/10/13, Eric Hodel wrote:

>
>I see a thread in ruby-dev about importing RubyGems, but I cannot  
>read Japanese.

I haven't read the whole thread in detail, but one of the latest
developments was a discussion on how exactly to include RubyGems
in 1.9. There was quite some support for default inclusion
(i.e. no longer any need to do "require 'rubygems'"), but there
was one big problem, namely that "require 'rubygems'", including
the libraries used, would need a very serious amount of memory.

There was therefore the proposal for a 'diet' version of rubygems
that would only include the bare necessities for then including
other rubygems by 'require'. It's not yet clear how exactly they
plan to integrate rubygems, but I guess it would help move the
decision process ahead if you could give any idea of how easy/quickly
you (or somebody else) could produce such a 'diet' version, and
what size you would expect.

Regards,    Martin.


#-#-#  Martin J. Du"rst, Assoc. Professor, Aoyama Gakuin University
#-#-#  http://www.sw.it.aoyama.ac.jp       mailto:duerst@it.aoyama.ac.jp

Hi,

In message "Re: Import gem to Ruby 1.9"
    on Sat, 13 Oct 2007 15:01:36 +0900, Eric Hodel 
<drbrain@segment7.net> writes:

|Are there any other outstanding issues before I go ahead?
|
|I see a thread in ruby-dev about importing RubyGems, but I cannot  
|read Japanese.

We are discussing how we call rubygems after the merge.  Possible
options are:

  (a) just merge; if one wants to load libraries from gems, he needs
      to require 'rubygems' first.  very traditional, but introduces
      unnecessary normal library/gem'ed library separation.

  (b) silently require 'rubygems' in initialization process.  rubygems
      requires a lot of other libraries, so that memory consumption
      might be an issue.

  (c) create (or separate) tiny library only for finding gems, and let
      it be loaded in the initialization (from prelude.rb).  but no
      one knows how easy/difficult to make such a library (yet).

How do you think?

              matz.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

Yukihiro Matsumoto wrote:
> 
>   (c) create (or separate) tiny library only for finding gems, and let
>       it be loaded in the initialization (from prelude.rb).  but no
>       one knows how easy/difficult to make such a library (yet).
> 
> How do you think?

I needed to confirm the sense of core team about (c) first.  Sorry not
posting this to ruby-dev first.  I was waiting ko1 posting about this 
topic.

Matz, ko1 and akr, do you think that (c) includes gem version control,
for example 'gem "soap4r", "1.5.5"' ?  Need to confirm the requirement
of core team first.

Drbrain, sorry about this confusion.  Core team raised an objection to
[ruby-core:11803]
http://blade.nagaokaut.ac.jp/cgi-bin/vframe.rb/ruby/ruby-core/11625?11588-12637+split-mode-vertical
at this late date.  They think that gem loading feature should be
enabled by default. (Topic 4-1 arrived to the conclusion (a) above.)
OK, I know it's better than not discussed before imported.  Core team
should have been busy on M17N I think.

I may write my opinion later but basically the intent of RubyGems team
must come first.  We won't be able to maintain rubygems without
cooperation by RubyGems team.

Regards,
// NaHi

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Cygwin)

iQEVAwUBRxCJCR9L2jg5EEGlAQK/Mwf/SBr9A3NhXQZ2UsAy0OmRoBsO/mHnbEfI
Me2Pe0pQF0gLpuFP1m1OrOwghi0XyWnhyZbla1VigjJF6VfLcUeD4LoVjkNrOpkL
sxrNMoEF8kF9Wx1O0EGAXhW9T4tJWTFj0ofXHnyGEUP7JTY9w0vHs5tHLeqDQ71X
QvEIPE8kPuQGbqVvdtgzpsiosFLpwQwucnxxAkssB5jpvCbHxISYQmaKdwJx4WnC
5F33mxeo3RYGAvy2GXJIHGQzoA7vkcOL0raPX84RqQgUdpTr/vZ1GZmIBtMbyskP
SBoNpEBSYu8fgzGFXSyfnPbUVA8r0PPh0+BG1FECwdu55bLaYu9Pnw==
=lRqY
-----END PGP SIGNATURE-----

On Oct 13, 2007, at 01:24 , Yukihiro Matsumoto wrote:
> options are:
>       it be loaded in the initialization (from prelude.rb).  but no
>       one knows how easy/difficult to make such a library (yet).
>
> How do you think?

I would like to do (b), but (a) is acceptable to me.  I think it is
possible to remove most of the files RubyGems loads with require
'rubygems' using caching, which would be an easier alternative to
(c), however this would be probably not be much easier.

To provide more information, RubyGems has changed the amount of files
it loads for require 'rubygems' since 0.9.4 to reduce the amount of
memory used and to decrease startup time.  Most of the files loaded
in 0.9.4 were completely unnecessary.

I have 48 gems installed, and these are the differences between
RubyGems 0.9.4 and 0.9.4.5.

RubyGems 0.9.4:
1.34MB for ruby18
5.36MB for ruby18 + require 'rubygems'
6.63MB for ruby18 + require 'rubygems' + require 'zentest'

RubyGems 0.9.4.5:
1.34MB for ruby18
3.26MB for ruby18 + require 'rubygems'
3.85MB for ruby18 + require 'rubygems' + require 'zentest'

(I only have ten gems installed in 1.9, so I didn't think it was fair
to compare.  Ruby 1.9 seems to use more memory than 1.8 in this small
test, though.)

In total, RubyGems 0.9.4.5 requires 21 additional files on 1.9.  The
following non-RubyGems files are loading with require 'rubygems':

rbconfig.rb
thread.rb
forwardable.rb
rational.rb
date/format.rb
time.rb

On OS X, these six files add ~ 1.5MB.

On Oct 13, 2007, at 02:00 , NAKAMURA, Hiroshi wrote:
> >       requires a lot of other libraries, so that memory consumption
> posting this to ruby-dev first.  I was waiting ko1 posting about  
> this topic.
>
> Matz, ko1 and akr, do you think that (c) includes gem version control,
> for example 'gem "soap4r", "1.5.5"' ?  Need to confirm the requirement
> of core team first.

Please see my notes in [ruby-core:12644].  If RubyGems itself makes
improvements in the capabilities of require 'rubygems' it benefits
all RubyGems users, including those still using Ruby 1.8, so I would
like to explore making RubyGems lighter as much as possible before
resorting to a Ruby 1.9-only solution.

> Drbrain, sorry about this confusion.  Core team raised an objection to
> [ruby-core:11803]
> http://blade.nagaokaut.ac.jp/cgi-bin/vframe.rb/ruby/ruby-core/11625? 
> 11588-12637+split-mode-vertical
> at this late date.  They think that gem loading feature should be
> enabled by default. (Topic 4-1 arrived to the conclusion (a) above.)
> OK, I know it's better than not discussed before imported.  Core team
> should have been busy on M17N I think.

I think that loading rubygems by default is acceptable.  Since it is
for Ruby 1.9, such a change is ok.

> I may write my opinion later but basically the intent of RubyGems team
> must come first.  We won't be able to maintain rubygems without
> cooperation by RubyGems team.

I am interested in making RubyGems work well for everybody, and will
certainly consider any issues the core team raises.

>>   (b) silently require 'rubygems' in initialization process.  rubygems
>>       requires a lot of other libraries, so that memory consumption
>>       might be an issue.
don't forget performance (interpreter startup time)...on my machine,
-rubygems (v0.9.4) makes Ruby starting nearly 10x slower.

but I haven't tested 0.9.4.5.
[murphy]

On Oct 13, 2007, at 03:16 , murphy wrote:
>>>   (b) silently require 'rubygems' in initialization process.   
>>> rubygems
>>>       requires a lot of other libraries, so that memory consumption
>>>       might be an issue.
> don't forget performance (interpreter startup time)...on my machine,
> -rubygems (v0.9.4) makes Ruby starting nearly 10x slower.
>
> but I haven't tested 0.9.4.5.

The following times a representative of the steady state after the OS
has cached everything.  0.9.4.5 starts up twice as fast as 0.9.4.

$ time ruby -v -rubygems -e 'p Gem::RubyGemsVersion'
ruby 1.8.6 (2007-09-23 patchlevel 5000) [powerpc-darwin8.10.0]
"0.9.4.5"

real    0m0.136s
user    0m0.068s
sys     0m0.025s
$ time ruby19 -v -rubygems -e 'p Gem::RubyGemsVersion'
ruby 1.9.0 (2007-10-13 patchlevel 0) [powerpc-darwin8.10.0]
"0.9.4.5"

real    0m0.181s
user    0m0.110s
sys     0m0.030s
$ time ruby -v -Ilib -rubygems -e 'p Gem::RubyGemsVersion'
ruby 1.8.6 (2007-09-23 patchlevel 5000) [powerpc-darwin8.10.0]
"0.9.4"

real    0m0.280s
user    0m0.184s
sys     0m0.050s

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

Eric Hodel wrote:
>> Matz, ko1 and akr, do you think that (c) includes gem version control,
>> for example 'gem "soap4r", "1.5.5"' ?  Need to confirm the requirement
>> of core team first.
> 
> Please see my notes in [ruby-core:12644].  If RubyGems itself makes
> improvements in the capabilities of require 'rubygems' it benefits all
> RubyGems users, including those still using Ruby 1.8, so I would like to
> explore making RubyGems lighter as much as possible before resorting to
> a Ruby 1.9-only solution.

Definitely.  I want to make sure what they call 'tiny library only for
finding gems'.  If it means 'RubyGems minus gem command', 0.9.4.5 should
be enough though I've not yet checked it.  If not?

I think ruby/1.9.1 should bundle RubyGems as is, whether it's
enabled/disabled by default.

Regards,
// NaHi

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Cygwin)

iQEVAwUBRxDdWx9L2jg5EEGlAQJCTQgAsN+Ru3olTsFGadrVbKR0Q94tQyU7HtHo
SWUApKVsgckIYegrZGUJun6oWtWxtZ6vupPkBErfVzDGFTObl1I96+nJ98lvzmv3
sXhKQ49BOAw2E4qK+rM1oIMUiSSORg+T9q6K6qcTnudcYvJXFZftStLGvAW1RzBo
URevxsJBi9NwIvvFZPjjDCYYMH6zUhJRmKY91Rjb+4hIpHXPvd6SOROhDb2Gwn2i
4B1FTKvowv8Iw6PK5GYmbRtPE4BHjgoL71bgmf1GqmKXkCXQEoStmo783dM8GL5q
yi7ytPIgCnKAMZaeSGqOTKRojImDlK1+JEeBrcH9a4gVz9SfULOtBw==
=e5VP
-----END PGP SIGNATURE-----

On 10/13/07, NAKAMURA, Hiroshi <nakahiro@sarion.co.jp> wrote:
> > Please see my notes in [ruby-core:12644].  If RubyGems itself makes
> enabled/disabled by default.
>
> Regards,
> // NaHi

How about something as simple as a 'gem' method defined in Kernel by
default, that will require rubygems and then the gem (this may require
making the gem method load files again).  This gives us the benefit of
rubygems being "transparent"  and not taking a huge performance hit if
it isn't going to be used.

On Oct 13, 2007, at 08:00 , NAKAMURA, Hiroshi wrote:
>
> Definitely.  I want to make sure what they call 'tiny library only for
> finding gems'.  If it means 'RubyGems minus gem command', 0.9.4.5  
> should be enough though I've not yet checked it.  If not?

Ok.  Please keep me up to date on this.  If 0.9.4.5 is not good
enough I can explore making it even lighter.

On Oct 13, 2007, at 10:18 , Chris Carter wrote:
> How about something as simple as a 'gem' method defined in Kernel by
> default, that will require rubygems and then the gem (this may require
> making the gem method load files again).  This gives us the benefit of
> rubygems being "transparent"  and not taking a huge performance hit if
> it isn't going to be used.

I think the goal is to make Kernel#require do what you mean when your
require 'somefile', whether it is in stdlib, sitelib, a gem, -I or
RUBYLIB.  If you have to call a different method, it isn't doing what
you mean.

Compared to previous RubyGems, I don't feel there is a huge
performance hit.  The core team may think differently, though.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

Eric Hodel wrote:
>> Definitely.  I want to make sure what they call 'tiny library only for
>> finding gems'.  If it means 'RubyGems minus gem command', 0.9.4.5
>> should be enough though I've not yet checked it.  If not?
> 
> Ok.  Please keep me up to date on this.  If 0.9.4.5 is not good enough I
> can explore making it even lighter.

Following is my opinion about enabling rubygems by default.  I posted it
to ruby-dev at [ruby-dev:32061].  It's an awkward translation of mine.

  /  /  /

I don't get response from core team to [ruby-core:12643] so I don't know
what core team calls 'tiny library only for finding gems'.  Here're my
reason for objecting to enable rubygems by default.

1. If 'tiny library only for finding gems' includes replacing
Kernel.require and Kernel.gem method (everything) with diet version.

1-1. RubyGems includes packaging, deployment and feature loading
systems.  I don't think it's bad and it's useful in fact.  But if we let
it be an 'official Ruby language specification' by enabling it by
default, it's not always a good thing.  For example;

1-1-1. It's said that as a packaging and deployment system it interacts
badly with some other existing packaging system.  I heard about it from
an official Ruby package maintainer of Debian.  I know almost nothing
about actual problem but it should not be too late to enable rubygems by
default, after bundling rubygems and seeing how it goes.

1-1-2. As a feature loading system, letting 'multiple versions aware
feature loading' be an official Ruby language specification imposes too
much restriction to unborn feature loading systems.

1-2. Is it possible to call it's not an official Ruby language spec even
if it's enabled by default?

2. If 'tiny library only for finding gems' only includes replacing
Kernel.require (does not include version control feature of RubyGems).
This means that we bundle RubyGems as the packaging and deployment
system of ruby but do not accept it as a feature loading system as is.

2-1. We should not bundle a library this way.  When we think it's good
thing, we should go RubyGems list at first and propose the change.

2-2. Anyway, granting that RubyGems team accepts this proposal, it's too
late for ruby/1.9.1.  From 1.9.2 at best I think.

  /  /  /

Though I didn't write it to ruby-dev, I don't think 2 is acceptable for
RubyGems team.  Gem.activate with version constraints + dependency
control by gem command with .gemspec should be a key advantage feature
of RubyGems I think.  Kernel.require and Kernel.gem both depends
Gem.activate.

It's still ongoing topic on ruby-dev.  I'm waiting for some thoughts on
this topic.

I'll update again.

Regards,
// NaHi

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Cygwin)

iQEVAwUBRxN1px9L2jg5EEGlAQLeHQf/TXXsEBgDGkdLrLL3wEEQkgwJoH6nPB6t
4Gw+4Std4nP8/K8JysZdaMthqzNoCPK/v468SDZCrNSCd569Jq2nE8SL0jpSx3vj
Wwa9mhewOMID9tCn4R8oPXdllSwEOV9RO2gA+W+2S255jqo9+aCboIcTOkLRAABa
D65CviTsJ1/GUau6U25x8Ka3+5vWOJ+nhYUlVLfruUhStq8xoH2S943NVIWBkb2J
XsM3uPDBSCHEdmNOgN4aHTCKZO3YYFE5/h0KL3ZxTIWRrR/1xxvMf4vlqfLX1/Es
VE3nBQKvGKptHl6YfUqVgP1/s1moAh/OPWP4QcB1gPmzMw1tmHPeGQ==
=ODaz
-----END PGP SIGNATURE-----

On Oct 15, 2007, at 07:14 , NAKAMURA, Hiroshi wrote:
> posted it to ruby-dev at [ruby-dev:32061].  It's an awkward  
>
> late to enable rubygems by default, after bundling rubygems and  
> seeing how it goes.

We've found that making RubyGems work with other package systems
difficult.  Each one has slightly different requirements for how
things should work.  Making one system happy makes another unhappy.

I know that FreeBSD integrates with RubyGems and the `gem` tool, and
there haven't been any discussions between the RubyGems and FreeBSD
people.

If a third party packaging system wants to repackage a gem, it is
easy, the gem format is tar file with a .tar.gz inside.

> 1-1-2. As a feature loading system, letting 'multiple versions  
> aware feature loading' be an official Ruby language specification  
> imposes too much restriction to unborn feature loading systems.

How so?  Or do you mean vs. other possible multiple version aware
feature loading systems.

> 1-2. Is it possible to call it's not an official Ruby language spec  
> even if it's enabled by default?

I think not.

> 2-2. Anyway, granting that RubyGems team accepts this proposal,  
> It's still ongoing topic on ruby-dev.  I'm waiting for some  
> thoughts on this topic.


On IRC last night, matz, Koichi, akr and myself decided that RubyGems
would be imported into trunk, but without require 'rubygems' by
default.  (Correct me if I am wrong.)

We didn't make a plan for when/how this would happen.

Eric Hodel wrote:
> I would like to do (b), but (a) is acceptable to me.  I think it is 
> possible to remove most of the files RubyGems loads with require 
> 'rubygems' using caching, which would be an easier alternative to (c), 
> however this would be probably not be much easier.

My only concern with (b) is that it changes the semantics of require
across the board to include additional paths and searching logic out of
the box. Those changes seem like something that most would be more
comfortable opting into rather than having always present, especially
since non-RubyGems require semantics are well understood.

- Charlie

On 10/17/07, Eric Hodel <drbrain@segment7.net> wrote:
> On Oct 15, 2007, at 07:14 , NAKAMURA, Hiroshi wrote:
> > 1-1-2. As a feature loading system, letting 'multiple versions
> > aware feature loading' be an official Ruby language specification
> > imposes too much restriction to unborn feature loading systems.
>
> How so?  Or do you mean vs. other possible multiple version aware
> feature loading systems.

Since there seems to be some confusion, here's a re-translation of
this part of [ruby-dev:32061]

1-1-2: enabling rubygems by default, and hence endorsing it as part of
the official Ruby language specification, imposes restrictions on
future possible feature loading system implementations. Specifically,
such new systems will need to support version aware future loading.
(... possibly raising the bar too high?)

hi,

there is another feature that gem doesnt handle nicely atm:
with the standard ruby directory layout you can share the tree via nfs
for multiple architectures as the native extensions are in an arch
specific path. within an installed gem they are directly inside the
"lib" subdir. i wonder if gem should use the arch specific subdirs below
its "lib" subdir aswell.

    darix

On Oct 17, 2007, at 05:44 , Marcus Rueckert wrote:
> there is another feature that gem doesnt handle nicely atm:
> with the standard ruby directory layout you can share the tree via nfs
> for multiple architectures as the native extensions are in an arch
> specific path. within an installed gem they are directly inside the
> "lib" subdir. i wonder if gem should use the arch specific subdirs  
> below
> its "lib" subdir aswell.

Could you file a feature request in the RubyGems tracker?

Leonard Chin wrote:
> 
> 1-1-2: enabling rubygems by default, and hence endorsing it as part of
> the official Ruby language specification, imposes restrictions on
> future possible feature loading system implementations. Specifically,
> such new systems will need to support version aware future loading.
> (... possibly raising the bar too high?)

I agree wholeheartedly with this part. A better require/plugin mechanism
needs to be created before committing to having RubyGems loaded by 
default.

- Charlie

On Oct 17, 2007, at 1:28 PM, Charles Oliver Nutter 
<charles.nutter@sun.com
 > wrote:

>> 1-1-2: enabling rubygems by default, and hence endorsing it as part  
>> of
>> the official Ruby language specification, imposes restrictions on
>> future possible feature loading system implementations. Specifically,
>> such new systems will need to support version aware future loading.
>> (... possibly raising the bar too high?)
>
> I agree wholeheartedly with this part. A better require/plugin  
> mechanism needs to be created before committing to having RubyGems  
> loaded by default.
>

Charlie, how would you define "better"  in this context?

Chad

On Oct 17, 2007, at 12:28 , Charles Oliver Nutter wrote:
>> 1-1-2: enabling rubygems by default, and hence endorsing it as  
>> part of
>> the official Ruby language specification, imposes restrictions on
>> future possible feature loading system implementations. Specifically,
>> such new systems will need to support version aware future loading.
>> (... possibly raising the bar too high?)
>
> I agree wholeheartedly with this part. A better require/plugin  
> mechanism needs to be created before committing to having RubyGems  
> loaded by default.

You mean like the one in r13580 mentioned in [ruby-core:13232]?

Eric Hodel wrote:
> You mean like the one in r13580 mentioned in [ruby-core:13232]?

No, like the one Nobu mentioned that they were still discussing:

 > Nobu and I are still
 > talking about generic 'require-hook' feature to avoid ad-hoc
 > Kernel#require rewriting so any comments are welcome about this.

- Charlie

Charles Oliver Nutter wrote:
> Eric Hodel wrote:
>> You mean like the one in r13580 mentioned in [ruby-core:13232]?
> 
> No, like the one Nobu mentioned that they were still discussing:
> 
>  > Nobu and I are still
>  > talking about generic 'require-hook' feature to avoid ad-hoc
>  > Kernel#require rewriting so any comments are welcome about this.

Sorry, that was Hiroshi talking about Nobu...

- Charlie

On Oct 17, 2007, at 14:53 , Charles Oliver Nutter wrote:
> Charles Oliver Nutter wrote:
>> Eric Hodel wrote:
>>> You mean like the one in r13580 mentioned in [ruby-core:13232]?
>> No, like the one Nobu mentioned that they were still discussing:
>>  > Nobu and I are still
>>  > talking about generic 'require-hook' feature to avoid ad-hoc
>>  > Kernel#require rewriting so any comments are welcome about this.
>
> Sorry, that was Hiroshi talking about Nobu...

I think they went with r13580 instead...

On Oct 17, 2007, at 3:20 PM, Eric Hodel wrote:

>
> I think they went with r13580 instead...

Thats only one part of the solution though. Everything that wants to
hook into Kernel#require is still required to alias the existing
implementation, implement their own Kernel#require as a wrapper.

Have some kind of simple hook infrastructure that Kernel#require used
would eliminate everyone patching Kernel#require.

  - Evan Phoenix

Evan Phoenix wrote:
> 
> On Oct 17, 2007, at 3:20 PM, Eric Hodel wrote:
>> I think they went with r13580 instead...
> 
> Thats only one part of the solution though. Everything that wants to 
> hook into Kernel#require is still required to alias the existing 
> implementation, implement their own Kernel#require as a wrapper.
> 
> Have some kind of simple hook infrastructure that Kernel#require used 
> would eliminate everyone patching Kernel#require.

...and frequently breaking everyone else's hook in the process.

- Charlie

Hi,

At Wed, 17 Oct 2007 21:44:29 +0900,
Marcus Rueckert wrote in [ruby-core:12724]:
> there is another feature that gem doesnt handle nicely atm:
> with the standard ruby directory layout you can share the tree via nfs
> for multiple architectures as the native extensions are in an arch
> specific path. within an installed gem they are directly inside the
> "lib" subdir. i wonder if gem should use the arch specific subdirs below
> its "lib" subdir aswell.

Strongly agreed.

Nobuyoshi Nakada wrote:
> 
> Strongly agreed.

This also affects JRuby when sharing a gem dir.

- Charlie

In article <4710890A.3020009@sarion.co.jp>,
  "NAKAMURA, Hiroshi" <nakahiro@sarion.co.jp> writes:

> Matz, ko1 and akr, do you think that (c) includes gem version control,
> for example 'gem "soap4r", "1.5.5"' ?  Need to confirm the requirement
> of core team first.

I don't expect version control by default.  It's the feature
of RubyGems, not Ruby itself.

I expect "require" loads installed libraries which may be
gem or not.

I think the tiny library is easily implementable.

I wrote simple patch to prelude.rb.  Maybe it can be
refined, it works well.

It rescue LoadError at "require" and retry after RubyGems is
loaded.  However it checks gems directory before RubyGems is
loaded.  So, if no libraries are installed as gem, no
libraries including RubyGems itself are not loaded.  So
memory bloat is avoided (except this tiny code).

It also checks environment variables such as GEM_PATH, etc.
If user specifies them, RubyGems is loaded even if gem
default directory doesn't have the library.

Note that the path for Dir.glob should be generated using
rbconfig at compile time.

Also note that if RubyGems provides a library to replace
"require" method but not define "gem" method,
it is prefered over require('rubygems').

Index: prelude.rb
===================================================================
--- prelude.rb  (revision 13715)
+++ prelude.rb  (working copy)
@@ -23,3 +23,25 @@
   end
 end

+module Kernel
+  module_function
+  alias original_require require
+  def require(feature)
+    begin
+      original_require feature
+    rescue LoadError
+      want_rubygems =
+        ENV['GEM_PATH'] ||
+        ENV['GEMCACHE'] ||
+        ENV['GEM_SKIP'] ||
+        ENV['GEM_HOME'] ||
+ 
!Dir.glob("/home/src/ruby/gems19/lib/ruby/gems/1.9/gems/*/lib/{i686-linux/,}#{feature}*").empty?
+      if want_rubygems && require('rubygems')
+        require feature
+      else
+        raise
+      end
+    end
+  end
+end

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

Good to hear your opinion.  We ML people heard that you 4 guys 'decided'
that RubyGems would be imported into trunk, but without require
'rubygems' by default, without any reason why. [ruby-core:12712]

Tanaka Akira wrote:
> In article <4710890A.3020009@sarion.co.jp>,
>   "NAKAMURA, Hiroshi" <nakahiro@sarion.co.jp> writes:
> 
>> Matz, ko1 and akr, do you think that (c) includes gem version control,
>> for example 'gem "soap4r", "1.5.5"' ?  Need to confirm the requirement
>> of core team first.
> 
> I don't expect version control by default.  It's the feature
> of RubyGems, not Ruby itself.

All: That's what I should accually write in 1-1-2 in [ruby-core:12691].
(Thanks for translating, Leo!)

> I expect "require" loads installed libraries which may be
> gem or not.
> 
> I think the tiny library is easily implementable.
> 
> I wrote simple patch to prelude.rb.  Maybe it can be
> refined, it works well.

To decide what directories should be added to $LOAD_PATH, we need to
check Gem::Specifications in specification directory.  ruby-debug gem
defines 'cli' directory as a requrie_path for example.  Here's an
implementation what should be done by 'gem load only' gem handler.
http://dev.ctor.org/vtr
It supports version control but it should be removable easily.

Matz, akr, and ko1, you wamt to just include Kernel.require of rubygems
and no Kernel.gem and version control, right?  What is different from
just installing files to site_ruby dir then by rpm or deb?  What is the
point you want to include from rubygems?

Regards,
// NaHi

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Cygwin)

iQEVAwUBRxhwjB9L2jg5EEGlAQI7Vwf/Tw4e4mA/m4yltzY6Oxsd9u8C6YzU6zY4
QDpwYnTi1nel00cMirEGsOJTkW5uf9Pc46kOho056wCeh/lP8sr660B+iQfYivH8
xQuC2Db9kEgSpMTCsFK/tjk7SpYr7LI0Rr9v89eew7S47mhrCx06m8xTyz3g2+DJ
jsVU8jCMBEmybY1rrrr/o2rpqrvOt8jLykHfryB5m1gQ5ZSKRrb1vCbhBJ2NXn+v
NN3JwdTcAIeD+2yJo7tNEsCDtHgLBujoc0K3GDuw7Sy/AQal5vM6vWqwls+RwVSk
g8nH8rggLSUXM1n6rRQ2zN0PCN8kUkkVQLmZiU673hwaJ8Sbsbe1HQ==
=Uji4
-----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

I think there's a consensus about the need.  Nobu's idea is here;
(translation of [ruby-dev:31321])

  Following is an abstract processes of the current behavior of require;

    def require_internal(feature)
      exts = loadable_exts(feature)
      found = nil
      $LOAD_PATH.find {|path| found = find_file(path, feature, exts)} or
        raise LoadError, "no such file to load -- %s", feature
      load_file(found)
    end

  And a proposal is that when a path from $LOAD_PATH is not a String, do
  the following.

    path.find_file(feature, exts)
    found.load_file()

Charles Oliver Nutter wrote:
>> Thats only one part of the solution though. Everything that wants to
>> hook into Kernel#require is still required to alias the existing
>> implementation, implement their own Kernel#require as a wrapper.
>>
>> Have some kind of simple hook infrastructure that Kernel#require used
>> would eliminate everyone patching Kernel#require.
> 
> ...and frequently breaking everyone else's hook in the process.

Do you have an actual example?  RubyGems replaces require and
Rails(ActiveSupport) overrides it but they live together.  I think Rails
took care of rubygems' behavior.

Regards,
// NaHi

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Cygwin)

iQEVAwUBRxh/mh9L2jg5EEGlAQIPDQgAmJkpc//D6+hei7rlsa1ad30WKomqCX1s
yc9zZsLrLUtlFU4HPx70Krrio8x+YLUXYbusqqIvmz/9GfRBQUp6Anmkr7hGlzST
Iic5SIIOzbH3soc8ChZ3pqlC+Dzu/Mx3OyKUDKgKeHhaAX8guxbNhq1UEhaHMRqj
LDg5XPNNZxazl/mb7ZE8bR1cpwa+FgCRY0JQlQUT0ae/ur9uC0e6ScBL2dZQWSjO
MXvSUAMEtEp+XEB31Sbyaz7hqJQ6KfSdbtoGmhOg5ggDYZF30YNSLd/tS6UWGXnM
VGqxHGOmsk9wWzsglyuu00OvOZ1G1JeTyc/JNYL7UBHRrDz2/1QcFg==
=UGvB
-----END PGP SIGNATURE-----

In article <4718708D.3050001@sarion.co.jp>,
  "NAKAMURA, Hiroshi" <nakahiro@sarion.co.jp> writes:

> To decide what directories should be added to $LOAD_PATH, we need to
> check Gem::Specifications in specification directory.  ruby-debug gem
> defines 'cli' directory as a requrie_path for example.  Here's an
> implementation what should be done by 'gem load only' gem handler.
> http://dev.ctor.org/vtr
> It supports version control but it should be removable easily.

Oh, I didn't know that.  Thank you for the information.
However I don't like loading all gemspecs.

Since the condition can be conservertive, the glob can be changed to
  Dir.glob(".../lib/ruby/gems/1.9/gems/**/#{feature}*")
for supporting such libraries.

Do you know a library which cannot be supported by the glob
above?

> Matz, akr, and ko1, you wamt to just include Kernel.require of rubygems
> and no Kernel.gem and version control, right?  What is different from
> just installing files to site_ruby dir then by rpm or deb?  What is the
> point you want to include from rubygems?

Someone might install a library as gem.  I don't want to
care the library is installed as gem or not.  This is my
reason.  Is it not enough?

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

Eric Hodel wrote:
> 
> I know that FreeBSD integrates with RubyGems and the `gem` tool, and
> there haven't been any discussions between the RubyGems and FreeBSD people.
> 
> If a third party packaging system wants to repackage a gem, it is easy,
> the gem format is tar file with a .tar.gz inside.

On ruby-dev list, Ruby package maintainers are discussing what can be
issues how they can reduce the problem.  When they find an issue that
rubygems can help to solve it, they or I'll write here.

>> 1-1-2. As a feature loading system, letting 'multiple versions aware
>> feature loading' be an official Ruby language specification imposes
>> too much restriction to unborn feature loading systems.
> 
> How so?  Or do you mean vs. other possible multiple version aware
> feature loading systems.

Other version aware feature loading system is a candidate.  Other than
that, reloadable feature loader, signature checking feature loader, Jar
based feature loader for JRuby, etc. I think.  All these are non-YAGNI
things so it doesn't matter for me that 'just add something as a Ruby
language specification and remove it when it won't work in the future'
as a pragmatic programmer. :-)

Regards,
// NaHi

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Cygwin)

iQEVAwUBRxi8Kh9L2jg5EEGlAQKFfQgAxSesy1fAJcXgHQQvEJbizx2xsng9NiiG
smSRWbugFhpUM6Avg/uoFT5N+Ud+5pEF4MlKYxVUAsFqmGCd1xT4jtAwgpjPRZMZ
DDRaUlhI8GKt4IBQC4mYE/0T2DnL+1p82AYnBhiUSTOsSYtgkk+BRpg76NIidZwn
0jFS7nd2a/44gLuvg0ZJUUwYLFRa4CXVDs4l6toX5ijY5BErEgIQwgWbLLm7CC+S
jgN/JTuRoOWKFioTJZQmPrhEkjIBxlDcTPuvwhoXJ9z0kJdFF9vQDxAg9joq0wRf
GBEtiwkwssgaBPE2eS+v1jB7lylYG8doq4nmVxQENkE/zAvLlqwiig==
=NWlR
-----END PGP SIGNATURE-----

On 2007-10-19 23:16:46 +0900, NAKAMURA, Hiroshi wrote:
> > should work.  Making one system happy makes another unhappy.
> > 
> > I know that FreeBSD integrates with RubyGems and the `gem` tool, and
> > there haven't been any discussions between the RubyGems and FreeBSD people.
> > 
> > If a third party packaging system wants to repackage a gem, it is easy,
> > the gem format is tar file with a .tar.gz inside.
> 
> On ruby-dev list, Ruby package maintainers are discussing what can be
> issues how they can reduce the problem.  When they find an issue that
> rubygems can help to solve it, they or I'll write here.

like debian/redhat packagers? the discussion is in japanese most likely
right?:) if it is about distro packages i could contribute my
experiences aswell.[1]

    darix


[1] the opensuse buildservice ruby project is mainly maintained by me.
    http://en.opensuse.org/Ruby
    http://download.opensuse.org/repositories/ruby/

NAKAMURA, Hiroshi wrote:
>     def require_internal(feature)
>       exts = loadable_exts(feature)
>       found = nil
>       $LOAD_PATH.find {|path| found = find_file(path, feature, exts)} or
>         raise LoadError, "no such file to load -- %s", feature
>       load_file(found)
>     end

What would find_file look like?

def find_file(feature, exts)
   exts.each {|ext| file = ext.find_feature(feature); return file if 
file}
   nil
end

So there's one way...and then a given ext would just need to implement
find_feature (or whatever the name is).

This seems like the right approach, since it explicitly takes the
chaining out of the hands of require extenders.

>   And a proposal is that when a path from $LOAD_PATH is not a String, do
>   the following.
> 
>     path.find_file(feature, exts)
>     found.load_file()

This is interesting. Won't it require checking every object in every
require along the way?

> Do you have an actual example?  RubyGems replaces require and
> Rails(ActiveSupport) overrides it but they live together.  I think Rails
> took care of rubygems' behavior.

I don't have one in-hand, no, but the prevailing recommendation I've
seen is "don't override require because you'll break something". Rails
and RubyGems work fine together, but I've seen other folks override
require and break both, largely getting scared off. The problem is the
implicit requirement to call the next hook in the chain by aliasing and
calling require, which the above proposal eliminates.

- Charlie

In article <47187F9A.1080308@sarion.co.jp>,
  "NAKAMURA, Hiroshi" <nakahiro@sarion.co.jp> writes:

>       load_file(found)
>     end
>
>   And a proposal is that when a path from $LOAD_PATH is not a String, do
>   the following.
>
>     path.find_file(feature, exts)
>     found.load_file()

I think it is good feature.

1. simplification

aliasing and redefining require method is a chain of
responsibility pattern.  $LOAD_PATH is a chain of
responsibility pattern too.  nobu's proposal unify them.  So
it is a kind of simplification.

2. various operations on the chain

alias-and-redefine implements a chain of responsibility
pattern.  However it is difficult to implement operations
such as inspection, deletion, etc.  The operations easy to
implement are only insert-handler-at-first and
insert-handler-at-last.

For example, my tiny RubyGems loader [ruby-core:12766] is
useless after RubyGems is loaded.  So I want to delete it
when loading RubyGems.  However it is difficult if some
library alias-and-redefine require method.

nobu's proposal makes the deletion easy:
$LOAD_PATH.delete(my_tiny_rubygems_loader).

3. Kernel#require and Kernel.require

There are two require method: Kernel#require and
Kernel.require.  Kernel.require is tend to be forgotten for
redefinition.

nobu's proposal makes them to behave consistently.

4. name crash

aliased name may be crashed.

Although it is possible to avoid the crash using
local variable and define_method, they make
inspection/deletion almost impossible.

nobu's proposal is a clean way to avoid the crash.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

Tanaka Akira wrote:
> Since the condition can be conservertive, the glob can be changed to 
>   Dir.glob(".../lib/ruby/gems/1.9/gems/**/#{feature}*")
> for supporting such libraries.
> 
> Do you know a library which cannot be supported by the glob
> above?

No.  It should be enough for that purpose.  Usually gemspec writers do
the same thing.

  Gem::Specification.new do |s|
    ...
    s.files = Dir.glob("{bin,lib,test}/**/*")
    s.require_path = "lib"
    ...
  end

But then 'require "doc/index.html"' causes to load rubygems you know.  I
know that this fallback checking can be conservative but I don't like
this kind of uncertain behavior.  What's wrong when we just do the
following?

  begin
    original_require feature
  rescue LoadError
    require 'rubygems'
    original_require feature
  end

Worried about memory footprint?  When I use gemloader.rb (at
http://dev.ctor.org/vtr), process VSZ/RSS is as follows.  ruby 1.8.6
(2007-10-15 patchlevel 5000) [i686-linux] on CentOS 5 + Linux kernel
2.6.18-8.1.14.el5.  (sorry for it's not ruby 1.9.  gems I have is not
run under 1.9)

   VSZ /RSS
- ----------------------------------------------------------------------
A) 3136/1504 ... just started a ruby process
B) 3284/1804 ... loaded 'gemloader.rb'
C) 4036/2676 ... loads all gemspecs I have. 28 gemspecs, total 160KB.
D) 4300/2868 ... activates 'initializer' in Rails/1.2.5.

# bear in mind that D is not the whole size of Rails app.  ActiveSupport
# delays loading of tons of necessary features.

Here's a script;

  p 'A) base'; STDIN.gets

  require 'gemloader'
  p 'B) loaded'; STDIN.gets

  Gem::LoadPathWeaver.repository
  p 'C) gemspecs cached'; STDIN.gets

  require 'initializer'
  p 'D) activated'; STDIN.gets

- From memory footprint point of view, I think rubygems can be slimmed 
as
well as gemloader.rb by targeting 'extract features just for gem
loading'.  I won't say they should do that because it may introduce
duplicates and complexity in the code.  Anyway it's up to RubyGems team.

akr, do you think above footprint is acceptable?

>> Matz, akr, and ko1, you wamt to just include Kernel.require of rubygems
>> and no Kernel.gem and version control, right?  What is different from
>> just installing files to site_ruby dir then by rpm or deb?  What is the
>> point you want to include from rubygems?
> 
> Someone might install a library as gem.  I don't want to
> care the library is installed as gem or not.  This is my
> reason.  Is it not enough?

Thanks for the explanation.  I agree with your claim.  Before writing my
opinion; You really don't want to include Kernel.gem and version control
by default, right?  According to the previous proposal of yours (require
'rubygems' fallback), Kernel.gem is not enabled by default but rubygems
loads dependency libraries according to version requirement 
specifications.

When we install an application/library package via gem, it has a version
dependency even if users won't see it explicitly.  So including rubygems
without Kernel.gem and version control is not enough for your claim I 
think.

Correct me if I misunderstand your point.  Your proposal on
[ruby-dev:31944] was 'when we include rubygems, it should be enabled by
default'.  Your proposal may include Kernel.gem fallback, too?

Regards,
// NaHi

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Cygwin)

iQEVAwUBRxoXHh9L2jg5EEGlAQKxjAf/T2Leekbq7x8eiUSzkDMnBodv9uaxwwn/
86D9eG3nFDIxJwoTZf+ELUIAGo076jPf0a87vsr/CifWHhfVHbpT86HgxYkXiFEz
67wj3ZGu+dvVfezxZbmhKP4nLyH3GiGp6C4fsvff+t9NGF8EDFzC7D4I2gXg6h6G
V6GdxV1LnhQAv3JBRS1J0JEWvfzzo1+KfI8HTP8R2rq5f+vuKvjGd4Ig8mpWlWau
w8P7TNPchW+3bCE6Altd23rzWWPIyVqg196Vi/TVo5gFArxpyPU/UfVMlVqsbD+Y
8B7VeknNhfanNroUASHNjs7n3xQs8me+KWu6wY2rhu50B3hF4LcCpA==
=FIxQ
-----END PGP SIGNATURE-----

In article <471A1720.4080606@sarion.co.jp>,
  "NAKAMURA, Hiroshi" <nakahiro@sarion.co.jp> writes:

> But then 'require "doc/index.html"' causes to load rubygems you know.  I

If nobu's proposal [ruby-core:12770] is accepted, the loader
can check loadable extensions.  So doc/index.html can be
ignored because .html is not a loadable extension.

Hmm, this is 5th benefit of the proposal.

> know that this fallback checking can be conservative but I don't like
> this kind of uncertain behavior.  What's wrong when we just do the
> following?
>
>   begin
>     original_require feature
>   rescue LoadError
>     require 'rubygems'
>     original_require feature
>   end

I think there are too many cases loading RubyGems
unnecessary if no additional check.  For example,
lib/tmpdir.rb checks Win32API using LoadError.

> Worried about memory footprint?  When I use gemloader.rb (at
> http://dev.ctor.org/vtr), process VSZ/RSS is as follows.  ruby 1.8.6
> (2007-10-15 patchlevel 5000) [i686-linux] on CentOS 5 + Linux kernel
> 2.6.18-8.1.14.el5.  (sorry for it's not ruby 1.9.  gems I have is not
> run under 1.9)

I'm not sure the threshold between acceptable and
unacceptable memory consumption.  However lower memory
consumption is better if the solution is good enough.
I feel my tiny loader is good enough.

Another concern is about maintainance.  Do you maintain
gemloader.rb?

> Thanks for the explanation.  I agree with your claim.  Before writing my
> opinion; You really don't want to include Kernel.gem and version control
> by default, right?  According to the previous proposal of yours (require
> 'rubygems' fallback), Kernel.gem is not enabled by default but rubygems
> loads dependency libraries according to version requirement specifications.

I don't want to care the library is installed as gem or not.
I assume a library may be installed as not gem.
gem method doesn't have effect for such library.

So, there are no reason to use gem method (in the situation
I don't care gem or not).

> When we install an application/library package via gem, it has a version
> dependency even if users won't see it explicitly.  So including rubygems
> without Kernel.gem and version control is not enough for your claim I think.

Since gem method depends on RubyGems, using gem method makes
a source code RubyGems dependent.  It is what I don't want.

However version control is accomplished in some way.  For
example, an administrator chooses appropriate version when
a library is installed under site_ruby.  gemspec may be
another way.

> Correct me if I misunderstand your point.  Your proposal on
> [ruby-dev:31944] was 'when we include rubygems, it should be enabled by
> default'.  Your proposal may include Kernel.gem fallback, too?

[ruby-dev:31944] is just a question.

My proposal is 'when we include rubygems, require method can
load a library installed as gem'.

gem method is not important for me.  I like it is not
defined until an application require 'rubygems' explicitly,
though.

On Thu, Oct 18, 2007 at 10:24:23AM +0900, Charles Oliver Nutter wrote:
> Evan Phoenix wrote:
>> On Oct 17, 2007, at 3:20 PM, Eric Hodel wrote:
>>> I think they went with r13580 instead...
>> Thats only one part of the solution though. Everything that wants to hook 
>> into Kernel#require is still required to alias the existing 
>> implementation, implement their own Kernel#require as a wrapper.
>> Have some kind of simple hook infrastructure that Kernel#require used 
>> would eliminate everyone patching Kernel#require.
> 
> ...and frequently breaking everyone else's hook in the process.

I thought I'd also mention that the aliasing of `require` is a big
problem for freakyfreaky sandbox as well, since I have to be able to
get to Ruby's original require in order to flash a new interpreter.
Somehow, I need to load the sandbox before require is aliased, you
know?

_why

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

Quick reply for the point about gemloader.rb.

Tanaka Akira wrote:
> Another concern is about maintainance.  Do you maintain
> gemloader.rb?

Not at all.  That's a proof of concept for asking core team about
requirement feature and memory footprint of rubygems.  If it's enough
for core team and if it's acceptable for rubygems team, it might be
implemented by rubygems team.

vtr has no test.  No test, no work.

Regards,
// NaHi
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Cygwin)

iQEVAwUBRxv6xh9L2jg5EEGlAQKSlAgAhP9sfnve6rxB9C31SX/GDHFjvbiyU61R
DVLoNqYt/E0pLS1AgEfHK5UxtkpPDA3IilVU0myI3c1uxY2KHK0IxSFKHtvdBmCa
ybsD27dx8X1c6wtympXWv9mbVD6FPVNXyyx23ZSI4m4PdkjTtDflvkDtXFr4FvqD
qoYW+kajW2V9z12KyG8Ee5qrJcyZiGFrGgH5AztySdHjlkOhmyG5sgpkGdofaUQN
yAPcvuT6a9TlIMd2hDbLEywxAFA9aJSnuYoyk8sWARFauuiypTFaIS0GVLzfkwIj
uialHOzc1W4ID3DSaRspSrDbyJfFdVLQkwmX1Vi6jYCWfxc+TY2VSA==
=5Zl9
-----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

I happened to notice this...

Eric Hodel wrote:
> On IRC last night, matz, Koichi, akr and myself decided that RubyGems
> would be imported into trunk, but without require 'rubygems' by
> default.  (Correct me if I am wrong.)
> 
> We didn't make a plan for when/how this would happen.

You 4 decided that RubyGesm would be imported into trunk without require
'rubygems' by default.

Does this mean you're planning to add the LoadError fallback something
like akr is proposing?  Or you're planning to let users use RUBYOPT as
you wrote in [ruby-core:11818] ?

[ruby-core:11818]
| Not everybody will need RubyGems.  For example many Rails
| installations run with everything unpacked into a local directory for
| easy deployment.
|
| I would prefer users use RUBYOPT instead of enabling RubyGems by
| default.

Regards,
// NaHi

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Cygwin)

iQEVAwUBRxwuSR9L2jg5EEGlAQI4DQf8DD0SDq8fdbeaDWs/6Cap5UJvh4ZeXW6d
zjnqHwefDVDEUeJL4OAhHZfvuh1f5bB1gokMSMchtlJtrZI4iO+8O0w8Wv5J4KeY
iUZBx+unBVtnFNckKV+EZpiYH0djN9bfFax9xEBGPUZYAtwNue0ssEQ4XdgRAZhZ
Xe7BmLJUV2Pimh1mSH688Wx6Jm4zEUVBu8oqkJyzQi6TCNgsbnmBMAgm55FDx4Zn
nZ1RwSkkbuxuV3MApG4mwCymns6wOpulrIe6WGf8INUyZ/y8f3W97tfo4JPyHSFA
QZ6lICsA6+7z9FYtjkWJxcWEG6z0SGf8R6zt0kQ6ke5iGd49MB+AZg==
=umt0
-----END PGP SIGNATURE-----

In article <471C2E4B.1010703@sarion.co.jp>,
  "NAKAMURA, Hiroshi" <nakahiro@sarion.co.jp> writes:

>> On IRC last night, matz, Koichi, akr and myself decided that RubyGems
>> would be imported into trunk, but without require 'rubygems' by
>> default.  (Correct me if I am wrong.)
>> 
>> We didn't make a plan for when/how this would happen.
>
> You 4 decided that RubyGesm would be imported into trunk without require
> 'rubygems' by default.

I re-read the IRC session.

I think the decision is because release schedule.
(ko1 strongly claim it.)

If we have good enough solution now, I think there is no
reason to postpone enabling RubyGems's require by default.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

Tanaka Akira wrote:
>> But then 'require "doc/index.html"' causes to load rubygems you know.  I
> 
> If nobu's proposal [ruby-core:12770] is accepted, the loader
> can check loadable extensions.  So doc/index.html can be
> ignored because .html is not a loadable extension.
> 
> Hmm, this is 5th benefit of the proposal.

Can be.  But it is less related to the original topic, isn't it?

I agree that general loader extension point is good but it changes core
part of current require hacks.  Let's take it for granted that we adopt
Nobu's proposal as is, now $: is just a loader queue and there's no need
for loaders to take care about $".  A loader just find a file and load
it.  No need to return control to require.

The next generation of RubyGems or another gem loader will utilize the
new extension point but it's not the RubyGems we are discussing to
include in 1.9.1, right?

Note that I don't want to stop the discussion about general loader
extension point.

> 
> I think there are too many cases loading RubyGems
> unnecessary if no additional check.  For example,
> lib/tmpdir.rb checks Win32API using LoadError.

Sure.  It's too much used for feature detection.  New feature detection
scheme can be discussed as well.

We agreed that ruby should load rubygems when it's necessary.  My
opinion is 'let user to decide with RUBYOPT' but it's not a topic of
this discussion.

My opinion on the basis that enabling rubygems by default is, allow to
load gemspecs for creating feature manifest.  It's better than
'wild-guessing-and-load-rubygems-then-pray-it's-included' for me.  Oh
man, I must not be able to express my mind correctly in English.

> So, there are no reason to use gem method (in the situation
> I don't care gem or not).

OK.  Thanks.

>> When we install an application/library package via gem, it has a version
>> dependency even if users won't see it explicitly.  So including rubygems
>> without Kernel.gem and version control is not enough for your claim I think.
> 
> Since gem method depends on RubyGems, using gem method makes
> a source code RubyGems dependent.  It is what I don't want.

I see.

> However version control is accomplished in some way.  For
> example, an administrator chooses appropriate version when
> a library is installed under site_ruby.  gemspec may be
> another way.

RubyGems offers versions control unlike this kind of static version
dependency solving by administrator.  It offers dynamic version
dependency solving at runtime.  Say rails-1.2.3 and rails-1.2.5 are
installed on a system.  There also be activerecord-1.15.3 and 1.15.5.
One Rails application runs with rails-1.2.3 + AR-1.15.3 and another
application runs with rails-1.2.5 + AR-1.15.5.  Even granting that we
don't offer gem method by default, rubygems loads feature in a different
manner.  I think you might not like this behavior.

Regards,
// NaHi
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Cygwin)

iQEVAwUBRxyv3h9L2jg5EEGlAQKyBQgAmjKbRpIvI9jahRVDgZnhe2VVsEIX6IU0
ImTKNEYX7pOo/ehZWsISvslyttn2EooikRV92v7H498JtrAXKJh48CKPSQcgELXm
oT6nvz8KR9a8GjwYGYcQQvvvpSwS46pdYctzj8aB3VdeTZp7To/7M5iueg0j/UfW
9w80mQPxeKV2ttcXkIw1Y8Ar/W5vUrPTHs77pAgz6OaY0KHjNCUf2yJYdqnutXbX
3FQyvHkx0Qccjwdxr5vZmJfCUjERv7gaXeGs/3e/LrsJNTP7ajtU+nzLxZ70Ehxd
zPW1ClezAmcMolts0z3t2hzz3dDQfs0Cjd+MhpaRG4qpI4VIOYqGQw==
=vTgy
-----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

Marcus Rueckert wrote:
>> On ruby-dev list, Ruby package maintainers are discussing what can be
>> issues how they can reduce the problem.  When they find an issue that
>> rubygems can help to solve it, they or I'll write here.
> 
> like debian/redhat packagers? the discussion is in japanese most likely
> right?:) if it is about distro packages i could contribute my
> experiences aswell.[1]

Please do so here.  Rubygems-developers ML is good I think.  knu from
FreeBSD ports posted a request there.  An archive is here...  Hmm.
Archive at
http://rubyforge.org/pipermail/rubygems-developers/2007-October/thread.html
does not include the post from knu.  Something is wrong.  Please find
the article from
http://rubyforge.org/pipermail/rubygems-developers/2007-October.txt

Regards,
// NaHi

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Cygwin)

iQEVAwUBRxy1tx9L2jg5EEGlAQK50gf+LBDZnPgeYsRihVfh+69XbGc2CuwBeg26
K08qwJ3QGAlBUipiELGmvT3Q4evCexzGbK3JBdKdEeMwM1D5I0298ZwTUIazr5/W
Xtz0TbiannwkqMd1cPJmSWF75Uk440KklvSKXHH6QPZYEU6f88zN4Dz3hC/0nD4M
5g+f/aeaZRwqOPlPfLnNX3/vG32/GinO0pghpbXdIVu2aZwoFXmaHX8S7g+w1ImN
uuVJl0LUqKqtkfFjCYk3S+GcbkuxUVXcJUlaUjWDlrakydyOcPn5amgceR8GGLzm
X2prVw9K510OQJ2xHfdsuAbZht4g+fMS+nEm255DnoKhcAo/d9DxJg==
=XrsH
-----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

_why wrote:
> I thought I'd also mention that the aliasing of `require` is a big
> problem for freakyfreaky sandbox as well, since I have to be able to
> get to Ruby's original require in order to flash a new interpreter.
> Somehow, I need to load the sandbox before require is aliased, you
> know?

I don't know freakyfreaky well but how do you flash a new interpreter
with require hack?

Regards,
// NaHi

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Cygwin)

iQEVAwUBRxy3Bh9L2jg5EEGlAQJcsQgAhVhOHGa6fAL7EdtET860DhPQTQx0MDw1
Q2vt93nC8F3mGNPEb1hVsdZ+2pfSN0nFWqhpTINxBuBr0ECtGHkR8ssDQBC5GYTK
wL/Xub2I+ho2xQDZacbMbie7RXB0srCTr+f/dfcVZ6sypEtW22ybJ737pizzj/DE
F0Nyj2htbeDz7SaXuo4jHjtM/eqJ1EfeKogdRE6NsV8d/it2MARFKVCGSjF3k/oO
C9B9wU63r+WHc9lBPEXtwVNU/jo12noGJb3pWgOgSgR5idtCn+eGY5RWDs/rjPrL
KQ2ymhK6yxdZjdy4AACdnRppC5EwPq22TsLvHNU1cS/+gZj3vp7khg==
=n8Rp
-----END PGP SIGNATURE-----

On Mon, Oct 22, 2007 at 11:44:29PM +0900, NAKAMURA, Hiroshi wrote:
> _why wrote:
> > I thought I'd also mention that the aliasing of `require` is a big
> > problem for freakyfreaky sandbox as well, since I have to be able to
> > get to Ruby's original require in order to flash a new interpreter.
> > Somehow, I need to load the sandbox before require is aliased, you
> > know?
> 
> I don't know freakyfreaky well but how do you flash a new interpreter
> with require hack?

Well, in freakyfreaky's C extension, there's a method called
sandbox_copy_method, which is found in sand_hacks.c:

http://code.whytheluckystiff.net/sandbox/browser/trunk/ext/sand_table/sand_hacks.c

The sandbox_copy_method will grab the function pointers from the
Kernel#require's NODE and create a new method inside the sandbox using
those same pointers.  But not just `require`, all methods in Kernel
and throughout Ruby's stdlib.

I could just assign to rb_f_require directly, though.  It's okay.

_why

In article <471CAFE0.2070104@sarion.co.jp>,
  "NAKAMURA, Hiroshi" <nakahiro@sarion.co.jp> writes:

> My opinion on the basis that enabling rubygems by default is, allow to
> load gemspecs for creating feature manifest.  It's better than
> 'wild-guessing-and-load-rubygems-then-pray-it's-included' for me.  Oh
> man, I must not be able to express my mind correctly in English.

I'm not sure what you want.  However you can use any
RubyGems feature using require 'rubygems'.

> RubyGems offers versions control unlike this kind of static version
> dependency solving by administrator.  It offers dynamic version
> dependency solving at runtime.  Say rails-1.2.3 and rails-1.2.5 are
> installed on a system.  There also be activerecord-1.15.3 and 1.15.5.
> One Rails application runs with rails-1.2.3 + AR-1.15.3 and another
> application runs with rails-1.2.5 + AR-1.15.5.  Even granting that we
> don't offer gem method by default, rubygems loads feature in a different
> manner.  I think you might not like this behavior.

I think I like it.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

Tanaka Akira wrote:
> In article <471CAFE0.2070104@sarion.co.jp>,
>   "NAKAMURA, Hiroshi" <nakahiro@sarion.co.jp> writes:
> 
>> My opinion on the basis that enabling rubygems by default is, allow to
>> load gemspecs for creating feature manifest.  It's better than
>> 'wild-guessing-and-load-rubygems-then-pray-it's-included' for me.  Oh
>> man, I must not be able to express my mind correctly in English.
> 
> I'm not sure what you want.

It means;

require 'gemloader'

in prelude.rb, if we need to enable gem loading by default.

Bear in mind that my vote is RUBYOPT (not enable rubygems by default).

>> RubyGems offers versions control unlike this kind of static version
>> dependency solving by administrator.  It offers dynamic version
>> dependency solving at runtime.  Say rails-1.2.3 and rails-1.2.5 are
>> installed on a system.  There also be activerecord-1.15.3 and 1.15.5.
>> One Rails application runs with rails-1.2.3 + AR-1.15.3 and another
>> application runs with rails-1.2.5 + AR-1.15.5.  Even granting that we
>> don't offer gem method by default, rubygems loads feature in a different
>> manner.  I think you might not like this behavior.
> 
> I think I like it.

But gem method is not important for you?  Why don't you add gem method
fallback to prelude.rb as well?

Regards,
// NaHi

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Cygwin)

iQEVAwUBRx1NHh9L2jg5EEGlAQIF2gf8DngH2ShVIwoQnzAK36UQhnuFuJeulcJE
cql04msOb1bGLc7aiMQsrJ359f2yHJPgDqN/YNIrkg87JIwo3VY08OknKGCioTE0
/fzosQUuXFzdHD7McivEkphUkzMUJxD83ZqMTfscG3bYwPdakyR9xGQ3srrT4J3j
Mf74zZDMJ+MwIj3J/6dg/o99w6SLrFF5A/BdsTb9zDUFezELT4UtrNe+BykzQ53C
6KkyU3Y7556q6wLC2Yq/Ff9pgmnU8WI+L+eyRFeP9YBJjb9VhpuMyevUPgY0S5TA
v3RULTNo5kWFn2EUq3eh84TGgo+fTPsByGAnAYs2zdi/otX9W/wUhQ==
=egbm
-----END PGP SIGNATURE-----

In article <471D4D1F.5050006@sarion.co.jp>,
  "NAKAMURA, Hiroshi" <nakahiro@sarion.co.jp> writes:

> It means;
>
> require 'gemloader'
>
> in prelude.rb, if we need to enable gem loading by default.

What's advantage over mine?

> But gem method is not important for you?  Why don't you add gem method
> fallback to prelude.rb as well?

I can use any version selection scheme including RubyGems.

I may use RubyGems.  I may use others.  It depends on
situation.

gem method depends RubyGems.  It causes loading RubyGems
unnecessary when I don't use RubyGems.  If I know I use
RubyGems and needs gem method, I use it after
require 'rubygems'.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

Tanaka Akira wrote:
>> It means;
>>
>> require 'gemloader'
>>
>> in prelude.rb, if we need to enable gem loading by default.
> 
> What's advantage over mine?

2 points.  Your proposal and mine above both enables RubyGems' version
dependecy resolution, right?  But the above is;

1. does not have uncertain behavior.  No require 'doc/index' problem I
wrote at [ruby-core:12792].

2. allow gem method to work.  And gem method is usable always.  With
yours, it depends on a context of previous LoadError whether a library
can use gem method or not.  I don't like this statefullness, too.

> RubyGems and needs gem method, I use it after
> require 'rubygems'.

Your proposal of prelude.rb enables RubyGems.  As I wrote, RubyGems uses
their 'version selection scheme' independent from whether gem method
exists or not.  So you cannot use any other scheme.  RUBYOPT solution
allow you to do the above.

Anyway, I agree with you that we should not add 'gem' method by default.
 I prefer RUBYOPT from the very begining of this thread.

Regards,
// NaHi

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Cygwin)

iQEVAwUBRx1WYx9L2jg5EEGlAQIl4QgAxpa4kS0YM7mw58za4+7vOv1KsqqtJda2
YEZsoLCr55LhYQSbby8pbPqwsjlPaYhtMYOI0G5OeQt2jEd0n88zaYNdc0RzC+1G
FegR7VGib4sk6wFrROYR07jzkpwLad22rX6q40FMm0uTiJzh+VAgaUr4ZEHbmf1Q
7M5Qz6op5me3dkgkj54ArPu7Z/2ZioBzmwUuigZIxLX5RkD6G5/XtID45oPm1Vfb
J3zt8+HeVfXoK73yLQvFH4YvQyW6q+y+o/hTzPaUAUe+eASUmr/7oenVbBRqO48C
jAPqIMfxxl5eqeB+W++TDOzYXAuhZR+YKvb1KRw2EnyDtIRHhRoU7g==
=l94L
-----END PGP SIGNATURE-----

In article <471D5665.5040209@sarion.co.jp>,
  "NAKAMURA, Hiroshi" <nakahiro@sarion.co.jp> writes:

> 2. allow gem method to work.  And gem method is usable always.  With
> yours, it depends on a context of previous LoadError whether a library
> can use gem method or not.  I don't like this statefullness, too.

In the IRC session, drbrain said that separating Kernel.gem
to independent library is possible.  (matz asked it.)

So my tiny loader will load RubyGems without gem method.
The statefullness will be removed.

> 1. does not have uncertain behavior.  No require 'doc/index' problem I
> wrote at [ruby-core:12792].

I agree that mine is not accurate as yours or RubyGems
itself.

In contrast, the disadvantage of yours are memory
consumption and maintenance.

I think maintenance is critical problem.  Your gemloader.rb
is not appropriate for incorporating to Ruby until some
maintainer available.

The accuracy and memory consumption is trade off.  For
accuracy, all gemspecs must be loaded and analyzed.  But it
consumes memory: analyzer code and loaded gemspecs.

Since RubyGems itself does accurate analysis, its loader can
be conservative.  I think memory consumption justify the
conservativeness.

> Your proposal of prelude.rb enables RubyGems.  As I wrote, RubyGems uses
> their 'version selection scheme' independent from whether gem method
> exists or not.  So you cannot use any other scheme.  RUBYOPT solution
> allow you to do the above.

I can install any library under site_ruby.  Does RubyGems
prohibit it?

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

Tanaka Akira wrote:
>> 2. allow gem method to work.  And gem method is usable always.  With
>> yours, it depends on a context of previous LoadError whether a library
>> can use gem method or not.  I don't like this statefullness, too.
> 
> In the IRC session, drbrain said that separating Kernel.gem
> to independent library is possible.  (matz asked it.)
> 
> So my tiny loader will load RubyGems without gem method.
> The statefullness will be removed.

Good.  Now #2 is not the problem.

I cannot see the discussion details held on IRC so I cannot make my
decison whether it's good or not until the change RubyGems team will
make.  But I'm watching svn repository and developer-maillist of
RubyGems and write here when I'll find something to be noted.  Never
mind, let's go ahead.

>> 1. does not have uncertain behavior.  No require 'doc/index' problem I
>> wrote at [ruby-core:12792].
> 
> I agree that mine is not accurate as yours or RubyGems
> itself.
> 
> In contrast, the disadvantage of yours are memory
> consumption and maintenance.

Don't list 'maintenance' here.  When it's good enough, RubyGems team
will do it as I wrote in [ruby-core:12810].

> The accuracy and memory consumption is trade off.  For
> accuracy, all gemspecs must be loaded and analyzed.  But it
> consumes memory: analyzer code and loaded gemspecs.
> 
> Since RubyGems itself does accurate analysis, its loader can
> be conservative.  I think memory consumption justify the
> conservativeness.

We've written our thought on this trade off many times each other,
without concrete grounds.  I still don't have any number to show so I
don't repeat writing my thought on this again. :-)  No other comments
from ruby-core.  The decision is up to core team now.

One additional thought for akr's LoadError fallback.

Issue #1 is for solving the problem how we map feature -> path to be
added to $LOAD_PATH.  Java uses MANIFEST in jar.  RubyGems uses gemspec
in spec dir.  akr's LoadError fallback uses file system wild globbing.
 Default Ruby require uses file system accurately.

Can gem command of RubyGems can create a MANIFEST in gemspec dir?  Then
akr's LoadError fallback can grep it for finding rubygems to be loaded
or not.  It won't be needed as a part of RubyGems though.

akr, let me hear your thought on this.

>> Your proposal of prelude.rb enables RubyGems.  As I wrote, RubyGems uses
>> their 'version selection scheme' independent from whether gem method
>> exists or not.  So you cannot use any other scheme.  RUBYOPT solution
>> allow you to do the above.
> 
> I can install any library under site_ruby.  Does RubyGems
> prohibit it?

I mean that you can use site_ruby but cannot use others.

Regards,
// NaHi

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Cygwin)

iQEVAwUBRx/zsB9L2jg5EEGlAQKnpgf+NZc9KXPkC4xr1fCsygUeY0xQG+MlLlMg
ftnAuR+j6F6GY2+WfSKH/iaDWpnzFvsI8juqTjmgI5pGcJ2xHl1t88YQ1LMTy2n8
xHGcEQzJL7TCZErb+b2zT8X+PjkYTOJbvA76nfYf9HlY8hoPmQsFlsegkcsfpZvG
3qidHWLIIgG0MfsuWc4MwvV5qk67L9gTh6sG/387jMNHbpGf9lxovz9MgUChnqv0
CMdx9LRdGTSEuszIKqOZLhQD1j9/IIV0LH54yem7sNXB0+b5l8Zun5Oqjli/BdsD
Dx3zEQM3/kQSwLC/dsvjQZcte16VgJ0DVVUxUmeqOvH1Qf4JaJzKhw==
=GGV8
-----END PGP SIGNATURE-----

In article <471FF3B1.3060103@sarion.co.jp>,
  "NAKAMURA, Hiroshi" <nakahiro@sarion.co.jp> writes:

> Don't list 'maintenance' here.  When it's good enough, RubyGems team
> will do it as I wrote in [ruby-core:12810].

The maintenance problem is too important to ignore.

I couldn't find a mail to mention RubyGems developper will
do it.  Point me it.  Or, is it private discussion between
you and RubyGems developper?

> We've written our thought on this trade off many times each other,
> without concrete grounds.  I still don't have any number to show so I
> don't repeat writing my thought on this again. :-)  No other comments
> from ruby-core.  The decision is up to core team now.

I don't know the definition of "core team".  What is it?

> Issue #1 is for solving the problem how we map feature -> path to be
> added to $LOAD_PATH.  Java uses MANIFEST in jar.  RubyGems uses gemspec
> in spec dir.  akr's LoadError fallback uses file system wild globbing.
>  Default Ruby require uses file system accurately.
>
> Can gem command of RubyGems can create a MANIFEST in gemspec dir?  Then
> akr's LoadError fallback can grep it for finding rubygems to be loaded
> or not.  It won't be needed as a part of RubyGems though.
>
> akr, let me hear your thought on this.

I'm happy to use such information if RubyGems provide it.

> I mean that you can use site_ruby but cannot use others.

I think I can use RubyGems if I install a library as gem.
I don't understand what you mean.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

Tanaka Akira wrote:
> you and RubyGems developper?
You misunderstand me.  As I wrote in [ruby-core:12810], if it's good way
to go, RubyGems team will adopt it.  If RubyGem team won't adopt it, it
means that it's not good for RubyGems.  From the very biginning of this
thread, I worked for bundling RubyGems as a part of trunk as is.

>> We've written our thought on this trade off many times each other,
>> without concrete grounds.  I still don't have any number to show so I
>> don't repeat writing my thought on this again. :-)  No other comments
>> from ruby-core.  The decision is up to core team now.
> 
> I don't know the definition of "core team".  What is it?

It's said for dicision maker(s) of MRI & YARV.  Matz and ko1, and some
comitters for each area who Matz is relying on?  I don't know exactly.

> 
> I'm happy to use such information if RubyGems provide it.

Thanks.  What do you think, drbrain?

>> I mean that you can use site_ruby but cannot use others.
> 
> I think I can use RubyGems if I install a library as gem.
> I don't understand what you mean.

[ruby-core:12854]
| I may use RubyGems.  I may use others.  It depends on
| situation.

Your proposal don't allow to use others I wrote in [ruby-core:12774] for
example, right?  1.9.1 uses rubygems when a LoadError given.

Regards,
// NaHi

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Cygwin)

iQEVAwUBRyANcx9L2jg5EEGlAQIRDggAvPH5ycQDOrSzV1O1hQEwfy7pPW6njn7h
H050hz/tzIfFtj8lwafnPHLeAOjNCFTLVuQnoHjdZkSv98bpdj6a1kDXJAEJYfDx
Xou0I393qT9w/iCDxMjQQItRDULm+XEfytcw2wUZnVs4WHtxq0JmgIh0XZhUHDsN
JOylTv49G8/4gqANkDogWISNX1TS0/QLhSgXNpjOVFqQ304o/dMNbldNbfzH/ra0
bO8NeMfXbDP1DaqK8UZ88Yz9tGTeVIjjO/MtqUyGBtvdL89/p+2dnBCrwbsdQNkp
WbUkeqWGxk3dYJ5cism2O1UnDAikOrLe5qNuomXlEYkq2jokfDD+2Q==
=C/DU
-----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi again,

NAKAMURA, Hiroshi wrote:
> means that it's not good for RubyGems.  From the very biginning of this
> thread, I worked for bundling RubyGems as a part of trunk as is.

Oh I forgot to mention.  I don't have any other channel than here to
talk to RubyGems developers.

Regards,
// NaHi

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Cygwin)

iQEVAwUBRyAOlB9L2jg5EEGlAQKBnwf+NSWmgV1cToZIjzI3NgIP60AjgeAHIzEo
BJ4qBHD2voXFuinzjJ45w/ZJ1VZLky6zdwPnZewV9/LH+WNYAsY0QMeHCWDs0FwI
F/Y3CZf8eJoKMopizgnhW7euwwKe9PtjALuEiKU5t2aY6hMLuSNmulM1omp0HWVH
X3QzyIpxwlPuMwruD3H88muvAS/YqzBSUyUKmc3xFITzrF4fTesrAMlCalnA4tdG
amv8NydLWOWX4oGSJT8BIc4SS4ECn2m1Bsz4rdgEXZ6k7OOr7V4mpABYjuWYt6UW
DEyW0UnR9zkY1Ut9kb5CkV2CcyzAY3Y5zH4y9fDmNDZp3yYGJwhPFw==
=EJsy
-----END PGP SIGNATURE-----

In article <47200D74.6020202@sarion.co.jp>,
  "NAKAMURA, Hiroshi" <nakahiro@sarion.co.jp> writes:

> You misunderstand me.  As I wrote in [ruby-core:12810], if it's good way
> to go, RubyGems team will adopt it.  If RubyGem team won't adopt it, it
> means that it's not good for RubyGems.  From the very biginning of this
> thread, I worked for bundling RubyGems as a part of trunk as is.

I see.  The maintenance problem exists until RubyGems
developper adopt it.

> Your proposal don't allow to use others I wrote in [ruby-core:12774] for
> example, right?  1.9.1 uses rubygems when a LoadError given.

I understand.  I assumed "other version selection schemes" as
packaging system provided by OS: dpkg/rpm/ports/pkgsrc/etc.
But you think it differently.

Hmm.  For your loader, we can provide
disable-rubygems-loader.rb:

module Kernel
  remove_method :require
  alias require original_require
  remove_method :original_require
end

RUBYOPT='-rdisable-rubygems-loader -ryour-favorite-loader'
disables RubyGems and enables your favorite loader.

In article <87ve8vxl9n.fsf@fsij.org>,
  Tanaka Akira <akr@fsij.org> writes:

>> Your proposal don't allow to use others I wrote in [ruby-core:12774] for
>> example, right?  1.9.1 uses rubygems when a LoadError given.

> Hmm.  For your loader, we can provide
> disable-rubygems-loader.rb:

On second thought, disable-rubygems-loader.rb may be
unnecessary.

My tiny loader or RubyGems causes LoadError.  So simply
wrapping require method may work.

Would you show me a loader which doesn't work well with my
loader?

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

Tanaka Akira wrote:
>> You misunderstand me.  As I wrote in [ruby-core:12810], if it's good way
>> to go, RubyGems team will adopt it.  If RubyGem team won't adopt it, it
>> means that it's not good for RubyGems.  From the very biginning of this
>> thread, I worked for bundling RubyGems as a part of trunk as is.
> 
> I see.  The maintenance problem exists until RubyGems
> developper adopt it.

Sure.  And your LoadError fallback has the same issue.  The hack should
be maintained by RubyGems team because the check scheme depends on
RubyGems specification.  For example, you need to check APPLE_GEM_HOME
in rbconfig if RubyGems team accepts the patch from Apple for Leopard:
http://rubyforge.org/pipermail/rubygems-developers/2007-October/003112.html

According to DRY principle, we should not such logic in prelude.rb.

>> Your proposal don't allow to use others I wrote in [ruby-core:12774] for
>> example, right?  1.9.1 uses rubygems when a LoadError given.
> 
> I understand.  I assumed "other version selection schemes" as
> packaging system provided by OS: dpkg/rpm/ports/pkgsrc/etc.
> But you think it differently.

I see.

> Hmm.  For your loader, we can provide
> disable-rubygems-loader.rb:

> RUBYOPT='-rdisable-rubygems-loader -ryour-favorite-loader'
> disables RubyGems and enables your favorite loader.

I don't like this sort of negative spell.

> On second thought, disable-rubygems-loader.rb may be
> unnecessary.
> 
> My tiny loader or RubyGems causes LoadError.  So simply
> wrapping require method may work.
> 
> Would you show me a loader which doesn't work well with my
> loader?

module Kernel
  alias foo_hooked_require require

  def require(feature)
    begin
      foo_hooked_require feature
    rescue LoadError => e
      p "do something good"
      require feature
    end
  end
end

With RUBYOPT=-rfooloader, we can hook Kernel#require before RubyGems is
activated.  The require chain after a LoadError is as follows.

  rubygems -> foo -> rubygemsloader -> original_require

So foo can intercept a LoadError from rubygems.  But when an user
activates a gem with rubygems, say rails, dependency libraries are
activated by rubygems, too.  Now foo cannot intercept LoadError for
these libraries.

Till ruby equips sophisticated feature loading extension point, mixing
two or more loaders should not be expected to work.  Incorporating your
LoadError fallback means that Ruby allows RubyGems to be used as a only
custom loader.  I hope I can explain myself well.

Regards,
// NaHi

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Cygwin)

iQEVAwUBRyL+ox9L2jg5EEGlAQKtSAf/fghl+RBly7acz3LBv/SXXTQrJTi446aC
hRdPsfNinx7KwuYNKJC7lIPh3FmADTJEu4qMkjp3P7qBfO/08vrzhiTG4uSeuTvK
6kZV7ZTyQPv7cPV0sArO1+zpU/O2xiKjWiNQQ6FRvsB4VS1XE8PYOavny/+rds0b
/WQhk9OhVW/7LTkrbxmlcgQWFwQHzrVz3i+hycbt2WsrIQaedpOM+0sp85sGDnGP
LKsPUzS/A0RoeIBMMalHYUHBRaxRn7hZfMY5JrmkiciB1IFnFvTQK8z/hOWN8v7I
NktifiMkBzQkZFK3y32bhh1PWmJTdmQ0c5Nsdmr3BlgQIcvFNLNGtA==
=r0zx
-----END PGP SIGNATURE-----

In article <4722FEA4.6040509@sarion.co.jp>,
  "NAKAMURA, Hiroshi" <nakahiro@sarion.co.jp> writes:

>> I see.  The maintenance problem exists until RubyGems
>> developper adopt it.
>
> Sure.  And your LoadError fallback has the same issue.  The hack should

I maintain it.  So my tiny RubyGems loader has no such issue.

However I'm glad if it is maintained by RubyGems
developer.  But I maintain it until that.

> be maintained by RubyGems team because the check scheme depends on
> RubyGems specification.  For example, you need to check APPLE_GEM_HOME
> in rbconfig if RubyGems team accepts the patch from Apple for Leopard:
> http://rubyforge.org/pipermail/rubygems-developers/2007-October/003112.html

Thank you for the information.  I'll update my tiny RubyGems
loader if it is incorporated.

> According to DRY principle, we should not such logic in prelude.rb.

Sometimes an principle is difficult to apply.

>> RUBYOPT='-rdisable-rubygems-loader -ryour-favorite-loader'
>> disables RubyGems and enables your favorite loader.
>
> I don't like this sort of negative spell.

What's the reason?

>   end
> end

It causes infinite recursion.
Please show a working example.

> With RUBYOPT=-rfooloader, we can hook Kernel#require before RubyGems is
> activated.  The require chain after a LoadError is as follows.
>
>   rubygems -> foo -> rubygemsloader -> original_require
>
> So foo can intercept a LoadError from rubygems.  But when an user
> activates a gem with rubygems, say rails, dependency libraries are
> activated by rubygems, too.  Now foo cannot intercept LoadError for
> these libraries.

If an application and/or an administrator use RubyGems
explicitly, RubyGems should work.  So RubyGems should work
for Rails.

If an application use gem method to activate a library,
foo cannot intercept LoadError because LoadError doesn't
raised.  So foo may not work even if a RubyGems loader
is not used by default.  foo needs to know RubyGems to work
with a library installed as a gem.

> Till ruby equips sophisticated feature loading extension point, mixing
> two or more loaders should not be expected to work.

It is very difficult to combine extensions developed
individually.

Although it is interesting problem, I don't expect it will be
solved in near future.

Do you have a concrete idea for the sophisticated feature
loading extension point?

> Incorporating your
> LoadError fallback means that Ruby allows RubyGems to be used as a only
> custom loader.  I hope I can explain myself well.

I don't understand.  What is "custom loader"?

In article <47200D74.6020202@sarion.co.jp>,
  "NAKAMURA, Hiroshi" <nakahiro@sarion.co.jp> writes:

>> I don't know the definition of "core team".  What is it?
>
> It's said for dicision maker(s) of MRI & YARV.  Matz and ko1, and some
> comitters for each area who Matz is relying on?  I don't know exactly.

I talked to ko1.  He is surprised that you think he is a
member of the core team.

I think the term "core team" is too ambiguous.  It should be
defined before using it.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

Tanaka Akira wrote:
>>> I see.  The maintenance problem exists until RubyGems
>>> developper adopt it.
>> Sure.  And your LoadError fallback has the same issue.  The hack should
> 
> I maintain it.  So my tiny RubyGems loader has no such issue.

Oh, I didn't think you maintain it.  But let's wait for what RubyGems
team think about the way you go.

>> According to DRY principle, we should not such logic in prelude.rb.
> 
> Sometimes an principle is difficult to apply.

Definitely.  But it's a built-in logic of a ruby interpreter/Ruby
language.  We should be cautious.

>>> RUBYOPT='-rdisable-rubygems-loader -ryour-favorite-loader'
>>> disables RubyGems and enables your favorite loader.
>> I don't like this sort of negative spell.
> 
> What's the reason?

All spell are evil and spell for denying somethig are worse because it
generally don't work well and causes incomprehensible behavior.

>>   end
>> end
> 
> It causes infinite recursion.
> Please show a working example.

Sorry for confusing you.  You should know you can do anything at 'p "do
something good"' line because it's excerpted from your tiny rubygems
loader.  For example, replace;
  p "do something good"
with;
  raise unless require 'gemloader'

This causes Name crash between rubygems and gemloader but it's not the
issue I want to point out as I wrote in the previous mail.

> If an application and/or an administrator use RubyGems
> explicitly, RubyGems should work.  So RubyGems should work
> for Rails.

Sure.

> If an application use gem method to activate a library,
> foo cannot intercept LoadError because LoadError doesn't
> raised.  So foo may not work even if a RubyGems loader
> is not used by default.  foo needs to know RubyGems to work
> with a library installed as a gem.

Yes.

And those 2 behavior are not the point I wrote above.  Even if an user
want use another custom loader, your tiny rubygems loader loads rubygems
and a LoadError can activate gems.  As the result, the custom loader
cannot intercept the LoadError.  Am I misunderstand something?

  /  /  /

> loading extension point?
Agreed to all.  I don't have nothing to write down but I'll post a reply
to [ruby-core:12783] when I give a think about it.

Regards,
// NaHi

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Cygwin)

iQEVAwUBRyUyrh9L2jg5EEGlAQJO+ggAhT1Cw5pqB0ym2aYUW1oJ8cjB6phl68m+
iNawOISKaZJapoHRPo/3BqJ7Ah5tdAD/FHtmoiwXF8i3xAfG5Cb0ZGdcVEkpp///
xUwhSsJv0uko+yMXoYU3rfycyZX/ZJwQJ7Z2B2PsPdzanj3GxVWGguuwNcusBvYJ
0xEeVaNgwJF5nCqdKo8Is8CWwUGgSJe7SGppHLUQx6Pi4yBm1PEDjSIUEgCD1bcO
Hm3dVjBvyXNkpfwjbdJ7p/qQXanzh7+1a3rOfQlPyRDyxfHWZ2HyknId7CAddgWb
lrpeDciWBG7ZOxO49iKFnVly0HATgdGFVsB/JVDwJBdJV0ZaBwcTAg==
=4JFi
-----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

Tanaka Akira wrote:
>>> I don't know the definition of "core team".  What is it?
>> It's said for dicision maker(s) of MRI & YARV.  Matz and ko1, and some
>> comitters for each area who Matz is relying on?  I don't know exactly.
> 
> I talked to ko1.  He is surprised that you think he is a
> member of the core team.

What the point he is surprised with?

> I think the term "core team" is too ambiguous.  It should be
> defined before using it.

If I remember correctly it's already said in ruby-talk and I just gave
it a definition.  The definition above is not appropriate?  You mean
"don't use the term because core_team == Matz" ?

Regards,
// NaHi

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Cygwin)

iQEVAwUBRyU0OB9L2jg5EEGlAQKyQQf+KSyo4MzNmji8G6LM+V5PNYE1NQwI5rJA
wJROv2yccb/svqd7U1LCGps0PzLcPFBRTa1b0NhTQkVzVnexryeokyPWZDtqFCuC
5cV0lN/I6V3JrJhLjKUXmH1RLuigEMaCHGvb13nUG2r5baoEmPg2dB++u9jp95f5
0QMIgHQJr8Y0txaAxNspCMUTJLsAWlEhQG8Nzcq2/yG7yTmhm/bxWL4ujLx7JXPq
dChdeI+7cM6hE1wKzVdqaYqANNZN0a25hU1/4r4ZfNIaChPmE5fk64sLcUMKSsga
nzTsFX7LS6ugAY6NzyEpCYXF0C8BbP5tmeoAJfyNRo8bA7q4BV+iwQ==
=wQ3n
-----END PGP SIGNATURE-----

In article <472532B0.2060600@sarion.co.jp>,
  "NAKAMURA, Hiroshi" <nakahiro@sarion.co.jp> writes:

> Oh, I didn't think you maintain it.  But let's wait for what RubyGems
> team think about the way you go.

RubyGems developers can choose to maintain or not.  I don't
think there is a problem.

This is a critical difference of your proposal and mine.

> Definitely.  But it's a built-in logic of a ruby interpreter/Ruby
> language.  We should be cautious.

Do you have a concrete problem?

> All spell are evil and spell for denying somethig are worse because it
> generally don't work well and causes incomprehensible behavior.

I don't think it is evil or worse.  Do you have a concrete
problem?

Since it is intended to used as at first in RUBYOPT, I think
it works.

> Sorry for confusing you.  You should know you can do anything at 'p "do
> something good"' line because it's excerpted from your tiny rubygems
> loader.  For example, replace;
>   p "do something good"
> with;
>   raise unless require 'gemloader'
>
> This causes Name crash between rubygems and gemloader but it's not the
> issue I want to point out as I wrote in the previous mail.

My tiny RubyGems loader and your above code doesn't conflict.

% cat fooloader.rb
module Kernel
  alias foo_hooked_require require

  def require(feature)
    begin
      foo_hooked_require feature
    rescue LoadError => e
      raise unless require 'gemloader'
      require feature
    end
  end
end
% RUBYOPT=-rfooloader ./ruby -e 'require "redcloth"; p RedCloth'
RedCloth

The above ruby has my loader in prelude.rb.  RedCloth is
installed as a gem.  RUBYOPT=-rfooloader doesn't cause a
trobule: RedCloth can be required as usual, as fooloader
expected.

I know fooloader doesn't run in this case because my loader
run.  However the result is identical: RedCloth is loaded.
The two extensions doesn't conflicts because they does
exactly same task.

Please show working and conflicting example.

> And those 2 behavior are not the point I wrote above.  Even if an user
> want use another custom loader, your tiny rubygems loader loads rubygems
> and a LoadError can activate gems.  As the result, the custom loader
> cannot intercept the LoadError.  Am I misunderstand something?

I guess I don't understand the your point.

If an administrator installs a library as a gem, the library
is expected to use.  If the library is not installed,
LoadError is raised.

Do you think about that an administrator installs a library
as a gem but an user doesn't want to use it?

In article <47253439.1060006@sarion.co.jp>,
  "NAKAMURA, Hiroshi" <nakahiro@sarion.co.jp> writes:

> What the point he is surprised with?

I think he doesn't think he is a member of "core team".

> If I remember correctly it's already said in ruby-talk and I just gave
> it a definition.  The definition above is not appropriate?  You mean
> "don't use the term because core_team == Matz" ?

I couldn't find mail in ruby-talk.  Please show a reference.

Your definition in [ruby-core:12917] is not enough for
RubyGems issue.  Who is Matz relying on this area?

Since I don't know it, I can't understand "core team" in
this issue.  So I can't understand what you say.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

Tanaka Akira wrote:
>> Oh, I didn't think you maintain it.  But let's wait for what RubyGems
>> team think about the way you go.
> 
> RubyGems developers can choose to maintain or not.  I don't
> think there is a problem.

I think that RubyGems team possibly does not like this kind of logic
duplication.  Note that it's just a thought of mine.  I've heard nothing
about this from RubyGems team.

>> Definitely.  But it's a built-in logic of a ruby interpreter/Ruby
>> language.  We should be cautious.
> 
> Do you have a concrete problem?

Code duplication and independent maintainer as I wrote above.  My
concern will disappear once RubyGems team says they maintains it.

  require 'rubygems/gem_loadable_check'
  if Gem.gem_loadable?(feature)
    require 'rubygems'
  end
in prelude.rb is more better.

>> All spell are evil and spell for denying somethig are worse because it
>> generally don't work well and causes incomprehensible behavior.
> 
> I don't think it is evil or worse.  Do you have a concrete
> problem?

If it's not evil or worse, just let users use RUBYOPT as a charm to use
rubygems.  What's the difference?  But...

> Since it is intended to used as at first in RUBYOPT, I think
> it works.

Sorry, I don't understand this.

> My tiny RubyGems loader and your above code doesn't conflict.
>       require feature
>     end
>   end
> end
> % RUBYOPT=-rfooloader ./ruby -e 'require "redcloth"; p RedCloth'
> RedCloth
> 
> The above ruby has my loader in prelude.rb.  RedCloth is
> installed as a gem.  RUBYOPT=-rfooloader doesn't cause a
> trobule: RedCloth can be required as usual, as fooloader
> expected.

Here's my try;

0% cat prelude.rb

# Mutex

class Mutex
  def synchronize
    self.lock
    begin
      yield
    ensure
      self.unlock
    end
  end
end

# Thread

class Thread
  MUTEX_FOR_THREAD_EXCLUSIVE = Mutex.new
  def self.exclusive
    MUTEX_FOR_THREAD_EXCLUSIVE.synchronize{
      yield
    }
  end
end

module Kernel
  alias original_require require
  def require(feature)
    original_require feature
  rescue LoadError
    if require 'rubygems'
      require feature
    else
      raise
    end
  end
end

0% cat foo.rb
module Kernel
  alias foo_hooked_require require

  def require(feature)
    begin
      foo_hooked_require feature
    rescue LoadError => e
      raise unless require 'gemloader'
      require feature
    end
  end
end

0% ls /usr/local/lib/ruby/gems/1.9/specifications
httpclient-2.1.2.gemspec

0% RUBYOPT=-rfoo ruby19 -e 'require "httpclient"'
/usr/local/lib/ruby/site_ruby/1.9/gemloader.rb:44: warning: already
initialized constant OPS
/usr/local/lib/ruby/site_ruby/1.9/gemloader.rb:94: warning: already
initialized constant Requirement
gemloader.rb intercepted rubygems loading
/usr/local/lib/ruby/site_ruby/1.9/rubygems/specification.rb:629:in
`method_missing': undefined method `<<' for {}:Hash (NoMethodError)
        from
/usr/local/lib/ruby/site_ruby/1.9/rubygems/specification.rb:629:in
`add_dependency'

OT: I removed module_function from prelude.rb to allow users to use
Kernel.require directly.

The last exception is the 'Name crash' I wrote in the previous mail.

> I know fooloader doesn't run in this case because my loader
> run.  However the result is identical: RedCloth is loaded.
> The two extensions doesn't conflicts because they does
> exactly same task.
>
> Please show working and conflicting example.

Now foo.rb loads gemloader.rb but think when I want to use one of a
custom loader (not existing now) I wrote in [ruby-core:12774].  Say
signed tarball loader.  It might have another repository which has
signed tarballs but RubyGems loads a gem instead of it, isn't it?

> 
> Do you think about that an administrator installs a library
> as a gem but an user doesn't want to use it?

Hmm.  I didn't think about it in detail but yes, it should be a
situation I'm wondering about.  For example, a Leopard user may want to
use mydevelgemloader.rb which only loads gems from ~/mydevel/gems.

Regards,
// NaHi

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Cygwin)

iQEVAwUBRybE7h9L2jg5EEGlAQIwrAf/YYHf9ARuW6ntO2j4c4EjhtEPNTOeWqK4
7UTt02kcGeGI7U/rl6hKnnUu3cQ/z7YwEqf1BMi7lW9RG79D8hYD3tOIc6+prO7Q
ikTOmdjHC0ciihotRZgV39CPgfhu4MPU6BPJ7zUQ0hB5QWIEN2DadydT8+cBHlbj
iVaIIW6+ZKneQDk19U9iZrz+APP7tS5J2riVQUgz2jVge9K7Mwnbxbe2VNjuUXhI
oUokC1NIFoyPwKEyGAIsNYz6mUWKznt4oSrTkdgcTEozivZEZphWI3+EmJtyckUN
fjdXwIwmlbilsxJO+Rh8AvWi12MY4IWTg38yGIQwVpml1KDC0jc6oQ==
=Jz+I
-----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

Tanaka Akira wrote:
>> What the point he is surprised with?
> 
> I think he doesn't think he is a member of "core team".

He's not one of a decision maker of MRI & YARV?  It should be true for
MRI (1.8) but true for YARV (1.9), too?  Hmm.

>> If I remember correctly it's already said in ruby-talk and I just gave
>> it a definition.  The definition above is not appropriate?  You mean
>> "don't use the term because core_team == Matz" ?
> 
> I couldn't find mail in ruby-talk.  Please show a reference.

Searching "core team" from Google's archive
http://groups.google.co.jp/group/comp.lang.ruby/topics
gives me a list.  The latest seems to be [ruby-talk:264008].  Hmm.
I found [ruby-talk:251626] is interesting.  Is the term from
'ruby-core team' ?

Regards,
// NaHi

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Cygwin)

iQEVAwUBRybIRh9L2jg5EEGlAQJ+vwf+JAMljRCaOsKB7Nmjx1Osav2mLZlbeBpo
flaR7tMBtwhBDRYTrSvjq0QVNd19B0VlivLiPJeCU9phdEXNOEg66k1Gj9/D3yw6
jszAM7hIItsQ6Q+/fOWG1F9VUvHKT6ktvbQcZgeGUMzaUPOST4bYcSWSLzUzIjpN
W+PMrGKjFu969N1mli/LhYP6SpZUG3oK3mThsFfdcq8Rus9jIi+xKV84XDvz1Gj1
KkEuwYItiO5jWS7rpIyM7cSemhtRtl84neLEdLxxb76Lo+kMJoXZtp7B/OY7M4qS
OzjcDkZsEbKnAd8g2tQYHkik41jWPkT0u+tMl2TwsStXlwJeFMdW7g==
=DoT1
-----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi

Tanaka Akira wrote:
> This is a critical difference of your proposal and mine.

By the way, my proposal is, from the very beginning of this thread,
disabling rubygems by default and just bundle it.  Let users use RUBYOPT
as they are doing now.

Regards,
// NaHi

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Cygwin)

iQEVAwUBRybKMB9L2jg5EEGlAQKh7Qf/e8HzCcga1arDNSbMl3Aot++1sp0IwUv9
baBWnlSfomnN/GSsHeUb3/YK5wma7ud2d5ikGK0Ml5m/MnCrDUCpk/WykW4A0nre
NKAvxXSSyc/yHbJE0Bigtfpn0iqXs7NQsj3Mu3y3XR8ydGiUu8CwRcuB550vG0SA
6JgLYc5K7f1XVOxatxUaKa9wSUGDU2JJrsMnz114NaO9TTw3Wb91J1zXQavUWk1W
06oD2rO7jEk5B6P4zaPzDH+h8eRQ6trm/QyyZuTPga2J+FKVn5Vc1titlpn4cThX
9tXRNRWYsHXYI9Md5Fd43w80SMgn2OQOfyAJBNcwir84luT1LYCm2g==
=DjEW
-----END PGP SIGNATURE-----

In article <4726C4EF.7060605@sarion.co.jp>,
  "NAKAMURA, Hiroshi" <nakahiro@sarion.co.jp> writes:

> I think that RubyGems team possibly does not like this kind of logic
> duplication.  Note that it's just a thought of mine.  I've heard nothing
> about this from RubyGems team.

The logic is not identical.  RubyGems itself (and yours) is
accurate but memory consuming.  My tiny RubyGems loader is
not accurate but not memory saving

They are difficult to unify them.

> Code duplication and independent maintainer as I wrote above.

They cause maintainance cost, but I take it.  Any other
problem?

> If it's not evil or worse, just let users use RUBYOPT as a charm to use
> rubygems.  What's the difference?  But...

RUBYOPT is not required for using RubyGems.
disable-rubygems-loader.rb is used for disabling RubyGems.

I know you try to force us to use RUBYOPT for RubyGems.  But
I don't want to define RUBYOPT to simply use a installed
library.

> Here's my try;

Oops.  I tryed with internal version which try to remove
RubyGems loader.

However I can't reploduce your problem.

>   end
> end

I modified this to check LoadError at require 'rubygems'.

module Kernel
  alias _original_require require
  def require(feature)
    _original_require feature
  rescue LoadError => e
    if begin
         _original_require 'rubygems'
       rescue LoadError
         raise e
       end
      require feature
    else
      raise
    end
  end
end

> 0% cat foo.rb

foo.rb is exactly as yours.

> `add_dependency'
% ./ruby -rfoo -e 'require "redcloth"; p RedCloth'
/home/src/ruby/gems19/ruby/gemloader.rb:44: warning: already initialized 
constant OPS
/home/src/ruby/gems19/ruby/gemloader.rb:94: warning: already initialized 
constant Requirement
RedCloth
% ./ruby -rfoo -e 'require "httpclient"'
/home/src/ruby/gems19/ruby/gemloader.rb:44: warning: already initialized 
constant OPS
/home/src/ruby/gems19/ruby/gemloader.rb:94: warning: already initialized 
constant Requirement

The exception is not raised.

Note that I used gemloader.rb r13.
(Actually I used r12 supporting 1.9 myself at first.  It is
almost identical with r13.)

> The last exception is the 'Name crash' I wrote in the previous mail.

I'm not sure what causes your exception.

However I'm thinking to update my proposal to use
[ruby-dev:32161].

> Now foo.rb loads gemloader.rb but think when I want to use one of a
> custom loader (not existing now) I wrote in [ruby-core:12774].  Say
> signed tarball loader.  It might have another repository which has
> signed tarballs but RubyGems loads a gem instead of it, isn't it?

If a gem is installed, yes.  If a gem is not installed, no.

>> Do you think about that an administrator installs a library
>> as a gem but an user doesn't want to use it?
>
> Hmm.  I didn't think about it in detail but yes, it should be a
> situation I'm wondering about.  For example, a Leopard user may want to
> use mydevelgemloader.rb which only loads gems from ~/mydevel/gems.

In the condition under conflicts between an administrator
and an user, it is very difficult to solve the problem.

I think (or hope) such conflicting situation is rare.
An administrators intention, use a installed gem, should be
used by default.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

Tanaka Akira wrote:
>> I think that RubyGems team possibly does not like this kind of logic
>> duplication.  Note that it's just a thought of mine.  I've heard nothing
>> about this from RubyGems team.
> 
> The logic is not identical.  RubyGems itself (and yours) is
> accurate but memory consuming.  My tiny RubyGems loader is
> not accurate but not memory saving

Agreed.

> They are difficult to unify them.

Might true.  The way for letting it be unifyable can introduce some
other constraints I think.  For example, adding a rule for Gem
specification that 'gem creator must use 'lib' directory for
require_path in Gem::Specification' or so.

>> Code duplication and independent maintainer as I wrote above.
> 
> They cause maintainance cost, but I take it.  Any other
> problem?

No other problem I have for now.  But I'm still thinking RubyGems team
might not want it.  I heard that authors want to keep rubygems for 1.8
and 1.9 in sync.  I respect their vision.

Why don't you wait for the response?  Can somebody ask it in RubyConf?

>> If it's not evil or worse, just let users use RUBYOPT as a charm to use
>> rubygems.  What's the difference?  But...
> 
> RUBYOPT is not required for using RubyGems.
> disable-rubygems-loader.rb is used for disabling RubyGems.
> 
> I know you try to force us to use RUBYOPT for RubyGems.  But
> I don't want to define RUBYOPT to simply use a installed
> library.

I can understand it.  I too don't want to define RUBYOPT to use my
custom loader instead of RubyGems even if gems are installed by an
administrator.

It's about time we should summarize pros and cons of the ways we can
take.  It's a balancing issue, isn't it?

>> Here's my try;
> 
> Oops.  I tryed with internal version which try to remove
> RubyGems loader.
> 
> However I can't reploduce your problem.

>> The last exception is the 'Name crash' I wrote in the previous mail.
> 
> I'm not sure what causes your exception.

Sorry.  I wrongly set GEM_PATH to another point.  This repository
contains gems which has dependencies and causes
Gem::Specification#add_dependency crash.  Now I get the same result as
yours.

0% ruby19 -rfoo -e 'require "httpclient"'
/usr/local/lib/ruby/site_ruby/1.9/gemloader.rb:44: warning: already
initialized constant OPS
/usr/local/lib/ruby/site_ruby/1.9/gemloader.rb:94: warning: already
initialized constant Requirement
gemloader.rb intercepted rubygems loading
gemloader.rb intercepted rubygems loading
gemloader.rb intercepted rubygems loading
gemloader.rb intercepted rubygems loading
gemloader.rb intercepted rubygems loading

Anyway, the name crash is not important.  Back to the topic,

[ruby-core:12935]
> Would you show me a loader which doesn't work well with my
> loader?

I wrote a proof-of-concept custom loader 'unloadable.rb'.
http://dev.ctor.org/vtr/browser/trunk/lib/unloadable.rb
It (incompletely for now) allows user to unload a library.  See the head
of unloadable.rb for the definition of 'unload'.

Here's a script.  soap/soap is (still) installed in libdir of 1.9.
httpclient is installed as a gem.  prelude.rb is not changed (does not
include infinite loop detection but it won't affect I think).

0% ruby19 -d -runloadable -e 'require "soap/soap"; unload "soap/soap"'
[...]
removing constants: [:XSD, :Iconv, :SOAP]
[...]

0% ruby19 -d -runloadable -e 'require "httpclient/cookie"; unload
"httpclient/cookie"'
[...]
removing constants: [:RbConfig, :Config, :CROSS_COMPILING, :Gem,
:ConditionVariable, :Queue, :SizedQueue, :Forwardable,
:SingleForwardable, :Rational, :Date, :DateTime, :FileUtils, :Etc, :URI,
:WebAgent, :ParseDate]
[...]

RubyGems features are unexpectedly unloaded in the second execution.
For this issue, enabling RubyGems by default without tiny loader solves
the problem.

0% ruby19 -d -rubygems -runloadable -e 'require "httpclient/cookie";
unload "httpclient/cookie"'
[...]
removing constants: [:FileUtils, :Etc, :URI, :WebAgent, :ParseDate]
[...]

> However I'm thinking to update my proposal to use
> [ruby-dev:32161].

Sure.  I'll reply to ruby-dev.  We run this thread too much so no one
seems to be following here.

It's the best I think if we have such an extension point for custom
loaders and if RubyGems use it instead of replacing require.  But as I
wrote in [ruby-core:12827], we don't have enough time to do before 1.9.1
I think.  Do you think it happen?

>> Now foo.rb loads gemloader.rb but think when I want to use one of a
>> custom loader (not existing now) I wrote in [ruby-core:12774].  Say
>> signed tarball loader.  It might have another repository which has
>> signed tarballs but RubyGems loads a gem instead of it, isn't it?
> 
> If a gem is installed, yes.  If a gem is not installed, no.

Sure.

For reality, I wrote another proof-of-concept custom loader
'signedloader.rb'
http://dev.ctor.org/vtr/browser/trunk/lib/signedloader.rb
It tries to restrict to load features from configured dirs (libdir,
archdir, sitelibdir, sitearchdir, not including "." and others)

Here's a script.  ruby is ruby_1_8 version.  ruby19 is trunk version.
For 1.8, httpclient is installed in sitedir.  For 1.9, httpclient is
installed as a gem (no httpclient in sitedir).

0% ruby -rsignedloader -e 'require "httpclient/cookie"; p $".find_all {
|f| /httpclient/ =~ f }'
["signedloader:httpclient/cookie.rb"]

0% ruby19 -rsignedloader -e 'require "httpclient/cookie"; p $".find_all
{ |f| /httpclient/ =~ f }'
["/usr/local/lib/ruby/gems/1.9/gems/httpclient-2.1.2/lib/httpclient/cookie.rb"]

I use signedloader.rb to intercept RubyGems to load httpclient so the
second execution should raise a LoadError.

> An administrators intention, use a installed gem, should be
> used by default.

I can understand your thought (or hope).  Once Ruby "core team" (should
be Matz, this time) will decide this rather tight merge with RubyGems as
a design, with knowing this kind of rare issues, I'll live with that in
other way, for example, developing 'easily unloadable loader extension
point' such as your [ruby-dev:32161].

Regards,
// NaHi

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Cygwin)

iQEVAwUBRyk72x9L2jg5EEGlAQLPDQf/TbmqdHAAjDlTa7XlqdydwKmJJg+RbTzI
bRb1YoknAqlOI2C3T69mnHFYOcTxMsBgQcN9ytEtxaAh2Gs5Z/ZgD08nqmUrAROr
5K1Y6PRtREHdZHUKdmupVlkGD3BWy5gnuKo5OiDDQqrRAktAM0GYmA2k+Vw3O3yE
cnKleTRdzOYSwVjyXspcaw5Eiq1JFUtXWQwgtoBsqHlzWf/VNsL3aRn0rgG2k3f7
PIlasjHdknM3rxQ7JiNm0x2xccPkwJectQd7orX2f8S/jyvSAk244rGPfnu+RoJy
NGAoRGnxc10pO1G6EzifUBqeBWvGFJiIKHZ+5ODRclvEFWXp6B4w/w==
=p2lo
-----END PGP SIGNATURE-----

On Oct 31, 2007, at 9:37 PM, NAKAMURA, Hiroshi wrote:

> We run this thread too much so no one seems to be following here.

I can't speak for everyone, but I've very much enjoyed reading this
thread.

I sometimes feel like all the neat decisions about Ruby's future are
made in Japan and many of the rest of us just read about them in the
commit messages.  To be clear, I don't mean that negatively.  I know
you all try very hard to include us.

So, I haven't been able to think of anything intelligent enough to
add to this thread, but I am reading along and trying to understand.
I'm glad you had the discussion here and made that possible.

Thank you.

James Edward Gray II

On Oct 24, 2007, at 18:37 , NAKAMURA, Hiroshi wrote:
> will do it as I wrote in [ruby-core:12810].
> without concrete grounds.  I still don't have any number to show so I
>
> Can gem command of RubyGems can create a MANIFEST in gemspec dir?   
> Then
> akr's LoadError fallback can grep it for finding rubygems to be loaded
> or not.  It won't be needed as a part of RubyGems though.

It could.  I have considered speeding up RubyGems' require, since it
scans so many paths looking for files.  I never found it worthwhile
to implement, since files only get required around startup time.

>>> Your proposal of prelude.rb enables RubyGems.  As I wrote,  
>>> RubyGems uses their 'version selection scheme' independent from  
>>> whether gem method exists or not.  So you cannot use any other  
>>> scheme.  RUBYOPT solution allow you to do the above.
>>
>> I can install any library under site_ruby.  Does RubyGems prohibit  
>> it?
>
> I mean that you can use site_ruby but cannot use others.

If you install a library under site_ruby, a duplicate installation
with RubyGems will not be loaded without Kernel.gem.

On Oct 28, 2007, at 18:16 , NAKAMURA, Hiroshi wrote:
>> I think the term "core team" is too ambiguous.  It should be  
>> defined before using it.
>
> If I remember correctly it's already said in ruby-talk and I just gave
> it a definition.  The definition above is not appropriate?  You mean
> "don't use the term because core_team == Matz" ?

For English-speaking rubyists, "core team" is matz + other frequent
committers to ruby.  matz, nobu, usa, ko1, akr, knu, ...

My confusion comes from not knowing who is responsible for what parts
of ruby.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

I don't have enough vocabularies on this kind of... non ruby related
topic though...  Thank you for your kind message.

James Edward Gray II wrote:
> I sometimes feel like all the neat decisions about Ruby's future are
> made in Japan and many of the rest of us just read about them in the
> commit messages.  To be clear, I don't mean that negatively.  I know you
> all try very hard to include us.

I created a list of bundled library maintainer [ruby-core:12535] and I
realized we have few non-Japanese maintainer.  Though it's my individual
thought but there should be more.

Oh, you should take over a CSV namespace (not csv.rb) maintainer from
me.  I'll create a time to discuss about it soon.  I'm very sorry for
keeping you waiting long.

> So, I haven't been able to think of anything intelligent enough to add
> to this thread, but I am reading along and trying to understand.  I'm
> glad you had the discussion here and made that possible.

I found an audience here, at least.  Let me try to keep updating
ruby-core and ruby-dev.

Regards,
// NaHi

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Cygwin)

iQEVAwUBRysVQR9L2jg5EEGlAQLIFQf8CVbx+zHKNEvTBZyDTX3aNwsVzQrQDZGJ
qavz4NOvFNDw8W6BvagOKwjDbPRYHPuhJvb7JfYYQO7f7rziDJ3KNhpJ/UHI9gDD
zOtlflosA4hEz+GTgj7TlgWaEuDfpI0Jtw/dcqv2EccTsVbcsb/FTRAb3FQfHPWD
bOFZ0YEsV6duLPLHpaYhHaemM9aaHWgjDp0cS8qzj/z3PPVkV+xp+bxUSLQmCkwn
ny58rdTnRiV7/QiyjEiTpXwcHb4gvYkU4yD4G+TNkpuUBFVykzs+GY+9eHcy8dq8
wg4Nph4R0OXPrcSBImybk+3Wers5pM7YRtENVAa2DtLHwiJSOsURfw==
=QArt
-----END PGP SIGNATURE-----

On Nov 2, 2007, at 7:17 AM, NAKAMURA, Hiroshi wrote:

> Oh, you should take over a CSV namespace (not csv.rb) maintainer from
> me.  I'll create a time to discuss about it soon.  I'm very sorry for
> keeping you waiting long.

There is no hurry.  We can talk about it whenever you like.  I'll be
here.

James Edward Gray II