Forum: Ruby-core [ruby-trunk - Feature #7375][Open] embedding libyaml in psych for Ruby 2.0

Posted by tenderlovemaking (Aaron Patterson) (Guest)
on 2012-11-17 00:43
(Received via mailing list)
Issue #7375 has been reported by tenderlovemaking (Aaron Patterson).

----------------------------------------
Feature #7375: embedding libyaml in psych for Ruby 2.0
https://bugs.ruby-lang.org/issues/7375

Author: tenderlovemaking (Aaron Patterson)
Status: Open
Priority: Normal
Assignee:
Category:
Target version:


People have a hard time using psych (and thus rubygems) because it 
depends on libyaml. We can ease upgrades by embedding libyaml in order 
to eliminate the library dependency.

libyaml is MIT license, so it shouldn't impact Ruby's license.

Matz, may I embed libyaml for preview2?
Posted by mame (Yusuke Endoh) (Guest)
on 2012-11-17 03:06
(Received via mailing list)
Issue #7375 has been updated by mame (Yusuke Endoh).

Status changed from Open to Assigned
Assignee set to matz (Yukihiro Matsumoto)
Target version set to 2.0.0


----------------------------------------
Feature #7375: embedding libyaml in psych for Ruby 2.0
https://bugs.ruby-lang.org/issues/7375#change-33006

Author: tenderlovemaking (Aaron Patterson)
Status: Assigned
Priority: Normal
Assignee: matz (Yukihiro Matsumoto)
Category:
Target version: 2.0.0


People have a hard time using psych (and thus rubygems) because it 
depends on libyaml. We can ease upgrades by embedding libyaml in order 
to eliminate the library dependency.

libyaml is MIT license, so it shouldn't impact Ruby's license.

Matz, may I embed libyaml for preview2?
Posted by Yusuke Endoh (Guest)
on 2012-11-17 03:10
(Received via mailing list)
Looks good.  Matz, could you accept this?

$B$^$D$b$H$5$s!"(B2.0.0 $B$O(B libyaml $B$,;v<B>eI,?\$J$N$G(B 
($B$J$$$H(B rubygems $B$,F0$+$J$$(B) $B!"(B
$B%S%k%I$,LLE]$H$$$&(B FAQ $B$,=P$=$&$G$9!#(B
aaron $B$KAjCL$7$?$H$3$m!"(Blibyaml $B$N%=!<%9%3!<%I$r$^$k$4$H(B psych 
$B$K%P%s%I%k$9$k(B
$B%Q%C%A$r:n$C$F$/$l$^$7$?!#(B

libyaml $B$O(B MIT 
$B%i%$%;%s%9$J$N$G%i%$%;%s%9>e$NLdBj$O$J$$$H;W$$$^$9!#(B
$B>5G'$7$FD:$1$^$9$+!)(B

--
Yusuke Endoh <mame@tsg.ne.jp>


2012/11/17 tenderlovemaking (Aaron Patterson) 
<aaron@tenderlovemaking.com>:
Posted by Vít Ondruch (vo_x)
on 2012-11-17 19:09
(Received via mailing list)
Issue #7375 has been updated by vo.x (Vit Ondruch).


I hope this proposal will be rejected. Here are some reasons:

https://fedoraproject.org/wiki/Packaging:No_Bundle...
http://wiki.gentoo.org/wiki/Why_not_bundle_dependencies

And here are policies regarding bundling for several Linux 
distributions. All of them forbids bundling

https://fedoraproject.org/wiki/Packaging:Guideline...
http://www.debian.org/doc/debian-policy/ch-source....
http://en.opensuse.org/openSUSE:Packaging_guidelin...
----------------------------------------
Feature #7375: embedding libyaml in psych for Ruby 2.0
https://bugs.ruby-lang.org/issues/7375#change-33027

Author: tenderlovemaking (Aaron Patterson)
Status: Assigned
Priority: Normal
Assignee: matz (Yukihiro Matsumoto)
Category:
Target version: 2.0.0


People have a hard time using psych (and thus rubygems) because it 
depends on libyaml. We can ease upgrades by embedding libyaml in order 
to eliminate the library dependency.

libyaml is MIT license, so it shouldn't impact Ruby's license.

Matz, may I embed libyaml for preview2?
Posted by jeremyevans0 (Jeremy Evans) (Guest)
on 2012-11-17 19:24
(Received via mailing list)
Issue #7375 has been updated by jeremyevans0 (Jeremy Evans).


vo.x (Vit Ondruch) wrote:
> I hope this proposal will be rejected. Here are some reasons:
>
> https://fedoraproject.org/wiki/Packaging:No_Bundle...
> http://wiki.gentoo.org/wiki/Why_not_bundle_dependencies
>
> And here are policies regarding bundling for several Linux distributions. All of 
them forbids bundling
>
> 
https://fedoraproject.org/wiki/Packaging:Guideline...
> http://www.debian.org/doc/debian-policy/ch-source....
> 
http://en.opensuse.org/openSUSE:Packaging_guidelin...

There are two fairly easy ways to fix that, hopefully Aaron will choose 
one of them:

1) Default to using the system implementation if present, using the 
embedded one only if no system implementation is found.
2) Default to using the embedded implementation, but have a separate 
configure flag for using the system implementation.

I'd prefer 1), since it doesn't change things for existing packagers, 
while making it easier on other people who build ruby from source.
----------------------------------------
Feature #7375: embedding libyaml in psych for Ruby 2.0
https://bugs.ruby-lang.org/issues/7375#change-33028

Author: tenderlovemaking (Aaron Patterson)
Status: Assigned
Priority: Normal
Assignee: matz (Yukihiro Matsumoto)
Category:
Target version: 2.0.0


People have a hard time using psych (and thus rubygems) because it 
depends on libyaml. We can ease upgrades by embedding libyaml in order 
to eliminate the library dependency.

libyaml is MIT license, so it shouldn't impact Ruby's license.

Matz, may I embed libyaml for preview2?
Posted by Aaron Patterson (tenderlove)
on 2012-11-17 21:20
(Received via mailing list)
On Sun, Nov 18, 2012 at 03:05:50AM +0900, vo.x (Vit Ondruch) wrote:
>
> Issue #7375 has been updated by vo.x (Vit Ondruch).
>
>
> I hope this proposal will be rejected. Here are some reasons:
>
> https://fedoraproject.org/wiki/Packaging:No_Bundle...
> http://wiki.gentoo.org/wiki/Why_not_bundle_dependencies

Thanks for the links.  Unfortunately these lists of rules don't help
solve the problems I enumerated in the ticket.

> And here are policies regarding bundling for several Linux distributions. All of 
them forbids bundling
>
> 
https://fedoraproject.org/wiki/Packaging:Guideline...
> http://www.debian.org/doc/debian-policy/ch-source....
> 
http://en.opensuse.org/openSUSE:Packaging_guidelin...

Again, doesn't help with the problems we're having.
Posted by Aaron Patterson (tenderlove)
on 2012-11-17 21:20
(Received via mailing list)
On Sun, Nov 18, 2012 at 03:23:49AM +0900, jeremyevans0 (Jeremy Evans) 
wrote:
> > 
https://fedoraproject.org/wiki/Packaging:Guideline...
> > http://www.debian.org/doc/debian-policy/ch-source....
> > 
http://en.opensuse.org/openSUSE:Packaging_guidelin...
>
> There are two fairly easy ways to fix that, hopefully Aaron will choose one of 
them:
>
> 1) Default to using the system implementation if present, using the embedded one 
only if no system implementation is found.
> 2) Default to using the embedded implementation, but have a separate configure 
flag for using the system implementation.
>
> I'd prefer 1), since it doesn't change things for existing packagers, while 
making it easier on other people who build ruby from source.

#1 seems totally reasonable.  I could do that pretty easily. :-)
Posted by "Vít Ondruch" <v.ondruch@gmail.com> (Guest)
on 2012-11-17 21:34
(Received via mailing list)
Dne 17.11.2012 21:19, Aaron Patterson napsal(a):
> Again, doesn't help with the problems we're having.

I don't understand what problems. Was this issue triggered by
[ruby-core:49416]? May be there could be better error message then

/test_gem_command_manager.rb:
cannot load such file -- psych


Also the if the point

However, I'm able to build and install `ruby`
without `libyaml-dev`

made there is valid, how it comes that the build does not fail if the
libyaml is not present on the system, while it seems to be hard
dependency of some officially supported Ruby feature?

And what about educational factor? You bundle once and everybody will
think it is good practice, while it is not. Nobody learns nothing :/


Vit
Posted by Benoit Daloze (Guest)
on 2012-11-17 21:51
(Received via mailing list)
On 17 November 2012 21:34, Vt Ondruch <v.ondruch@gmail.com> wrote:
>
> Vit
Agreed, it does not make sense to me to bundle to avoid this problem.
They are already many other dependencies which are of course not 
bundled.
`rvm requirements` lists amongst others readline, iconv, zlib,
libyaml, libffi, openssl.
I would be for a failing build if libyaml is not present (except if
explicitly specified, like --disable-libyaml), or a very visible
warning at least, since it is a dependency of a standard library
always installed by default (rubygems).
I always thought missing extensions due to missing libraries were too
hard to notice in the build process, maybe a good summary of what was
built and what failed at the end would solve that?
Posted by Yusuke Endoh (Guest)
on 2012-11-18 15:42
(Received via mailing list)
Hello,

2012/11/18 Benoit Daloze <eregontp@gmail.com>:
> Agreed, it does not make sense to me to bundle to avoid this problem.
> They are already many other dependencies which are of course not bundled.
> `rvm requirements` lists amongst others readline, iconv, zlib,
> libyaml, libffi, openssl.

There are three practical reasons why libyaml should be bundled:

  - rubygems strongly depends on libyaml.
  - libyaml is not so popular. compared to readline, zlib, openssl, etc.
    I think there are many platforms that has no libyaml.
  - libyaml is actually so small.

I can understand the policy of the Linux distribution.  But, Ruby source
distribution is mainly used for those who intend to use Ruby with no
packaing system.

Note that there is a precident: nkf stdlib actually bundles the whole
source code of nkf project.


Vit, can you create a distro package of Ruby depending on libyaml 
package,
rathar than bundling it, even if Ruby source distribution bundles 
libyaml?
I'm sorry to trouble you, but I would greatly appreciate your 
cooperation.


> On 17 November 2012 21:34, Vt Ondruch <v.ondruch@gmail.com> wrote:
>> I don't understand what problems. Was this issue triggered by
>> [ruby-core:49416]? May be there could be better error message then

BTW: The trigger is a draft of 2.0.0 upgrade notes that I'm writing:

http://bugs.ruby-lang.org/projects/ruby/wiki/200Up...

It shows a cumbersome process to install libyaml to use rubygems.
I asked Aaron to check the process, and he counter-proposed bundling
libyaml.
Posted by "Vít Ondruch" <v.ondruch@gmail.com> (Guest)
on 2012-11-18 16:49
(Received via mailing list)
Dne 18.11.2012 15:41, Yusuke Endoh napsal(a):
> Hello,
>
> 2012/11/18 Benoit Daloze <eregontp@gmail.com>:
>> Agreed, it does not make sense to me to bundle to avoid this problem.
>> They are already many other dependencies which are of course not bundled.
>> `rvm requirements` lists amongst others readline, iconv, zlib,
>> libyaml, libffi, openssl.
> There are three practical reasons why libyaml should be bundled:
>
>    - rubygems strongly depends on libyaml.

No doubt about it.

>    - libyaml is not so popular. compared to readline, zlib, openssl, etc.
>      I think there are many platforms that has no libyaml.

How that come that somebody could not have libyaml on some platform but
would be able to compile libyaml bundled with psych? This argument seems
moot. Or are you going to fork libyaml because of that?

>    - libyaml is actually so small.

It is even smaller if you don't bundle it and use the system one :)

>
> I can understand the policy of the Linux distribution.  But, Ruby source
> distribution is mainly used for those who intend to use Ruby with no
> packaing system.

Who are you referring to? If there is somebody like that, he/she should
have probably some level of understanding, because it will be needed
sooner or later.

>
> Note that there is a precident: nkf stdlib actually bundles the whole
> source code of nkf project.

I'll definitely look at nkf. Thanks for the tip. Also note that there
are other bundled code, such as rubygems, rake, rdoc, oniguruma,
minitest. But I am not sure why Ruby should use bad examples as a
justification for bundling.

>
>
> Vit, can you create a distro package of Ruby depending on libyaml package,
> rathar than bundling it, even if Ruby source distribution bundles libyaml?
> I'm sorry to trouble you, but I would greatly appreciate your cooperation.

If the implementation will follow proposal 1) from Jeremy, then probably
quite easily. But this is not technical question, if I can or cannot.
Posted by Yusuke Endoh (Guest)
on 2012-11-18 18:38
(Received via mailing list)
Hello Vit,

2012/11/19 Vt Ondruch <v.ondruch@gmail.com>:
>> I can understand the policy of the Linux distribution.  But, Ruby source
>> distribution is mainly used for those who intend to use Ruby with no
>> packaing system.
>
>
> Who are you referring to? If there is somebody like that, he/she should have
> probably some level of understanding, because it will be needed sooner or
> later.


As you might guess, I'm concerned about those who build Ruby source
tarball themselves.
Probably, they are not a majority.  But we hold them in high regard.
They are the first users for us.

Of course, many of them will address the dependency lack.
But we can easily fix the lack ourselves.  I don't like to waste
their time to perform the following useless process:

  - build and install Ruby tarball,
  - (after a bit) notice that gem does not work,
  - find the cause and workaround with impatience,
  - find, download, build and install libyaml, and
  - rebuild and reinstall Ruby tarball again!
Posted by "Vít Ondruch" <v.ondruch@gmail.com> (Guest)
on 2012-11-18 22:20
(Received via mailing list)
Dne 18.11.2012 18:37, Yusuke Endoh napsal(a):
Posted by Aaron Patterson (tenderlove)
on 2012-11-19 00:07
(Received via mailing list)
On Mon, Nov 19, 2012 at 12:48:36AM +0900, V
Posted by Yusuke Endoh (Guest)
on 2012-11-19 04:40
(Received via mailing list)
Hello Vit,

2012/11/19 Vt Ondruch <v.ondruch@gmail.com>:
> But anyway. I'll try to refrain from further comments to this thread, since
> I made all my points.

You might have a coronary if you saw trunk/LEGAL.
Not only nkf but also the Ruby core itself bundles, in whole or in part,
many source files of other projects.
Posted by matz (Yukihiro Matsumoto) (Guest)
on 2012-11-20 06:59
(Received via mailing list)
Issue #7375 has been updated by matz (Yukihiro Matsumoto).


My only concern for bundling third party library is that leaving behind 
unmaintained after years.
So give me a promise that Aaron will keep it for foreseeable future. 
Then I will trust him.

Matz.

----------------------------------------
Feature #7375: embedding libyaml in psych for Ruby 2.0
https://bugs.ruby-lang.org/issues/7375#change-33169

Author: tenderlovemaking (Aaron Patterson)
Status: Assigned
Priority: Normal
Assignee: matz (Yukihiro Matsumoto)
Category:
Target version: 2.0.0


People have a hard time using psych (and thus rubygems) because it 
depends on libyaml. We can ease upgrades by embedding libyaml in order 
to eliminate the library dependency.

libyaml is MIT license, so it shouldn't impact Ruby's license.

Matz, may I embed libyaml for preview2?
Posted by Aaron Patterson (tenderlove)
on 2012-11-21 02:06
(Received via mailing list)
On Tue, Nov 20, 2012 at 02:58:58PM +0900, matz (Yukihiro Matsumoto) 
wrote:
>
> Issue #7375 has been updated by matz (Yukihiro Matsumoto).
>
>
> My only concern for bundling third party library is that leaving behind 
unmaintained after years.
> So give me a promise that Aaron will keep it for foreseeable future. Then I will 
trust him.

I'm not sure exactly what you're asking, but I have no plans to quit
maintenance for the foreseeable future.  :-)
Posted by mame (Yusuke Endoh) (Guest)
on 2012-11-24 03:33
(Received via mailing list)
Issue #7375 has been updated by mame (Yusuke Endoh).

Priority changed from Normal to High

Thanks matz!  Aaron, go ahead.

--
Yusuke Endoh <mame@tsg.ne.jp>
----------------------------------------
Feature #7375: embedding libyaml in psych for Ruby 2.0
https://bugs.ruby-lang.org/issues/7375#change-33746

Author: tenderlovemaking (Aaron Patterson)
Status: Assigned
Priority: High
Assignee: matz (Yukihiro Matsumoto)
Category:
Target version: 2.0.0


People have a hard time using psych (and thus rubygems) because it 
depends on libyaml. We can ease upgrades by embedding libyaml in order 
to eliminate the library dependency.

libyaml is MIT license, so it shouldn't impact Ruby's license.

Matz, may I embed libyaml for preview2?
Posted by mame (Yusuke Endoh) (Guest)
on 2012-11-24 06:21
(Received via mailing list)
Issue #7375 has been updated by mame (Yusuke Endoh).

Assignee changed from matz (Yukihiro Matsumoto) to tenderlovemaking 
(Aaron Patterson)


----------------------------------------
Feature #7375: embedding libyaml in psych for Ruby 2.0
https://bugs.ruby-lang.org/issues/7375#change-33793

Author: tenderlovemaking (Aaron Patterson)
Status: Assigned
Priority: High
Assignee: tenderlovemaking (Aaron Patterson)
Category:
Target version: 2.0.0


People have a hard time using psych (and thus rubygems) because it 
depends on libyaml. We can ease upgrades by embedding libyaml in order 
to eliminate the library dependency.

libyaml is MIT license, so it shouldn't impact Ruby's license.

Matz, may I embed libyaml for preview2?
Posted by tenderlovemaking (Aaron Patterson) (Guest)
on 2012-11-28 19:27
(Received via mailing list)
Issue #7375 has been updated by tenderlovemaking (Aaron Patterson).

Status changed from Assigned to Closed
% Done changed from 0 to 100

I fixed this in r37919, but it didn't seem to close.  I'm closing now.
----------------------------------------
Feature #7375: embedding libyaml in psych for Ruby 2.0
https://bugs.ruby-lang.org/issues/7375#change-34090

Author: tenderlovemaking (Aaron Patterson)
Status: Closed
Priority: High
Assignee: tenderlovemaking (Aaron Patterson)
Category:
Target version: 2.0.0


People have a hard time using psych (and thus rubygems) because it 
depends on libyaml. We can ease upgrades by embedding libyaml in order 
to eliminate the library dependency.

libyaml is MIT license, so it shouldn't impact Ruby's license.

Matz, may I embed libyaml for preview2?
Posted by zzak (Zachary Scott) (Guest)
on 2012-11-28 19:50
(Received via mailing list)
Issue #7375 has been updated by zzak (Zachary Scott).


Aaron, you have to add the ticket in the message, like:
[Feature #7375]
----------------------------------------
Feature #7375: embedding libyaml in psych for Ruby 2.0
https://bugs.ruby-lang.org/issues/7375#change-34091

Author: tenderlovemaking (Aaron Patterson)
Status: Closed
Priority: High
Assignee: tenderlovemaking (Aaron Patterson)
Category:
Target version: 2.0.0


People have a hard time using psych (and thus rubygems) because it 
depends on libyaml. We can ease upgrades by embedding libyaml in order 
to eliminate the library dependency.

libyaml is MIT license, so it shouldn't impact Ruby's license.

Matz, may I embed libyaml for preview2?
Please log in before posting. Registration is free and takes only a minute.
Existing account (Switch to SSL-encrypted connection)
NEW: Do you have a Google/GoogleMail or Yahoo account? No registration required!
Log in with Google account | Log in with Yahoo account
No account? Register here.