Well folks, it was a great experiment, but I'm thinking maybe we should bail on the idea of integrating maven directly into RubyGems. There's a number of reasons I think we should ditch it: * It would bring us closer to running stock RubyGems, so you can all just use update --system to get up-to-date RubyGems as versions come out. * It has never worked perfectly; there are dependency graphs that can't be represented well, Maven versions that don't map at all to RubyGems versions, and Maven and RubyGems have different ways of specifying "soft" version dependencies. * It has never worked on Windows; we chose poorly when we went with : as a group:artifact separator. * RubyGems folks have never really been interested in incorporating our changes or supporting Maven directly. * Bundler folks have never really been interested in supporting Maven directly, and would have run into the same issues as above. It was a really nice feature that we just never were able to implement well, given RubyGems versioning and dependency logic. Is anyone using RubyGems Maven support in JRuby? Would you miss it? - Charlie
on 2012-09-12 03:19
on 2012-09-12 03:50
On Wed, Sep 12, 2012 at 2:17 AM, Charles Oliver Nutter <email@example.com>wrote: > Well folks, it was a great experiment, but I'm thinking maybe we > should bail on the idea of integrating maven directly into RubyGems. > > Maybe, it would be better with a tool for generating a gem from jar files. For example, if a-0.1.jar depends on b-0.2.jar, and this depends on c-0.3.jar, we will have 3 gems (a-0.1gem, b-0.2.gem, c-0.3.gem). Any other jar which depends on c-0.3.jar could reuse the c-0.3.gem This requires a some manual work for JRuby users, but it will work in any environment where RubyGems/Bundler is ready.
on 2012-09-12 06:37
I want to add one more thought about the rubygems maven support. when we create a gem out of a jar we mapped the jar version somehow to rubygems version - given it some version range. since maven version do resolve totally differently from rubygems, the need arose to make the mapped version range bigger and bigger. to accommodate jar like joda-time where version 2.1 is backward compatible with version 1.6.x then the mapping would basically means ">0" or at least [<3, >=1] which means to almost omit version constraints altogether. the version mapping between jar and rubygems does not work. the conclusion from that is to use the dependency resolution libraries which do the right job. but that needs to lives outside of rubygems and following the idea of bundler the project jbundler tries to fill that hole. (in case you really want to have a look: https://github.com/mkristian/jbundler - but you do not need to, there are (java) projects which resolves their deps manually) - Kristian
on 2012-09-12 07:18
This is coming from someone who has only used JRuby with rails - I can imagine a possibility where maven could be used with a project and asked to dump the jars in a particular directory from where the application does a blind 'require everything'. Maven appears to be very configurable. All the directory locations are configurable, it can be made to point to some empty directory for any directory that is required. There will need to exist a full fledged pom.xml at the application root. It already has a 'copy-dependencies' task where it dumps all dependencies at a configurable location. It can also be made to write out a file which contains the classpath, including all the jar names in the format where it can be called from command line. All the other lifecycles can be made to do nothing. People can use whatever they want from RubyGems and maven stuff can be loaded from another place. There might be some gotchas for sure but overall it all doesn't sound too out of the ordinary to me right now.
on 2012-09-12 07:31
great idea Pradeep. but we also need to find a solution to have gems depending on jars instead of vendoring them as t done currently. here my proposal for solving the "gem depends on a jar" problem: https://github.com/mkristian/jbundler/wiki/Gem-wit... here maven would need something more. -Kristian On Wed, Sep 12, 2012 at 10:47 AM, Pradeep Singh
on 2012-09-12 07:35
What if gems were OSGI bundles?
on 2012-09-12 17:52
Maven has a lot of cruft that is not useful for a JRuby project. What we really need is the transitive dependencies resolution provided by Maven. That way a JRuby can just defined a handful of Jars and have all their transitive Jars automatically installed. This is the goal of LockJar - <https://github.com/mguymon/lock_jar>http://mguymon... Similar to Bundler, the flow is to create a Jarfile that is used to generated a Jarfile.lock . The Jarfile.lock can be used to automatically install Jars to the local system and populate the classpath. The Jar dependencies from the Jarfile.lock can even be installed when a Gem is installed. Once Bundler gets plugins ( https://github.com/carlhuda/bundler/issues/1945 ) I can add a plugin that will sync up LockJar with Bundler. Then the Jarfile.lock will not have to managed separately.
on 2012-09-18 00:19
Kristian and I are starting to collaborate on a broader solution for Gems with Jars, which should cover the goals of Kristian's proposal. We are still kicking around ideas and use cases that need covered. Btw, LockJar will handle Jar dependencies for libraries. Even install the Jars when a Gem installs. The only thing a lib needs to do is call the LockJar.lock when it is required: #get jarfile relative the gem dir lockfile = File.expand_path( "../Jarfile.lock", __FILE__ ) LockJar.load( :lockfile => lockfile ) https://github.com/mguymon/lock_jar#gem-integration --M
on 2012-09-18 03:14
hi Theo, I just want to pick on rubygems. it loads gems lazy, whenever you require 'something' it will looks in its gems to find 'something.rb' and "activate" any gem needed on the way. if you require 'something_else' then it will do the same for something else. in case it needs a gem with a different version of an "active" gem, it bails off with an error "already activate gems . . . ." (or so). in can be that things are just fine when you first require 'something_else' and then require 'something' rubygems gems resolution is also changing over time, i.e. it first tries to load the latest version of a needed gem. so one day you use gem A with version 2.0 and the next you will use version 2.1 because on of your new gems pulled in the newer version. I can not even be sure if someone else uses the gems as I since meanwhile there is even newer version available. so gems resolution with rubygems depends on the order at which gems gets activated, i.e. also the execution order within your code can matter. it is changing over time. IMO not really you want to rely on. this would be also true if rubygems could resolve jar dependencies. this lead to a lot of headache and then first rails (2.x) made their own gem-dependeny-management and later bundler replaced it with rails (3.x). the reason why bundler works much better is that look at a set of dependencies and resolve them all together, it guarantees reproducibility (over time). I do see for gem projects following Gemfile source :rubygems gemspec so just manage you gems with bundler for that project. for me rubygems is a convenient tool for trying out things with or without irb. it is not useful for manage the deps of a project, library or application. regards, Kristian PS @Theo can I add you to our little discussion with Michael - more view points are appreciated ? @Micheal there is already an email in the pipeline but I am not so fast right now :( On Mon, Sep 17, 2012 at 10:45 AM, Michel Guymon
on 2012-09-18 03:45
Hi, (sorry if you get this twice, I had problems with my list subscription) I'd love to be more deeply involved in this discussion. It's something I've been struggling with a lot since most of the gems I build work on top of Java libraries. About Rubygems: don't get me wrong, I'm not saying I want Rubygems to handle gem activation, I'm saying that for any solution to succeed Rubygems needs to be involved in the install-time dependency resolution. Everyone uses Bundler for version management, and many use RVM for sandboxing, but those work on top of Rubygems dependency resolution (whether or not Bundler is there managing exactly which versions gets installed). I may not have gotten the exact technical definitions correct there, but I hope you can parse out what I'm getting at. In my opinion JRuby must be able to to the following, out of the box: install all dependencies for a gem, including any JAR/Maven/whatever dependencies of gems that the gem depends on. This means that if gem A depends on JAR X, and gem B depends on gem A, then installing B will install X, without the user having to specify that X is needed. Adding Bundler, or some other tool, into the mix, it can help making sure no incompatible versions end up getting installed, or required in a bad order -- but the important thing is that the JAR got installed, and that the user didn't have to specify that a dependency of A has to be installed when she installs B. Consider a similar situation: Bundler has an awesome feature that lets you specify a git repository as the source of a gem. However, the gem in that repository cannot in turn depend on another git repository, because Rubygems doesn't support the feature, and Bundler doesn't replace Rubygems (for example, Bundler could look for a Gemfile in the repository and discover transitive dependencies that way, but it doesn't). This means that if you want to live on the bleeding edge and install a gem from a repository, and have all its dependencies installed from their respective repositories, you have to specify all of the dependencies in your Gemfile, and all the dependencies of all of the dependencies (and so on) too. Focusing on Bundler, JBundler, or LockJar or any other tool like that does not solve that problem, unless we want to invent a parallel dependency management environment to Rubygems. Being able to do what Bundler can do with git repositories is trivial, what is needed is something that handles the transitivity of installation, not just the transitivity of version resolution. The proposal to use the "requirements" property in gemspecs is nice, because it works with Rubygems, but in my opinion it needs to go into Rubygems itself, or at least into JRuby so that it works out of the box. LockJar and JBundler are nice, but this problem needs to be solved at a lower level. T#
on 2012-09-18 04:25
On Mon, Sep 17, 2012 at 2:12 PM, Theo <firstname.lastname@example.org> wrote: > Hi, > > (sorry if you get this twice, I had problems with my list subscription) > that's why I did not receive your first email - I just saw today the reply of Micheal :) OK, I see there are people around who just want to rubygems with rvm or by setting GEM_HOME/GEM_PATH per project, or similar ;) since the maven_gem support never made it into rubygems the question is what can be done to provide the desired functionality. the only way see right now is something like $ gem install_with_jars mygem this could at least make sure there are some versions of the required jars installed. the thing is to "activate" those jars. the need of custom code depending on a tools really ties the tool (which should be external IMO) any ideas ? - Kristan
on 2012-09-18 10:59
On 09/17/2012 05:22 AM, kristian wrote: > or by setting GEM_HOME/GEM_PATH per project, or similar ;) > > since the maven_gem support never made it into rubygems the question > is what can be done to provide the desired functionality. > > the only way see right now is something like > > $ gem install_with_jars mygem What about using the Gemspec's external to handle downloading the Jar dependencies when the Gem install? The external can be set to a Rakefile, the only juggling act is that Rakefile's default must be used for the Gemspec external call. While not ideal (native support would be nice), it does at least give the ability to package Gems and have them install all dependencies. A gem plugin is a good idea. To be able to see what jars a Gem is dependent on, re download Jars, etc.
on 2012-09-18 12:32
just thinking aloud ;) a gem plugin could be a way to hook something into the "gem install". whenever you call a gem command all plugins get loaded. maybe it is possible to do so in way which survives a few updates of rubygems. - Kristian
on 2012-09-24 20:38
On Mon, Sep 17, 2012 at 10:58 PM, kristian <email@example.com> wrote: > just thinking aloud ;) > and led to some discussions but a proof of concept: https://github.com/mkristian/jbundler/wiki/rubyjam... https://github.com/mkristian/jbundler/tree/next_generation a little demo is here: $ jruby -S gem install rubyjams $ jruby -S gem install nokogiri-maven $ jruby -rubygems -e 'require "rubyjams";gem "nokogiri-maven"; require "nokogiri"'` so the install of nokogiri-maven downloads and installs dependent jars. the gem 'nokogiri-maven' chooses the right gem (and not just nokogiri) and the last require will add the dependent jars to the classloader. so the idea is to mimic rubygems for jars. keeping things as symmetrically as possible to people find themselves on familiar ground. of course it is not at all perfect and might fail rather soon. but if you need more control over the jars you need to use a tool like jbundler. regards, Kristian