Wrapping an existing java library

cremes · June 2, 2009, 7:17pm

In my daily work I need to wrap an existing java library and put a
pretty little ruby interface on it. I thought I’d take this
opportunity to write up how to do it and add it to the wiki.

However, I don’t know how to do it.

Here are the ways I think I can do it (in no particular order).

Apply the Decorator pattern and layer a ruby interface over the
java library. All calls eventually delegate to the java classes. This
is coded in ruby.
Subclass each of the java classes and add a few ruby idioms to the
subclasses to “sugar” it.
Write the ruby interface in java using the JRubyMethod annotations
just like JRuby does internally for its own stuff. Essentially the
same as #1 but coded in java.

I imagine #3 would be the best performer since it would keep a
majority of the code in java.

Any other approaches? Does anyone have any recommendations on existing
projects I could look at to figure this out?

cr

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

cremes · June 2, 2009, 8:05pm

Chuck,

I had a java library that I recently integrated into a JRuby + Sinatra +
Rack app. It was a client library I’d written that had service interface
classes and implementations, etc.

At first I went down the road of messing with the Java interface from
Ruby, or thinking about having a Ruby class that abstracted the Java
layer, but I quickly decided just to use the Java service implementation
class from Ruby, because JRuby already creates a Ruby class that
abstracts the java classes, and in my past experience (a few years doing
telecom API integrations) you end up with a lot more work long-term by
putting too many layers in between the UI and the end-of-the-line
service you are hitting that will inevitably change.

So for example in the ruby name_of_service.rb I have (I generified the
names):

include Java

import ‘pkg.name.of.client.service.impl.MyServiceV1Impl’
import ‘pkg.name.of.client.service.impl.MyServiceV2Impl’
import ‘pkg.name.of.client.service.model.MyServiceV1Method1Request’
import ‘pkg.name.of.client.service.model.MyServiceV1Method1Response’
import ‘pkg.name.of.client.service.model.MyServiceV1Method2Request’
import ‘pkg.name.of.client.service.model.MyServiceV1Method2Response’
import ‘pkg.name.of.client.service.model.MyServiceV2Method1Request’
import ‘pkg.name.of.client.service.model.MyServiceV2Method1Response’

import ‘java.util.ArrayList’

(then down in the code, here is a generified and modified example)

    my_service_v2 = MyServiceV2Impl.new
    ...
    some_array_list = ArrayList.new()
    some_array_list.add(Java::JavaLang::Integer.new(2))
    some_array_list.add(Java::JavaLang::Integer.new(4))
    some_array_list.add(Java::JavaLang::Integer.new(6))
    ...
    my_req = MyServiceV2Method1Request.new
    my_req.setSomeArrayList(some_array_list)
    ...
    my_resp = my_service_v2.someMethod1(my_req)

Note that you can either use PascalCase or camelCase that would match
the Java methods/classnames or you can use
ruby_naming_that_has_underscores. I also didn’t need the () after new
above (not needed in Ruby), but left it there to show you could use it
if you want. Notice that you can either use import to use the ClassName
notation or the Java:PackageGoesHereInPascalCase::ClassName notation.
You’ll want to use the latter for String, Integer, and Float since those
conflict with Ruby’s and inevitably cause confusion (as they did for
me).

Also be aware that the number conversions don’t work exactly as you’d
imagine (at least when I tested with 1.2.0- haven’t tested conversions
in 1.3.0RC2). Here are my notes on that (sorry my blog was recently
retemplated and looks bad, but still better than before):

http://stufftohelpyouout.blogspot.com/2009/05/jruby-numbers-and-their-corresponding.html
http://stufftohelpyouout.blogspot.com/2009/05/fun-with-jruby-irb-console.html
http://stufftohelpyouout.blogspot.com/2009/05/point-at-which-jruby-loses-precision-on.html

So basically my thought would be a:

Don’t do anything extra. JRuby can just use the existing Java

Hope that helps!

Gary

Chuck R. wrote:

is coded in ruby.

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

cremes · June 2, 2009, 8:18pm

While JRuby can use the existing Java, I find that this is generally
best practice if the Java interface is incredibly simple. The “Java Way”
has always involved too many objects representing nearly infinite
extensibility (but for all purposes impractical) functionality.

I have a terrible example here, but it shows what I am talking about:
I created an interface that is slightly interchangeable with Ruby’s
RestClient gem that wraps HttpClient:

See jruby_restclient.rb

This is not a great example because it can be wildly inefficient,
instantiating more objects than is actually needed, not using
String#from_java_bytes, etc - but it should have some working examples
of how to call Java methods and include Java classes.

I’d avoid the annotations method in general because that requires
compilation, and Ruby can be so much nicer to work with.

Ikai

cremes · June 2, 2009, 8:31pm

Gary and Ikai,

thanks for your responses. I’ve already done a bunch of work with
JRuby and Java so I’m pretty comfortable with #import, the calling
conventions, converting from String to RubyString, etc.

Your answers helped me look at this in a new light. I agree with Ikai
that the “java way” produces lots and lots of extraneous classes and
interfaces. I wanted to “hide” some of that complexity and put a nice
ruby face on the library. However, I also liked Gary’s comment about
how JRuby already wraps each java class and additional wrappers get in
the way.

I think I’ll take a bit of a hybrid approach. The primary service
classes don’t need any wrapping. They can be used “as is” from JRuby
using whatever convention the user prefers (CamelCase versus
snake_case, etc). I bet I can reopen those proxy classes and add some
syntatic sugar for some of the weirdness in the original java class.
Further, I can wrap up a bunch of the extra utility classes into a
single ruby class with a nice idiomatic interface.

I’ll start hacking on this now. Who knows what I’ll really do until
I get elbow deep into it though…

In case anyone is interested, I’m going to attempt to wrap the Esper
[1] java library.

cr

[1] http://esper.codehaus.org

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

cremes · June 2, 2009, 9:12pm

Ruby rocks at creating DSLs that can glue things together, but the
tradeoff in creating a clean DSL is to limit the amount of options
available. Contrast this with Java, where the philosophy is to creating
software that gives users as many options as possible. There’s a
stereotype that a lot of Ruby folks are Java expats, and the general
feeling amongst these vets is that Java is overengineered and
overdesigned because it evolved by way of committee. If you consider
what needs to be done to read input from a file using just the Java
standard library or the amount of boilerplate setup needed to do basic
functionality and compare this to the type of programming interfaces
that Rubyists strive for, there is a clear and recognizable difference
in approach. This is the interface for Ruby’s RestClient:

response = RestClient.get “http://www.cnn.com”

That’s it, and it covers most use cases. Does it do retries? No. Session
management? No.

When I say that Java is “infinitely extensible”, what I mean is that
usually, it seems as if Java interfaces are meant to give client
developers as much choice and as much power as possible even if only a
small minority of users will use this functionality. As a result,
programming interfaces become clunky and require quite a lot of setup
that is nothing at all related to the business logic of an application.

And that’s why JRuby is such an amazing project. There are very few
things that Java can’t do, and very many things that Ruby cannot do.
We’ll use Ruby and common software packages to build out our business
logic, and when we approach the part of our project where we need a
little bit more functionality that is readily available in the Java
world, we inline Java.

Ikai

cremes · June 2, 2009, 10:26pm

On Tue, 2009-06-02 at 12:12 -0500, Chuck R. wrote:

In my daily work I need to wrap an existing java library and put a
pretty little ruby interface on it. I thought I’d take this
opportunity to write up how to do it and add it to the wiki.

However, I don’t know how to do it.

Here are the ways I think I can do it (in no particular order).

Any other approaches? Does anyone have any recommendations on existing
projects I could look at to figure this out?

I think you should start by finding your motivation for wrapping the
library. Step 1 should be to package the unchanged library as a gem,
ready for use with JRuby without any glue (ref jdbc-derby). Step 2
would be to identify typical USAGE of the library, and collect
convenience methods/converters/adapters in a separate gem that depends
on the first gem.

This approach has been recommended for FFI wrappers also: Supply a
barebone low-level version of the C library using FFI in one gem, and a
rubyfied wrapper layer in a second gem.

The main reason I also recommend this approach, is that we have been
using lots of Java libraries in our systems for the last 18 months, and
it is much easier to find commonly used code than trying to figure out
what we need in advance.

cremes · June 2, 2009, 10:35pm

Ikai,

Good points.

Note that I think that a few reasons that Java has so many options (so
many libraries, so many classes, so many methods) are that:

Java has been in use by a greater percentage of the development
community for a long time. (I know Ruby has been around for almost the
same length of time, but it has not been as widely-adopted for as long,
nor is it today.)
With Java and Java libraries, it is a more difficult to go in and make
changes than it is in Ruby. (For example our team makes changes to
Rails/Rake, etc. to debug, etc. without thinking about it, but although
I’ve written Ant tasks and Maven mojos, I wouldn’t just go in and change
Ant source code to fix something (except for a small contribution I made
years ago ).

If Ruby were to increase its adoption rate for a long enough period of
time, then theoretically it (like many languages) would have many more
options both in what you could do in the code and via a plethora of
libraries and many frameworks that would at that point would have been
developed (although there are already very many!). So my thought is
there- “Stick with any language long enough, and it will no longer be
simple.”

It is definitely a good idea of trying to take advantage of what is out
there already that is developed in Java within JRuby, but hopefully for
Ruby’s and JRuby’s sake, many more of those things would be developed in
Ruby.

Early on (but not long ago) I had the misconception of JRuby as already
having taken advantage of more of the mainstream Java
libraries/frameworks like Hibernate and Spring and making greater use of
them, but interfacing with them via ways that a Ruby developer would be
more familiar with. But that wasn’t correct. If I understand it
correctly, JRuby seems to be more about Ruby running on the JVM and as a
JRuby (Ruby) developer, you might call Java at times, but that’s not its
primary purpose, nor is the implementation very much about Java reuse.
Kind of like Java’s primary purpose is not for the average developer to
need to be aware of how to interface with C libraries. While Grails,
also a good project, is taking the other route of being heavy on the
Java focus (to take advantage of Spring, Hibernate, etc.). I think that
has its merits as well, but if I understand correctly, JRuby’s goal is
more Ruby and less J. I’m probably wrong though- I haven’t been around
these parts very long.

Thanks!
Gary

Ikai L. wrote:

functionality and compare this to the type of programming interfaces
developers as much choice and as much power as possible even if only a

Ikai

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

cremes · June 2, 2009, 8:53pm

Ikai,

Not that I’m an expert by any means on JRuby, but I disagree with the
following statement:

The “Java Way” has always involved too many objects representing
nearly infinite extensibility (but for all purposes impractical)
functionality.

Ruby is more extensible than Java. I am a long time Java developer and
would never say that Java has infinite extensibility. You can redefine
so much in Ruby/JRuby that it can easily make one’s head spin. And if
its objects’ functionality were impractical, I don’t think that there
would be so many people using them.

Java jobs on Indeed currently are 3% of all postings indeed indexes:
http://www.indeed.com/jobtrends?q=java&l=
Ruby jobs on Indeed currently are 0.2% of all postings indeed indexes:
http://www.indeed.com/jobtrends?q=ruby&l=

That’s not to say that Java is better. Ruby is paving the way for the
future. But “Java’s objects’ functionality” are certainly not
impractical. In fact, if you wrote a JRuby library to interface with
something written in Java, aren’t you contradicting yourself?

Appreciate your comment though, and agree that if the purpose is to have
a JRuby library that is partially or wholly interchangeable with
another, and you wanted the inner implementation to use a java library
such as the Apache Jakarta project HttpClient (near and dear to my heart
as I’ve used it even before it was in the Jakarta sandbox I used to
write “screen-scrapers” (ok, really “web-data” extractors, or whatever
you want to call them) after all) then that would seem to me to be a
fine example of something needing a layer of JRuby classes between the
other JRuby and the Java.

Like I said though, I’m no expert on JRuby, so I’d be glad to hear what
everyone thinks.

Gary

Ikai L. wrote:

GitHub - ikai/jruby-rest-client: RestClient for JRuby

Ikai

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

cremes · June 2, 2009, 10:56pm

On Tue, 2009-06-02 at 16:34 -0400, Gary W. wrote:

Early on (but not long ago) I had the misconception of JRuby as already
having taken advantage of more of the mainstream Java
libraries/frameworks like Hibernate and Spring and making greater use of
them, but interfacing with them via ways that a Ruby developer would be
more familiar with. But that wasn’t correct. If I understand it
correctly, JRuby seems to be more about Ruby running on the JVM and as a
JRuby (Ruby) developer, you might call Java at times, but that’s not its
primary purpose, nor is the implementation very much about Java reuse.

The JRuby project is very much about Java library reuse. However the
core JRuby project “only” makes all Java libraries in the world
available to your Ruby program. You can use them directly. You don’t
need any wrapper.

You are right that the JRuby project is focused on you developing in
Ruby. The recent developments in ruby2java will however make writing
Java libraries in Ruby and make them available to Java applications a
lot easier. (Haven’t tried it yet, though)

Kind of like Java’s primary purpose is not for the average developer to
need to be aware of how to interface with C libraries. While Grails,
also a good project, is taking the other route of being heavy on the
Java focus (to take advantage of Spring, Hibernate, etc.). I think that
has its merits as well, but if I understand correctly, JRuby’s goal is
more Ruby and less J. I’m probably wrong though- I haven’t been around
these parts very long.

I really do not see JRuby that way. Two major differences vs Groovy:

Ruby already has a large collection of libraries, so the focus on
library reuse is split between Ruby and Java. You can choose to use
ActiveRecord, Sequel, etc. or Hibernate, JDO, etc.

A LOT of Java libraries like Spring core solve problems that are not
present in the Ruby world, like Aspect Oriented Programming and
dependency injection. So using Ruby simply removes the problem instead
of solving it

cremes · June 2, 2009, 11:09pm

Er, using Hadoop, that is. It’s more a science project than anything,
because at that scale it might not make sense to run Ruby for intense
data processing.

Ikai

cremes · June 2, 2009, 11:07pm

When the JRuby core developers asked everyone what we wanted most, for
some reason everyone screamed, “Performance!” Performance is nice, but I
would have liked more tight integration with Java. For instance, we
still haven’t figured out how a trivially easy way of doing MapReduce
with JRuby.

(Would LOVE to hear how to do this if anyone has done it)

Ikai

cremes · June 3, 2009, 12:42am

Ikai L. wrote:

When the JRuby core developers asked everyone what we wanted most, for
some reason everyone screamed, “Performance!” Performance is nice, but I
would have liked more tight integration with Java. For instance, we
still haven’t figured out how a trivially easy way of doing MapReduce
with JRuby.

I’m afraid you’ve lost me here; how is tighter integration with java
going to
provide a trivial way of doing MapReduce? Is it trivial in Java? (I’ll
admit
to a sore lack of knowledge here.)

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

cremes · June 3, 2009, 4:46am

Chuck R. wrote:

In my daily work I need to wrap an existing java library and put a
pretty little ruby interface on it. I thought I’d take this opportunity
to write up how to do it and add it to the wiki.

However, I don’t know how to do it.

Here are the ways I think I can do it (in no particular order).

My rough recommendation is below…

Apply the Decorator pattern and layer a ruby interface over the java
library. All calls eventually delegate to the java classes. This is
coded in ruby.

Do this first, and sprinkle a little “Delegator” into it. Use normal JI,
wrap the existing stuff, and get it working well.

Subclass each of the java classes and add a few ruby idioms to the
subclasses to “sugar” it.

Don’t do this from Ruby if you can help it right now. Subclassing works
but for any intensive use it starts to show its seams. We’re hoping to
improve it in the future but for the moment I’d only recommend it for
very specific targeted use.

Write the ruby interface in java using the JRubyMethod annotations
just like JRuby does internally for its own stuff. Essentially the same
as #1 but coded in java.

Do this second, when you know where bottlenecks arise in the
wrappers/delegators implemented using #1 above. If you don’t run into
any bottlenecks, you won’t need to do anything. And hopefully in JRuby
1.4 we’ll be able to do enough work to solve most of the rough bits of
Java integration so what you have will work fine.

Charlie

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

cremes · June 3, 2009, 5:17am

Ikai L. wrote:

When the JRuby core developers asked everyone what we wanted most, for
some reason everyone screamed, “Performance!” Performance is nice, but I
would have liked more tight integration with Java. For instance, we
still haven’t figured out how a trivially easy way of doing MapReduce
with JRuby.

(Would LOVE to hear how to do this if anyone has done it)

Here’s my #1 goal for JRuby 1.4 (or whatever the JRuby that results from
a summer of work is called):

Reduce the gap between Java and Ruby.

This can be broken down into several subgoals:

Better compiler that produces the most optimized, idiomatic code
possible given modern compiler techniques and the complexities of the
Ruby language.

This means a move to a more modern compiler design, where we produce an
intermediate representation and then do several optimization passes over
it. Initially this would be static optimizations, like turning Fixnum
loops into int loops, eliminating closer dispatch for literal iterations
like [1,2,3].each and 5.times, and so on. Later, it could mean profiled
optimizations, like determining which call we’re always making and being
smarter about doing that call. This would help bring execution
performance parity with some of the newer Ruby impls.

Gutting the internals of Java integration to reduce object churn,
weakref usage, call protocols, and coercion semantics.

This would be a moderately big job, but the payoff would be huge. JI is
expensive these days mostly because of all the object wrapping and
coercing we have to do, as well as a number of unoptimized paths. Fixing
that would improve JI perf uniformly and reduce the number of hops from
a Ruby call site to the actual Java code. We’d also probably have an
easier path to using invokedynamic for Java invocation and an easier
path to step 3 below.

Begin moving away from IRubyObject toward Object call paths.

This is the long-talked-about “lightweights” support, which would allow
us to pass Java objects through Ruby without ever coercing them or
wrapping them. This is a very large job, but I think it can be bitten
off in pieces. The approach would be to work backward through JI and
allow Java objects to propagate further and further up that particular
part of the call path. As they propagate higher and higher before being
coerced or wrapped, more and more of the Object call logic will fall
into place.

Begin formalizing what Ruby types are and what Java types are and how
the two can interact.

This is work on ruby2java, for example, where we’re making it easier to
represent a Ruby class as some kind of “real” Java class. Initially we
could do things like always generate a stub Java class for every Ruby
class, so that in memory profiles, etc, we’d actually see those class
names. As we move forward, it could improve toward lazily generating a
class when first constructed, instantiating signatures for all Java
methods that exist at that point. This gives us the potential for
core-class call performance for all those methods as well as
reflectability.

I want to do all this plus take some vacation time and hit two
conferences I may need help.

Charlie

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

cremes · June 3, 2009, 6:09pm

On Jun 2, 2009, at 3:25 PM, Uwe K. wrote:

This approach has been recommended for FFI wrappers also: Supply a
barebone low-level version of the C library using FFI in one gem,
and a
rubyfied wrapper layer in a second gem.

I like this suggestion a lot. Thanks for sharing.

cr

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

cremes · June 10, 2009, 8:02pm

I personally like #1 the best, although I don’t think of it as a
decorator for a Java class, I think of it as a Ruby class that does
useful stuff that as an implementation detail calls out to a Java
class. Since you can call Java directly with many Ruby’isms (naming,
getter/setter conversion, etc.) I don’t see the need for any sort of
thin shim layer. I prefer going straight to the “optimal API” stage
and then figuring out how that connects to the Java class that’s
actually doing the work. Maybe in that process you decide a few
wrappers help out the implementation but I think top-down is easier
with Ruby, where bottom up is typically easier with Java. If in the
end it’s too slow, then you recode your nice pretty Ruby interface in
Java as you point out in #3. This is a pretty reasonable thing to do
although I have not often needed to do it.

David K.

On Jun 2, 2009, at 10:12 AM, Chuck R. wrote:

This is coded in ruby.

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email