Jruby on rails scheduling options

Rails
1

Hi, I’m using JRuby 1.5.6 on Rails to build myself an application that
will periodically go away and retrieve any RSS podcasts that I have
subscribed too.

I’ve chosen JRuby primarily because I’m familiar with Java, wish to
utilise the Rails framework and most importantly I’m able to perform the
“heavy lifting” tasks in Java when Ruby falls short of my requirements.
Up to now (and I’m still in the early stages of development) this hybrid
approach has been working extremely well.

I’m now at a point where I’m needed to implement scheduling of periodic
and long running tasks to a background process. My requirements are to
have a database backed scheduling system that is, ideally, well
documented, currently maintained and clean.

My problem now is that after many days of researching suitable off the
self gem packaged solutions, I appear to be left with very few options
because of my use of JRuby.

Some of the gems I’ve tried…

rufus scheduler

Having used this before I’m happy with it’s interface and documentation,
however there is a lack of database persistence, hence a deal breaker
for my requirements.

delayed_job

My ideal solution would be delayed_job. Good documentation, still being
maintained and database backed, however, breaks under JRuby due to
ObjectSpace being turn off (we can however fix this by re-enabling) but
more fatally the dependence on the daemons gem which throws a “fork is
unsafe and disabled by default on JRuby” error due to limitations within
the JRuby implementation.

There is a fork on github that doesn’t have a dependence on daemons,
however I’m not happy switching to a fork off the main development
branch and I’m still left with the ObjectSpace issue which I’m unsure as
to it’s performance impact.

quartz-jruby

While there have been various quartz based gems before, this very recent
offering (GitHub - artha42/quartz-jruby: A better quartz gem for jruby) is another attempt at
providing a slick ruby-like interface. There is however minimal
documentation and I’m unsure as to if this can be database backed, my
gut feeling is that it is not.

The problem

While I’ve only highlighted 3 options here, I’m aware that there are
others available. I’ve however not been able to find a solution to tick
all 3 requirement boxes (docs, maintained, database backed).

So the questions are…

Has anyone else been in this situation and come up with a solution?

Has anyone managed to get delayed_job working in whatever form?

Are there any better solutions out there that I’ve overlooked and will
satisfy my needs?

2

Phil O. wrote in post #967209:

Hi, I’m using JRuby 1.5.6 on Rails to build myself an application that
will periodically go away and retrieve any RSS podcasts that I have
subscribed too.

I’ve chosen JRuby primarily because I’m familiar with Java, wish to
utilise the Rails framework and most importantly I’m able to perform the
“heavy lifting” tasks in Java when Ruby falls short of my requirements.
Up to now (and I’m still in the early stages of development) this hybrid
approach has been working extremely well.
[…]
My problem now is that after many days of researching suitable off the
self gem packaged solutions, I appear to be left with very few options
because of my use of JRuby.
[…]

I’m going to suggest that you experiment with switching away from JRuby.
I think you have chosen it for what appear to be the wrong reasons: if
you are using Ruby correctly, you generally won’t need Java for “heavy
lifting”. As a Ruby developer familiar with Java (and using JRuby on a
non-Rails project), I would say that Java is far more likely to fall
short of your requirements than Ruby is.

Have you actually written any Java code in this project? If so, what
for?

Also, as you’ve discovered, JRuby support for gems with native
extensions is not that great. You will find MRI/YARV/REE far less
difficult to work with, I think.

Best,

Marnen Laibow-Koser
http://www.marnen.org
[email protected]

3

On Dec 8, 2010, at 12:55 PM, Marnen Laibow-Koser wrote:

I’m going to suggest that you experiment with switching away from JRuby.
I think you have chosen it for what appear to be the wrong reasons: if
you are using Ruby correctly, you generally won’t need Java for “heavy
lifting”.

I don’t know a lot about the use case, and there are certainly downsides
with Rails on JRuby. One use case for JRuby is when you’ve got a large,
long lived project and want to clearly define an API that a bunch of
devs are going to work on by putting richer type information around some
core classes - whether in Java or a lower ceremony option like Scala.
It’s also interesting if you want (for example) some Clojure code
(although I’m not too sure how seamless the interop would be). I know
about using proper testing to replace the “nanny compiler”, and
personally I’m almost always working in a dynamically typed language,
but I do find use cases where static typing and all of the formality and
tooling support it brings can add value. That said, I’m not sure a Rails
app munging RSS feeds is necessarily that use case . . .

I have heard some credible people recommend JRuby for running Rails apps
from a performance perspective. Any particular take on that (other than
don’t pre-optimize by worrying about performance?!). I’m still very new
to the Rails world although for the small projects I’m currently playing
with in Rails I’m just throwing them up on Heroku.

Best Wishes,
Peter

4

Further to note, I run on Windows so suffer from a lack of cron.

In defence of JRuby’s support for gems, I have never, up to now, come
across an instance where a gem hasn’t worked in JRuby. I’m aware I don’t
use JRuby as much as some guys out there, but I’d have expected to have
hit problems long before now if support was in a generally poor state
(how would this be defined? a percentage of incompatibility?).

The only problems I’ve faced are the ones listed here, although I admit
they appear to fairly serious ones for the scope of the project.

5

The “heavy lifting” tasks I refer too are heavy data processing or IO
heavy tasks that have performed poorly in early prototypes using a Ruby
only approach.

For example, once a RSS feed is downloaded and parsed, it’s elements
(usually mp3’s of 100mb+ each) are then to be downloaded and stored away
on a NAS array. Very quick prototypes suggested that standard open-uri
within Ruby was not up to this task as the file being downloaded was
buffered completely into memory before being written to disk on
completion. I accept there may be ways to overcome this (suggestions
welcome), that research has not been undertaken yet so I couldn’t
comment on what they are.

Beyond this requirement, there is a need to transcode audio (and
possibly video in the future), support popular audio formats such as
WMA, MP3, OGG with metadata editing of those files and enable a
streaming api for audio.

There are many mature java libs to perform these tasks already out there
that I wished to use to provide that “heavy lifting” support when Ruby
doesn’t give me the tools I need (and arguably Ruby isn’t design with
these tasks in mind).

However, I digress. JRuby is the weapon of choice (reasons aside), so I
bring us back to the original question. Is there a scheduling solution
for JRuby that suits my needs?

6

Please quote when replying.

Phil O. wrote in post #967248:

The “heavy lifting” tasks I refer too are heavy data processing or IO
heavy tasks that have performed poorly in early prototypes using a Ruby
only approach.

For example, once a RSS feed is downloaded and parsed, it’s elements
(usually mp3’s of 100mb+ each) are then to be downloaded and stored away
on a NAS array. Very quick prototypes suggested that standard open-uri
within Ruby was not up to this task as the file being downloaded was
buffered completely into memory before being written to disk on
completion. I accept there may be ways to overcome this (suggestions
welcome), that research has not been undertaken yet so I couldn’t
comment on what they are.

Beyond this requirement, there is a need to transcode audio (and
possibly video in the future), support popular audio formats such as
WMA, MP3, OGG with metadata editing of those files and enable a
streaming api for audio.

There are many mature java libs to perform these tasks already out there

There are also many mature Ruby libraries to do this sort of thing,
probably with C extensions.

that I wished to use to provide that “heavy lifting” support when Ruby
doesn’t give me the tools I need

Ruby does give you the tools you need. You will probably get along
better with Ruby if you use it as a language in its own right, not an
abstraction layer on top of Java.

(and arguably Ruby isn’t design with
these tasks in mind).

Arguably you are wrong. :slight_smile: It really sounds like you’re making a
largely unjustified assumption about what Ruby can and cannot do
efficiently.

However, I digress. JRuby is the weapon of choice (reasons aside), so I
bring us back to the original question.

No…that’s my point. I think you’ve chosen the wrong weapon here.

Is there a scheduling solution
for JRuby that suits my needs?

Best,

Marnen Laibow-Koser
http://www.marnen.org
[email protected]

7

Phil O. wrote in post #967256:

Further to note, I run on Windows so suffer from a lack of cron.

Rails doesn’t work well on Windows, so you’re making your life harder
here too.

In defence of JRuby’s support for gems, I have never, up to now, come
across an instance where a gem hasn’t worked in JRuby. I’m aware I don’t
use JRuby as much as some guys out there, but I’d have expected to have
hit problems long before now if support was in a generally poor state
(how would this be defined? a percentage of incompatibility?).

See

and http://www.railsplugins.org (which is primarily about Rails 3
compatibility, but also includes JRuby compatibility information).
Normally I’d refer you to http://www.isitjruby.com , but that appears to
be down.

Basically, gems with C extensions don’t work on JRuby. Much of what you
are doing probably requires C extensions. It’s doubtful that anyone’s
done the work to write equivalent Java extensions.

The only problems I’ve faced are the ones listed here, although I admit
they appear to fairly serious ones for the scope of the project.

Right. And you can avoid them entirely by giving up JRuby as I’ve
suggested.

Again: you are making your life harder by using JRuby for this project
– and by not trusting Ruby to work for you. Doctor, it hurts when I do
that. Don’t do that. :slight_smile:

Best,

Marnen Laibow-Koser
http://www.marnen.org
[email protected]