Performance question about ruby/rails

I have a program I’d like to write and I’d like to get off of Java and
start using something more productive. I have played around with
Rails, and I think I could the same app in a fraction of the time.

There’s one part of the app though that I am not if Rails/Ruby would
be a good fit… so I’d like to get some expert advice on this one
portion.

There’s one part of the application that needs to check a thesaurus
about 100 or 200 times per request. There might be a lot of these
requests. There is also a grammar checker that is going to be used
(although I need to find one that works with Ruby).

I can imagine that these are some very expensive requests. In the java
world, I can simply put the thesaurus and grammar checker (with rules)
entirely in memory as Java beans in a spring container. When the app
starts up, we initialize these two services and they just stay in
memory, waiting for requests.

When user requests come in that need to use the thesaurus or grammar
checker, they happen extremely quickly because they are just sitting
in memory the entire time.

Since Ruby is a scripted language, does that mean this would not be
possible? Would there be a way to make these extremely performant
using the Rails paradigm? Or is this a bad fit for Ruby and Rails?

Thank you.

On Apr 28, 3:39pm, egervari [email protected] wrote:

Since Ruby is a scripted language, does that mean this would not be
possible?

Does sound to me like this would be a problem - it’s entirely possible
to set something like this up during app startup.
The only issue you might have is that (unless you go down the jruby
world) with ruby you’d have multiple rails application processes (MRI
never runs more than one thread at once), and so each one would have a
copy of this thesaurus, you might end up using a lot of memory

Fred

Fred

On Apr 28, 10:39am, egervari [email protected] wrote:

requests. There is also a grammar checker that is going to be used
in memory the entire time.
You may want to consider going with a hybrid approach - keep the heavy
lifting in Java, with a lightweight API on top that the Ruby code can
call. There are plenty of examples of this sort of thing - Solr is
precisely this technique applied (with some enhancements) to the
Lucene libraries.

–Matt J.

Ruby already has may c apis that run very fast with a ruby wrapper
around them c code. Also, you can run services that respond to all
this, it would work like memcached does, extremely fast service with a
ruby client. I dont recommend having the thesaurus as a pure ruby
gems. If you can work around that there will be no difference between
using rails or java

Hi Ken,

On Thu, Apr 28, 2011 at 9:39 AM, egervari [email protected]
wrote:

I have a program I’d like to write and I’d like to get off of Java and
start using something more productive. I have played around with
Rails, and I think I could the same app in a fraction of the time.

Welcome! I think you wouldn’t have to look far to find any number of
ex-Java developers on this list who could provide you with personal
experience supporting your thinking. :wink:

world, I can simply put the thesaurus and grammar checker (with rules)
entirely in memory as Java beans in a spring container. When the app
starts up, we initialize these two services and they just stay in
memory, waiting for requests.

I, too, will recommend a hybrid approach, though I suggest looking at
an IMDB approach to the shared thesaurus. Here’s a link to one
approach that looks ready-made for integration with a RoR app.
http://www.themomorohoax.com/2009/03/15/activerecord-sqlite-in-memory-db-without-rails

HTH,
Bill

There’s some really interesting approaches in all of these emails.

I like this one the best. I haven’t learned how to setup 2 databases
in rails… but is this complicated to do? I guess I could make a
database just like this person is doing, put all the thesaurus lookups
in it, and then keep it running just like any other database.

I want to avoid doing a ruby<->java bridge if I can. The last thing I
want is some bloated java process needing 512 megabytes of ram just to
do its think without throwing some kind of out of permgem space or out
of memory errors :confused: At least if it’s all in rails, I can just dedicate
all of extra ram to rails and my database.

Are the in-memory databases fairly quickly, such as sqlite? I’ve not
used it mysql. If it can take do at least 1000 individual select
statements very quickly (like less than a second), then that could
work.

On Thu, Apr 28, 2011 at 1:55 PM, egervari [email protected]
wrote:

of memory errors :confused: At least if it’s all in rails, I can just dedicate
all of extra ram to rails and my database.

Are the in-memory databases fairly quickly, such as sqlite? I’ve not
used it mysql. If it can take do at least 1000 individual select
statements very quickly (like less than a second), then that could
work.

If a thesaurus is just a grouping of words then you can use memcached,
is a
shared key value store, fast and is integrated to rails, you can do
Rails.cache.fetch(word){ Words.thesaurus(word) } and the first time it
will
hit the db write the result to memcached and there after it will pull it
from memcached. Memcached can be clustered and the key are shared among
the
servers.

http://media.railscasts.com/videos/115_caching_in_rails_2_1.mov

http://railslab.newrelic.com/2009/02/19/episode-8-memcached

On Apr 28, 2011, at 5:04 PM, egervari wrote:

Ooo, nice. Yeah, that could solve it.

I also was doing some other research and JRuby completely slipped my
mind. Is that also a reasonable alternative too? Do you guys have any
experience using rails 3 under Jruby? Is it still easy and straight-
forward, or is the mismatch (like getting rails to run on tomcat and
using jdbc, among other things) going to be frustrating?

I’m using JRuby for a number of Rails projects. Generally it works very
well with the very occasional glitch (just had one running capistrano
which depends on highline which requires the ffi-ncurses gem under
JRuby.

Generally it’s pretty good though. And you can just use rvm like you
would with anything else, so it’s like using MRI - just with a slower
startup time in dev mode.

Best Wishes,
Peter

Actually, this might the best alternative because the way I would do
it in Java is just load the entire thesaurus into memory (all the
nouns, verbs, adj’s, etc.) I didn’t care too much about memory, but
I’m reasonable certain that 90% of the values were ever hit :wink: I guess
it might be a bit slower at first, but caching the values that end up
getting used a lot is actually quite a bit much better use of
memory :wink:

I’m think I’m going to go with Rails. I want to do something different
than Java entirely. It’ll be fun for a change. I haven’t had any fun
in Java in years. I want to have fun AND make money. With java
development, it’s always been this torturous thing to bear, and money
was the only reason I could find for continuing to do it :frowning:

On Thu, Apr 28, 2011 at 11:18 PM, egervari [email protected]
wrote:

in Java in years. I want to have fun AND make money. With java
development, it’s always been this torturous thing to bear, and money
was the only reason I could find for continuing to do it :frowning:

Why don’t you just run a thesaurus app as a standalone ruby application
which you connect to via tcp sockets?
This way you only use the memory for the thesaurus once and use rails to
send requests to your standalone server?
And, if even necessary, you could look at making it multithreaded…

Andrew

Ooo, nice. Yeah, that could solve it.

I also was doing some other research and JRuby completely slipped my
mind. Is that also a reasonable alternative too? Do you guys have any
experience using rails 3 under Jruby? Is it still easy and straight-
forward, or is the mismatch (like getting rails to run on tomcat and
using jdbc, among other things) going to be frustrating?

So if I use memcached, the memory would be used up multiple times?

nop, what is more awesome is that you can pass an ip (or an array) to
the
config and it will use that (those) server’s memcached so in the future
scaling your app is just a matter of putting the memcached server on
another box(check the doc to find out how easy is to cluster) and
restarting
your web server, it would take you 3 minutes

Why don’t you just run a thesaurus app as a standalone ruby application
which you connect to via tcp sockets?
This way you only use the memory for the thesaurus once and use rails to
send requests to your standalone server?
And, if even necessary, you could look at making it multithreaded…

Andrew

Well, before i didn’t even have a thesaurus app. I just had 4 text
files and wrote some code to load them up in memory, creating the hash
maps with all the data. I put all of this in a facade/service that hid
it all away, and clients just called 1 simple method in order to get
the list of words that related.

So if I use memcached, the memory would be used up multiple times?

On Fri, Apr 29, 2011 at 1:25 AM, radhames brito [email protected]
wrote:

Memcached is a simple server process that stores a hash - if thats all
you
need, that would definitely be the easiest solution.

On Thu, Apr 28, 2011 at 7:10 PM, Andrew M. [email protected] wrote:

your web server, it would take you 3 minutes

Memcached is a simple server process that stores a hash - if thats all you
need, that would definitely be the easiest solution.

Agreed.

My initial recommendation was based on the assumption that you’re
going to need to store the data somewhere and a db is, to me, the
logical place for it. So, to me, memcached is a second step, not a
first. YMMV.

To answer your earlier question about multiple database connections,
it can be as simple as adding an entry for it to database.yml and a
before_filter in application.rb to establish a connection to same for
every Rails instance.

HTH,
Bill

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs