First, I feel compelled to point out that Glassfish gem is Glassfish
server stripped down to just the stuff that you need to serve Jruby
apps, so it’s at least closer to a real app server than mongrel is. It
will handle multiple concurrent users, pool non-threadsafe rails
applications for you, etc. That little rant out of the way…
The first thing to check in any case where Java is using more memory
than you expect it to is your heap settings. If you run Java with
-Xms512m, for example, then Java will set the minimum heap size to
512MB, and it will reserve half a gig of memory, whether it needs it or
not. The GC will only run when it’s filled the entire 512MB with
garbage, and it will look like Java is hogging memory, even if it could
easily be happy living in 64MB. If it’s taking hours for the GC to have
to run, that’s a good sign that you’ve given it too much heap.
I’d suggest launching a rails app and then pointing jconsole (included
with JDK 5.0+) at it, so that you can see how much of your allocated
heap your applications are actually using. I’m somewhat curious as to
how marc was able to get it to idle at 12mb with two runtimes, but my
generic test application idles at around 40MB per runtime, which is
about what I’m used to seeing. 250-350MB definitely seems high, to me.
You can play around with the -Xmx and -Xms (maximum and minimum heap
sizes, respectively) settings a bit if you want to, but I find that in
general setting the max to the maximum that you are willing to let it
take and leaving the VM’s memory allocator to figure out the rest works
best.
Setting -Xms higher (probably with a maximum of your expected idle size)
will improve startup (since resizing the heap forces a full garbage
collection), but otherwise won’t affect things. Setting -Xmx higher will
reduce time between garbage collections, but will make the app take up
more memory. To a point, that’s good: If you have 8GB RAM and it’s a
dedicated server, you want to be using as much of that 8GB of RAM as
possible (with a bit of free space, maybe 512MB, since you never want
to swap to disk). However, RAM beyond what the application needs to only
be doing a young generation collection every few minutes won’t improve
performance at all, so more RAM generally means more apps running on the
server… Until you become CPU-bound.
<Java resource usage and GC lecture/rant>
Especially in rails apps, where most objects are extremely short-lived,
all the garbage collections that you run are going to be essentially
instant. The Hotspot GC young generation (objects less than one
collection old) collections scale based on how many objects survive, and
with rails that increases with the number of currently active requests
when the GC goes off. I actually have GC logs from Glassfish running a
Rails app (with images, AJAX, etc.) at a few thousand requests per
second, and it only has about 12MB of new live data per collection,
which means GCs in the hundredths of seconds. Those collections were
happening every three seconds, but they don’t have to stop the
application and .02 seconds really isn’t very long. Eventually, the
surviving objects will pile up and a full collection will need to be run
to clean them out, and that will take about a second. The full GCs need
to stop everything for a bit, but not for the full second. The stress
test needed a full collection roughly every 45 minutes on a 1G heap, so
a second of GC every 45 minutes is not bad at all. This is also on an
app serving a few thousand requests per second, so you’ll likely have
much lower traffic.
I had a point, I promise (that’s why I labeled this as a rant). When you
are sizing your server instances, you’ll want to figure out where the
best compromise between number of runtimes and frequency of garbage
collections is. If you set the heap too small, you’ll be doing full
collections more often, which will eventually noticeably impact
performance: If you’re running a 1-second collection every minute, it’s
not that big a deal. If you’re running a 1-second collection every three
seconds, it’s a problem. If you actually run out of heap, you can be
doing a 1-second collection every second, which will grind your app to a
halt. But, if you set the heap too large, you won’t be able to fit as
many runtimes into the space that you have, and thus won’t be able to
use the machine’s resources optimally. If, for example, you had a choice
between 128MB heaps with GCs every minute and 256MB heaps with GCs every
two minutes, you’d probably want to go with the 128MB heaps as long as
you had idle CPU time, since that means (potentially) twice as many apps
on the machine at once. Once you become CPU-bound (CPUs at 100% under
your “heavy” load), then you might as well increase the heaps to fill
the available memory, since adding more apps would decrease overall
performance.
Always remember that the processors and memory on your computers are
there to be used, and if you can go to maximal load and still have idle
CPU and free memory, you can add more load and thus get more out of the
computer.
rant tl;dr: You want the largest heaps that you can get on as many apps
as it takes to use all of the available processor when fully loaded, as
long as your full GC times are reasonable in those conditions. If they
aren’t, reduce the number of apps and increase the heap size on the
remaining ones until full GC times are reasonable (or add more memory to
the machine, if possible).
marc wrote:
anything easy we can do to minimize the memory usage of these separate JRuby
a bug/problem-tracking Rail app - using the glassfish gem.
The idle size is around 12Mb, and even when three or four folk are
working on it, it rarely peaks over 40Mb. Performance is better too.
I have no idea whether this is useful info or not, especially in your
case. This whole area seems to be a complex black art, and I think it
would be time well-spent simplifying it.
To unsubscribe from this list, please visit:
http://xircles.codehaus.org/manage_email
