Page/action caching across multiple machines

Rails Deployment
1

Does anyone have a preferred way of doing page/action caching across
multiple machines? I know with fragment caching you can use
memcached. But page caching usually goes in the document root, so it
is not usually in a shared directory.

I saw one suggestion which was to change the page cache directory to /
public/cache, and then modify the Apache re-write rules to check for
static content in /public first, and then /public/cache second. But
that seems klunky.

Another crazy idea I heard was to possibly NFS/GFS mount the entire
application directory, and then your second application machine has
everything shared with the first one. (Only PID file locations would
need to be kept separate.)

Does anyone have any experience or preferred methods on this? There
is suprisingly little documentation on scaling Rails to multiple
boxes.

thanks,
Jeff

co-founder, cto

2

On Mar 25, 2008, at 6:45 AM, Jeff wrote:

Jeff

co-founder, cto
www.patientslikeme.com

We use GFS to share a /data partition between multiple servers. The
rails apps and assets + page caches are available in a consistent way
across multiple boxes this way. It works very well. GFS has context
dependant symlinks so you can have private directories for pid and log
files on each machine even though they all write to the same path.

I must warn you that getting GFS and red hat cluster suite setup and
working properly is a battle though and took us a long time to get
completely stable, in the end it’s totally worth it though.

Another option if you don’t have a SAN is to use memcached for your
page caches and use the memcached module in nginx so nginx will hit
memcached first for cached pages and if the cache misses it will fall
back to your rails app which serves the page and stores it in
memcached. You use the uri for the memcached key. This is much easier
to setup than GFS, but GFS performs much better than memcached for
page cached pages in our test. On one of our larger clients clusters
we are able to get 25kreq/sec for a GFS page cache and only 8kreq/sec
for a memcached page cache. Still both perform very well so it’s just
a matter of how much time you want to spend getting things set up.

Of course all engineyard slices come with GFS and SAN storage :wink:

Cheers-

3

On Mar 26, 2008, at 7:23 AM, Jeff wrote:

Thanks for the reply, Ezra. We actually have an NFS mount set up, but
I’ve read your warnings against NFS, so I’d like to try memcached.
The only kicker is we are running Apache, which I know how to change
the Rewrite rules to look for static content on the mount first, but
not sure how to do that with Apache + memcached. This may be the
excuse I’ve been looking for to dump Apache, anyway.

thanks again for the help,
jeff

Hey Jeff-

It’s a great excuse to dump apache :wink: But before you do that take a
look at this module:

http://tangent.org/608/mod_memcached.html

I have not used this and I don’t know much about it but it can
probably do the same thing nginx mod_memcached can do I would imagine.
In case you decide to try nginx let me save you a little time with the
magic memcached config options:

Say you have all page caches in /article

location /article {

 proxy_intercept_errors  on;
 default_type text/html;
 charset UTF-8;

 if ($request_method = GET) {
     # Say nginx to try to fetch some key from memcache:
    # serve with correct mime type and charset
     set $memcached_key "app-production:article:$uri";
     memcached_pass memcached_cluster;
     # If no info would be found in memcache or memecache would be

dead, go to
# /fallback location
error_page 404 502 504 = @fallback;
break;
}

 if (!-f $request_filename) {
   proxy_pass http://mongrels;
   break;
 }

}

location @fallback {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_redirect false;
proxy_max_temp_file_size 0;
# Pass request to mongrel
proxy_pass http://mongrels;
}

Cheers-

4

Thanks for the reply, Ezra. We actually have an NFS mount set up, but
I’ve read your warnings against NFS, so I’d like to try memcached.
The only kicker is we are running Apache, which I know how to change
the Rewrite rules to look for static content on the mount first, but
not sure how to do that with Apache + memcached. This may be the
excuse I’ve been looking for to dump Apache, anyway.

thanks again for the help,
jeff