I’m looking for comments, suggestions, best practices for handling
page_caching with shared storage.
This is the scenario:
nginx box in front of multiple mongrel_cluster boxes (Xen instances
actually)
At the moment each mongrel_cluster has its own public folder, though
sub-folders of this are shared via symlinks, so the file data
(principally images uploaded via attachment_fu, and cached fragments)
can be shared between the mongrel_cluster instances and nginx.
I’m now adding some limited page_caching, principally for the RSS feeds
to start with. What do you think is the best way to handle this:
– share the whole public folder across everything
– put the cached pages into a subdirectory in public and have some sort
of test in the nginx config file to see if the pages exist in there
before passing on the request to the mongrel clusters
At the moment each mongrel_cluster has its own public folder, though
I’d recommend you look at clustered filesystems, rather than something
–
– Tom M., CTO
– Engine Y., Ruby on Rails Hosting
– Support, Scalability, Reliability
– (866) 518-YARD (9273)
Thanks Tom
By clustered filesystems (apologies for my ignorance) do you mean some
sort of SAN. At the moment, I’m using one of the Xen instances as NFS,
which works OK), and though I was thinking I’ll probably need to upgrade
the storage as the site grows, it hasn’t so far been a problem.
If you shared RAILS_ROOT in its entirety does this increase problems
with the various servers trying to access the same files (obviously
logging could be a problem – though that could presumably be solved by
going to a Syslog solution – at the moment I’ve got each
mongrel_cluster handling its own logging)?
Any pointers for this self-taught sysadmin gratefully received.
By clustered filesystems (apologies for my ignorance) do you mean some
sort of SAN. At the moment, I’m using one of the Xen instances as NFS,
which works OK), and though I was thinking I’ll probably need to upgrade
the storage as the site grows, it hasn’t so far been a problem.
You don’t necessarily need a SAN to implement a clustered filesystem,
but it helps.
As an alternative, you could potentially use gnbd or vlbade on a Xen
instances (or instances) to supply the SANishness required.
If you shared RAILS_ROOT in its entirety does this increase problems
with the various servers trying to access the same files (obviously
logging could be a problem – though that could presumably be solved by
going to a Syslog solution – at the moment I’ve got each
mongrel_cluster handling its own logging)?
We’ve found logging to a different disk to be ideal. In our case we
generally log to the /var partition.
While it’s convenient to have your logs in one place, we’ve found the
performance penalty to be too high.
–
– Tom M., CTO
– Engine Y., Ruby on Rails Hosting
– Support, Scalability, Reliability
– (866) 518-YARD (9273)
going to a Syslog solution – at the moment I’ve got each
– Tom M., CTO
– Engine Y., Ruby on Rails Hosting
– Support, Scalability, Reliability
– (866) 518-YARD (9273)
Thanks again for that Tom. I’ll look into the systems you suggest. From
what you’re saying, though, it seems there’s no straightforward solution
for the middle ground in which I’m in, though perhaps in the short-term
(to get page-caching working properly, i.e. having the pages served by
nginx rather than the mongrels), sharing the public folder on the NFS
might be the best solution (with the mongrel_clusters still logging
locally).
This will allow me to gradually scale things (and thus concentrate on
the app for a bit, not to mention marketing, maintenance, keeping up to
speed with developments, etc).
