Distributed caching techniques?

Hi,

I need to cache various things at server startup (parsed CSS selectors,
parsed Liquid template objects, Liquid drop object registry etc…) and
I’d
like a single cache location, instead of a cache on each web node. A
dedicated memcached node looks like a good idea but I’m worried that the
cost of marshaling my objects on each cache get will out-weight the
speed
advantages gained by using the cache in the first place.

Does anyone have any tips on what to do in this situation?

Cheers
Ian.

On Sep 18, 2006, at 8:11 AM, Ian L. wrote:

A dedicated memcached node looks like a good idea but I’m worried
that the cost of marshaling my objects on each cache get will out-
weight the speed advantages gained by using the cache in the first
place.

Does anyone have any tips on what to do in this situation?

Yes.

Test your assumptions about speed. Memcached is easy enough to setup
and experiment with.

Also: remember that memcached is not persistent and if you want to
cache things at startup which hang around indefinitely, you’re using
it wrong. What’s so bad about keeping a local copy of the cache if
it’s never going to change and consistent across all nodes?


Chris W.

Ah, I didn’t know memcached wasn’t persistent. Having a cache per node
isn’t
a big problem, it’s just that i’ll need a mechanism in place to ensure
the
caches don’t get out of sync. Once I start clustering Mongrel, I’m going
to
need something that I can use to share objects among each ruby process.

OK, Drb is what I was looking for. A few wrapper script here and there
and I
can use my objects as if they were local.

For anyone doing this, remember to include DRbUndumped into any classes
your
server will return instances of, you’ll be unable to transparently
access
the remote methods otherwise.