Concurrency problem with memcached fragments mongrel cluster

Hi,

I have the following question: What happens when you memcache fragments
and you are running with a mongrel cluster. In the case when we check
read_fragment() in our action/controller and by the time Rails rechecks
to see if the fragment exists in the view, the fragment could have
possibly expired in memcache, or could have gotten deleted if you are
running some expiration sweeper of some sort. This means that none of
the instance variables or SQL was run since read_fragment() returned
true saying there is a fragment. Does this mean view rendering will
crash because it would try to create the cache block? How does one fix
this window if this is a problem? How do multiple mongrels synchronize
around this shared fragment?

Thanks.

around this shared fragment?
This could very well happen, but if your memcache ram size is large
enough
and you’ve told it not to expire, probably won’t.

But…memcache is not permanent storage… they make that very clear.
So if you’re relying on it being there long term, well… don’t.

:slight_smile:

Yes this could happen as well, but if your cache size is large enough,
it’ll
happen probably about one in a million times. Can’t think of a way
around
this.

Vish

My understanding is that memcached uses LRU, so calling read_fragment()
will
push the fragment to the top of he used list. This should give it a
good
chance of surviving until you use it in the view. However, you should
check
out extended_fragment_cache [1]. Though I have never used it, I believe
when calling read_fragment(), it saves the cached fragment in an
in-process
cache. This would in theory cut down on the number of cache queries,
and
possibly guarantee that the fragment exists (in the in-process cache)
when
you hit the view. I found this extension through Tyler Kovaks’ slides
titled “Efficient Memcached Usage” [2] (check out slide 8/11).

[1]
http://www.gemjack.com/gems/extended_fragment_cache-0.0.1/files/home/tony/projects/gemjack/gems/tmp/lib/extended_fragment_cache_rb.html
[2] http://sparecycles.org/efficient_memcached_usage/index.html