[ANN] memcache-client extensions plugin

Rails
1

From:
http://blog.zvents.com/2006/11/1/rails-plugin-memcacheclient-extensions

=MemCacheClient Extensions

== About

The memcache-client_extensions plugins adds three new commands to the
memcache client API:

  1. get_multi : retrieve more than 1 key in parallel
  2. stats : retrieve server performance and utilization statistics
  3. flush_all : empty all information stored in memcached

== Installation

  1. This plugin requires that the memcache-client gem is installed.

    gem install memcache-client

  2. Install the plugin or the gem
    $ script/plugin install
    svn://rubyforge.org/var/svn/zventstools/projects/memcache-client_extensions

  • OR -

    gem install memcache-client_extensions

== get_multi

Retrieve multiple values from memcached in parallel, if possible. The
memcached protocol supports the ability to retrieve multiple keys in a
single request. Pass in an array of keys to this method and it will:
a. map the key to the appropriate memcached server
b. send a single request to each server that has one or more key
values

Returns a hash of values.

CACHE[“a”] = 1
=> 1
CACHE[“b”] = 2
=> 2
CACHE.get_multi([“a”,“b”])
=> {“a”=>1, “b”=>2}

Here’s a benchmark showing the speedup:

CACHE[“a”] = 1
CACHE[“b”] = 2
CACHE[“c”] = 3
CACHE[“d”] = 4
keys = [“a”,“b”,“c”,“d”,“e”]
Benchmark.bm do |x|
x.report { for i in 1…1000; keys.each{|k| CACHE.get(k);} end }
x.report { for i in 1…1000; CACHE.get_multi(keys); end }
end

returns:
user system total real
0.180000 0.130000 0.310000 ( 0.459418)
0.200000 0.030000 0.230000 ( 0.269632)

There’s a fair amount of non-DRY between get_multi and get (and
threadsafe_cache_get/multi_threadsafe_cache_get and
cache_get/multi_cache_get for that matter) but I think it’s worth it
since the extra overhead to handle multiple return values is unneeded
for a single-key get (which is by far the most common case).

== stats

The stats method returns statistics for each memcached server. An
explanation of the statistics can be found in the memcached docs:
http://cvs.danga.com/browse.cgi/wcmtools/memcached/doc/protocol.txt?rev=HEAD&content-type=text/plain

Example:

CACHE.stats
=> {“localhost:11211”=>{“pid”=>“20188”, “bytes”=>“4718”,
“connection_structures”=>“4”, “time”=>“1162278121”,
“pointer_size”=>“32”, “limit_maxbytes”=>“67108864”, “version”=>“1.2.0”,
“cmd_get”=>“14532”, “cmd_set”=>“32”, “bytes_written”=>“432583”,
“uptime”=>“1557”, “curr_items”=>“4”, “curr_connections”=>“3”,
“total_connections”=>“19”, “get_misses”=>“0”,
“rusage_user”=>“0.119981”, “rusage_system”=>“0.313952”,
“total_items”=>“32”, “get_hits”=>“14532”, “bytes_read”=>“190619”}}

== flush_all

The flush_all method empties all cache namespaces on all memcached
servers. This method is very useful for testing your code with
memcached since you normally want to reset the cache to a known
(empty) state at the beginning of each test.

== Bugs, Code and Contributing

There’s a RubyForge project set up at:

http://rubyforge.org/projects/zventstools/

Anonymous SVN access:

$ svn checkout svn://rubyforge.org/var/svn/zventstools

Author: Tyler K. (tyler dot kovacs at gmail dot com)

2

Cool :slight_smile:

Slightly OT here, but is there a way to access all values from a
namespace
in memcached? Like “get all my books” or something. Seems like a good
extension to your get_multi function.

But perhaps the memcached protocol doesn’t allow for this. That was my
finding the last time I read through it. Somebody should fix this
though:
clearing/adding-to/retrieving a namespace is a real good feature to add.

Vish

3

No, the memcached protocol doesn’t support a “get all” method. Here’s
a thread from the memcached mailing that covers the issue:

http://lists.danga.com/pipermail/memcached/2006-April/002112.html

You could memcache your dataset in an array (in addition to memcaching
each individual item for single-item retrieval), but that probably
wouldn’t be a good idea for anything other than very small datasets.