Fastcgi_cache_key with cookie

dubstep · December 27, 2011, 6:14am

I’m trying to cache per user Drupal using this config :

fastcgi_ignore_headers Cache-Control Expires; #tried ‘__utma’
‘Set-Cookie’
fastcgi_cache drupal;
fastcgi_cache_key $http_cookie$request_method$scheme$host$request_uri;
fastcgi_cache_methods GET HEAD;
fastcgi_cache_valid any 1s;
fastcgi_cache_use_stale error timeout invalid_header updating http_500;

fastcgi_max_temp_file_size 2M;
add_header X-Micro-Cache $upstream_cache_status;

I can get this working without google analytics, but when I enable
google analytics it adds several cookies __utma __utmb __utmc __utmz.
Nginx looks at $http_cookie and sees that its different so it passes it
to the backend. Preferably I would like to just use the cache key as
$cookie_SESS which is the drupal session cookie, however the full cookie
name is encoded to something like this

SSESSae42ac488e03c2e2xxxxxda6ce2f5ee7=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

I’ve omitted some letters and numbers. So my question is how do I just
extract the Drupal Session cookie? Or failing that just strip all
cookies? I have tried fastcgi_ignore header ‘__utma’ ‘Set-Cookie’ but
it doesn’t seem to work.

Posted at Nginx Forum:

zenw · December 27, 2011, 7:38am

On 27 Dez 2011 05h13 WET, [email protected] wrote:

I’m trying to cache per user Drupal using this config :

fastcgi_ignore_headers Cache-Control Expires; #tried ‘__utma’
‘Set-Cookie’
fastcgi_cache drupal;
fastcgi_cache_key $http_cookie$request_method$scheme$host$request_uri;
fastcgi_cache_methods GET HEAD;
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
This is not needed. This is the default setting. Cf.
Module ngx_http_fastcgi_module

fastcgi_cache_valid any 1s;
fastcgi_cache_use_stale error timeout invalid_header updating http_500;

fastcgi_max_temp_file_size 2M;
add_header X-Micro-Cache $upstream_cache_status;

I can get this working without google analytics, but when I enable
google analytics it adds several cookies __utma __utmb __utmc
__utmz. Nginx looks at $http_cookie and sees that its different so
it passes it to the backend. Preferably I would like to just use
the cache key as

cookie_SESS which is the drupal session cookie, however the full
cookie
name is encoded to something like this

SSESSae42ac488e03c2e2xxxxxda6ce2f5ee7=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Remove that key and replace by:

fastcgi_cache_key $scheme$host$request_uri;

At the http level do:

map $http_cookie $no_cache {
default 0;
~SESS 1;
}

Then add:

If we have a cookie we should bypass the cache. The same if we have a

fastcgi_cache_bypass $no_cache;
fastcgi_no_cache $no_cache;

To the above FCGI cache config.

Also the first stop for drupal and Nginx stuff is the g.d.o group

http://groups.drupal.org/nginx

— appa

zenw · December 27, 2011, 7:54am

Thx for replying, but I have already tried this config from drupal.org.

With this config it will not cache the page correct? What I’m trying to
do is set every page to be cached whether logged in or not, or more
specifically to maintain a separate cache for each user. But I need to
differentiate the cache key based on the session cookie.

I need to optimize for authenticated performance, our website has 90%
logged in users, our non authenticated speed for nginx is more than
sufficient now. On my crappy laptop I can get 4000+ req/s for non
authenticated pages, for authenticated pages I get 2 req/sec without any
caching at all. With fastcgi caching I can get 3000+ req/sec, but this
is without google analytics, as soon as I enable it my speed plunges
down to 2req/sec

Posted at Nginx Forum:

zenw · December 27, 2011, 7:57am

Oh you are Perusio I got your config from drupal.org hehe and
tried your drupal-with-nginx config from github

Posted at Nginx Forum:

zenw · December 27, 2011, 8:34am

On 27 Dez 2011 06h53 WET, [email protected] wrote:

Thx for replying, but I have already tried this config from
drupal.org.

With this config it will not cache the page correct? What I’m
trying to do is set every page to be cached whether logged in or
not, or more specifically to maintain a separate cache for each
user. But I need to differentiate the cache key based on the
session cookie.

Why not implement a small module that sets a DRUPAL_UID cookie and use
that as a cache key? Much simpler than using the session cookie.

Take a look at what the boost module does. I can put up a simple
module that does that. Are you on drupal 6/pressflow?

I need to optimize for authenticated performance, our website has
90% logged in users, our non authenticated speed for nginx is more
than sufficient now. On my crappy laptop I can get 4000+ req/s for
non authenticated pages, for authenticated pages I get 2 req/sec
without any caching at all. With fastcgi caching I can get 3000+
req/sec, but this is without google analytics, as soon as I enable
it my speed plunges down to 2req/sec

Ok. Then what you need is a better key. If you have the DRUPAL_UID
cookie then set:

fastcgi_cache_key $http_cookie_DRUPAL_UID$scheme$host$request_uri;

That’s a cool idea for a very simple module.

There’s a hack here for setting a DRUPAL_UID cookie but it involves
killing kittens: hacking core

http://groups.drupal.org/node/194333

— appa

PS: Here’s a very simple module that sets a header, not a cookie. It’s
untested YMMV, caveat emptor, and all that:
Simple module to add a header with the user ID · GitHub

The .info file is missing.

PPS: Use this key with the above module.

 fastcgi_cache_key $http_x_drupal_uid$scheme$host$request_uri;

zenw · December 27, 2011, 8:47am

I’ve also tried this using the userid module I set a custom cookie like
so

userid on;
userid_name “testid”;
userid_expires 100s;

and set the fastcgi_cache_key like so

fastcgi_cache_key $cookie_testid$request_method$scheme$host$request_uri

the problem with this setup is when I login or logout, since the key is
based on only $cookie_testid without $http_cookie, then the user will
still see the logged in page even after logging out. Alternatively how
would I user the cache purge module to purge the key when the user hits
the login or logout button? this is the sample config

http {
proxy_cache_path /tmp/cache keys_zone=tmpcache:10m;

server {
    location / {
        proxy_pass         http://127.0.0.1:8000;
        proxy_cache        tmpcache;
        proxy_cache_key    $uri$is_args$args;
    }

    location ~ /purge(/.*) {
        allow              127.0.0.1;
        deny               all;
        proxy_cache_purge  tmpcache $1$is_args$args;
    }
}

}

not sure how to rewrite the url.

Posted at Nginx Forum:

zenw · December 27, 2011, 9:01am

On 27 Dez 2011 07h46 WET, [email protected] wrote:

$cookie_testid$request_method$scheme$host$request_uri

the problem with this setup is when I login or logout, since the key
is based on only $cookie_testid without $http_cookie, then the user
will still see the logged in page even after logging out.
Alternatively how would I user the cache purge module to purge the
key when the user hits the login or logout button? this is the
sample config

I think you can do it with map. Untested:

map $http_cookie $logged_in {
default 0;
~SESS 1;
}

Define two locations each with a different cache (anon/authenticated):

The default cache is the anon:

location @anon-cache {
error_page 418 @auth-cache;
if ($logged_in) {
return 418;
}
userid_expires 0s;
… # fastcgi cache stuff for anon users
}

location @auth-cache {
# set the user id expires here to a positive value
# fastcgi cache for authenticated users
}

I never played with the userid module. So…

— appa

zenw · December 27, 2011, 9:24am

Sorry I dont quite understand whats going on in your config.

error_page 418 @auth-cache;
if ($logged_in) {
return 418;

what does this do? looked up http error 418 “I’m a teapot” ROFL. so
going line by line

Posted at Nginx Forum:

zenw · December 27, 2011, 8:48am

I suppose when the user hits login or logout button the url needs to be

http://example.com/purge

if my default logged in page is http://example.com/

Posted at Nginx Forum:

zenw · December 27, 2011, 10:06am

On 27 Dez 2011 08h24 WET, [email protected] wrote:

Sorry I dont quite understand whats going on in your config.

error_page 418 @auth-cache;
if ($logged_in) {
return 418;

what does this do? looked up http error 418 “I’m a teapot” ROFL.
so going line by line

Yes, hence it’s innocuous to use You get to keep all other for real
status codes.

The map sets the value of the $logged_in variable. If there’s a cookie
with SESS it sets it to 1. The default is 0.

If $logged_in is 1 then return a 418. The error_page directive sets
the 418 error page to be the @auth-cache location.

— appa

zenw · December 27, 2011, 6:02pm

zenw Wrote:

fastcgi_cache_valid any 1s;
to just use the cache key as $cookie_SESS which is
‘__utma’ ‘Set-Cookie’ but it doesn’t seem to work.
Maybe extract the session cookie with something like this:

map $http_cookie $session_cookie {
default “”;
~^.(?SESS.+;).$ $key;
}

and

fastcgi_cache_key
$session_cookie$request_method$scheme$host$request_uri;

I’m sure the regex could be written better, but that does work.

Posted at Nginx Forum:

zenw · December 28, 2011, 2:16am

Awesome this works great, thanks so much for your help dude!!

Posted at Nginx Forum:

zenw · December 28, 2011, 2:35am

Oh and your too brian mercer, you have been a great help with all your
useful articles on nginx and drupal

Posted at Nginx Forum:

zenw · December 28, 2011, 5:15am

António P. P. Almeida Wrote:

fastcgi_ignore_headers Cache-Control Expires;
fastcgi_max_temp_file_size 2M;
it
xxxxxxxxxxxxxxxxxx
like this:
map $http_cookie $session_id {
default ‘’;
~SESS(?<session_guid>[[:alnum:]]+)
$session_guid;
}

— appa

That regex will only get you the cookie name (md5 of domain) without the
cookie value. You will want the equal sign in there so you get both
name and value:

~^.SESS(?[\w=]+).$ $key;

or just the value:

~^.SESS.+=(?[\w]+).$ $key;

Posted at Nginx Forum:

zenw · December 27, 2011, 8:52pm

On 27 Dez 2011 17h01 WET, [email protected] wrote:

i;
at $http_cookie and sees that its different so it
Session cookie? Or failing that just strip all
cookies? I have tried fastcgi_ignore header
‘__utma’ ‘Set-Cookie’ but it doesn’t seem to work.

Maybe extract the session cookie with something like this:

map $http_cookie $session_cookie {
default “”;
~^.(?SESS.+;).$ $key;
}

Yes. We can remove the SESS string and keep just the unique part of the
id.

map $http_cookie $session_id {
default ‘’;
~SESS(?<session_guid>[[:alnum:]]+) $session_guid;
}

— appa

zenw · December 28, 2011, 5:29am

Many thanks Perusio

Posted at Nginx Forum:

zenw · December 28, 2011, 8:03am

On 28 Dez 2011 04h15 WET, [email protected] wrote:

the cookie value. You will want the equal sign in there so you get
both name and value:

~^.SESS(?[\w=]+).$ $key;

or just the value:

~^.SESS.+=(?[\w]+).$ $key;

MD5 of the domain? How? AFAIK it’s much more elaborate than that.

See:
http://svn.php.net/viewvc/php/php-src/branches/PHP_5_3_8/ext/session/session.c?revision=315335&view=markup

— appa

zenw · December 28, 2011, 2:56pm

António P. P. Almeida Wrote:

— appa
or just the value:

— appa

nginx mailing list
[email protected]
nginx Info Page

Drupal session cookie is in the form:
session_name = session_id
e.g.
SESSc4f381f0375dba8f9670647c311307a9=b051bb99415e65d3d921f7891f3ca91f

The session_name is either
session_name(‘SESS’ . md5($cookie_domain))
from your settings file or
session_name(‘SESS’ . md5(explode(‘://’, $base_url, 2)))
see
http://api.drupal.org/api/drupal/includes--bootstrap.inc/function/conf_init/6

Your regex was only getting the session_name and not the session_id

Posted at Nginx Forum:

zenw · December 28, 2011, 5:53pm

On 28 Dez 2011 13h55 WET, [email protected] wrote:

}
~^.SESS(?[\w=]+).$ $key;
_5_3_8/ext/session/session.c?revision=315335&view=
SESSc4f381f0375dba8f9670647c311307a9=b051bb99415e65d3d921f7891f3ca91f

The session_name is either session_name(‘SESS’
. md5($cookie_domain)) from your settings file or
session_name(‘SESS’ . md5(explode(‘://’, $base_url, 2))) see
conf_init | bootstrap.inc | Drupal 6 | Drupal API

Your regex was only getting the session_name and not the session_id

If that’s the case then why get the cookie name? Just get the value
which is the session ID.

map $http_cookie $session_id {
default ‘’;
~SESS[[:alnum:]]+=(?<session_guid>[[:alnum:]]+) $session_guid;
}

— appa