Forum: NGINX memcache key double escaped

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
Xavier G. (Guest)
on 2009-02-26 08:43
(Received via mailing list)
Hello all,

I have trouble with nginx memcached module with urls containing
special char, for instance :

GET /list.xml?fulltext=testé&sort=rating&dir=DESC

will store the following key in memcache :

<25 set brh:262:68:/list.xml?fulltext=test%C3%A9&sort=rating&dir=DESC
0 21600 7006
 >25 STORED

If I query back the same URL I have the following message in nginx
debug log :

2009/02/26 07:16:38 [info] 14614#0: *4684 key: "brh:262:68:/list.xml?
fulltext=test%25C3%25A9&sort=rating&dir=DESC" was not found by
memcached while reading response header from upstream, client:
10.0.0.177, server: devel.internal.net, request: "GET /list.xml?
fulltext=test%C3%A9&sort=rating&dir=DESC HTTP/1.0", upstream:
"memcached://10.0.0.177:11211", host: "devel.internal.net"

The request matches the key, eq : fulltext=test%C3%A9
But the memcache module is double escaping will trying to get the
query : fulltext=test%25C3%25A9

I don't know if this is the correct behaviour, of course this can be
bypasse using the following code in python for exemple :

urllib.quote("/list.xml?fulltext=test
%C3%A9&sort=rating&dir=DESC",safe="&/?=")

$memcached_key is set like this in nginx.conf :

set $memcached_key brh:$prefix:$request_uri;

Regards,

xav
Maxim D. (Guest)
on 2009-02-26 13:22
(Received via mailing list)
Hello!

On Thu, Feb 26, 2009 at 07:33:41AM +0100, Xavier G. wrote:

> 21600 7006
> "memcached://10.0.0.177:11211", host: "devel.internal.net"
>
> $memcached_key is set like this in nginx.conf :
>
> set $memcached_key brh:$prefix:$request_uri;

It's expected as nginx escapes $memcached_key before passing it to
memcached.  It escapes " ", "%", %00-%1F.

Maxim D.
Xavier G. (Guest)
on 2009-02-26 14:54
(Received via mailing list)
Hello,

>
> It's expected as nginx escapes $memcached_key before passing it to
> memcached.  It escapes " ", "%", %00-%1F.
>

Thanks for your reply.  Yes I could see from source that memcache
module escape the key.

But seems to me weird behaviour to escape values that are already
escaped. It's not a problem if memcache user notice this. I'm pretty
sure this leads to lots of cache miss.

xav
Maxim D. (Guest)
on 2009-02-26 15:26
(Received via mailing list)
Hello!

On Thu, Feb 26, 2009 at 01:40:58PM +0100, Xavier G. wrote:

> But seems to me weird behaviour to escape values that are already
> escaped.

The memcached module have to be able to deal with mostly arbitrary
$memcached_key, and therefore it has to escape space and control
characters to avoid violation of memcached protocol.  To make this
escaping one-to-one it's also required to escape '%'.

> It's not a problem if memcache user notice this. I'm pretty
> sure this leads to lots of cache miss.

I agree that this deserves a bit more documentation.

Maxim D.
This topic is locked and can not be replied to.