Nginx Cache and WP canonical URL create infinite loop (bug ?)

[dubstep]

Hello everybody.

Just want to report this to all people who have an NGINX that cache
request in front a Wordpress installation

Scenario

• Nginx is configured as reverse proxy in front of an Apache that run
  Wordpress.
• proxy_cache_key is not defined so is $scheme$proxy_host$request_uri;

Let’s say a client require the page http://www.mysite.com/my-page// (yes
double slash at the end) that is not cached

1. Nginx pass the request to Apache/Wordpress
2. Apache/Wordpress process the request and reply with a 301 and sends
   the client to http://www.mysite.com/my-page/ because is the canonical
   URL
3. Nginx save in the cache the response with these data
   KEY: http://www.mysite.com/my-page/
   HTTP/1.0 301 Moved Permanently
   […]
   Location: http://www.mysite.com/my-page/
   […]
   As you can see the KEY is equal to Location
4. The user/browser send a request for http://www.mysite.com/my-page/
   (just one slash)
5. Nginx find the key in the cache and so redirect the browser to
   http://www.mysite.com/my-page/ causing an infinite loop

Conclusions

• All the users that require the page http://www.mysite.com/my-page/
  while is cached are send in the loop
• This happen of course also for requests with a number of slashes at
  the end > 2
• I can’t call this bug but is just an “unexpected” behaviour, for sure
  a malicious user can overload your site or make pages un-available with
  these kind of requests.
• There are probably other type of URL that create this situation. Have
  found any until now, can someone point one out ?
• Is there a way to avoid this ?

Thanks

–
Simone

[hpatoio]

Hello!

On Wed, Nov 23, 2011 at 05:26:54PM +0100, Simone F. wrote:

5. Nginx find the key in the cache and so redirect the browser to
   http://www.mysite.com/my-page/ causing an infinite loop

Conclusions

• All the users that require the page http://www.mysite.com/my-page/ while is
  cached are send in the loop
• This happen of course also for requests with a number of slashes at the end >
  2
• I can’t call this bug but is just an “unexpected” behaviour, for sure a
  malicious user can overload your site or make pages un-available with these kind
  of requests.
• There are probably other type of URL that create this situation. Have found
  any until now, can someone point one out ?
• Is there a way to avoid this ?

Could you please show your config? By default nginx will use the
same uri in cache as it will use in request to upstream, and this
shouldn’t happen.

There are some nuances though, at least when using proxy_pass with
variables (see http://trac.nginx.org/nginx/ticket/34, patch is
awaiting Igor’s review).

Maxim D.

[hpatoio]

Could you please show your config?

Nothing special, here we go (only important parts)

server {

server_name www.mysite.com
listen 1.2.3.4:80

proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_store_access user:rw group:rw all:r;

location / {

# If logged in, don't cache.
if ($http_cookie ~* 

“comment_author_|wordpress_(?!test_cookie)|wp-postpass_” ) {
set $do_not_cache 1;
}

# Response in not cached
proxy_no_cache $do_not_cache;

# Response is not taken from the cache
proxy_cache_bypass $do_not_cache;

proxy_cache MY_CACHE;
proxy_pass http://ALL_backend;

}

location ~ /purge(/.*) {
proxy_cache_purge MY_CACHE “$scheme://$host$1$is_args$args”;
}

}

By default nginx will use the same uri in cache as it will use in request to
upstream, and this shouldn’t happen

AFAIK when I require http://www.mysite.com/my-page// the key for the
cache is set to http://www.mysite.com/my-page/

Ciao

–
Simone

[hpatoio]

On 11/23/2011 06:41 PM, Maxim D. wrote:

With this config, the cache key for the request to
“http://www.mysite.com/my-page//” will be
“http://ALL_backend/my-page//”.

Well, I was wrong. In another file, on main level, I’ve

proxy_cache_key “$scheme://$host$request_uri”;

–
Simone

[hpatoio]

Hello!

On Thu, Nov 24, 2011 at 09:50:10AM +0100, Simone F. wrote:

On 11/23/2011 06:41 PM, Maxim D. wrote:

With this config, the cache key for the request to
“http://www.mysite.com/my-page//” will be
“http://ALL_backend/my-page//”.

Well, I was wrong. In another file, on main level, I’ve

proxy_cache_key “$scheme://$host$request_uri”;

With this proxy_cache_key the key (for the same request) will be
“http://www.mysite.com/my-page//”.

Maxim D.

[hpatoio]

Hello!

On Wed, Nov 23, 2011 at 06:11:41PM +0100, Simone F. wrote:

proxy_set_header Host $host;

location ~ /purge(/.*) {
proxy_cache_purge MY_CACHE “$scheme://$host$1$is_args$args”;
}

}

By default nginx will use the same uri in cache as it will use in request to
upstream, and this shouldn’t happen

AFAIK when I require http://www.mysite.com/my-page// the key for the cache is
set to http://www.mysite.com/my-page/

With this config, the cache key for the request to
“http://www.mysite.com/my-page//” will be
“http://ALL_backend/my-page//”.

Maxim D.

[hpatoio]

On 11/24/2011 11:17 AM, Maxim D. wrote:

With this proxy_cache_key the key (for the same request) will be
“http://www.mysite.com/my-page//”.

Well, actually when I require http://www.mysite.com/my-page// the key
I’ve in the cache is http://www.mysite.com/my-page/
I’m on NGINX 0.8.54 and I’ve installed “Cache purge” module
http://labs.frickle.com/nginx_ngx_cache_purge/

Also when I require a URL like http://www.mysite.com/my-page/###### in
the cache the KEY is http://www.mysite.com/my-page/

If I’ve an entry in the cache with the key
http://www.mysite.com/my-page/ and I require
http://www.mysite.com/my-page/###### the request hit the cache

Regards

–
Simone