Fastcgi_cache_bypass and 502 Bad Gateway

Using Nginx 0.8.54.

I’m trying to use fastcgi_cache_bypass with an HTTP header for Nginx to
request a new response from the backend and cache it.

if ($http_cookie ~*
"comment_author_|wordpress_(?!test_cookie)|wp-postpass_" ) {
  set $donot_cache 1;
}

fastcgi_cache   cachetitle;
fastcgi_cache_key   $scheme$host$request_uri;
fastcgi_no_cache   $donot_cache; # do no cache at all
fastcgi_cache_bypass   $http_cache_bypass; # bypass cache and cache the
new response
fastcgi_cache_use_stale  error timeout invalid_header http_500;
fastcgi_cache_min_uses  1;

It does seem to work – I see a fresh page returned from the backend,
but on the second request without the header I get a 502 Bad Gateway
response.

I also noticed that on Nginx wiki page there is no reference to
fastcgi_cache_bypass, although I saw it there just a few days ago.

Is this feature being removed or postponed for later? Is it available in
the latest dev version?

Kaspars

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,173795,173795#msg-173795

Anyone using fastcgi_cache_bypass with success?

I have looked through the changelogs and have not seen it being removed
as a feature. The documentation in russian still contains it, while the
english wiki does not.

Kaspars

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,173795,174131#msg-174131

kaspars Wrote:

if ($http_cookie ~*
“comment_author_|wordpress_(?!test_cookie)|wp-post
pass_” ) {
set $donot_cache 1;
}

AFAIK, the '$donot_cache ’ you set here will not contain anything
outside the if block. and therefore your no_cache test will always be
done on an empty variable.

Variable scopes are not documented and only trial and error and fumbling
in the dark reveals stuff.

Would be great if they just had global scopes so that once it is set, it
is there to be used everywhere. However, I think I read something
negative once on this from Maxim D. (who seems to be one of the
devs) so I think no matter how much we might want this sort of thing, it
will not happen.

A bit of a pain but that is how it is and most of us don’t know enough
to fill in the gaps in the documentation.

I had a similar problem and had to use the 3rd party lua module to run
the test in a way that the variable set was available after:
http://forum.nginx.org/read.php?2,161969,168394#msg-168394

In your case, why not just use ‘fastcgi_no_cache
$cookie_COOKIE-1$cookie_COOKIE-2$cookie_COOKIE-3’ and skip the whole
‘if’ test?

The advice is to avoid using ‘if’ unless absolutely required. See:
http://wiki.nginx.org/IfIsEvil

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,173795,174161#msg-174161

AFAIK, the '$donot_cache ’ you set here
will not contain anything outside the if block.

You are right, I forgot to put “set $donot_cache = 0;” right before the
if block, but it doesn’t seem to be the problem.

Everything is fine regarding the availability of those variables, and I
verified it by adding them as an HTTP header to all responses.

In your case, why not just use ‘fastcgi_no_cache
$cookie_COOKIE-1$cookie_COOKIE-2$cookie_COOKIE-3’
and skip the whole ‘if’ test?

Because those cookie names change – they have random numbers appended
to them, so I need to do a regex check for their prefixes.

I’ll be sure to post an update if I find something.

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,173795,174184#msg-174184

[quote]$test will always be equal to 0 as any variable set within the
‘if’ block is not exposed outside the block AFAIK.[/quote]

No. I am already using this check successfully for the fastcgi_no_cache
rule.

The real problem is with the fastcgi_cache_bypass. My understanding is
that if “fastcgi_cache_bypass 1”, then Nginx will request a fresh
response from the backend and cache that response for further use where
“fastcgi_cache_bypass 0”.

Is that how fastcgi_cache_bypass should work?

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,173795,174194#msg-174194

The point I was trying to make is that in this case for instance, …

set $donot_cache 0;
if ($http_cookie ~*
"comment_author_|wordpress_(?!test_cookie)|wp-postpass_" ) {
  set $donot_cache 1;
}
set $test $donot_cache;

… $test will always be equal to 0 as any variable set within the ‘if’
block is not exposed outside the block AFAIK.

So when you run …

set $donot_cache 0;
if ($http_cookie ~*
"comment_author_|wordpress_(?!test_cookie)|wp-postpass_" ) {
  set $donot_cache 1;
}
...
fastcgi_no_cache   $donot_cache;

… ‘$donot_cache’ will always resolve to null.

Again, this is AFAIK based on stuff I have read here and there.

See here for instance:
http://forum.nginx.org/read.php?2,25197,25197#msg-25197

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,173795,174192#msg-174192

kaspars Wrote:

My understanding is that
if “fastcgi_cache_bypass 1”, then Nginx will
request a fresh response from the backend and
cache that response for further use where
“fastcgi_cache_bypass 0”.

Is that how fastcgi_cache_bypass should work?

No idea as it is not documented in the English docs and I can’t read
Russian.

Assuming it works like the proxy version though (and the English
documentation for that is really unclear as it seems to suggest to just
use the ‘cache_bypass’ version and forget about the ‘no_cache’ version),
then my understanding is:

  1. ?_no_cache = get from backend but cache response
  2. ?_cache_bypass = do not cache response

So if you want to bypass the cache and also not cache that response,
then you should be using the two together.

Note though that as said, the English documentation does not state this
but again I think I have read Igor posting this somewhere.

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,173795,174207#msg-174207

This is known issue and I believe Igor worked on patching this.
Avoid using different _cache_bypass/_no_cache until this is
fixed.

Thanks, Maxim! That’s good to know.

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,173795,174250#msg-174250

On 11 Fev 2011 16h57 WET, [email protected] wrote:

[/code]
set $donot_cache 1;
}

This is because as Maxim, and agentzh have explained before: an if is
an implicit location. So what you’re requesting above is for a
variable with a value changed at a lower level (implicit location) be
visible at an upper level (containing location).

My understanding of Nginx scoping is that it works in one direction,
i.e., you can only be sure that this works global -> http -> server ->
location -> if in location.

There’s no reverse path for inheriting variables set at a lower level
on a higher level.

Can someone more knowledgeable than I concur and straighten me if I’m
incorrect?

Thanks,
— appa

Hello!

On Fri, Feb 11, 2011 at 09:49:54PM +0000, António P. P. Almeida wrote:

}
if ($http_cookie ~*
My understanding of Nginx scoping is that it works in one direction,
i.e., you can only be sure that this works global -> http -> server ->
location -> if in location.

There’s no reverse path for inheriting variables set at a lower level
on a higher level.

Can someone more knowledgeable than I concur and straighten me if I’m
incorrect?

You are incorrect. Variables are property of request. They are
either set during request processing or not, and if set - they are
visible to all operations with request.

Maxim D.

Hello!

On Fri, Feb 11, 2011 at 12:07:52PM -0500, kaspars wrote:

Is that how fastcgi_cache_bypass should work?

Yes, it’s how it should work. But this isn’t really how it works
now - due to bug behaviour is undefined if fastcgi_cache_bypass
isn’t the same as fastcgi_no_cache (and the same applies to
proxy_cache_bypass/proxy_no_cache and so on).

This is known issue and I believe Igor worked on patching this.
Avoid using different _cache_bypass/_no_cache until this is
fixed.

Maxim D.

On 12 Fev 2011 00h06 WET, [email protected] wrote:

Hello!

You are incorrect. Variables are property of request. They are
either set during request processing or not, and if set - they are
visible to all operations with request.

So there’s no concept of scope in the usual programming language
sense. It just depends on the request. If the request visits all
locations where variables are set, then the values are set
independently of the context level at which the assignment
instructions exist.

Is this correct?

Thanks,
— appa

António P. P. Almeida Wrote:

Is this correct?
I think this is correct but this to me essentially create a variable
scope since as Maxim has explained elsewhere, the if block creates a
nested location block and since only one location block is processed in
the request flow, things set in there may not be visible, to the rest of
the flow.

The problem is know what exactly is visible after and what is not.

So far, according to the ifisevil page, it seems ‘return’, and rewrite
commands are safe but anything else may result in errors.

It may be the case that ‘set’ is safe too and a list of what is and
isn’t would be really great.

For me, I just use the two listed stuff on the ifisevil page (return and
rewrites) and agetnzh’s lua module to do other if tests.

To the OP, perhaps try using the same variable on both fastcgi params
which from Maxim’s explanation, is not susceptible to the bug.

fastcgi_no_cache $http_cache_bypass; # do no cache at all
fastcgi_cache_bypass $http_cache_bypass; # bypass cache <— Not sure
how you get the user’s browser to send this header

At a suitable point in your application, you could also do something
like
EQUIVALENT-OF if ($http_cookie ~*
“comment_author_|wordpress_(?!test_cookie)|wp-postpass_” ) {
header(Cache-Control: private, no-store, no-cache);
}
so that nginx will not cache the response.

Basically, try to share the load between the application and nginx.

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,173795,174371#msg-174371

[quote]fastcgi_no_cache $http_cache_bypass; # do no cache at all
fastcgi_cache_bypass $http_cache_bypass; # bypass cache <— Not sure
how you get the user’s browser to send this header[/quote]

On each page I would have an AJAX call to a very small PHP script that
checks for the cache version number and makes a GET (or HEAD) request
with Cache-Bypass header if the page needs to be regenerated.

The idea is similar to the one described here:
http://konstruktors.com/blog/wordpress/2521-ajax-cache-purge-cookie-plugin/

But instead of using a version number for cache key (set by a cookie), I
would use it for cache invalidation.

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,173795,174729#msg-174729

On Sat, Feb 12, 2011 at 8:21 AM, Antnio P. P. Almeida [email protected]
wrote:

So there’s no concept of scope in the usual programming language
sense. It just depends on the request. If the request visits all
locations where variables are set, then the values are set
independently of the context level at which the assignment
instructions exist.

Is this correct?

Correct as long as the “request” in your context is an atomic
request, that is, excluding any subrequests that it issues directly or
indirectly.

Even though by default, nginx subrequests also share all the
variables of the parent requests (as well as all of its siblings), and
ngx_auth_request, ngx_eval, ngx_addition, and ngx_ssi all follow this
default behavior, some notable exceptions, however, exist in the wild.

Almost all of our nginx modules, for example, disable the sharing
because of potential bad effects similar to use of global variables in
a general-purpose programming language. Our ngx_echo, ngx_srcache, and
ngx_lua modules fall into this category, just to name a few :slight_smile:

Cheers,
-agentzh

On 12 Fev 2011 09h15 WET, [email protected] wrote:

instructions exist.
So far, according to the ifisevil page, it seems ‘return’, and
bug.

fastcgi_no_cache $http_cache_bypass; # do no cache at all
fastcgi_cache_bypass $http_cache_bypass; # bypass cache <— Not
sure how you get the user’s browser to send this header

Hmm, IIRC I’ve read somewhere that (some?) fastcgi directives do
behave like a “standard” programming language thing in the sense
there’s a scope and the usual rules apply. But I may be wrong. It
would be great if Igor, Maxim or agentzh could elaborate further and
help put this issue to rest, at least make it clear what’s a hack, and
what’s not hack, i.e., behaves much more in the declarative language
sense than in the imperative programming sense.

At a suitable point in your application, you could also do something
like
EQUIVALENT-OF if ($http_cookie ~*
“comment_author_|wordpress_(?!test_cookie)|wp-postpass_” ) {
header(Cache-Control: private, no-store, no-cache);
}
so that nginx will not cache the response.

Or send the ‘X-Accel-Expire: 0’ header.

— appa

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs