sent_http_HEADER Volatile under Nginx 1.2.4

Im in the process of making some amends to an environment, where my
upstream servers are sending a custom header (X-No-Cache), which I need
to detect and alter caching rules within the configuration.

The custom header is visible within the output, and I can re-output it
as another header through configuration (i.e. add_header
X-Sent-No-Cache $sent_http_x_no_cache; ).

However, as soon as I perform any type of testing of this custom header,
it disappears.

For example, if I was to perform a map on the custom header, try to set
an Nginx variable to the value of the header, or test within an IF
statement, any future call to this header is no longer possible.
Additionally any setting or testing of the header fails.

Unfortunately I have little control of the upstream, so cannot use an
alternative method (such as proper Cache-Control headers).

Has anyone experienced similar behaviour, or have any pearls of wisdom?

Thanks in advance,

Paul

Hello!

On Thu, Dec 12, 2013 at 07:19:56PM +0000, Paul T. wrote:

header, it disappears.

For example, if I was to perform a map on the custom header, try
to set an Nginx variable to the value of the header, or test
within an IF statement, any future call to this header is no
longer possible. Additionally any setting or testing of the
header fails.

Both “set” and “if” directives you mentioned are executed before
a request is sent to upstream, and at this point there is no
X-No-Cache header in the response. Due to this, using the
$sent_http_x_no_cache variable in “set” or “if” will result in an
empty value, and this value will be cached for later use.

It’s not clear what you are trying to do so I can’t advise any
further, but certainly using the $sent_http_x_no_cache variable in
“if” or “set” directives isn’t going to work, and this is what
causes behaviour you see.

Just a map{} should work fine though - as long as you don’t try to
call the map before the X-No-Cache header is actually available.
E.g., something like this should work fine:

map $sent_http_x_no_cache $foo {
    ""                    empty;
    default               foo;
}

add_header X-Foo $foo;

It might be also a goo idea to use $upstream_http_x_no_cache
variable instead, see here:

http://nginx.org/en/docs/http/ngx_http_upstream_module.html#variables


Maxim D.
http://nginx.org/

Hi Maxim,
Thanks for your response. Youre right! Using the map did work (I thought
Id tried that, but must have been tired!).
So, now I have one other challenge, the value of $foo that you define
below is needed to identify whether to cache the response of not. The
only issue is that I have a number of other directives that I also need
to add into the mix - therefore I use the set_by_lua code to
nest/combine OR within an if statementcode below (Ive kept the variable
name as foo, so its clear which Im referring to):
map $upstream_http_x_no_cache $foo {
“” 0;
default 1;
}
set_by_lua $bypass_cache ’
local no_cache_dirs = tonumber(ngx.var.no_cache_dirs) or 0
local logged_in = tonumber(ngx.var.logged_in) or 0
local no_cache_header = tonumber(ngx.var.foo) or 0

if((no_cache_dirs == 1) or (no_cache_header == 1) or (logged_in == 1))
then
return 1;
end

return 0;
';
Now when I make the Lua local variable declaration in order to use it,
the value of $upstream_http_x_no_cache is reset to 0, even when it was
set as 1 originally. If I comment out the line declaring the local
variable within the Lua call, it returns to being a value of 1 again.
Am I getting the sequencing of events wrong again? Is there any way that
I can get the value of $upstream_http_x_no_cache into this Lua block, or
would I need to do it another way?
Thanks very much for your help so far Maxim.
Paul


Hello!

On Thu, Dec 12, 2013 at 07:19:56PM +0000, Paul T. wrote:

header, it disappears.

For example, if I was to perform a map on the custom header, try
to set an Nginx variable to the value of the header, or test
within an IF statement, any future call to this header is no
longer possible. Additionally any setting or testing of the
header fails.

Both “set” and “if” directives you mentioned are executed before
a request is sent to upstream, and at this point there is no
X-No-Cache header in the response. Due to this, using the
$sent_http_x_no_cache variable in “set” or “if” will result in an
empty value, and this value will be cached for later use.

It’s not clear what you are trying to do so I can’t advise any
further, but certainly using the $sent_http_x_no_cache variable in
“if” or “set” directives isn’t going to work, and this is what
causes behaviour you see.

Just a map{} should work fine though - as long as you don’t try to
call the map before the X-No-Cache header is actually available.
E.g., something like this should work fine:

map $sent_http_x_no_cache $foo {
    ""                    empty;
    default               foo;
}

add_header X-Foo $foo;

It might be also a goo idea to use $upstream_http_x_no_cache
variable instead, see here:

http://nginx.org/en/docs/http/ngx_http_upstream_module.html#variables

Hello!

On Thu, Dec 12, 2013 at 11:36:21PM +0000, Paul T. wrote:

map $upstream_http_x_no_cache $foo {
return 1;
way that I can get the value of $upstream_http_x_no_cache into
this Lua block, or would I need to do it another way?

Are you going to use the result in proxy_no_cache? If yes, you
can just use multiple variables there, something like this should
work:

proxy_no_cache $upstream_http_x_no_cache
               $no_cache_dirs
               $logged_in;

See here for details:

http://nginx.org/r/proxy_no_cache

within the configuration.
within an IF statement, any future call to this header is no
further, but certainly using the $sent_http_x_no_cache variable in
}
http://nginx.org/


nginx mailing list
[email protected]
nginx Info Page


Maxim D.
http://nginx.org/

Yup, again, youre right! Ive moved the config around, so that Im testing
for any true value in the proxy_no_cache & proxy_bypass_cache directives
(removing the existing set_by_lua block).

However, its still not behaving as Id expect.

In the following scenario (note comments):

map $upstream_http_x_no_cache $no_cache_header {
“” 0;
default 1;
}

proxy_cache_bypass $no_cache_dirs $logged_in; # $no_cache_header;
proxy_no_cache $no_cache_dirs $logged_in; # $no_cache_header;

X-Cache-Status value is MISS, which is correct. Output of
$no_cache_header is 1 (as set in the map).

However, when adding back in the compare on $no_cache_header:

proxy_cache_bypass $no_cache_dirs $logged_in $no_cache_header;
proxy_no_cache $no_cache_dirs $logged_in $no_cache_header;

X-Cache-Status value is still MISS, which is not correct, as it should
be BYPASS. Output of $no_cache_header is 0.

Unless Im missing something, it still looks like touching the variable
kills it?

Thanks again,

Paul

Hello!

On Mon, Dec 16, 2013 at 09:22:25AM +0000, Paul T. wrote:

X-Cache-Status value is still MISS, which is not correct, as it should be
BYPASS. Output of $no_cache_header is 0.

Unless I’m missing something, it still looks like touching the variable kills
it?

The proxy_cache_bypass directive is expected to be checked before
a request is sent to a backend - it is to control whether a
request will be served from a cache or passed to a backend.

That is, what you see is actually expected behaviour - there are
no reasons X-Cache-Status to be BYPASS, and the cached
$no_cache_header value to be different from 0.


Maxim D.
http://nginx.org/

Hi Maxim,

Ok, thanks for the clarification.

So to confirm, we are looking for the value of the sent header from the
upstream, to identify whether the content should be served from the
cache, or the upstream. Does this therefore mean that the code that we
have below, will check for the X-No-Cache header, and if present, will
always render the content from the upstream (no cache), and that if not
present, will enable the result to be cacheable? If so, and it is only
the reporting of the X-Cache-Status value that is rendering a false
value, then this will give us what we want?

If not, then what suggestions would you have for caching only on the
basis of this sent http header being present?

Thanks againnearly there :wink:

Paul

On 16 December 2013 12:38, Paul T. [email protected] wrote:

X-Cache-Status value that is rendering a false value, then this will give us
what we want?

If not, then what suggestions would you have for caching only on the basis
of this sent http header being present?

I may have missed in this thread why the earlier suggestion of

proxy_no_cache $upstream_http_x_no_cache

doesn’t work for you. It does seem to meet your requirement of
“caching only on the basis of this sent [== upstream?] http header
being present” …

J