Setting cache parameters via if directives

Hello,

We have a particular application (gitweb) that performs a particular,
extraordinarily slow function when the home page is loaded. As the
number of repositories has increased, this has grown to take several
minutes per page view (yes, ugh).

To combat this, we tried setting up cache so that this only occurs once
an hour, but it’s still causing too much of a problem…both because it
doesn’t always last an hour (sometimes we see the delay happen again
sooner) and because when it does expire, it simply takes too long (and
if multiple clients connect at that same time making that request, it
gets even worse).

As a result, what I’d like to do is to have the cache expire after an
hour, but once every twenty or thirty minutes manually refresh the
cache. That way clients can use the existing cache while the new page is
being rebuilt, which will then save the new values into the cache and
reset the timer.

So my thought was that I could do something like the code below, but it
doesn’t work since I can’t put the directives I want into the if block,
and the proxy_pass in the if block (if I remove the cache values) honors
the cache values in the outside location block anyways:

location = / {
    if ($remote_addr = "1.2.3.4") {
        proxy_pass http://127.0.0.1:80;
        proxy_cache gitweb;
        proxy_cache_valid 200 20m;
        proxy_read_timeout 300;
        proxy_send_timeout 300;
    }
    proxy_pass http://127.0.0.1:80;
    proxy_cache gitweb;
    proxy_cache_valid 200 60m;
    proxy_cache_use_stale off;
    proxy_read_timeout 60;
    proxy_send_timeout 60;
}

Any help much appreciated.

Thanks,
Jeff

On Thu, Jan 27, 2011 at 11:08 PM, Jeff M. [email protected]
wrote:

   proxy_cache_valid 200 60m;
   proxy_cache_use_stale off;
   proxy_read_timeout 60;
   proxy_send_timeout 60;

}

Any help much appreciated.

I do something similar using the conditinal setting of variables
inside the if block to set Cache-Control values, perhaps you can use a
similar method to set the values of proxy_cache_valid. Since setting
variables is one of the few “safe” things to do inside an if block,
and most directives can take variables as agurments, it is generally
useful.

location / {
set $mycc = “private, max-age=0”;

if ($foo = “bar”) {
set $mycc = “public, max-age=3600”
}

proxy_pass http://backend;
add_header Cache-Control $mycc;
}


RPM

On 01/29/2011 03:19 AM, Jeff M. wrote:

if ($foo = “bar”) {
set $mycc = “public, max-age=3600”
}

proxy_pass http://backend;
add_header Cache-Control $mycc;
}

Great idea, thank you.

Unfortunately, this doesn’t work:

    set $cachevalidval "60m";
    if ($remote_addr = "1.2.3.4") {
        set $cachevalidval "1m";
    }
    proxy_pass http://127.0.0.1:80;
    proxy_cache gitweb;
    proxy_cache_valid any $cachevalidval;

I get complaints that $cachevalidval is an invalid time value. I’ve
tried doing things like having $cachevalidval only have a number and
then using "$cachevalidval"m but nothing works so far…

Thanks,
Jeff

On 01/28/2011 10:55 AM, Ryan M. wrote:

if ($foo = “bar”) {
set $mycc = “public, max-age=3600”
}

proxy_pass http://backend;
add_header Cache-Control $mycc;
}

Great idea, thank you.

–Jeff

Hello Jeff…

On Fri, Jan 28, 2011 at 3:08 AM, Jeff M. [email protected]
wrote:

sooner) and because when it does expire, it simply takes too long (and
if multiple clients connect at that same time making that request, it
gets even worse).

I think you look the wrong way at the problem you have. You should fix
these
issues you mention which are NOT the expected behavior before trying to
fix
your situation with your proposed patch of allowing a different time to
a
special IP and leave a bot requesting the page.

First problem: early renewal of page.

This is not a mistery why it happens: Either your server it’s returning
a
header that tells nginx to refresh the cache, or you have a problem
inside
your configuration. If you tell nginx to update each 60 min, it will
always
respect that except backend server tells nginx to refresh the cache.

Second problem: multiple clientes renewing cache.

This can easily be fixed by using proxy_cache_stale, you can set it up
to
return users the last cached page while the backend ends updating the
first
request that expired. This means that after a requests pass to the
backend,
while that request does not complete, nginx will return the last cached
(which already expired) version to any new request for that page.

If you fix those both problems then I think you won’t need all that
workaround you want to do.

I’m not nginx guru and maybe it’s me looking the wrong way at your
problem
but in case it help this is how I dealt before with the same problem you
described.

Good Luck!

Guzmn B.

On Fri, Jan 28, 2011 at 11:55 PM, Ryan M. [email protected]
wrote:

if ($foo = “bar”) {
set $mycc = “public, max-age=3600”
}

proxy_pass http://backend;
add_header Cache-Control $mycc;
}

I’m amazed to see the config above works for you. I’m not sure if
you’re using some ancient versions of nginx that I know very little
about (like 0.6.x or 0.5.x or even earlier).

Consider the following config snippet:

location /test {
    set $a 3;
    if ($a = 3) {
        set $a 4;
    }
    echo $a;
}

GET /test will give you nothing for nginx 0.7.21 ~ 0.9.4. Don’t tell
me that the ngx_echo module’s echo directive is buggy because it’s too
dead simple to get wrong.

Basically for “modern” versions of nginx, the location “if” block
creates an anonymous nested location and will trap the execution flow,
that is, once you get in, you’ll never go out of it to reach the
content phase. And that’s one of the reasons why people call nginx’s
“if” an evil monster :wink:

Cheers,
-agentzh

On Tue, Feb 1, 2011 at 4:47 PM, agentzh [email protected] wrote:

GET /test will give you nothing for nginx 0.7.21 ~ 0.9.4.
Forgot to mention that we’ve been in favor of ngx_lua’s set_by_lua and
rewrite_by_lua directives to do something like this:

location /test {
    set $a 3;
    set_by_lua $a '
        if ngx.var.a == "3" then
            return 4
        end
    ';
    echo $a;
}

and GET /test will give you the expected answer, “4” and the
performance remains outstanding especially when you’ve enabled LuaJIT
2.0 :wink:

Cheers,
-agentzh

On Tue, Feb 1, 2011 at 2:47 AM, agentzh [email protected] wrote:

you’re using some ancient versions of nginx that I know very little
}

It does in fact work in production on nginx 0.7.6x. Below is my actual
configuration (trimmed to the essentials and with a few substitutions
of actual URIs).

location /MyApp {
proxy_pass http://backendTomcat;
proxy_set_header Host $host;
proxy_read_timeout 900;
proxy_next_upstream error timeout;
proxy_redirect http://example.com/ $scheme://example.com/;

#hide caching related headers from backend
#the backend is dumb about this but we can’t change it
proxy_hide_header Pragma;
proxy_hide_header Cache-Control;
proxy_hide_header Expires;

#these variables are the new values for caching headers

set $mycc “private,max-age=0”;
set $myexp “Fri, 01 Jan 1990 00:34:56 GMT”;

#create a variable that includes both scheme and URI so we
#can redirect in and out of SSL if needed
set $schemeuri $scheme$uri;

#add exception for dynamic CSS stylesheet, make it cache for 1 hour
if ($request_uri ~* uiframework.Theme) {
set $mycc “private,max-age=3600”;
set $myexp “”;
}
if ($schemeuri = “http/MyApp/changePassword”) {
rewrite ^(.*)$ https://$host$1 redirect;
}

#add our new headers
add_header “Cache-Control” $mycc;
add_header “Expires” $myexp;
}


RPM

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs