Invalidate cache for static files?

Server NGINX
1

What would be the simplest way to invalidate browser cache of static
files served by nginx?

For instance, we have a website which is in active development. We get a
reasonable amount of traffic, but we often get complaints when we update
the live files because some browsers are working with a mix of fresh
(live) and stale (cached) js/css files. We’d like to inform/force
browsers that the file is new and should be updated.

Phillip B Oldham
The Activity People
[email protected] mailto:[email protected]

Policies

This e-mail and its attachments are intended for the above named
recipient(s) only and may be confidential. If they have come to you in
error, please reply to this e-mail and highlight the error. No action
should be taken regarding content, nor must you copy or show them to
anyone.

This e-mail has been created in the knowledge that Internet e-mail is
not a 100% secure communications medium, and we have taken steps to
ensure that this e-mail and attachments are free from any virus. We must
advise that in keeping with good computing practice the recipient should
ensure they are completely virus free, and that you understand and
observe the lack of security when e-mailing us.

2

That is a tricky thing with various browsers. The best sollution I’ve
experienced for this, is to assume the files are cached indefinetly, and
you modify somekind of unique folder, to the same location. Ie:

/css/[0-9]+.[0-9]+/(.+) => /css/real/$1 (somebody else will have to do
a valid nginx rewrite rule)

This way the browser always loads the new files, as long as you modify
the dirname in your software, when you want to update the clients.

/css/1.00/style.css => /css/real/style.css
/css/1.01/style.css => /css/real/style.css
…

Just add another + 0.01 to your application, when you want to reload
with the new css/js files.

With a purely development server, you can use a dynamic variable (php’s
time() function for example) in the directory name, so that it always
reloads the relevant code.

This should be a more elegant and a cross-platform sollutions for all
browsers. In fact, i’ve seen it been recommended several times.

BR

3

Hello!

On Mon, Jul 21, 2008 at 01:57:06PM +0100, Phillip B Oldham wrote:

What would be the simplest way to invalidate browser cache of static
files served by nginx?

For instance, we have a website which is in active development. We get a
reasonable amount of traffic, but we often get complaints when we update
the live files because some browsers are working with a mix of fresh
(live) and stale (cached) js/css files. We’d like to inform/force
browsers that the file is new and should be updated.

Just use normal web development practices, e.g. add “?v=”
to js/css urls. Nothing special, nginx is just web server.

Alternatively you may force nginx to disable cache for specific
urls via expires directive, see
http://wiki.codemongers.com/NginxHttpHeadersModule#expires. But
this is probably not what you want since this will:

 a) Disable cache for all requests, not only when something

get’s changed on server.
b) Sometimes ignored by browsers.

Maxim D.

4

Would it help moving them into a different folder instead of appending
query string?

Sent from my iPhone

5

Yes, that’s the idea.

You replace a part of the url with a version string, this can be a
folder, or it can be the filename itself.

Moving them is a lazy man’s sollution to a rewrite rule, which handles
this for you, so you don’t have to copy files around on each revision /
version.

This is a paradigm that can be applied to any web server with support
for rewriting rules (apache, lighttpd, nginx,…).

P.s. off-topic, will you publish your slides on optimisation, from the
php 08 conference? I missed your talk, and I would really like to see
what you had to say, since the videolectures aren’t uploaded yet.

BR, Tit

6

Maxim D. wrote:

mix of fresh (live) and stale (cached) js/css files. We’d like to
inform/force browsers that the file is new and should be updated.

Just use normal web development practices, e.g. add “?v=”
to js/css urls. Nothing special, nginx is just web server.
I’ve tried this before, it is problematic since cache doesn’t behave
correctly and in some cases browsers or even caching proxies in front of
backend servers never cache the resource, because they percieve the link
as dynamic.

Having unique urls, like I suggested in the previous email, solves
theese problems. Also, you can then set the Expires & Cache-control
headers for static content far in the future and encourage browser
caching, since you have an effective method of always reloading them
without fail. Caches and browsers will keep such objects, while the
suggested sollution with the “?v=version” will not be cached on many of
them, since they percieve the url as dynamic and always try a reload,
ignoring possible Expires & Cache-control headers.

For a small setup, where you don’t care about resources, your sollution
is mostly fine, but is problematic in the real world with tens of
thousands of users.

BR

7

On Mon, Jul 21, 2008 at 04:35:30PM +0100, Phillip B Oldham wrote:

I don’t think moving/renaming files would be a great option. There are a
lot of files to manage, and a lot of links to them in the source. Would
be a bit of a nightmare.

It would be useful if nginx supported Etags based on last modified time.
Especially if the etag could be cached and a customer signal could be
passed to nginx to clear this cache.

nginx supports Last-Modified header. However, Last-Modified as well as
ETag
do not help to invalide browsers’ cache.

Maxim D. wrote:

with a mix of fresh (live) and stale (cached) js/css files. We’d
Having unique urls, like I suggested in the previous email, solves
tens of thousands of users.
[email protected] mailto:[email protected]
This e-mail has been created in the knowledge that Internet e-mail is
not a 100% secure communications medium, and we have taken steps to
ensure that this e-mail and attachments are free from any virus. We must
advise that in keeping with good computing practice the recipient should
ensure they are completely virus free, and that you understand and
observe the lack of security when e-mailing us.

8

I don’t think moving/renaming files would be a great option. There are a
lot of files to manage, and a lot of links to them in the source. Would
be a bit of a nightmare.

It would be useful if nginx supported Etags based on last modified time.
Especially if the etag could be cached and a customer signal could be
passed to nginx to clear this cache.

Denis Arh wrote:

Hello!

like to inform/force browsers that the file is new and should be
theese problems. Also, you can then set the Expires & Cache-control

BR

–

Phillip B Oldham
The Activity People
[email protected] mailto:[email protected]

Policies

This e-mail and its attachments are intended for the above named
recipient(s) only and may be confidential. If they have come to you in
error, please reply to this e-mail and highlight the error. No action
should be taken regarding content, nor must you copy or show them to
anyone.

This e-mail has been created in the knowledge that Internet e-mail is
not a 100% secure communications medium, and we have taken steps to
ensure that this e-mail and attachments are free from any virus. We must
advise that in keeping with good computing practice the recipient should
ensure they are completely virus free, and that you understand and
observe the lack of security when e-mailing us.

9

Hi

Have you tried suppressing the Last-Modified response header ?
Usually, when a web server serves a static page, it uses the mtime of
the file to send back the Last-Modified response header.

When the browser loads the static file, let’s say at time X, it may get
back the response header Last-Modified: Y

Then, subsequently, when the browser wants the page again, it sends the
If-Modified-Since request header, containing X
And, if the file has not been modified in the interval since X, then the
web server sends back the HTTP not modified response (304)

Does NGINX send back the Last-Modified header for static resources ? I
guess it should.
Also, if NGINX allows you to suppress a response header on the way out,
then you should use that config directive to suppress the Last-Modified
header.

I believe that should cause the browser to reload the resource every
time, though one can’t be sure about these things

Regards,
Mansoor

----- “Denis Arh” [email protected] wrote:

| From: “Denis Arh” [email protected]
| To: [email protected]
| Sent: Monday, July 21, 2008 8:31:16 PM GMT +05:30 Chennai, Kolkata, Mumbai, New Delhi
| Subject: Re: Invalidate cache for static files?
|
| Would it help moving them into a different folder instead of appending
|
| query string?
|
| Sent from my iPhone
|
| On 21.7.2008, at 16:55, Tit P. [email protected] wrote:
|
| >
| >
| > Maxim D. wrote:
| >> Hello!
| >>
| >> On Mon, Jul 21, 2008 at 01:57:06PM +0100, Phillip B Oldham wrote:
| >>
| >>> What would be the simplest way to invalidate browser cache of
| >>> static files served by nginx?
| >>>
| >>> For instance, we have a website which is in active development. We
|
| >>> get a reasonable amount of traffic, but we often get complaints
|
| >>> when we update the live files because some browsers are working
|
| >>> with a mix of fresh (live) and stale (cached) js/css files. We’d
|
| >>> like to inform/force browsers that the file is new and should be
|
| >>> updated.
| >>
| >> Just use normal web development practices, e.g. add “?v=”
| >> to js/css urls. Nothing special, nginx is just web server.
| > I’ve tried this before, it is problematic since cache doesn’t behave
|
| > correctly and in some cases browsers or even caching proxies in
| > front of backend servers never cache the resource, because they
| > percieve the link as dynamic.
| >
| > Having unique urls, like I suggested in the previous email, solves
|
| > theese problems. Also, you can then set the Expires & Cache-control
|
| > headers for static content far in the future and encourage browser
|
| > caching, since you have an effective method of always reloading them
|
| > without fail. Caches and browsers will keep such objects, while the
|
| > suggested sollution with the “?v=version” will not be cached on many
|
| > of them, since they percieve the url as dynamic and always try a
| > reload, ignoring possible Expires & Cache-control headers.
| >
| > For a small setup, where you don’t care about resources, your
| > sollution is mostly fine, but is problematic in the real world with
|
| > tens of thousands of users.
| >
| > BR
| >

10

Oh I forgot.

In case the browser gets back a 304 response for a static resource, then
it loads it from its cache
But you may have trouble with “certain” browsers even after
theory says it should re-fetch

M

----- “Mansoor P.” [email protected] wrote:

| From: “Mansoor P.” [email protected]
| To: [email protected]
| Sent: Monday, July 21, 2008 9:13:55 PM GMT +05:30 Chennai, Kolkata, Mumbai, New Delhi
| Subject: Re: Invalidate cache for static files?
|
| Hi
|
| Have you tried suppressing the Last-Modified response header ?
| Usually, when a web server serves a static page, it uses the mtime of
| the file to send back the Last-Modified response header.
|
| When the browser loads the static file, let’s say at time X, it may
| get back the response header Last-Modified: Y
|
| Then, subsequently, when the browser wants the page again, it sends
| the If-Modified-Since request header, containing X
| And, if the file has not been modified in the interval since X, then
| the web server sends back the HTTP not modified response (304)
|
| Does NGINX send back the Last-Modified header for static resources ? I
| guess it should.
| Also, if NGINX allows you to suppress a response header on the way
| out, then you should use that config directive to suppress the
| Last-Modified header.
|
| I believe that should cause the browser to reload the resource every
| time, though one can’t be sure about these things
|
| Regards,
| Mansoor
|
| ----- “Denis Arh” [email protected] wrote:
|
| | From: “Denis Arh” [email protected]
| | To: [email protected]
| | Sent: Monday, July 21, 2008 8:31:16 PM GMT +05:30 Chennai, Kolkata,
| Mumbai, New Delhi
| | Subject: Re: Invalidate cache for static files?
| |
| | Would it help moving them into a different folder instead of
| appending
| |
| | query string?
| |
| | Sent from my iPhone
| |
| | On 21.7.2008, at 16:55, Tit P. [email protected] wrote:
| |
| | >
| | >
| | > Maxim D. wrote:
| | >> Hello!
| | >>
| | >> On Mon, Jul 21, 2008 at 01:57:06PM +0100, Phillip B Oldham wrote:
| | >>
| | >>> What would be the simplest way to invalidate browser cache of
| | >>> static files served by nginx?
| | >>>
| | >>> For instance, we have a website which is in active development.
| We
| |
| | >>> get a reasonable amount of traffic, but we often get complaints
| |
| | >>> when we update the live files because some browsers are working
| |
| | >>> with a mix of fresh (live) and stale (cached) js/css files.
| We’d
| |
| | >>> like to inform/force browsers that the file is new and should
| be
| |
| | >>> updated.
| | >>
| | >> Just use normal web development practices, e.g. add
| “?v=”
| | >> to js/css urls. Nothing special, nginx is just web server.
| | > I’ve tried this before, it is problematic since cache doesn’t
| behave
| |
| | > correctly and in some cases browsers or even caching proxies in
| | > front of backend servers never cache the resource, because they
| | > percieve the link as dynamic.
| | >
| | > Having unique urls, like I suggested in the previous email, solves
| |
| | > theese problems. Also, you can then set the Expires &
| Cache-control
| |
| | > headers for static content far in the future and encourage browser
| |
| | > caching, since you have an effective method of always reloading
| them
| |
| | > without fail. Caches and browsers will keep such objects, while
| the
| |
| | > suggested sollution with the “?v=version” will not be cached on
| many
| |
| | > of them, since they percieve the url as dynamic and always try a
| | > reload, ignoring possible Expires & Cache-control headers.
| | >
| | > For a small setup, where you don’t care about resources, your
| | > sollution is mostly fine, but is problematic in the real world
| with
| |
| | > tens of thousands of users.
| | >
| | > BR
| | >