I am currently implementing a caching proxy with many short-lived items,
expiring at a specific date (Expires header set at an absolute time
10 and 30 seconds in the future by the origin). For various reasons, my
cache is multi-level (edge, intermediate 2, intermediate 1, origin) and
needs to make the items expire at the edge at 2exactly the time set in
Expires header. When the item expires, i want an updated version of the
to be available at the same URL.
I have been able to make it work, and I’m using proxy_cache_lock at
cache level to ensure i’m not hammering the origin servers nor the proxy
server. As documented, this works perfectly for items not present in the
I am also using proxy_cache_use_stale updating to avoid this hammering
in the case of already in-cache items.
My problems begin when an in-cache item expires. 2 top-level caches
name them E1,E2 for example) request an updated fragment from the below
level (INT). The fragment is requested by INT from below for the first
request, but for the second request, a stale fragment is sent (according
proxy_cache_use_stale setting, with the UPDATING status). So far, all is
working according to the docs. The problem is that fragments in the
status are stale, and cannot be cached at all by E1,E2…, and this can
very impacting for INT, because all the requests made on E1/E2 are now
proxied to INT directly, until INT has a fresh version of the item
in cache (this is quite a short duration, but in testing this generate
bursts of 15 to 50 requests in the mean time).
Is there a way to implement the proxy_cache_lock to make it work also
expired in-cache items in the configuration ? If not, can you suggest a
to implement this (i’m not familiar to nginx’s source, but i’m willing
dig into it) ?
Posted at Nginx Forum: