Is it correct that when $content_length > client_body_buffer_size, then $request_body == "" ? If so this would be worth documenting at request_body. I am using: proxy_cache_methods POST; proxy_cache_key "$request_method$request_uri$request_body"; Which works for small requests, but for large requests clients got very strange results due to $request_body being empty and hence getting false cache hits for completely different form posts. Is there something available like $body_hash that can be used as a caching key even for large request bodies? Or alternatively, how would I configure nginx to not cache requests when content_length is larger than client_body_buffer_size?
on 2013-09-14 02:57
on 2013-09-14 15:49
Hello! On Fri, Sep 13, 2013 at 05:56:51PM -0700, Jeroen Ooms wrote: > Is it correct that when $content_length > client_body_buffer_size, > then $request_body == "" ? If so this would be worth documenting at > request_body. Yes, it's intended behaviour. If a request body is larger than client_body_buffer_size, it's written to disk and not available in memory, hence no $request_body. The limitation is more or less obvious, and it's also explicitly documented here in the $r->request_body() method documentation: http://nginx.org/en/docs/http/ngx_http_perl_module... It might worth adding some short reference into $request_body variable description though. > caching key even for large request bodies? Or alternatively, how > would I configure nginx to not cache requests when content_length > is larger than client_body_buffer_size? The proxy_no_cache $request_body_file; should do the trick, see http://nginx.org/r/proxy_no_cache. -- Maxim Dounin http://nginx.org/en/donation.html
on 2013-09-14 20:16
@ Maxim Dounin Thanks! This is very helpful. I have also set: client_body_buffer_size 1m; Could this setting have any side effects? I am not expecting too many large POST request. From what I read, client_body_buffer_size is actually the maximum amount of memory allocated. Does this mean that for small requests (e.g. without a body) there is no additional overhead introduced by raising this value?
on 2013-09-14 21:24
Hello! On Sat, Sep 14, 2013 at 11:15:39AM -0700, Jeroen Ooms wrote: > overhead introduced by raising this value? Yes, it's not allocated if there is no request body, and only needed buffer is allocated if a request body is known to be smaller. On the other hand, it can be used as a DoS vector if an attacker is allowed to open many connections but you can't afford them all to allocate client_body_buffer_size buffer. Additionally, using such a big $request_body in proxy_cache_key implies various overheads. In particular, proxy_buffer_size should be set big enough to be able to contain cache header with a key. Not even talking about reading/writing cache files with such keys. -- Maxim Dounin http://nginx.org/en/donation.html
on 2013-09-26 02:42
One more question regarding this: > The > proxy_no_cache $request_body_file; > should do the trick, see http://nginx.org/r/proxy_no_cache. I tried this and get a warning: nginx: [warn] "proxy_no_cache" functionality has been changed in 0.8.46, now it should be used together with "proxy_cache_bypass" Do I just need to add an additional line: proxy_cache_bypass $request_body_file; It is not clear to me how proxy_cache_bypass is different from proxy_no_cache.
on 2013-09-26 14:38
Hello! On Wed, Sep 25, 2013 at 08:41:44PM -0400, Jeroen Ooms wrote: > Do I just need to add an additional line: > proxy_cache_bypass $request_body_file; > > It is not clear to me how proxy_cache_bypass is different from proxy_no_cache. The "proxy_no_cache" directive prevents caching of a response got from an upstream (i.e., do not save to a cache). In contrast, the "proxy_cache_bypass" directive prevents returing a response from a cache if it's there. The warning in question is an artifact from 0.8.x time when proxy_no_cache used to mean both "no cache" and "bypass". It probably should be removed. -- Maxim Dounin http://nginx.org/en/donation.html