Forum: NGINX x-accel-redirect enables caching for POST requests

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
Bb3d77926899cc3e2aa8ad6e34bac121?d=identicon&s=25 Руслан Закиров (Guest)
on 2015-12-23 13:54
(Received via mailing list)
Hi,

If location /a/ redirects a POST request to location /b/ using
X-Accel-Redirect then result of the POST request is cached.

Tested this and reproduced with nginx 1.6.3 and 1.9.7.

Setup:
https://gist.github.com/ruz/4a66ee78fedf27181799

Logs from hitting /a/:
[1450873503.675] 200 HIT 127.0.0.1 "POST /a/ HTTP/1.1" [0.001, 0.001] {
127.0.0.1:5000}
[1450873504.113] 200 HIT 127.0.0.1 "POST /a/ HTTP/1.1" [0.001, 0.001] {
127.0.0.1:5000}
[1450873504.529] 200 HIT 127.0.0.1 "POST /a/ HTTP/1.1" [0.002, 0.002] {
127.0.0.1:5000}
[1450873567.648] 200 EXPIRED 127.0.0.1 "POST /a/ HTTP/1.1" [0.001 :
0.002,
0.003] {127.0.0.1:5000 : 127.0.0.1:5000}

Logs from hitting /b/ directly:
[1450875056.289] 200 - 127.0.0.1 "POST /b/ HTTP/1.1" [0.005, 0.005] {
127.0.0.1:5000}
[1450875058.073] 200 - 127.0.0.1 "POST /b/ HTTP/1.1" [0.001, 0.001] {
127.0.0.1:5000}

Looks like a bug to me. Do I miss something?
A8108a0961c6087c43cda32c8616dcba?d=identicon&s=25 Maxim Dounin (Guest)
on 2015-12-23 16:49
(Received via mailing list)
Hello!

On Wed, Dec 23, 2015 at 03:54:31PM +0300,   wrote:

> Logs from hitting /a/:
> [1450875056.289] 200 - 127.0.0.1 "POST /b/ HTTP/1.1" [0.005, 0.005] {
> 127.0.0.1:5000}
> [1450875058.073] 200 - 127.0.0.1 "POST /b/ HTTP/1.1" [0.001, 0.001] {
> 127.0.0.1:5000}
>
> Looks like a bug to me. Do I miss something?

X-Accel-Redirect changes a request from POST to GET.

--
Maxim Dounin
http://nginx.org/
Bb3d77926899cc3e2aa8ad6e34bac121?d=identicon&s=25 Руслан Закиров (Guest)
on 2015-12-23 17:11
(Received via mailing list)
On Wed, Dec 23, 2015 at 6:49 PM, Maxim Dounin <mdounin@mdounin.ru>
wrote:

> X-Accel-Redirect changes a request from POST to GET.
>

No, it doesn't. Getting request method POST on the backend and even form
data is intact.
Bb3d77926899cc3e2aa8ad6e34bac121?d=identicon&s=25 Руслан Закиров (Guest)
on 2015-12-23 17:15
(Received via mailing list)
On Wed, Dec 23, 2015 at 7:10 PM, Руслан Закиров <ruz@sports.ru> wrote:

>
> On Wed, Dec 23, 2015 at 6:49 PM, Maxim Dounin <mdounin@mdounin.ru> wrote:
>
>> X-Accel-Redirect changes a request from POST to GET.
>>
>
> No, it doesn't. Getting request method POST on the backend and even form
> data is intact.
>


Output from the psgi app (updated gist with data dumper):

127.0.0.1 - - [23/Dec/2015:19:05:46 +0300] "POST /a/ HTTP/1.0" 200 0 "-"
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36
(KHTML,
like Gecko) Chrome/47.0.2526.106 Safari/537.36"
{
  CONTENT_LENGTH => 7,
  CONTENT_TYPE => 'application/json, application/x-www-form-urlencoded',
  HTTP_ACCEPT => '*/*',
  HTTP_ACCEPT_ENCODING => 'gzip, deflate',
  HTTP_ACCEPT_LANGUAGE => 'en-US,en;q=0.8,ru;q=0.6',
  HTTP_CACHE_CONTROL => 'no-cache',
  HTTP_CONNECTION => 'close',
  HTTP_HOST => '127.0.0.1:5000',
  HTTP_ORIGIN => 'chrome-extension://fdmmgilgnpjigdojojpjoooidkmcomcm',
  HTTP_PRAGMA => 'no-cache',
  HTTP_USER_AGENT => 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106
Safari/537.36',
  PATH_INFO => '/b/',
  QUERY_STRING => '',
  REMOTE_ADDR => '127.0.0.1',
  REMOTE_PORT => 56137,
  REQUEST_METHOD => 'POST',
  REQUEST_URI => '/b/',
  SCRIPT_NAME => '',
  SERVER_NAME => 0,
  SERVER_PORT => 5000,
  SERVER_PROTOCOL => 'HTTP/1.0',
  'psgi.errors' => *::STDERR,
  'psgi.input' => bless( \*{'Stream::Buffered::PerlIO::$io'},
'FileHandle'
),
  'psgi.multiprocess' => '',
  'psgi.multithread' => '',
  'psgi.nonblocking' => '',
  'psgi.run_once' => '',
  'psgi.streaming' => 1,
  'psgi.url_scheme' => 'http',
  'psgi.version' => [
    1,
    1
  ],
  'psgix.harakiri' => 1,
  'psgix.input.buffered' => 1,
  'psgix.io' => bless( \*Symbol::GEN5, 'IO::Socket::INET' )
}
x=y&y=z
127.0.0.1 - - [23/Dec/2015:19:05:46 +0300] "POST /b/ HTTP/1.0" 200 22
"-"
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36
(KHTML,
like Gecko) Chrome/47.0.2526.106 Safari/537.36"
A8108a0961c6087c43cda32c8616dcba?d=identicon&s=25 Maxim Dounin (Guest)
on 2015-12-23 17:19
(Received via mailing list)
Hello!

On Wed, Dec 23, 2015 at 07:10:43PM +0300,   wrote:

> On Wed, Dec 23, 2015 at 6:49 PM, Maxim Dounin <mdounin@mdounin.ru> wrote:
>
> > X-Accel-Redirect changes a request from POST to GET.
> >
>
> No, it doesn't. Getting request method POST on the backend and even form
> data is intact.

It does,
http://hg.nginx.org/nginx/file/tip/src/http/ngx_ht...

            if (r->method != NGX_HTTP_HEAD) {
                r->method = NGX_HTTP_GET;
            }

Though it looks like it only does so for nginx itself, and this
indeed looks like a bug.  The code should be similar to one in
error_page handling:

        if (r->method != NGX_HTTP_HEAD) {
            r->method = NGX_HTTP_GET;
            r->method_name = ngx_http_core_get_method;
        }

If you want to get request and request method intact, use
X-Accel-Redirect to a named location instead.

--
Maxim Dounin
http://nginx.org/
Bb3d77926899cc3e2aa8ad6e34bac121?d=identicon&s=25 Руслан Закиров (Guest)
on 2015-12-23 22:17
(Received via mailing list)
On Wed, Dec 23, 2015 at 7:19 PM, Maxim Dounin <mdounin@mdounin.ru>
wrote:

>
> It does,
> http://hg.nginx.org/nginx/file/tip/src/http/ngx_ht...
>
>             if (r->method != NGX_HTTP_HEAD) {
>                 r->method = NGX_HTTP_GET;
>             }
>
> Though it looks like it only does so for nginx itself, and this
> indeed looks like a bug.  The code should be similar to one in
>

Would you create an issue for this in tracker or do I need to so it
doesn't
disappear in archives?


> error_page handling:
>
>         if (r->method != NGX_HTTP_HEAD) {
>             r->method = NGX_HTTP_GET;
>             r->method_name = ngx_http_core_get_method;
>         }
>
> If you want to get request and request method intact, use
> X-Accel-Redirect to a named location instead.


Understood and it works for our case. We actually don't want to send
POST
requests to location that issues internal redirects (our HRU service),
but
we haven't found good way to avoid it.
A8108a0961c6087c43cda32c8616dcba?d=identicon&s=25 Maxim Dounin (Guest)
on 2015-12-24 19:24
(Received via mailing list)
Hello!

On Thu, Dec 24, 2015 at 12:16:43AM +0300,   wrote:

> > > No, it doesn't. Getting request method POST on the backend and even form
> > indeed looks like a bug.  The code should be similar to one in
> >
>
> Would you create an issue for this in tracker or do I need to so it doesn't
> disappear in archives?

No real need to open tickets.  I've submitted a patch for an
internal review here.  Just in case, patch below.


# HG changeset patch
# User Maxim Dounin <mdounin@mdounin.ru>
# Date 1450981046 -10800
#      Thu Dec 24 21:17:26 2015 +0300
# Node ID 10e233c763566b8466f6e302511094866a14e77a
# Parent  78b4e10b4367b31367aad3c83c9c3acdd42397c4
Upstream: fixed changing method on X-Accel-Redirect.

Previously, only r->method was changed, resulting in handling of a
request
as GET within nginx itself, but not in requests to proxied servers.

See http://mailman.nginx.org/pipermail/nginx/2015-Dece....

diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c
--- a/src/http/ngx_http_upstream.c
+++ b/src/http/ngx_http_upstream.c
@@ -2499,6 +2499,7 @@ ngx_http_upstream_process_headers(ngx_ht

             if (r->method != NGX_HTTP_HEAD) {
                 r->method = NGX_HTTP_GET;
+                r->method_name = ngx_http_core_get_method;
             }

             ngx_http_internal_redirect(r, &uri, &args);


--
Maxim Dounin
http://nginx.org/
De7b680154f831d87d8ea48743852f14?d=identicon&s=25 gglater62 (Guest)
on 2016-02-09 19:06
(Received via mailing list)
Hi,

Are POST disallowed for X-Accel-Redirect?
There is setup:
nginx sends requests to Apache, which replies with X-Accel-Redirect and
X-Path-Info.
In X-Accel-Redirect there is path to php script, then nginx did do POST
or
GET to that script, but it stopped to work since 1.9.10.
Is there a possibility to preserve original $request_method? How to make
it
work without patching nginx?

Posted at Nginx Forum:
https://forum.nginx.org/read.php?2,263661,264437#msg-264437
A8108a0961c6087c43cda32c8616dcba?d=identicon&s=25 Maxim Dounin (Guest)
on 2016-02-09 19:15
(Received via mailing list)
Hello!

On Tue, Feb 09, 2016 at 01:06:21PM -0500, gglater62 wrote:

> Hi,
>
> Are POST disallowed for X-Accel-Redirect?
> There is setup:
> nginx sends requests to Apache, which replies with X-Accel-Redirect and
> X-Path-Info.
> In X-Accel-Redirect there is path to php script, then nginx did do POST or
> GET to that script, but it stopped to work since 1.9.10.
> Is there a possibility to preserve original $request_method? How to make it
> work without patching nginx?

Original request method is preserved when using X-Accel-Redirect
to a named location.

--
Maxim Dounin
http://nginx.org/
De7b680154f831d87d8ea48743852f14?d=identicon&s=25 gglater62 (Guest)
on 2016-02-09 21:54
(Received via mailing list)
I found a workaround:

set $method $request_method;
if ($request ~ ^POST) { set $method POST; }
fastcgi_param REQUEST_METHOD $method;

Posted at Nginx Forum:
https://forum.nginx.org/read.php?2,263661,264447#msg-264447
B3be4cc1eba56848a856d002b723eb60?d=identicon&s=25 Dan He (danma)
on 2017-02-09 17:00
Maxim Dounin wrote in post #1181273:
> Hello!
>
> On Tue, Feb 09, 2016 at 01:06:21PM -0500, gglater62 wrote:
>
>> Hi,
>>
>> Are POST disallowed for X-Accel-Redirect?
>> There is setup:
>> nginx sends requests to Apache, which replies with X-Accel-Redirect and
>> X-Path-Info.
>> In X-Accel-Redirect there is path to php script, then nginx did do POST or
>> GET to that script, but it stopped to work since 1.9.10.
>> Is there a possibility to preserve original $request_method? How to make it
>> work without patching nginx?
>
> Original request method is preserved when using X-Accel-Redirect
> to a named location.
>
> --
> Maxim Dounin
> http://nginx.org/

Hi, sorry to necro, would you mind elaborating on how to use the old
functionality with a named location? I've spent a while searching how to
use a named location and can't figure it out.

Here's my nginx config working with an older nginx version, before the
fix to enforce GET requests:

<code>
upstream gate_proxy {
    server 127.0.0.1:8889;
}

server {
    listen                  80 default_server;

    # Send all traffic to gate first
    location / {
        proxy_redirect          off;
        proxy_pass_header       Server;
        proxy_set_header        Host $http_host;
        proxy_set_header        X-Real-IP $remote_addr;
        proxy_set_header        X-Forwarded-For
$proxy_add_x_forwarded_for;
        proxy_set_header        X-Scheme $scheme;
        proxy_pass              http://gate_proxy;
    }

    # Proxy to Apache after X-Accel
    location /proxy-to-web-app/ {
        internal;
        proxy_redirect      off;
        proxy_pass_header   Server;
        proxy_set_header    Host $http_host;
        proxy_set_header    X-Real-IP $remote_addr;
        proxy_set_header    X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header    X-Scheme $scheme;

        # Make sure the trailing slash is maintained here, as this
affects the URI relayed.
        proxy_pass          http://127.0.0.1:8081/;
    }
}
</code>

Thanks for any advice :)
This topic is locked and can not be replied to.