Help with X-Accel-Redirect and "rewrite or internal redirection cycle" errors

Server NGINX
1

Hi,

I’m trying to use X-Accel-Redirect from a Ruby app to hand over
downloading of a file to Nginx, but it fails and the error in the logs
is:

2011/06/03 09:18:16 [error] 20500#0: *37 rewrite or internal redirection
cycle while internal redirect to “/files/file.m4a” while reading
response header from upstream, client: 10.0.1.23, server: hercules,
request: “GET
/download/e2a25db11bdbdd783531783eaa16eafdfc86355b/1307089096/file.m4a/
HTTP/1.1”, upstream:
“http://127.0.0.1:8679/download/e2a25db11bdbdd783531783eaa16eafdfc86355b/1307089096/file.m4a/”,
host: “fakehost.blob”, referrer: “http://fakehost.blob/music/file.m4a”

Here is the Nginx config from sites-available:

upstream fakehost.blob {
server 127.0.0.1:8679;
server 127.0.0.1:8680;
}

server {
listen 80;

server_name fakehost.blob;

access_log /var/log/nginx/fakehost.blob.log main;
error_log /var/log/nginx/fakehost.blob.error.log info;

location /files {
internal;
alias /Users/www/sites/$http_host/assets/files;
}

location / {
proxy_pass http://$http_host;
}

location = /50x.html {
root html;
}
}

and in the main nginx.conf:

http {
server_names_hash_bucket_size 64;
include mime.types;
default_type application/octet-stream;

log_format main '$remote_addr - $remote_user [$time_local]
$request ’
'“$status” $body_bytes_sent “$http_referer” ’
‘“$http_user_agent” “$http_x_forwarded_for”’;

sendfile on;

keepalive_timeout 65;## General Options
ignore_invalid_headers on;
server_name_in_redirect off;
server_tokens off;

#TCP
tcp_nopush on;
tcp_nodelay off;

Proxy options

proxy_buffering on;
#proxy_cache_min_uses 3;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;

proxy_connect_timeout 90;
proxy_send_timeout 90;
proxy_read_timeout 90;

proxy_buffer_size 4k;
proxy_buffers 4 32k;
proxy_busy_buffers_size 64k;
proxy_temp_file_write_size 64k;

If anyone is able to give me some insight or help with this I’d be very
grateful, as I’ve reached the limit (I’m far beyond it!:slight_smile: of my
knowledge with Nginx.

Regards,
Iain

2

Thanks for replying.

On 6 Jun 2011, at 14:14, Antnio P. P. Almeida wrote:

Problably your app is issuing redirects for that request.

I thought that with the /files location that the Ruby app wouldn’t be
touched once it handed off the download?

You should
issue the X-Accel-Redirect header with the the download location.

X-Accel-Redirect path/to/download/file

Only then will the Ruby handling of the file will be overriden and the
download handled by Nginx. Cf. http://wiki.nginx.org/NginxXSendfile

— appa

The Ruby code issues the path so it looks like a uri:

  headers( {'X-Accel-Redirect' => "/files/#{title}",
            'Content-Type' => "audio/#{File.extname(title)[1..-1]}",
            'Content-Disposition' =>  "attachment",
            'Content-Transfer-Encoding' =>

‘application/octet-stream’,
‘Content-Length’ => File.size( path_to_file ).to_s,
‘Cache-Control’ => ‘no-cache’,
‘Content-Description’ => ‘File Transfer’, })

Here’s an example of an actual header it sends:

D, [Mon 06-06-2011 1439 #82286] DEBUG – : to_send_to_nginx:
{“X-Accel-Redirect”=>“/files/file.m4a”, “Content-Type”=>“audio/m4a”,
“Content-Disposition”=>“attachment”,
“Content-Transfer-Encoding”=>“application/octet-stream”,
“Content-Length”=>“9032392”, “Cache-Control”=>“no-cache”,
“Content-Description”=>“File Transfer”}

so with this:

location /files {
internal;
alias /Users/www/sites/$http_host/assets/files;
}

it should end up with
Users/www/sites/fakehost.blob/assets/files/file.m4a ?

It seems to be correct for what I’ve read in the docs and examples -
does it look right to you?

Regards,
Iain

3

Hello!

On Mon, Jun 06, 2011 at 01:50:42PM +0100, Iain B. wrote:

I’m trying to use X-Accel-Redirect from a Ruby app to hand over
downloading of a file to Nginx, but it fails and the error in
the logs is:

2011/06/03 09:18:16 [error] 20500#0: *37 rewrite or internal redirection cycle
while internal redirect to “/files/file.m4a” while reading response header from
upstream, client: 10.0.1.23, server: hercules, request: “GET
/download/e2a25db11bdbdd783531783eaa16eafdfc86355b/1307089096/file.m4a/ HTTP/1.1”,
upstream:
“http://127.0.0.1:8679/download/e2a25db11bdbdd783531783eaa16eafdfc86355b/1307089096/file.m4a/”,
host: “fakehost.blob”, referrer: “http://fakehost.blob/music/file.m4a”

Please note “server: hercules”.

[…]

server {
listen 80;

server_name fakehost.blob;

This is not the server where request is processed.

Maxim D.

4

On 6 Jun 2011 13h50 WEST, [email protected] wrote:

/download/e2a25db11bdbdd783531783eaa16eafdfc86355b/1307089096/file.m4a/
server 127.0.0.1:8679;

location = /50x.html {
root html;
}
}

Problably your app is issuing redirects for that request. You should
issue the X-Accel-Redirect header with the the download location.

X-Accel-Redirect path/to/download/file

Only then will the Ruby handling of the file will be overriden and the
download handled by Nginx. Cf. http://wiki.nginx.org/NginxXSendfile

— appa

5

On 6 Jun 2011, at 15:07, Maxim D. wrote:

Please note “server: hercules”.

[…]

server {
listen 80;

server_name fakehost.blob;

This is not the server where request is processed.

Sorry, I made that up :slight_smile: I didn’t want to use the actual host/server
name so I filled in some fake bits, and the tasks of Hercules were on my
mind - probably because getting computers to do what you want can be
like pushing a rock up a hill endlessly :smiley:

I should probably have put server: fakehost.blob. Apologies for the
confusion.

Ah ha! Looking at the original, the server name is from one of the other
site’s being proxied to. Does this mean I should use an absolute uri for
the X-Accel-Redirect header?

I’m going to try it now.

Regards,
Iain

6

On 6 Jun 2011, at 15:55, Maxim D. wrote:

If you do care enough about your actual server names but still
need help with debugging - reproduce the problem in sandbox with
fake names.

Bit harsh, there was no difference to the result from my mistake, and
trying to back up a position with a false dilemma rarely works. I think
you’ll find it’s up to me how much information I give about my projects,
asking for your help doesn’t mean I owe you anything, and giving your
help doesn’t mean you get to be rude.

Ah ha! Looking at the original, the server name is from one of
the other site’s being proxied to. Does this mean I should use
an absolute uri for the X-Accel-Redirect header?

No. The problem is still the same: redirect loop happens not in
the server{} you provided, but in other one.

I’m finding entries for this in the access logs of a different server,
so I’ll see if whether on redirect the host name is being passed along.

You haven’t provided any info, so just wild guess: you want to
read about proxy_ignore_headers directive. See here:

Module ngx_http_proxy_module

Well, to say I haven’t provided any info is quite a stretch, and if
there was other information you thought I should give I think you would
be better to say what it is, but thankyou for your help anyway, I’ll
give it a look.

Regards,
Iain

7

Hello!

On Mon, Jun 06, 2011 at 03:23:09PM +0100, Iain B. wrote:

This is not the server where request is processed.

Sorry, I made that up :slight_smile: I didn’t want to use the actual
host/server name so I filled in some fake bits, and the tasks of
Hercules were on my mind - probably because getting computers to
do what you want can be like pushing a rock up a hill endlessly
:smiley:

I should probably have put server: fakehost.blob. Apologies for
the confusion.

There are two options:

  1. You understand things enough to mangle logs and configs, and
    you don’t need any help with debugging.

  2. You need help with debugging.

Choose any one. Obviously with (1) there are no reasons to post
anything here.

If you do care enough about your actual server names but still
need help with debugging - reproduce the problem in sandbox with
fake names.

Ah ha! Looking at the original, the server name is from one of
the other site’s being proxied to. Does this mean I should use
an absolute uri for the X-Accel-Redirect header?

No. The problem is still the same: redirect loop happens not in
the server{} you provided, but in other one.

You haven’t provided any info, so just wild guess: you want to
read about proxy_ignore_headers directive. See here:

http://wiki.nginx.org/HttpProxyModule#proxy_ignore_headers

Maxim D.