Nginx as a proxy, needs to follow 301 responses

Hi everyone,

I am using nginx as a proxy for custom URLs. I call URLs like
/proxy/www.example.com/favicon.ico and nginx acts as a proxy for
www.example.com/favicon.ico. This works fine as long as the response
from the upstream server is not a 301. If it is a 301, this gets
forwarded to the client which then calls the target location directly.

Is there a way nginx can resolve these 301s and deliver the contents to
the client? You can find the config below.

Thank you very much!
Hendrik

location ~* ^/proxy/(.?)/(.)$ {
resolver 8.8.8.8;
proxy_pass http://$1/$2;
server_name_in_redirect on;
}

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,121104,121104#msg-121104

Hi,
On 18/08/2010 22:31, hendrik wrote:

Hi everyone,

I am using nginx as a proxy for custom URLs. I call URLs like
/proxy/www.example.com/favicon.ico and nginx acts as a proxy for
www.example.com/favicon.ico. This works fine as long as the response
from the upstream server is not a 301. If it is a 301, this gets
forwarded to the client which then calls the target location directly.

Is there a way nginx can resolve these 301s and deliver the contents to
the client? You can find the config below.
I think you should be able to do this by compiling the headers-more
module (http://github.com/agentzh/headers-more-nginx-module) and have a
config like :

proxy.conf

resolver 8.8.8.8;
proxy_pass http://$1;
more_set_headers -s ‘301 302 303’ ‘Location:
/proxy/$upstream_http_location’; # you might want to add more 30x’s here

nginx.conf

location ~ ^/proxy/http://(.*)$ {

include proxy.conf;

}

location ~ ^/proxy/(.*)$ {

include proxy.conf;

}

though I’ve not tested it. I think that $upstream_http_location will be
set after the response from the source server, but I’m not certain.

A few notes :

  • Having a proxy without some kind of security check is probably not a
    good idea. You could do this with your proxy, but could also be done
    with iptables.
  • There’s probably not much point in trying to enforce ‘domain/url’ in
    the regex. It will slow things down, and you don’t really gain
    anything. Same with case-insensitive regexes.
  • Nginx buffers responses until the request has been completed before
    passing to the client. On small files, this shouldn’t be a problem, but
    on big ones it might. On continual audio/video streams, it will block
    (as far as I know, and it did the last time I tested it).
  • Depending on what you’re using it for, you might be better off using
    Apache Trafficserver as a high-performance forward proxy. The
    performance is good, and I think it’ll be able to do what you want, but
    the config files are really ugly.

Cheers,

Marcus.

Hi Marcus!

Thank you very much for your response! I’ll have a look into the module
you advised and leave some feedback.

Regarding your remarks:

  • I am aware that leaving this proxy open is a security issue and I’m
    thinking about doing some basic checks like referrer. But since this
    setup can only proxy single files and isn’t defined according to
    socks-standards or something, I don’t think this should be a real
    problem.
  • I tried different regexes and sticked with this, since I tried some
    configs where I needed the host. Thanks for reminding me to change this
    back! :slight_smile:
  • I only need to proxy little images (<10k). Do you know a way to permit
    only those sizes for nginx to proxy?

Again, thank you very much for your answer!
Hendrik

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,121104,121779#msg-121779

Hi,

I don’t know think there’s a way of limiting the response size from
proxied content, but there may be.

Also, I just noticed the proxy_max_temp_file_size, which if you set to 0
disables using temporary files for proxying, so my previous comments
about problems with large file sizes/streamed content can be ignored.

Marcus.

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs