Forum: NGINX Too Many Redirects - CDN Rewrite Rule

I have tried both ways to redirect my static files to a CDN (content
delivery network), but both ways result in the message "too many 
redirects."
Can anyone tell me what I am doing wrong here?

location ~* ^.+.(jpe?g|gif|css|png|js|ico)$ {
       rewrite ^ http://cdn.mydomain.com$request_uri? permanent;
       access_log off;
}


location ~* \.(jpg|jpeg|gif|png|flv|mp3|mpg|mpeg|js|css|ico|woff)$ {
       return 301 http://cdn.mydomain.com$request_uri;
       access_log off;
}

Posted at Nginx Forum: 
http://forum.nginx.org/read.php?2,237609,237609#msg-237609

On Wed, 2013-03-20 at 22:47 -0400, toddlahman wrote:
> location ~* \.(jpg|jpeg|gif|png|flv|mp3|mpg|mpeg|js|css|ico|woff)$ {
>        return 301 http://cdn.mydomain.com$request_uri;
>        access_log off;
> }
>
> Posted at Nginx Forum: 
http://forum.nginx.org/read.php?2,237609,237609#msg-237609
>
That should work ok. Are you sure your cdn isn't using the same ruleset?

Steve

It is possible my CDN is using the same ruleset. I am using MaxCDN, and 
they
use Nginx to serve static images. How would I write this ruleset to be
compatible with MaxCDN (aka NetDNA)?

Posted at Nginx Forum: 
http://forum.nginx.org/read.php?2,237609,237612#msg-237612

Why are you trying to rewrite your URLs at all? Why don't you simply 
endure that your HTML or dynamic content references images at 
cdn.mydomain.com?

Sent from my iPhone

On Thu, Mar 21, 2013 at 12:20:53AM -0400, toddlahman wrote:

Hi there,

> It is possible my CDN is using the same ruleset. I am using MaxCDN, and they
> use Nginx to serve static images. How would I write this ruleset to be
> compatible with MaxCDN (aka NetDNA)?

You shouldn't have to care.

If your server is configured to cause a redirect loop, it's your
problem. If their one is, it's their problem.

So: which server is causing the "Too Many Redirects"?

  curl -i http://yourserver/XXico

Do you get a http redirect? What happens when you "curl -i" the returned
Location: value?

Repeat until you can see which server (yourserver or the cdn) is causing
the redirect chain.

Then fix that server.

  f
--
Francis Daly        francis@daoine.org

I talked to NetDNA and they said my redirect would in fact cause a 
redirect
loop. Unfortunately the PHP project I am using the CDN for would take a 
long
time to convert URLs to the CDN for static files so another route may be 
a
better solution.

There has been mention of using proxy_pass to an upstream to pass static
files off to the upstream CDN. I've tried a few different ways to apply
this, but none have worked. Does anyone have a working proxy_pass 
solution
to offload static files to the CDN?

Posted at Nginx Forum: 
http://forum.nginx.org/read.php?2,237609,237651#msg-237651

On 21 March 2013 19:13, toddlahman <nginx-forum@nginx.us> wrote:
> I talked to NetDNA and they said my redirect would in fact cause a redirect
> loop. Unfortunately the PHP project I am using the CDN for would take a long
> time to convert URLs to the CDN for static files so another route may be a
> better solution.
>
> There has been mention of using proxy_pass to an upstream to pass static
> files off to the upstream CDN. I've tried a few different ways to apply
> this, but none have worked. Does anyone have a working proxy_pass solution
> to offload static files to the CDN?

Francis already told you what to do. Please follow his instructions
and report back how you get on, where you get stuck, or what your
final working config looks like so others can learn from it.

Jonathan
--
Jonathan Matthews // Oxford, London, UK
http://www.jpluscplusm.com/contact.html

On Thu, Mar 21, 2013 at 03:13:43PM -0400, toddlahman wrote:

Hi there,

> I talked to NetDNA and they said my redirect would in fact cause a redirect
> loop. Unfortunately the PHP project I am using the CDN for would take a long
> time to convert URLs to the CDN for static files so another route may be a
> better solution.

I'm a bit confused about how your configuration can have any affect on
someone else's CDN.

If you want to fetch "file.js" from the CDN, what url should you use?

Perhaps I misunderstand what NetDNA actually do.

  f
--
Francis Daly        francis@daoine.org

As I already said, the code doesn't work with NetDNA, so there is 
nothing to
fix. A different approach must be taken, and I am looking for solutions
other than that which is already broken.

Posted at Nginx Forum: 
http://forum.nginx.org/read.php?2,237609,237655#msg-237655

On 21 March 2013 19:39, toddlahman <nginx-forum@nginx.us> wrote:
> As I already said, the code doesn't work with NetDNA, so there is nothing to
> fix. A different approach must be taken, and I am looking for solutions
> other than that which is already broken.

You haven't explained *why* it's broken, or what you've done to
troubleshoot the problem.
You need to follow the instructions that Francis wrote earlier. Show
us the output of the curl invocations. Tell us what your CDN support
have told you about how you /should/ configure this - even if just
conceptually, and not in nginx-config language.

There isn't a single nginx "CDN solution" because there isn't just one
"CDN". They all implement their services differently, and those
differences will dictate how you solve this problem. Hence you need to
tell us everything they've told /you/.

Jonathan
--
Jonathan Matthews // Oxford, London, UK
http://www.jpluscplusm.com/contact.html

The reply I received from NetDNA after supplying the same information as 
I
did here is as follows:

"Too many redirects" is a legit message in this scenario - example: You 
are
redirecting domain.com/file.jpg TO cdn.domain.com/file.jpg ---> request
comes to CDN and CDN neds to cache this file from origin so it tries to
fetch from origin from location "domain.com/file.jpg" ---> request comes 
to
origin and redirect rule you made redirects this request back to CDN 
<---
this is where infinite loop starts.

This is not a proper way to implement CDN as you have to monitor who is
requesting your origin file so you could know whether it's a request you
want to redirect or not. The best way to handle this is to monitor our
anycast IPs and redirect everything except for those ips:

108.161.176.0/20
70.39.132.0/24
92.60.240.208/29
92.60.240.217/29
216.12.211.60/32
216.12.211.59/32

If you want to implement CDN this way, we can't support that 
implementation
as we don't really encourage our client to use this type of 
implementation.
The reason is quite simple: We are going to add new IP block that has to 
be
white listed and until you make update for your redirects rule you'll be
pushing infinite redirects to our servers causing service not to work 
until
you add new block as well.

Posted at Nginx Forum: 
http://forum.nginx.org/read.php?2,237609,237661#msg-237661

What netdna said is sensible and I imagine any cdn would say the same. 
Ultimately the ball is in your court.

If you want to use a CDN (and it's not compulsory) then change your app 
so that the image links are absolute links with the cdn domain name. 
There's no good reason for nginx to have any part of this

Sent from my iPhone

On Thu, Mar 21, 2013 at 06:48:43PM -0400, toddlahman wrote:

Hi there,

> The reply I received from NetDNA after supplying the same information as I
> did here is as follows:
>
> "Too many redirects" is a legit message in this scenario - example: You are
> redirecting domain.com/file.jpg TO cdn.domain.com/file.jpg ---> request
> comes to CDN and CDN neds to cache this file from origin so it tries to
> fetch from origin from location "domain.com/file.jpg" ---> request comes to
> origin and redirect rule you made redirects this request back to CDN <---
> this is where infinite loop starts.

Ah, okay, that makes sense. You don't upload to the CDN; it instead
mirrors your content on demand. (That was the part I had missed.)

As was mentioned, the correct solution is for all of your links to
actually be to things like http://cdn.domain.com/file.jpg; but as was
also mentioned, that is not a quick change for you.

The NetDNA suggestion -- to serve when it comes from them and to 
redirect
when it comes from others -- is reasonable; but as they also indicate,
"when it comes from them" is non-trivial to get right, and will be a
problem if you ever get it wrong in one direction.

Using proxy_pass in your nginx would defeat the main purpose of using
a CDN, as your own bandwidth would be used always.

An alternative possibility could be for you to set up another
server{} block with a server_name of (say) cdn-src.domain.com which
has the same document root as your main one, but which just serves
the static files.  Then, at the point where your CDN is configured to
map cdn.domain.com/file.jpg -> domain.com/file.jpg, map it instead to
cdn-src.domain.com/file.jpg.

With no change to your application, your users go to 
domain.com/something
and follow a link to domain.com/file.jpg; your "main" server redirects
them to cdn.domain.com/file.jpg; they get that, which fetches (now
successfully) cdn-src.domain.com/file.jpg and returns it to the user.

Does that sound like it might do what you want, without taking too
much effort to keep synchronised?

  f
--
Francis Daly        francis@daoine.org

_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx