Proxy_pass and $content_type

Hello all,

We’re wanting to use nginx as a proxy between a varnish and a S3
storage. We’re using debian lenny, nginx version is 0.6.32-3.

What we’re dowing:

server {
location / {
proxy_pass …
if ($content_type !~* “image/”) {
return 403
}
}
}

What happens ? well, 403 for all. We put in log “$content_type”, and saw
it’s set to “-”.

Is it normal? Is there another way to filter by content type?
Our final goal is:
S3 sends either the file if it can find it, or an XML (so a content_type
“text/xml” or smth like that). We don’t want to give the xml, as it
contains S3 bucket name…

If any of you has an idea/fixe/workaround…

Thanks in advance

Best regards,

C.

On Tue, Sep 15, 2009 at 01:40:06PM +0200, Cedric J. wrote:

  return 403

If any of you has an idea/fixe/workaround…
You should use 0.7.62, because

  1. it can cache content instead of varnish,

  2. it has image_filter, that tests that response has valid GIF, PNG,
    JPEG:

    location / {
    proxy_pass …
    image_filter test;
    error_page 415 =403 /403.html;
    }

    location /403.html {
    root path/to/page;
    }

Hello!

On Tue, Sep 15, 2009 at 01:40:06PM +0200, Cedric J. wrote:

  return 403
}

}
}

What happens ? well, 403 for all. We put in log “$content_type”, and saw it’s set to “-”.

You are testing request Content-Type, not response. It’s
unlikely to be set at all for most requests.

Additionally, this check happens at rewrite phase - i.e. before
anything was got from upstream.

Is it normal? Is there another way to filter by content type?
Our final goal is:
S3 sends either the file if it can find it, or an XML (so a content_type “text/xml” or smth like that). We don’t want to give the xml, as it contains S3 bucket name…

I believe S3 responds with appropriate http status code on errors,
and you are able to intercept them via proxy_intercept_errors +
error_page.

See

http://wiki.nginx.org/NginxHttpProxyModule#proxy_intercept_errors

for details.

Maxim D.

Hello,

Well, if it was so simple… No, S3 returns 200 whatever happens. it’s
either an xml with a listing, or the file…
So proxy_intercept_errors won’t work.

Ygor: well, lenny has legacy stable, and there’s no backport… and it
seems that we can’t (or won’t) use other sources than the official
debian ones. Any other way ?
For now, we’re filtering upstream in varnish…

Best regards,

C.

PS: sorry for my later mail (nginx-0.6.32-3: proxy_pass and
$content_type), it was dupple because of my gmail account… you can
forget it -.-

On Tue, 15 Sep 2009 16:33:19 +0400

On Tue, Sep 15, 2009 at 03:05:56PM +0200, Cedric J. wrote:

Hello,

Well, if it was so simple… No, S3 returns 200 whatever happens. it’s either an xml with a listing, or the file…
So proxy_intercept_errors won’t work.

Ygor: well, lenny has legacy stable, and there’s no backport… and it seems that we can’t (or won’t) use other sources than the official debian ones. Any other way ?

No way.

Hmm, for file not found yes. Well, I spoke with my colleague, the big
dill is for directory listing: it returns 200 (that’s correct) and an
XML (which we don’t want to show up). Sooo… if S3 can be configured
so that it won’t show some “default index”, it would be great, but
that’s not the case for now.

I contacted nginx debian maintainer, but I don’t know why, I’m pretty
sure he won’t upgrade lenny’s version. So we’re stuck with some nasty
post-filtering.

Thank you for your time anyway :slight_smile:

Best regards,

C.

Hello!

On Tue, Sep 15, 2009 at 03:05:56PM +0200, Cedric J. wrote:

Hello,

Well, if it was so simple… No, S3 returns 200 whatever happens. it’s either an xml with a listing, or the file…
So proxy_intercept_errors won’t work.

According to their docs they return correct status code at least for
REST interface. Seen here:

http://docs.amazonwebservices.com/AmazonS3/2006-03-01/index.html?UsingRESTError.html

Maxim D.

hmmm, yeah, we didn’t think about that.
It can do the trick, we’ll test it. It can lighten post-filtering,
even if S3 gives directory listing when there’s no trailing /.

Thanks a lot!

Best regards,

C.

Hello!

On Tue, Sep 15, 2009 at 05:53:41PM +0200, Cédric Jeanneret wrote:

Hmm, for file not found yes. Well, I spoke with my colleague, the big
dill is for directory listing: it returns 200 (that’s correct) and an
XML (which we don’t want to show up). Sooo… if S3 can be configured
so that it won’t show some “default index”, it would be great, but
that’s not the case for now.

So the solution is to just disallow directory listings via rewrite
rules. It should be simple enough at least if S3 do not try to
return listings for URL’s without trailing slash:

location / {
    if ($uri ~ "/$") {
        return 403;
    }
    proxy_pass ...
}

or using regex locations:

location / {
    proxy_pass ...
}

location ~ /$ {
    return 403;
}

Maxim D.

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs