Proxy_pass not passing to dynamic $host

Hi,

I need to do proxy_pass to host name passed in url and rewrite url as
well.
Since the host name is difference with each request, I cannot provide an
upstream for it. Below is the nginx configuration I am using but it
doesnt
do proxy pass and returns 404 error. The hostname resembles ec2…com.

location ~* ^(/ec2…*)$ {

try_files $uri $uri/index.html;

rewrite ^(/ec2…*)$ https://example.com:8080/test last;

proxy_read_timeout 300;
proxy_connect_timeout 300;
proxy_redirect off;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://:8080$1/test;

}

Thanks.

Posted at Nginx Forum:

On Thu, Oct 31, 2013 at 07:55:15PM -0400, nehay2j wrote:

Hi there,

I need to do proxy_pass to host name passed in url and rewrite url as well.
Since the host name is difference with each request, I cannot provide an
upstream for it. Below is the nginx configuration I am using but it doesnt
do proxy pass and returns 404 error. The hostname resembles ec2…com.

proxy_pass to a dynamic hostname taken from the request url works for
me.

What is an example request that you make that does not do what you want?

What is the proxy_pass line that that request sees, when you replace
the $variables with their values?

What do the nginx logs say happened?

f

Francis D. [email protected]

Thanks Francis.
I am making a GET call through browser like-
https://example.com/ec2..com

Error Logs-

2013/11/01 01:33:49 [error] 13086#0: *1 no host in upstream
“/ec2-xx-xxx-xxx-xxx…amazonaws.com:8080/test”,
client: 10.10.4.167, server: clarity-test.cloud.tibco.com, request: “GET
/ec2-xx-xx-xxx-xx…amazonaws.com HTT
P/1.1”, host: “example.com

Regards,
Neha

Posted at Nginx Forum:

Thanks Francis. I was able to get past this issue. Appreciate all the
help.

Now I am stuck at forwarding the POST parameters to this proxy server.

proxy_pass http://:8080$1/clarity;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_read_timeout 300;
proxy_connect_timeout 300;
proxy_redirect off;
#proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
#proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

If I uncomment ‘proxy_set_header Host’ and ‘proxy_set_header
X-Forwarded-For’ it ends into 404 error. Otherwise it rewrites and
proxies
fine but doesnt pass the POST parameter.

Regards,
Neha

Posted at Nginx Forum:

On Thu, Oct 31, 2013 at 09:39:54PM -0400, nehay2j wrote:

Hi there,

I am making a GET call through browser like-
https://example.com/ec2..com

So “$1” = “/ec2…com” and the proxy_pass argument is
http:///ec2…com/test

Error Logs-

2013/11/01 01:33:49 [error] 13086#0: *1 no host in upstream
“/ec2-xx-xxx-xxx-xxx…amazonaws.com:8080/test”,
client: 10.10.4.167, server: clarity-test.cloud.tibco.com, request: “GET
/ec2-xx-xx-xxx-xx…amazonaws.com HTT
P/1.1”, host: “example.com

In “http:///ec2…com/test”, the host is between the second and third /,
which is blank, hence “no host in upstream”.

Change something so that the proxy_pass argument is
http://ec2…com/test.

Either use “proxy_pass http:/$1/test”, or keep http://$1/test and remove
the / from $1 by putting it outside the ().

Then try again.

And if the error message indicates “resolver”, see
Module ngx_http_core_module.

f

Francis D. [email protected]

On Fri, Nov 01, 2013 at 12:29:24PM -0400, nehay2j wrote:

Hi there,

Thanks Francis. I was able to get past this issue. Appreciate all the help.

Now I am stuck at forwarding the POST parameters to this proxy server.

proxy_pass http://:8080$1/clarity;

If I uncomment ‘proxy_set_header Host’ and ‘proxy_set_header
X-Forwarded-For’ it ends into 404 error.

I presume that the 404 comes from your upstream server. Use
proxy_set_header to set whatever header values that upstream needs,
to handle the request.

I suspect that “proxy_set_header Host $1;” is what you want – or to
have no proxy_set_header directives at all so that the default applies.

Otherwise it rewrites and proxies
fine but doesnt pass the POST parameter.

It all works fine for me.

Can you provide evidence of where it fails to do what you expect?

Perhaps tcpdump the traffic, and see what does nginx send to upstream?
If
it is the expected correct client request, then you should check what
the upstream does with a similar request that you make using, say, curl.

Provide a specific example of what exactly you do, ideally using “curl”,
and what response you get, and what response you expect. That’s the best
way of making it easier for someone else to help.

f

Francis D. [email protected]

Thanks Francis. I could finally see the post parameters at server end.
Setting proxy_set_header Host $1; changes the browser url which we donot
want. Currently, the code looks like-

    location ~ /(?<ec2instance>[0-9].*) {
            rewrite  $(.*)$ https://$http_host/test last;
            proxy_pass http://$ec2instance:8080/test;
            proxy_set_header Host $http_host;
            proxy_set_header X-Real-IP  $remote_addr;
            proxy_set_header X-Forwarded-For

$proxy_add_x_forwarded_for;

}

If i set ‘proxy_set_header Host $http_host;’ to have $proxy_host
instead of
$http_host, all other things work fine other than cookies. Cookies does
not
get transferred. Having $http_host throws 404 error with the following
entry
in log-

2013/11/01 21:05:37 [error] 17515#0: *1 open() “/opt/nginx/html/test”
failed
(2: No such file or directory), client:
10.10.4.167, server: example.com, request: “GET /testHTTP/1.1”, host:
"example.com
", referrer: “http://example.com/test/test.html

I will try to get the tcpdump of traffic.

Regards,
Neha

Posted at Nginx Forum:

On Fri, Nov 01, 2013 at 09:45:18PM -0400, nehay2j wrote:

Hi there,

I did a curl on the url - curl -i https://example.com/23.23.234.234

HTTP/1.1 302 Found
Location: https://marketplace.example.com/marketplace/marketplace/login

Which is correct.

I confess that I don’t see how the configuration you show below can lead
to that response. But I don’t know what upstream is doing, so maybe it
all just works.

Do you get the same response from upstream if you avoid nginx?

curl -i -H Host:example.com http://23.23.234.234:8080/test

Also, are you aware that a HTTP 302 will cause the browser url to
change? Once the browser gets the above response, the next request goes
to marketplace.example.com and not to example.com.

But when I submit post request through browser, I gives me
a 404 with no error in logs.

If there’s nothing in the logs, that suggests that you weren’t talking
to nginx. Or that your nginx logging level is too low for what you are
trying to see.

What happens when you submit a post using curl, so you can see exactly
what happens without the browser getting in the way?

curl -i -d key=value https://example.com/23.23.234.234

Should i be using rewrite command here or not?

I would have thought “not”, but I don’t know what that rewrite directive
is intended to do in the first place.

If I remove rewrite command, it give file not found error as it picks
default root path always.

Here, you haven’t said what request you made or what result you
expected.

f

Francis D. [email protected]

Hi Francis,

I added rewrite command so that the url doesn’t show IP passed to the
nginx.
Curl gives a 302 because it doesnt have the sessionid with it. If there
is a
session id that is passed to the application running on
http://23.23.234.234:8080/test, it will take us to app. I can see in
application logs that jsessionid does not get there and hence it
redirects
to the login page.

Curl from upstream gave me-
[ec2-user@clarity-test conf]$ curl -i -H
ec2-54-208-198-229.compute-1.amazonaws.com:example.com http:/
/23.23.234.234:8080/test
HTTP/1.1 302 Found
Server: Apache-Coyote/1.1
Location:
https://marketplace-staging.cloud.tibco.com/marketplace/marketplace/login
Content-Length: 0
Date: Sat, 02 Nov 2013 14:13:05 GMT

[ec2-user@clarity-test conf]$ curl -i -d key=value
https://example.com/23.23.234.234
HTTP/1.1 302 Moved Temporarily
Content-Type: text/html
Date: Sat, 02 Nov 2013 14:11:53 GMT
Location: https://clarity-test.cloud.tibco.com:8080/clarity
Server: nginx/1.4.2
Content-Length: 160
Connection: keep-alive

302 Found

302 Found


nginx/1.4.2

If I remove the rewrite command and provides the html page in proxy_pass
i.e. proxy_pass http://:8080$ec2instance/test/test.html, it gets to the
test.html page of the application but does not load any .css .js files
and
says open failed "/opt/nginx/html/test/test.html.

Regards,
Neha

Posted at Nginx Forum:

On Sat, Nov 02, 2013 at 10:21:14AM -0400, nehay2j wrote:

Hi there,

I suspect you’ll get better help from someone else.

I’m unable to work out what it is that you want nginx to do.

I added rewrite command so that the url doesn’t show IP passed to the nginx.

What does that mean, in the problem-report format of “I use this
configuration; I make request A; I get response B; but I expect response
C”?

Curl gives a 302 because it doesnt have the sessionid with it. If there is a
session id that is passed to the application running on
http://23.23.234.234:8080/test, it will take us to app.

What does “take us to app” mean? The problem report should be in the
same format as above.

What happens when you add the sessionid to the curl request?

(The whole point of using curl is to make it easier to see what is
happening. If it doesn’t make it easier, don’t use curl. Instead, find
some other way of clearly describing the unwanted behaviour that you
see.)

If I remove the rewrite command and provides the html page in proxy_pass
i.e. proxy_pass http://:8080$ec2instance/test/test.html, it gets to the
test.html page of the application but does not load any .css .js files and
says open failed "/opt/nginx/html/test/test.html.

The same problem report format will probably help here too.

Each css or js request is a separate http request, so for each one you
can say “I make request A; I get response B; but I expect response C”.

Good luck with it,

f

Francis D. [email protected]

Hi,

I did a curl on the url - curl -i https://example.com/23.23.234.234

HTTP/1.1 302 Found
Date: Sat, 02 Nov 2013 01:37:47 GMT
Location: https://marketplace.example.com/marketplace/marketplace/login
Server: nginx/1.4.2
Content-Length: 0
Connection: keep-alive

Which is correct. But when I submit post request through browser, I
gives me
a 404 with no error in logs. Should i be using rewrite command here or
not?
If I remove rewrite command, it give file not found error as it picks
default root path always.

2013/11/01 20:35:08 [error] 17339#0: *11 open()
“/opt/nginx/html/test/main.html” failed (2: No such file or directory
), client: 10.10.4.167, server:example.com, request: “GET
/test/main.html
HTTP/1.1”, host: “example.com”, referrer:
http://example.com/test_console/test.html

location ~ /(?[0-9].*) {

alias http://:8080$ec2instance/test;

try_files $uri;

           rewrite  $(.*)$ https://$http_host:8080/test break;
            proxy_pass http://$ec2instance:8080/test;
            proxy_read_timeout 300;
            proxy_connect_timeout 300;
            proxy_set_header Host $http_host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For

$proxy_add_x_forwarded_for;
}

Regards,
Neha

Posted at Nginx Forum: