Forum: NGINX Debugging nginx Load Balancer

I'm using nginx 0.6.36 for a load balancing a cluster of GlassFish Gem.

Is there a debug level or statement that can be added to nginx.conf that
will log the instance serving the request ?

-Arun

Arun Gupta wrote:
> I'm using nginx 0.6.36 for a load balancing a cluster of GlassFish Gem.
> 
> Is there a debug level or statement that can be added to nginx.conf that
> will log the instance serving the request ?

1. nginx should be build with ./configure --with-debug

2. add ip to debug requests to nginx.conf:

events {
   debug_connection 10.10.1.1;
}

Thanks for the response!

nginx was built using --with-debug, here is my complete CLI:

./configure --prefix=/usr/local/nginx --sbin-path=/usr/sbin --with-debug 
--with-http_ssl_module

nginx and GlassFish gem are all running on the same box for my 
experiment, i.e. 127.0.0.1. So I updated the existing events entry as:

events {
    worker_connections  1024;
    debug_connection 127.0.0.1;
}

Saw no new messages in access.log and lots of messages in error.log but 
no information on the port being servicing the request.

Am I missing something ?

-Arun

Anton Yuzhaninov wrote:
> Arun Gupta wrote:
>> I'm using nginx 0.6.36 for a load balancing a cluster of GlassFish Gem.
>> 
>> Is there a debug level or statement that can be added to nginx.conf that
>> will log the instance serving the request ?
> 
> 1. nginx should be build with ./configure --with-debug
> 
> 2. add ip to debug requests to nginx.conf:
> 
> events {
>    debug_connection 10.10.1.1;
> }

Arun Gupta wrote:
> events {
>     worker_connections  1024;
>     debug_connection 127.0.0.1;
> }
> 
> Saw no new messages in access.log and lots of messages in error.log but 
> no information on the port being servicing the request.
> 

This should be in error.log

May be I misunderstood your question, but if you want to know upstream 
port, you may add to
access_log format variable $upstream_addr

http://wiki.nginx.org/NginxHttpUpstreamModule#.24upstream_addr
http://wiki.nginx.org/NginxHttpLogModule

> May be I misunderstood your question, but if you want to know upstream 
> port, you may add to
> access_log format variable $upstream_addr
> 
> http://wiki.nginx.org/NginxHttpUpstreamModule#.24upstream_addr
> http://wiki.nginx.org/NginxHttpLogModule
Ok, that took care of logging.

I created a cluster of 3 backend servers and the access.log now shows 
...

127.0.0.1 - [127.0.0.1:3002] - [29/Apr/2009:11:03:43 -0700] GET /runlogs 
HTTP/1.1 "200" 621 "-" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 
10_5_6; en-us) AppleWebKit/525.27.1 (KHTML, like Gecko) Version/3.2.1 
Safari/525.27.1" "-"
127.0.0.1 - [127.0.0.1:3000] - [29/Apr/2009:11:03:43 -0700] GET 
/stylesheets/scaffold.css?1240977992 HTTP/1.1 "200" 889 
"http://localhost/runlogs" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 
10_5_6; en-us) AppleWebKit/525.27.1 (KHTML, like Gecko) Version/3.2.1 
Safari/525.27.1" "-"
127.0.0.1 - [127.0.0.1:3001] - [29/Apr/2009:11:03:43 -0700] GET 
/favicon.ico HTTP/1.1 "200" 0 "http://localhost/runlogs" "Mozilla/5.0 
(Macintosh; U; Intel Mac OS X 10_5_6; en-us) AppleWebKit/525.27.1 
(KHTML, like Gecko) Version/3.2.1 Safari/525.27.1" "-"

The port is shown as the second parameter in each log line, that's good! 
But I expected the request to be served by only one instance, why all 3 
?

-Arun

Arun Gupta wrote:
> /favicon.ico HTTP/1.1 "200" 0 "http://localhost/runlogs" "Mozilla/5.0 
> (Macintosh; U; Intel Mac OS X 10_5_6; en-us) AppleWebKit/525.27.1 
> (KHTML, like Gecko) Version/3.2.1 Safari/525.27.1" "-"
> 
> The port is shown as the second parameter in each log line, that's good! 
> But I expected the request to be served by only one instance, why all 3 

It seems to be 3 different requests and they was served by different 
upstreams because of load
balancing  upstreams.

>> The port is shown as the second parameter in each log line, that's good! 
>> But I expected the request to be served by only one instance, why all 3 
> 
> It seems to be 3 different requests and they was served by different 
> upstreams because of load
> balancing  upstreams.
Interesting ...

If I curl the page then I see only one log entry but invoking the page 
from the browser still shows 3 log entries.

Any idea why 3 requests are made from the browser ?

-Arun

On Wed, 2009-04-29 at 20:42 +0200, Arun Gupta wrote:
> 
> Any idea why 3 requests are made from the browser ?

Most browsers will use several separate connections per domain for
downloading separate elements (Firefox defaults to 8 for normal
connections, 2 for persistent connections).  Your keepalive settings for
Nginx can affect this number.

I believe wget can only use one under all circumstances.

Cliff

> Any idea why 3 requests are made from the browser ?
>
> -Arun

It looks to me like three requests were made, because it's getting
three different resources:

    GET /runlogs
    GET /stylesheets/scaffold.css?1240977992
    GET /favicon.ico

Probably because the HTML at /runlogs refers to the stylesheet and
icon, and the browser goes and gets them to display them. No?

Ok that explains part of it!

curl (wget on Mac) is indeed using single request. I set 
keepalive_timeout to 0 but still see 3 requests being made.

-Arun

Cliff Wells wrote:
> On Wed, 2009-04-29 at 20:42 +0200, Arun Gupta wrote:
>> 
>> Any idea why 3 requests are made from the browser ?
> 
> Most browsers will use several separate connections per domain for
> downloading separate elements (Firefox defaults to 8 for normal
> connections, 2 for persistent connections).  Your keepalive settings for
> Nginx can affect this number.
> 
> I believe wget can only use one under all circumstances.
> 
> Cliff

> It looks to me like three requests were made, because it's getting
> three different resources:
> 
>     GET /runlogs
>     GET /stylesheets/scaffold.css?1240977992
>     GET /favicon.ico
> 
> Probably because the HTML at /runlogs refers to the stylesheet and
> icon, and the browser goes and gets them to display them. No?
Ok, that explains it!

I guess now static file caching needs to be enabled which will reduce 
the trip to back-end servers, right ?

-Arun

You need a block that will cause nginx to serve static files directly. 
Something like


    location ~* ^.+\.(css|js)$ {
        root /var/www/foo;
        access_log off;
        expires max;
        break;
    }

Arun Gupta wrote:
> I'm using nginx 0.6.36 for a load balancing a cluster of GlassFish Gem.
> 
> Is there a debug level or statement that can be added to nginx.conf that
> will log the instance serving the request ?
> 
> -Arun

you should switch over to a load master 2500 because the load master
2500 is an essential component of high-availability, clustering and
fault tolerance, all of which provide the infrastructure for reliable
Internet sites and corporate intranets.

<a 
href="http://www.kemptechnologies.com/load-balancer-2500.shtml?utm_source=ruby&utm_medium=pv&utm_term=mp&utm_campaign=home">http://www.kemptechnologies.com/load-balancer-2500.shtml</a>

Arun Gupta wrote:
> I'm using nginx 0.6.36 for a load balancing a cluster of GlassFish Gem.
> 
> Is there a debug level or statement that can be added to nginx.conf that
> will log the instance serving the request ?
> 
> -Arun

you should switch over to a load master 2500 because the load master
2500 is an essential component of high-availability, clustering and
fault tolerance, all of which provide the infrastructure for reliable
Internet sites and corporate intranets.

http://www.kemptechnologies.com/load-balancer-2500.shtml</a>