How do I disable DNS Caching and DNS Reverse Lookup in Nginx?

Hello all

Ive been assigned a task to set up an ipv6 to ipv4 reverse proxy for my
company. I decided to use nginx to do the job. I found the following
article
online which describes how to configure nginx as a reverse proxy :

So this is how i set up my reverse proxy. First off I installed RHEL 6.5
on
a VM and installed nginx on it. Second off I set up an AAAA record in
our
DNS as a test FQDN so that I could use that FQDN to connect through the
proxy to an IPV4 website. For example, the FQDN is ipv6.mycoolsite.com
and
the IPv4 website is www.yourcoolsite.com. I set up the default.conf file
as
such:

server {
listen [::]:80 default ipv6only=on;
server_name ipv6.mycoolsite.com;

#charset koi8-r;
access_log /var/log/nginx/log/ipv6.mycoolsite.com.access.log  main;
error_log  /var/log/nginx/log/ipv6.mycoolsite.com.error.log;
location / {
   # root   /usr/share/nginx/html;
   # index  index.html index.htm;
proxy_pass    http://www.yourcoolsite.com;
proxy_redirect   default;
proxy_set_header   X-Real-Host        $host;
proxy_set_header   X-Real-IP   $remote_addr;
proxy_set_header   X-Forwarded-Host   $host;
proxy_set_header   X-Forwarded-Server $host;
proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;
proxy_read_timeout 120;

    }

}

Here are the issues that I am currently having:

When I run the nginx service and I test the FQDN on an ipv6 enabled
computer, I am able to access the IPV4 website www.yourcoolsite.com. But
when I change the proxy_pass FQDN to a different IPV4 website in the
config
file and reload the service, ipv6.mycoolsite.com still connects to
www.yourcoolsite.com and not to the new IPV4 FQDN. I think its loading a
cached copy of www.yourcoolsite.com instead of loading the new IPV4
FQDN.
When it finallly does load the new site, it does so REALLY slowly. I
think
this is due to reverse DNS lookup occuring!

Now what I am trying to figure out here is what is causing the caching
to
occur and the slow loading times? How do I go about disabling DNS
caching as
well as the reverse DNS lookup? I want to be able to connect the IPV4
website specified in the default.conf file when ever I change the file
and
reload the service. I dont want to connect to a cached copy of the
previous
IPV4 entry !

any help will be greatly appreciated!!

Oh and when I check the access logs after I test the proxy, this is what
I
see:

- - [29/Dec/2013:01:31:13 -0500] "GET /commonspot/javascript/lightbox/window_ref.js HTTP/1.1" 200 11198 "http://ipv6.mycoolsite.com/" "Mozilla/5.0 (X11; Linux i686; rv:17.0) Gecko/20131023 Firefox/17.0" "-" - - [29/Dec/2013:01:31:13 -0500] "GET /commonspot/javascript/util.js HTTP/1.1" 200 64891 "http://ipv6.mycoolsite.com/" "Mozilla/5.0 (X11; Linux i686; rv:17.0) Gecko/20131023 Firefox/17.0" "-" - - [29/Dec/2013:01:31:13 -0500] "GET /commonspot/javascript/lightbox/lightbox.js HTTP/1.1" 200 59730 "http://ipv6.mycoolsite.com/" "Mozilla/5.0 (X11; Linux i686; rv:17.0) Gecko/20131023 Firefox/17.0" "-" - - [29/Dec/2013:01:31:14 -0500] "GET /global/images/chrome/logos/slogan.png HTTP/1.1" 404 8839 "http://ipv6.mycoolsite.com/global/css/style.css" "Mozilla/5.0 (X11; Linux i686; rv:17.0) Gecko/20131023 Firefox/17.0" "-" - - [29/Dec/2013:01:31:14 -0500] "GET /common/commonspot/templates/images/chrome/bg/results-bottom.png HTTP/1.1" 200 669 "http://ipv6.mycoolsite.com/" "Mozilla/5.0 (X11; Linux i686; rv:17.0) Gecko/20131023 Firefox/17.0" "-" - - [29/Dec/2013:01:31:14 -0500] "GET /images/2013Dec5.jpg HTTP/1.1" 404 8849 "http://ipv6.mycoolsite.com/" "Mozilla/5.0 (X11; Linux i686; rv:17.0) Gecko/20131023 Firefox/17.0" "-" - - [29/Dec/2013:01:31:14 -0500] "GET /images/2013Dec1.jpg HTTP/1.1" 404 8840 "http://ipv6.mycoolsite.com/" "Mozilla/5.0 (X11; Linux i686; rv:17.0) Gecko/20131023 Firefox/17.0" "-" - - [29/Dec/2013:01:31:14 -0500] "GET /images/2013Dec2.jpg HTTP/1.1" 404 8847 "http://ipv6.mycoolsite.com/" "Mozilla/5.0 (X11; Linux i686; rv:17.0) Gecko/20131023 Firefox/17.0" "-" - - [29/Dec/2013:01:31:14 -0500] "GET /images/2013Dec4.jpg HTTP/1.1" 404 8850 "http://ipv6.mycoolsite.com/" "Mozilla/5.0 (X11; Linux i686; rv:17.0) Gecko/20131023 Firefox/17.0" "-" - - [29/Dec/2013:01:32:08 -0500] "GET /images/2013Dec3.jpg HTTP/1.1" 404 8842 "http://ipv6.mycoolsite.com/" "Mozilla/5.0 (X11; Linux i686; rv:17.0) Gecko/20131023 Firefox/17.0" "-"

Why am I getting a 404 response in the log entry?

Also here is the error log

2013/12/27 13:13:01 [error] 6138#0: *248 upstream timed out (110:
Connection
timed out) while connecting to upstream, client:
xxxx:xxxx:x:xxxx::xxx:xxxx,
server: ipv6.mycoolsite.com, request: “GET
/commonspot/javascript/lightbox/lightbox.js HTTP/1.1”, upstream:
“http://[2001:1900:2302:2000::ff]:80/commonspot/javascript/lightbox/lightbox.js”,
host: “ipv6.mycoolsite.com”, referrer:
http://ipv6.mycoolsite.com/index.htm
2013/12/27 13:43:08 [error] 6138#0: *276 upstream timed out (110:
Connection
timed out) while connecting to upstream, client:
xxxx:xxxx:x:xxxx::xxx:xxxx,
server: ipv6.mycoolsite.com, request: “GET /index.htm HTTP/1.1”,
upstream:
“http://[2001:1900:2302:2000::ff]:80/index.htm”, host:
ipv6.mycoolsite.com
2013/12/29 01:14:03 [error] 13140#0: *402 upstream timed out (110:
Connection timed out) while connecting to upstream, client:
xxxx:xxxx:x:xxxx::xxx:xxxx, server: ipv6.mycoolsite.com, request: “GET
/global/js/libs/validation-engine.css HTTP/1.1”, upstream:
“http://[2001:1900:2300:1::ff]:80/global/js/libs/validation-engine.css”,
host: “ipv6.mycoolsite.com”, referrer: “http://ipv6.mycoolsite.com/
2013/12/29 01:14:03 [error] 13140#0: *406 upstream timed out (110:
Connection timed out) while connecting to upstream, client:
xxxx:xxxx:x:xxxx::xxx:xxxx, server: ipv6.mycoolsite.com, request: “GET
/global/js/jquery.scrollTo-min.js HTTP/1.1”, upstream:
“http://[2001:1900:2300:1::ff]:80/global/js/jquery.scrollTo-min.js”,
host:
ipv6.mycoolsite.com”, referrer: “http://ipv6.mycoolsite.com/
2013/12/29 01:14:03 [error] 13140#0: *410 upstream timed out (110:
Connection timed out) while connecting to upstream, client:
xxxx:xxxx:x:xxxx::xxx:xxxx, server: ipv6.mycoolsite.com, request: “GET
/global/js/libs/always-include-ie.js HTTP/1.1”, upstream:
“http://[2001:1900:2300:1::ff]:80/global/js/libs/always-include-ie.js”,
host: “ipv6.mycoolsite.com”, referrer: “http://ipv6.mycoolsite.com/
2013/12/29 01:14:04 [error] 13140#0: *404 upstream timed out (110:
Connection timed out) while connecting to upstream, client:
xxxx:xxxx:x:xxxx::xxx:xxxx, server: ipv6.mycoolsite.com, request: “GET
/images/2013Dec2.jpg HTTP/1.1”, upstream:
“http://[2001:1900:2300:1::ff]:80/images/2013Dec2.jpg”, host:
ipv6.mycoolsite.com”, referrer: “http://ipv6.mycoolsite.com/
2013/12/29 01:14:04 [error] 13140#0: *408 upstream timed out (110:
Connection timed out) while connecting to upstream, client:
xxxx:xxxx:x:xxxx::xxx:xxxx, server: ipv6.mycoolsite.com, request: “GET
/images/2013Dec4.jpg HTTP/1.1”, upstream:
“http://[2001:1900:2300:1::ff]:80/images/2013Dec4.jpg”, host:
ipv6.mycoolsite.com”, referrer: “http://ipv6.mycoolsite.com/
2013/12/29 01:15:34 [error] 13140#0: *410 upstream timed out (110:
Connection timed out) while connecting to upstream, client:
xxxx:xxxx:x:xxxx::xxx:xxxx, server: ipv6.mycoolsite.com, request: “GET
/global/css/colorbox.css HTTP/1.1”, upstream:
“http://[2001:1900:2300:1::ff]:80/global/css/colorbox.css”, host:
ipv6.mycoolsite.com”, referrer: “http://ipv6.mycoolsite.com/
2013/12/29 01:25:57 [error] 13140#0: *472 upstream timed out (110:
Connection timed out) while connecting to upstream, client:
xxxx:xxxx:x:xxxx::xxx:xxxx, server: ipv6.mycoolsite.com, request: “GET
/global/js/libs/intercept-include.js HTTP/1.1”, upstream:
“http://[2001:1900:2300:1::ff]:80/global/js/libs/intercept-include.js”,
host: “ipv6.mycoolsite.com”, referrer: “http://ipv6.mycoolsite.com/
2013/12/29 01:32:07 [error] 13140#0: *510 upstream timed out (110:
Connection timed out) while connecting to upstream, client:
xxxx:xxxx:x:xxxx::xxx:xxxx, server: ipv6.mycoolsite.com, request: “GET
/images/2013Dec3.jpg HTTP/1.1”, upstream:
“http://[2001:1900:2300:1::ff]:80/images/2013Dec3.jpg”, host:
ipv6.mycoolsite.com”, referrer: “http://ipv6.mycoolsite.com/

I had to blank out the IPV6 address for privacy’s sake. Also i have no
idea
how to paste code properly in mailing lists! lol

Sorry I am a bit new web servers so any help will be greatly
appreciated!

Posted at Nginx Forum:

Hello!

On Sun, Dec 29, 2013 at 01:08:18PM -0500, linuxr00lz2013 wrote:

[…]

occur and the slow loading times? How do I go about disabling DNS caching as
well as the reverse DNS lookup? I want to be able to connect the IPV4
website specified in the default.conf file when ever I change the file and
reload the service. I dont want to connect to a cached copy of the previous
IPV4 entry !

any help will be greatly appreciated!!

Most likely, what you are seeing is your browser’s caching. Try
cleaning your browser’s cache.


Maxim D.
http://nginx.org/

Hello Happy New year and thank you for the reply!

I dont think thats the cause, because I tried clearing the cache and it
was
still stlow! Is there a special directive that I have to use to get it
to
stop caching?

Posted at Nginx Forum:

Hello!

On Wed, Jan 01, 2014 at 10:54:13AM -0500, linuxr00lz2013 wrote:

Hello Happy New year and thank you for the reply!

I dont think thats the cause, because I tried clearing the cache and it was
still stlow! Is there a special directive that I have to use to get it to
stop caching?

Unfortunately, there is no magic directive “do it all right”.
There is no DNS caching in nginx which survives configuration
reload, and there are no reverse DNS lookups in http module at
all.

Unfortunately, you don’t show us real configuration and real logs,
so basically nobody here can help with debugging, but general tips
are:

  1. Make sure you are testing it right. This basically means
    you’ll have to forget about browsers as they are too complex to be
    usable as testing tools and use telnet or curl for basic tests.
    And make sure to watch logs while doing tests.

  2. Make sure you’ve configured it right. Make sure to understand
    what you write in your configuration, make sure to test what you
    wrote (“nginx -t” is your friend, as well as error log), and avoid
    stupid mistakes like infinite loops. See above for recommended
    testing tools.

  3. Avoid descriptive terms like “really”, “painfully”, “awfully” -
    measure instead. If a request takes 60 milliseconds - it may be
    either really fast or really slow, depeding on use case.
    Moreover, exact numbers are usually help a lot with debugging. If
    something takes 60 seconds - it usually means that there is 60
    second timeout somewhere (one of configure upstream servers can’t
    be reached?).

Happy New Year and happy debugging!


Maxim D.
http://nginx.org/

Hello!

On Mon, Jan 06, 2014 at 12:35:46PM -0500, linuxr00lz2013 wrote:

Hello thank you for your reply!

  1. I have shown you the real configuration and logs. All I changed was the
    FQDN’s because I dont know if I am allowed by my company to post them
    online.

The problem is that it makes configs and logs unusable for the
purpose of tracing typos and dump misconfigurations like proxy
loops.

General recommendation for those who don’t want to show names and
ips in public is to reporoduce a problem in some test environment
instead, and provide real configs and logs from this environment.

  1. Which tests do you recommend I run using telnet and curl? I am not too
    familiar with using curl so any guidance will be greatly appreciated!

Most trivial test is to do something like:

$ time curl -o /dev/null http://example.com

to see if it show the problem (i.e., if it’s slow, and how slow it
is).


Maxim D.
http://nginx.org/

Hello thank you for your reply!

  1. I have shown you the real configuration and logs. All I changed was
    the
    FQDN’s because I dont know if I am allowed by my company to post them
    online.

  2. Which tests do you recommend I run using telnet and curl? I am not
    too
    familiar with using curl so any guidance will be greatly appreciated!

Thanks!

Maxim D. Wrote:

stop caching?

  1. Make sure you are testing it right. This basically means
  2. Avoid descriptive terms like “really”, “painfully”, “awfully” -
    Maxim D.
    http://nginx.org/

nginx mailing list
[email protected]
nginx Info Page

Posted at Nginx Forum: