Forum: NGINX How do I disable DNS Caching and DNS Reverse Lookup in Nginx ?

2974d09ac2541e892966b762aad84943?d=identicon&s=25 linuxr00lz2013 (Guest)
on 2013-12-29 19:09
(Received via mailing list)
Hello all

Ive been assigned a task to set up an ipv6 to ipv4 reverse proxy for my
company. I decided to use nginx to do the job. I found the following
article
online which describes how to configure nginx as a reverse proxy :

http://www.kutukupret.com/2011/05/02/nginx-as-reve...

So this is how i set up my reverse proxy. First off I installed RHEL 6.5
on
a VM and installed nginx on it. Second off I set up an AAAA record in
our
DNS as a test FQDN so that I could use that FQDN to connect through the
proxy to an IPV4 website. For example, the FQDN is ipv6.mycoolsite.com
and
the IPv4 website is www.yourcoolsite.com. I set up the default.conf file
as
such:

<quote>

server {
    listen      [::]:80 default ipv6only=on;
    server_name  ipv6.mycoolsite.com;

    #charset koi8-r;
    access_log /var/log/nginx/log/ipv6.mycoolsite.com.access.log  main;
    error_log  /var/log/nginx/log/ipv6.mycoolsite.com.error.log;
    location / {
       # root   /usr/share/nginx/html;
       # index  index.html index.htm;
    proxy_pass    http://www.yourcoolsite.com;
    proxy_redirect   default;
    proxy_set_header   X-Real-Host        $host;
    proxy_set_header   X-Real-IP   $remote_addr;
    proxy_set_header   X-Forwarded-Host   $host;
    proxy_set_header   X-Forwarded-Server $host;
    proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;
    proxy_read_timeout 120;

        }

    }

</quote>

Here are the issues that I am currently having:

When I run the nginx service and I test the FQDN on an ipv6 enabled
computer, I am able to access the IPV4 website www.yourcoolsite.com. But
when I change the proxy_pass FQDN to a different IPV4 website in the
config
file and reload the service, ipv6.mycoolsite.com still connects to
www.yourcoolsite.com and not to the new IPV4 FQDN. I think its loading a
cached copy of www.yourcoolsite.com instead of loading the new IPV4
FQDN.
When it finallly does load the new site, it does so REALLY slowly. I
think
this is due to reverse DNS lookup occuring!

Now what I am trying to figure out here is what is causing the caching
to
occur and the slow loading times? How do I go about disabling DNS
caching as
well as the reverse DNS lookup? I want to be able to connect the IPV4
website specified in the default.conf file when ever I change the file
and
reload the service. I dont want to connect to a cached copy of the
previous
IPV4 entry !

any help will be greatly appreciated!!




Oh and when I check the access logs after I test the proxy, this is what
I
see:

<quote>
 - - [29/Dec/2013:01:31:13 -0500] "GET
/commonspot/javascript/lightbox/window_ref.js HTTP/1.1" 200 11198
"http://ipv6.mycoolsite.com/" "Mozilla/5.0 (X11; Linux i686; rv:17.0)
Gecko/20131023 Firefox/17.0" "-"
 - - [29/Dec/2013:01:31:13 -0500] "GET /commonspot/javascript/util.js
HTTP/1.1" 200 64891 "http://ipv6.mycoolsite.com/" "Mozilla/5.0 (X11;
Linux
i686; rv:17.0) Gecko/20131023 Firefox/17.0" "-"
 - - [29/Dec/2013:01:31:13 -0500] "GET
/commonspot/javascript/lightbox/lightbox.js HTTP/1.1" 200 59730
"http://ipv6.mycoolsite.com/" "Mozilla/5.0 (X11; Linux i686; rv:17.0)
Gecko/20131023 Firefox/17.0" "-"
 - - [29/Dec/2013:01:31:14 -0500] "GET
/global/images/chrome/logos/slogan.png HTTP/1.1" 404 8839
"http://ipv6.mycoolsite.com/global/css/style.css" "Mozilla/5.0 (X11;
Linux
i686; rv:17.0) Gecko/20131023 Firefox/17.0" "-"
 - - [29/Dec/2013:01:31:14 -0500] "GET
/common/commonspot/templates/images/chrome/bg/results-bottom.png
HTTP/1.1"
200 669 "http://ipv6.mycoolsite.com/" "Mozilla/5.0 (X11; Linux i686;
rv:17.0) Gecko/20131023 Firefox/17.0" "-"
 - - [29/Dec/2013:01:31:14 -0500] "GET /images/2013Dec5.jpg HTTP/1.1"
404
8849 "http://ipv6.mycoolsite.com/" "Mozilla/5.0 (X11; Linux i686;
rv:17.0)
Gecko/20131023 Firefox/17.0" "-"
 - - [29/Dec/2013:01:31:14 -0500] "GET /images/2013Dec1.jpg HTTP/1.1"
404
8840 "http://ipv6.mycoolsite.com/" "Mozilla/5.0 (X11; Linux i686;
rv:17.0)
Gecko/20131023 Firefox/17.0" "-"
 - - [29/Dec/2013:01:31:14 -0500] "GET /images/2013Dec2.jpg HTTP/1.1"
404
8847 "http://ipv6.mycoolsite.com/" "Mozilla/5.0 (X11; Linux i686;
rv:17.0)
Gecko/20131023 Firefox/17.0" "-"
 - - [29/Dec/2013:01:31:14 -0500] "GET /images/2013Dec4.jpg HTTP/1.1"
404
8850 "http://ipv6.mycoolsite.com/" "Mozilla/5.0 (X11; Linux i686;
rv:17.0)
Gecko/20131023 Firefox/17.0" "-"
 - - [29/Dec/2013:01:32:08 -0500] "GET /images/2013Dec3.jpg HTTP/1.1"
404
8842 "http://ipv6.mycoolsite.com/" "Mozilla/5.0 (X11; Linux i686;
rv:17.0)
Gecko/20131023 Firefox/17.0" "-"
</quote>


Why am I getting a 404 response in the log entry?

Also here is the error log

<quote>

2013/12/27 13:13:01 [error] 6138#0: *248 upstream timed out (110:
Connection
timed out) while connecting to upstream, client:
xxxx:xxxx:x:xxxx::xxx:xxxx,
server: ipv6.mycoolsite.com, request: "GET
/commonspot/javascript/lightbox/lightbox.js HTTP/1.1", upstream:
"http://[2001:1900:2302:2000::ff]:80/commonspot/jav...,
host: "ipv6.mycoolsite.com", referrer:
"http://ipv6.mycoolsite.com/index.htm"
2013/12/27 13:43:08 [error] 6138#0: *276 upstream timed out (110:
Connection
timed out) while connecting to upstream, client:
xxxx:xxxx:x:xxxx::xxx:xxxx,
server: ipv6.mycoolsite.com, request: "GET /index.htm HTTP/1.1",
upstream:
"http://[2001:1900:2302:2000::ff]:80/index.htm", host:
"ipv6.mycoolsite.com"
2013/12/29 01:14:03 [error] 13140#0: *402 upstream timed out (110:
Connection timed out) while connecting to upstream, client:
xxxx:xxxx:x:xxxx::xxx:xxxx, server: ipv6.mycoolsite.com, request: "GET
/global/js/libs/validation-engine.css HTTP/1.1", upstream:
"http://[2001:1900:2300:1::ff]:80/global/js/libs/va...,
host: "ipv6.mycoolsite.com", referrer: "http://ipv6.mycoolsite.com/"
2013/12/29 01:14:03 [error] 13140#0: *406 upstream timed out (110:
Connection timed out) while connecting to upstream, client:
xxxx:xxxx:x:xxxx::xxx:xxxx, server: ipv6.mycoolsite.com, request: "GET
/global/js/jquery.scrollTo-min.js HTTP/1.1", upstream:
"http://[2001:1900:2300:1::ff]:80/global/js/jquery....,
host:
"ipv6.mycoolsite.com", referrer: "http://ipv6.mycoolsite.com/"
2013/12/29 01:14:03 [error] 13140#0: *410 upstream timed out (110:
Connection timed out) while connecting to upstream, client:
xxxx:xxxx:x:xxxx::xxx:xxxx, server: ipv6.mycoolsite.com, request: "GET
/global/js/libs/always-include-ie.js HTTP/1.1", upstream:
"http://[2001:1900:2300:1::ff]:80/global/js/libs/al...,
host: "ipv6.mycoolsite.com", referrer: "http://ipv6.mycoolsite.com/"
2013/12/29 01:14:04 [error] 13140#0: *404 upstream timed out (110:
Connection timed out) while connecting to upstream, client:
xxxx:xxxx:x:xxxx::xxx:xxxx, server: ipv6.mycoolsite.com, request: "GET
/images/2013Dec2.jpg HTTP/1.1", upstream:
"http://[2001:1900:2300:1::ff]:80/images/2013Dec2.jpg", host:
"ipv6.mycoolsite.com", referrer: "http://ipv6.mycoolsite.com/"
2013/12/29 01:14:04 [error] 13140#0: *408 upstream timed out (110:
Connection timed out) while connecting to upstream, client:
xxxx:xxxx:x:xxxx::xxx:xxxx, server: ipv6.mycoolsite.com, request: "GET
/images/2013Dec4.jpg HTTP/1.1", upstream:
"http://[2001:1900:2300:1::ff]:80/images/2013Dec4.jpg", host:
"ipv6.mycoolsite.com", referrer: "http://ipv6.mycoolsite.com/"
2013/12/29 01:15:34 [error] 13140#0: *410 upstream timed out (110:
Connection timed out) while connecting to upstream, client:
xxxx:xxxx:x:xxxx::xxx:xxxx, server: ipv6.mycoolsite.com, request: "GET
/global/css/colorbox.css HTTP/1.1", upstream:
"http://[2001:1900:2300:1::ff]:80/global/css/colorb..., host:
"ipv6.mycoolsite.com", referrer: "http://ipv6.mycoolsite.com/"
2013/12/29 01:25:57 [error] 13140#0: *472 upstream timed out (110:
Connection timed out) while connecting to upstream, client:
xxxx:xxxx:x:xxxx::xxx:xxxx, server: ipv6.mycoolsite.com, request: "GET
/global/js/libs/intercept-include.js HTTP/1.1", upstream:
"http://[2001:1900:2300:1::ff]:80/global/js/libs/in...,
host: "ipv6.mycoolsite.com", referrer: "http://ipv6.mycoolsite.com/"
2013/12/29 01:32:07 [error] 13140#0: *510 upstream timed out (110:
Connection timed out) while connecting to upstream, client:
xxxx:xxxx:x:xxxx::xxx:xxxx, server: ipv6.mycoolsite.com, request: "GET
/images/2013Dec3.jpg HTTP/1.1", upstream:
"http://[2001:1900:2300:1::ff]:80/images/2013Dec3.jpg", host:
"ipv6.mycoolsite.com", referrer: "http://ipv6.mycoolsite.com/"

</quote>

I had to blank out the IPV6 address for privacy's sake. Also i have no
idea
how to paste code properly in mailing lists! lol

Sorry I am a bit new web servers so any help will be greatly
appreciated!

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,245904,245904#msg-245904
A8108a0961c6087c43cda32c8616dcba?d=identicon&s=25 Maxim Dounin (Guest)
on 2013-12-31 00:16
(Received via mailing list)
Hello!

On Sun, Dec 29, 2013 at 01:08:18PM -0500, linuxr00lz2013 wrote:

[...]

> occur and the slow loading times? How do I go about disabling DNS caching as
> well as the reverse DNS lookup? I want to be able to connect the IPV4
> website specified in the default.conf file when ever I change the file and
> reload the service. I dont want to connect to a cached copy of the previous
> IPV4 entry !
>
> any help will be greatly appreciated!!

Most likely, what you are seeing is your browser's caching.  Try
cleaning your browser's cache.

--
Maxim Dounin
http://nginx.org/
2974d09ac2541e892966b762aad84943?d=identicon&s=25 linuxr00lz2013 (Guest)
on 2014-01-01 16:55
(Received via mailing list)
Hello Happy New year and thank you for the reply!

I dont think thats the cause, because I tried clearing the cache and it
was
still stlow! Is there a special directive that I have to use to get it
to
stop caching?

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,245904,245945#msg-245945
A8108a0961c6087c43cda32c8616dcba?d=identicon&s=25 Maxim Dounin (Guest)
on 2014-01-02 03:07
(Received via mailing list)
Hello!

On Wed, Jan 01, 2014 at 10:54:13AM -0500, linuxr00lz2013 wrote:

> Hello Happy New year and thank you for the reply!
>
> I dont think thats the cause, because I tried clearing the cache and it was
> still stlow! Is there a special directive that I have to use to get it to
> stop caching?

Unfortunately, there is no magic directive "do it all right".
There is no DNS caching in nginx which survives configuration
reload, and there are no reverse DNS lookups in http module at
all.

Unfortunately, you don't show us real configuration and real logs,
so basically nobody here can help with debugging, but general tips
are:

1) Make sure you are testing it right.  This basically means
you'll have to forget about browsers as they are too complex to be
usable as testing tools and use telnet or curl for basic tests.
And make sure to watch logs while doing tests.

2) Make sure you've configured it right.  Make sure to understand
what you write in your configuration, make sure to test what you
wrote ("nginx -t" is your friend, as well as error log), and avoid
stupid mistakes like infinite loops.  See above for recommended
testing tools.

3) Avoid descriptive terms like "really", "painfully", "awfully" -
measure instead.  If a request takes 60 milliseconds - it may be
either really fast or really slow, depeding on use case.
Moreover, exact numbers are usually help a lot with debugging.  If
something takes 60 seconds - it usually means that there is 60
second timeout somewhere (one of configure upstream servers can't
be reached?).

Happy New Year and happy debugging!

--
Maxim Dounin
http://nginx.org/
2974d09ac2541e892966b762aad84943?d=identicon&s=25 linuxr00lz2013 (Guest)
on 2014-01-06 18:36
(Received via mailing list)
Hello thank you for your reply!

1) I have shown you the real configuration and logs. All I changed was
the
FQDN's because I dont know if I am allowed by my company to post them
online.

2) Which tests do you recommend I run using telnet and curl? I am not
too
familiar with using curl so any guidance will be greatly appreciated!

Thanks!



Maxim Dounin Wrote:
-------------------------------------------------------
> > stop caching?
> 1) Make sure you are testing it right.  This basically means
> 3) Avoid descriptive terms like "really", "painfully", "awfully" -
> Maxim Dounin
> http://nginx.org/
>
> _______________________________________________
> nginx mailing list
> nginx@nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,245904,246065#msg-246065
A8108a0961c6087c43cda32c8616dcba?d=identicon&s=25 Maxim Dounin (Guest)
on 2014-01-07 03:32
(Received via mailing list)
Hello!

On Mon, Jan 06, 2014 at 12:35:46PM -0500, linuxr00lz2013 wrote:

> Hello thank you for your reply!
>
> 1) I have shown you the real configuration and logs. All I changed was the
> FQDN's because I dont know if I am allowed by my company to post them
> online.

The problem is that it makes configs and logs unusable for the
purpose of tracing typos and dump misconfigurations like proxy
loops.

General recommendation for those who don't want to show names and
ips in public is to reporoduce a problem in some test environment
instead, and provide real configs and logs from this environment.

> 2) Which tests do you recommend I run using telnet and curl? I am not too
> familiar with using curl so any guidance will be greatly appreciated!

Most trivial test is to do something like:

$ time curl -o /dev/null http://example.com

to see if it show the problem (i.e., if it's slow, and how slow it
is).

--
Maxim Dounin
http://nginx.org/
Please log in before posting. Registration is free and takes only a minute.
Existing account

NEW: Do you have a Google/GoogleMail, Yahoo or Facebook account? No registration required!
Log in with Google account | Log in with Yahoo account | Log in with Facebook account
No account? Register here.