Forum: NGINX 98: Address already in use

2974d09ac2541e892966b762aad84943?d=identicon&s=25 slowgary (Guest)
on 2011-04-14 14:04
(Received via mailing list)
Hi all,

Nginx has been working fine for months on my server. Recently, however,
I see that "service nginx configtest" reports the following:

the configuration file /etc/nginx/nginx.conf syntax is ok
[emerg]: listen() to 0.0.0.0:17942, backlog 511 failed (98: Address
already in use)
configuration file /etc/nginx/nginx.conf test failed


I believe nginx has been updated recently, but I'm not sure.  It's
currently at version 0.8.53.  I've ran configtest before and have never
seen this error.  I need to add a new site in conf.d, but I fear that
reloading or restarting afterwards will fail and my live sites will be
down.

netstat -tlnp lists 0.0.0.0:17942 as in use by Program name "nginx.conf"
(which seems weird, shouldn't this just say "nginx").  Also, if I rename
the conf.d/site.conf file that includes the 17942 port entry, it just
fails on the next conf down the line on the following ip/port.

Now that I've typed out the entire problem, it occurs to me that the
problem could be the misnamed Program name.  Maybe nginx thinks that a
different program is occupying those ports because the name is not
"nginx".  Does that seem likely?  Also, was there a bug fixed somewhere
that resolved/caused the name change?  I have a hunch that restarting
nginx will solve the problem, but I don't want to risk it since it's a
live server.

Any ideas?  Thank you for reading.

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,191227,191227#msg-191227
A8108a0961c6087c43cda32c8616dcba?d=identicon&s=25 Maxim Dounin (Guest)
on 2011-04-14 15:59
(Received via mailing list)
Hello!

On Thu, Apr 14, 2011 at 08:04:03AM -0400, slowgary wrote:

> Hi all,
>
> Nginx has been working fine for months on my server. Recently, however,
> I see that "service nginx configtest" reports the following:
>
> the configuration file /etc/nginx/nginx.conf syntax is ok
> [emerg]: listen() to 0.0.0.0:17942, backlog 511 failed (98: Address
> already in use)
> configuration file /etc/nginx/nginx.conf test failed

Could you please provide details about OS you use?

EADDRINUSE should occur on bind() syscall earlier, and it's
correctly handled there (ignored during configuration testing, and
no listen() call done).

> I believe nginx has been updated recently, but I'm not sure.  It's
> currently at version 0.8.53.  I've ran configtest before and have never
> seen this error.  I need to add a new site in conf.d, but I fear that
> reloading or restarting afterwards will fail and my live sites will be
> down.

Reload/restart will likely succeed - listen() shouldn't be even
called in your situation normally.  It's not clear why bind()
doesn't report the same error though, most likely something
OS-related.

> nginx will solve the problem, but I don't want to risk it since it's a
> live server.

Name reported by netstat is completely unrelated (and it's not
under nginx control).

Maxim Dounin
2974d09ac2541e892966b762aad84943?d=identicon&s=25 slowgary (Guest)
on 2011-04-14 16:21
(Received via mailing list)
The OS is Centos 5.6 (Final)

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,191227,191272#msg-191272
2974d09ac2541e892966b762aad84943?d=identicon&s=25 slowgary (Guest)
on 2011-04-14 17:15
(Received via mailing list)
I don't think it has anything to do with my configs, since things have
been working correctly for a long time.  Just in case though, you can
see them here:

nginx.conf
http://pastebin.com/bJeur7Bq

mydomain.conf
http://pastebin.com/rAMuKgug

Thanks again.

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,191227,191291#msg-191291
C4121c306f8f0c6505033103b3aeeeb7?d=identicon&s=25 "Hüseyin Mert" <huseyinin@gmail.com> (Guest)
on 2011-04-14 19:47
(Received via mailing list)
Hello,
could you stop nginx and check whats running on port 80?
by:*
lsof -i :80*
or
netstat -antp|grep 80
--
Hseyin Mert
http://www.hmert.com
2974d09ac2541e892966b762aad84943?d=identicon&s=25 slowgary (Guest)
on 2011-04-14 20:20
(Received via mailing list)
"netstat -antp | grep 80" output can be found at:
http://pastebin.com/gtBUPckV

Thanks.

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,191227,191349#msg-191349
2974d09ac2541e892966b762aad84943?d=identicon&s=25 slowgary (Guest)
on 2011-04-14 20:21
(Received via mailing list)
Sorry, I forgot to mention that I did not stop nginx.  There are live
sites hosted on this server.  My ultimate goal here is to avoid
downtime, otherwise I would definitely restart nginx to see what
happens.

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,191227,191350#msg-191350
526b09f2b949aae346eda86df5827b1f?d=identicon&s=25 "António P. P. Almeida" <appa@perusio.net> (Guest)
on 2011-04-14 20:40
(Received via mailing list)
On 14 Abr 2011 19h21 WEST, nginx-forum@nginx.us wrote:

> Sorry, I forgot to mention that I did not stop nginx.  There are
> live sites hosted on this server.  My ultimate goal here is to avoid
> downtime, otherwise I would definitely restart nginx to see what
> happens.

There's no downtime if you restart it properly: See
http://wiki.nginx.org/NginxCommandLine#Upgrading_T...

--- appa
0f7a1240e82f744c6c607fa7081b99f7?d=identicon&s=25 Igor Sysoev (Guest)
on 2011-04-15 08:31
(Received via mailing list)
On Thu, Apr 14, 2011 at 08:04:03AM -0400, slowgary wrote:
>
>
> Now that I've typed out the entire problem, it occurs to me that the
> problem could be the misnamed Program name.  Maybe nginx thinks that a
> different program is occupying those ports because the name is not
> "nginx".  Does that seem likely?  Also, was there a bug fixed somewhere
> that resolved/caused the name change?  I have a hunch that restarting
> nginx will solve the problem, but I don't want to risk it since it's a
> live server.

Could you show output of
ps ax -o pid,ppid,%cpu,vsz,wchan,command|egrep '(nginx|PID)'


--
Igor Sysoev
2974d09ac2541e892966b762aad84943?d=identicon&s=25 slowgary (Guest)
on 2011-04-15 11:19
(Received via mailing list)
PID  PPID %CPU    VSZ WCHAN  COMMAND
 1867     1  0.0  14248 rt_sig nginx: master process /usr/sbin/nginx -c
/etc/nginx/nginx.conf
22231  1867  0.2  18704 -      nginx: worker process
22232  1867  0.2  27852 -      nginx: worker process
32653 32619  0.0   1932 pipe_w egrep (nginx|PID)


Thanks Igor.

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,191227,191456#msg-191456
A8108a0961c6087c43cda32c8616dcba?d=identicon&s=25 Maxim Dounin (Guest)
on 2011-04-15 18:18
(Received via mailing list)
Hello!

On Thu, Apr 14, 2011 at 10:20:31AM -0400, slowgary wrote:

> The OS is Centos 5.6 (Final)

Looking though linux code and testing on the exactly same centos -
it seems that linux may indeed return EADDRINUSE on listen().

But as far as I see the only situation when it may appear is two
bind()'s on conflicting ports followed by two listen()'s.  Second
listen() call will fail with EADDRINUSE.

This shouldn't happen in nginx unless you are running something
with 3rd party patches (or there is some other programm doing
bind()/listen() on the same ports, but this is racy and really
unlikely to be reproduceable).

Could you please show full config (with all listen's - the only
you've provided seems not contain ports you see in complaints at
all...), "nginx -V" output and "nginx -t" output (this should be
identical to "service nginx configtest", but just in case)?

Maxim Dounin
2974d09ac2541e892966b762aad84943?d=identicon&s=25 slowgary (Guest)
on 2011-04-18 19:23
(Received via mailing list)
Thanks Maxim.

I installed nginx from the EPEL repo, I don't know of any 3rd party
patches.  Is it common for 3rd party patched versions to exist in a
repo?  Also, my current configuration has worked fine for months.  I
believe this problem may have started after upgrading from CentOS 5.5 to
5.6.

=======================================================================
nginx -V output:

nginx version: nginx/0.8.53
built by gcc 4.1.2 20080704 (Red Hat 4.1.2-48)
TLS SNI support disabled
configure arguments: --user=nginx --group=nginx
--prefix=/usr/share/nginx --sbin-path=/usr/sbin/nginx
--conf-path=/etc/nginx/nginx.conf
--error-log-path=/var/log/nginx/error.log
--http-log-path=/var/log/nginx/access.log
--http-client-body-temp-path=/var/lib/nginx/tmp/client_body
--http-proxy-temp-path=/var/lib/nginx/tmp/proxy
--http-fastcgi-temp-path=/var/lib/nginx/tmp/fastcgi
--http-uwsgi-temp-path=/var/lib/nginx/tmp/uwsgi
--http-scgi-temp-path=/var/lib/nginx/tmp/scgi
--pid-path=/var/run/nginx.pid --lock-path=/var/lock/subsys/nginx
--with-http_ssl_module --with-http_realip_module
--with-http_addition_module --with-http_xslt_module
--with-http_image_filter_module --with-http_geoip_module
--with-http_sub_module --with-http_dav_module --with-http_flv_module
--with-http_gzip_static_module --with-http_random_index_module
--with-http_secure_link_module --with-http_degradation_module
--with-http_stub_status_module --with-http_perl_module --with-mail
--with-file-aio --with-mail_ssl_module --with-ipv6 --with-cc-opt='-O2 -g
-pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector
--param=ssp-buffer-size=4 -m32 -march=i386 -mtune=generic
-fasynchronous-unwind-tables' --with-cc-opt='-O2 -g -pipe -Wall
-Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector
--param=ssp-buffer-size=4 -m32 -march=i386 -mtune=generic
-fasynchronous-unwind-tables'


=======================================================================
nginx -t output:

the configuration file /etc/nginx/nginx.conf syntax is ok
[emerg]: listen() to 0.0.0.0:17942, backlog 511 failed (98: Address
already in use)
configuration file /etc/nginx/nginx.conf test failed



=======================================================================
configs

fastcgi_params
http://pastebin.com/bEYxpKhs


nginx.conf
http://pastebin.com/MX3aLQk4


phpmyadmin.conf
http://pastebin.com/UKvq4xPY


mydomain1.conf
http://pastebin.com/r96E1Nmm


mydomain2.conf
http://pastebin.com/Gku1irY1


shared_rewrite_rules
http://pastebin.com/633reaxP




Thank you for your help.

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,191227,192126#msg-192126
2974d09ac2541e892966b762aad84943?d=identicon&s=25 slowgary (Guest)
on 2011-04-19 15:15
(Received via mailing list)
Last night I stopped and started nginx and it resolved the issue.

nginx -t
the configuration file /etc/nginx/nginx.conf syntax is ok
configuration file /etc/nginx/nginx.conf test is successful


It really seems that somehow nginx couldn't realize that nginx itself
was bound to those addresses.

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,191227,192320#msg-192320
A8108a0961c6087c43cda32c8616dcba?d=identicon&s=25 Maxim Dounin (Guest)
on 2011-04-19 17:21
(Received via mailing list)
Hello!

On Tue, Apr 19, 2011 at 09:14:48AM -0400, slowgary wrote:

> Last night I stopped and started nginx and it resolved the issue.
>
> nginx -t
> the configuration file /etc/nginx/nginx.conf syntax is ok
> configuration file /etc/nginx/nginx.conf test is successful
>
>
> It really seems that somehow nginx couldn't realize that nginx itself
> was bound to those addresses.

As I already said - the problem is that EADDRINUSE error was
returned on listen() call for some reason, not on bind() as it
should.  This is itself POSIX violation, but I believe we don't
mind adding Linux-specific workaround if the problem is common and
understood.

So the real question is "how this may happen?".  So far I wasn't
able to reproduce it even on exactly the same OS.  Please let us
know if you will be able to reproduce it.

Maxim Dounin
2974d09ac2541e892966b762aad84943?d=identicon&s=25 Peleke (Guest)
on 2013-10-11 20:28
(Received via mailing list)
I have the same problem and even if I stop nginx with /etc/init.d/nginx
stop
it is running on Debian Wheezy:

  PID  PPID %CPU    VSZ WCHAN  COMMAND
 2709     1  0.0 127476 ep_pol nginx: worker process
 2710     1  0.0 127716 ep_pol nginx: worker process
 2711     1  0.0 127476 ep_pol nginx: worker process
 2713     1  0.0 127444 ep_pol nginx: worker process
 2714     1  0.0 125208 ep_pol nginx: cache manager process
 3458  3218  0.0   6392 pipe_w egrep (nginx|PID)

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,191227,243631#msg-243631
Please log in before posting. Registration is free and takes only a minute.
Existing account

NEW: Do you have a Google/GoogleMail, Yahoo or Facebook account? No registration required!
Log in with Google account | Log in with Yahoo account | Log in with Facebook account
No account? Register here.