How to track down an nginx crash

Payam_C · May 21, 2009, 10:29pm

Hey guys,

Nginx crashed 2 times on 2 of my proxy servers… no errors in logs
and i cant figure out why the crash took place. I do see from my rrd
files that there was about 20mbps of http traffic going to the box at
the time of the crash…

any ideas?
Thanks,

Payam_C · May 21, 2009, 10:46pm

all i can say is… wow.

nginx has never crashed on me. congrats?

i would provide your version info, that might help first off.

Payam_C · May 22, 2009, 12:41am

On Thu, May 21, 2009 at 1:39 PM, Michael S. [email protected]
wrote:

and i cant figure out why the crash took place. I do see from my rrd

nginx/0.7.54 with SSL

Payam_C · May 22, 2009, 1:33am

Igor would probably ask at this point for a debug log.

If this continues to happen I would compile nginx --with-debug and
have a way to log when it crashes if you can. You can limit who
triggers debugging by IP too by using debug_connection, otherwise you
will have immensely large logfiles, but at least you’d possibly be
able to track down the event and have the info Igor would probably be
able to pick apart.

Payam_C · May 22, 2009, 9:37am

On Thu, May 21, 2009 at 01:20:16PM -0700, Payam C. wrote:

Hey guys,

Nginx crashed 2 times on 2 of my proxy servers… no errors in logs
and i cant figure out why the crash took place. I do see from my rrd
files that there was about 20mbps of http traffic going to the box at
the time of the crash…

What do mean by the crash ? If nginx exists on single there is a alert
message in error_log.

Payam_C · May 22, 2009, 2:58am

On Thu, May 21, 2009 at 4:17 PM, Michael S. [email protected]
wrote:

nginx/0.7.54 with SSL

Great advice, ill setup the debugging tonight as im sure the issue
will return in time. I am using nginx as a part of ddos mitigation
system so it really pushes all resources to the limit

Thanks again

Payam_C · May 26, 2009, 5:55am

2009/5/22 Igor S. [email protected]:

message in error_log.

–
Igor S.
Igor Sysoev

Hey Igor,

Nginx process crashed without leaving any traces in the system logs or
error.log
Im going to rebuild with debug in there but I still an more sure why
it crashed (2 different boxes… both nginx crashed under 20mbps of
load / 8k-10k request/sec while the other 8 boxes did not)

Unfortunately I have not had much time to resume troubleshooting but
hope to start again in a few days