# Nginx segfault around ~600 requests per second (reverse proxy) **URL:** \ **Category:** NGINX\ **Created:** [January 26, 2010, 12:35am UTC](https://www.ruby-forum.com/t/nginx-segfault-around-600-requests-per-second-reverse-proxy/182282 "2010-01-26T00:35:46Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![yomuppet](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/y/b782af/32.png) [@yomuppet](https://www.ruby-forum.com/u/yomuppet)\ **Post date:** [January 26, 2010, 12:35am UTC](https://www.ruby-forum.com/t/nginx-segfault-around-600-requests-per-second-reverse-proxy/182282/1 "2010-01-26T00:35:46Z")
Hello, I have noticed that we seem to consistently get an nginx segault when reaching around 600 requests per second (reverse proxy). Here is the nginx build information: $ nginx -V nginx version: nginx/0.8.29 built by gcc 4.1.2 20080704 (Red Hat 4.1.2-44) TLS SNI support disabled configure arguments: –add-module=…/agentzh-chunkin-nginx-module-cb610a5 –with-http\_ssl\_module --with-http\_realip\_module –with-http\_addition\_module --with-http\_sub\_module –with-http\_dav\_module --with-http\_flv\_module –with-http\_gzip\_static\_module --with-http\_stub\_status\_module –with-http\_perl\_module Here is the segfault information: Jan 17 12:51:31 util1 kernel: nginx[27888]: segfault at 0000000000000008 rip 0000000000429061 rsp 00007fffcc260500 error 6 Jan 24 16:21:33 util1 kernel: nginx[3835]: segfault at 0000000000000008 rip 0000000000429061 rsp 00007fffcc260550 error 6 Jan 24 16:22:24 util1 kernel: nginx[8956]: segfault at 0000000000000008 rip 0000000000429061 rsp 00007fffcc260550 error 6 Jan 24 16:22:26 util1 kernel: nginx[8982]: segfault at 0000000000000008 rip 0000000000429061 rsp 00007fffcc260550 error 6 Jan 24 16:22:27 util1 kernel: nginx[9026]: segfault at 0000000000000008 rip 0000000000429061 rsp 00007fffcc260550 error 6 Does anyone know why this might be occurring, or any helpful information we can provide to better track it down? Thanks very much in advance. J Posted at Nginx Forum: > **[nginx segfault around ~600 requests per second (reverse proxy)](https://forum.nginx.org/read.php?2,45615,45615#msg-45615)** > > Hello, I have noticed that we seem to consistently get an nginx segault when reaching around 600 requests per second (reverse proxy). Here is the nginx build information: $ nginx -V nginx version: nginx/0.8.29 built by gcc 4.1.2 20080704 (Red Hat 4. ---
**Author:** ![yomuppet](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/y/b782af/32.png) [@yomuppet](https://www.ruby-forum.com/u/yomuppet)\ **Post date:** [January 26, 2010, 1:34am UTC](https://www.ruby-forum.com/t/nginx-segfault-around-600-requests-per-second-reverse-proxy/182282/2 "2010-01-26T01:34:18Z")
Hi, yomuppet wrote: > Have you tried compiling without the chunkin module? AFAIK it’s still experimental. > Here is the segfault information: > Jan 17 12:51:31 util1 kernel: nginx[27888]: segfault at 0000000000000008 rip 0000000000429061 rsp 00007fffcc260500 error 6 > Jan 24 16:21:33 util1 kernel: nginx[3835]: segfault at 0000000000000008 rip 0000000000429061 rsp 00007fffcc260550 error 6 > Jan 24 16:22:24 util1 kernel: nginx[8956]: segfault at 0000000000000008 rip 0000000000429061 rsp 00007fffcc260550 error 6 > Jan 24 16:22:26 util1 kernel: nginx[8982]: segfault at 0000000000000008 rip 0000000000429061 rsp 00007fffcc260550 error 6 > Jan 24 16:22:27 util1 kernel: nginx[9026]: segfault at 0000000000000008 rip 0000000000429061 rsp 00007fffcc260550 error 6 > > Does anyone know why this might be occurring, or any helpful information we can provide to better track it down? If compiling without the chunkin module doesn’t isolate the problem, perform a backtrace. Marcus. ---
**Author:** ![yomuppet](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/y/b782af/32.png) [@yomuppet](https://www.ruby-forum.com/u/yomuppet)\ **Post date:** [January 26, 2010, 4:52am UTC](https://www.ruby-forum.com/t/nginx-segfault-around-600-requests-per-second-reverse-proxy/182282/3 "2010-01-26T04:52:08Z")
On Tue, Jan 26, 2010 at 7:34 AM, yomuppet [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > Does anyone know why this might be occurring, or any helpful information we can provide to better track it down? It _may_ be a bug in ngx\_chunkin. I’ve just sent you a mail off-list explaining how to produce a backtrace for me to investigate this issue 🙂 Thanks for reporting it! -agentzh ---
**Author:** ![yomuppet](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/y/b782af/32.png) [@yomuppet](https://www.ruby-forum.com/u/yomuppet)\ **Post date:** [March 3, 2010, 3:15am UTC](https://www.ruby-forum.com/t/nginx-segfault-around-600-requests-per-second-reverse-proxy/182282/4 "2010-03-03T03:15:21Z")
Hello, Sorry for the delayed reply, it has been taking longer to hit a segfault (which is good), but still hitting some. Here is some backtrace output: (gdb) bt #0 0x0000003967cd3443 in \_\_epoll\_wait\_nocancel () from /lib64/libc.so.6 #1 0x000000000042890d in ngx\_epoll\_process\_events (cycle=0xc1e1050, timer=, flags=1) at src/event/modules/ngx\_epoll\_module.c:530 #2 0x0000000000421f08 in ngx\_process\_events\_and\_timers (cycle=0xc1e1050) at src/event/ngx\_event.c:245 #3 0x0000000000427740 in ngx\_worker\_process\_cycle (cycle=0xc1e1050, data=) at src/os/unix/ngx\_process\_cycle.c:795 #4 0x0000000000426087 in ngx\_spawn\_process (cycle=0xc1e1050, proc=0x427678 , data=0x0, name=0x46f721 “worker process”, respawn=2) at src/os/unix/ngx\_process.c:196 #5 0x0000000000427f3a in ngx\_master\_process\_cycle (cycle=0xc1e1050) at src/os/unix/ngx\_process\_cycle.c:612 #6 0x000000000040edfa in main (argc=22, argv=0xc1dff30) at src/core/nginx.c:399 (gdb) bt #0 0x0000003967cd3443 in \_\_epoll\_wait\_nocancel () from /lib64/libc.so.6 #1 0x000000000042890d in ngx\_epoll\_process\_events (cycle=0xc1e1050, timer=, flags=1) at src/event/modules/ngx\_epoll\_module.c:530 #2 0x0000000000421f08 in ngx\_process\_events\_and\_timers (cycle=0xc1e1050) at src/event/ngx\_event.c:245 #3 0x0000000000427740 in ngx\_worker\_process\_cycle (cycle=0xc1e1050, data=) at src/os/unix/ngx\_process\_cycle.c:795 #4 0x0000000000426087 in ngx\_spawn\_process (cycle=0xc1e1050, proc=0x427678 , data=0x0, name=0x46f721 “worker process”, respawn=2) at src/os/unix/ngx\_process.c:196 #5 0x0000000000427f3a in ngx\_master\_process\_cycle (cycle=0xc1e1050) at src/os/unix/ngx\_process\_cycle.c:612 #6 0x000000000040edfa in main (argc=22, argv=0xc1dff30) at src/core/nginx.c:399 (gdb) c Continuing. Program received signal SIGSEGV, Segmentation fault. ngx\_linux\_sendfile\_chain (c=0x2b31298aab70, in=0xc7bd768, limit=2147479551) at src/os/unix/ngx\_linux\_sendfile\_chain.c:131 131 iov-\>iov\_len += (size\_t) size; (gdb) bt #0 ngx\_linux\_sendfile\_chain (c=0x2b31298aab70, in=0xc7bd768, limit=2147479551) at src/os/unix/ngx\_linux\_sendfile\_chain.c:131 #1 0x000000000041186a in ngx\_chain\_writer (data=, in=) at src/core/ngx\_output\_chain.c:657 #2 0x0000000000412056 in ngx\_output\_chain (ctx=0xc7bd0b0, in=0x0) at src/core/ngx\_output\_chain.c:205 #3 0x000000000044540c in ngx\_http\_upstream\_send\_request (r=0xca55610, u=0xc7bd020) at src/http/ngx\_http\_upstream.c:1321 #4 0x0000000000445ef8 in ngx\_http\_upstream\_send\_request\_handler (r=0x0, u=0x0) at src/http/ngx\_http\_upstream.c:1422 #5 0x000000000044399f in ngx\_http\_upstream\_handler (ev=) at src/http/ngx\_http\_upstream.c:874 #6 0x00000000004220be in ngx\_event\_process\_posted (cycle=, posted=0x0) at src/event/ngx\_event\_posted.c:39 #7 0x0000000000421f80 in ngx\_process\_events\_and\_timers (cycle=0xc1e1050) at src/event/ngx\_event.c:272 #8 0x0000000000427740 in ngx\_worker\_process\_cycle (cycle=0xc1e1050, data=) at src/os/unix/ngx\_process\_cycle.c:795 #9 0x0000000000426087 in ngx\_spawn\_process (cycle=0xc1e1050, proc=0x427678 , data=0x0, name=0x46f721 “worker process”, respawn=2) at src/os/unix/ngx\_process.c:196 #10 0x0000000000427f3a in ngx\_master\_process\_cycle (cycle=0xc1e1050) at src/os/unix/ngx\_process\_cycle.c:612 #11 0x000000000040edfa in main (argc=22, argv=0xc1dff30) at src/core/nginx.c:399 (gdb) c Continuing. p iov p prev p cl-\>buf p cl-\>buf-\>pos up detach Program received signal SIGSEGV, Segmentation fault. ngx\_linux\_sendfile\_chain (c=0x2b21152bed40, in=0x48bfaf8, limit=2147479551) at src/os/unix/ngx\_linux\_sendfile\_chain.c:131 131 iov-\>iov\_len += (size\_t) size; (gdb) p iov $1 = (struct iovec \*) 0x0 (gdb) p prev $2 = (u\_char \*) 0x0 (gdb) p cl-\>buf Cannot access memory at address 0x0 (gdb) p cl-\>buf-\>pos Cannot access memory at address 0x0 (gdb) up #1 0x000000000041186a in ngx\_chain\_writer (data=, in=) at src/core/ngx\_output\_chain.c:657 657 ctx-\>out = c-\>send\_chain(c, ctx-\>out, ctx-\>limit); (gdb) detach LND: Sending signal 11 to Thread 0x2b2114c6b750 (LWP 21815) Detaching from program: /opt/nginx/sbin/nginx, process 21815 If anyone has further insight, would be much appreciated. Thanks. J Posted at Nginx Forum: > **[Re: nginx segfault around ~600 requests per second (reverse proxy)](https://forum.nginx.org/read.php?2,45615,59278#msg-59278)** > > Hello, I have noticed that we seem to consistently get an nginx segault when reaching around 600 requests per second (reverse proxy). Here is the nginx build information: $ nginx -V nginx version: nginx/0.8.29 built by gcc 4.1.2 20080704 (Red Hat 4.