# Worker process exited on signal 11

**URL:** <https://www.ruby-forum.com/t/worker-process-exited-on-signal-11/218966>\
**Category:** NGINX\
**Created:** [April 17, 2012, 3:44pm UTC](https://www.ruby-forum.com/t/worker-process-exited-on-signal-11/218966 "2012-04-17T15:44:41Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![dubstep](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/cdc98d/32.png) [@dubstep](https://www.ruby-forum.com/u/dubstep)\
**Post date:** [April 17, 2012, 3:44pm UTC](https://www.ruby-forum.com/t/worker-process-exited-on-signal-11/218966/1 "2012-04-17T15:44:41Z")

</div>

Dear all,

error logs show this error (worker process exited on signal 11). The  
error appears after ~1 hour, then with the half of the worker processes.  
No problems with 1.0-versions of nginx. Problem existed in 1.1.18  
already.

I’m using a custom rpm build with the following additional modules:  
–add-module=/root/rpm/SOURCES/agentzh-headers-more-nginx-module-3580526  
  
–add-module=/root/rpm/SOURCES/FRiCKLE-ngx\_cache\_purge-e34bd2c   
–add-module=/root/rpm/SOURCES/substitutions4nginx-read-only \

Here is the backtrace, any ideas?

Thank you.

2012/04/17 10:16:36 [alert] 20802#0: worker process 20808 exited on  
signal 11 (core dumped)  
2012/04/17 10:16:45 [alert] 20802#0: worker process 20805 exited on  
signal 11 (core dumped)  
2012/04/17 10:17:33 [alert] 20802#0: worker process 23616 exited on  
signal 11 (core dumped)  
2012/04/17 10:18:14 [alert] 20802#0: worker process 20806 exited on  
signal 11 (core dumped)  
2012/04/17 10:22:41 [alert] 20802#0: worker process 23892 exited on  
signal 11 (core dumped)

dmesg:  
nginx[20808]: segfault at 0000000000000020 rip 000000000042a47b rsp  
00007fff79073f80 error 4  
nginx[20805]: segfault at 0000000000000020 rip 000000000042a47b rsp  
00007fff79073f80 error 4  
nginx[23616]: segfault at 0000000000000020 rip 000000000042a47b rsp  
00007fff79073ff0 error 4  
nginx[20806]: segfault at 0000000000000020 rip 000000000042a47b rsp  
00007fff79073f80 error 4  
nginx[23892]: segfault at 0000000000000008 rip 000000000042cbe8 rsp  
00007fff79073f20 error 6

gdb /usr/sbin/nginx /var/local/dumps/core  
GNU gdb 6.6  
Copyright (C) 2006 Free Software Foundation, Inc.  
GDB is free software, covered by the GNU General Public License, and you  
are  
welcome to change it and/or distribute copies of it under certain  
conditions.  
Type “show copying” to see the conditions.  
There is absolutely no warranty for GDB. Type “show warranty” for  
details.  
This GDB was configured as “x86\_64-suse-linux”…  
Using host libthread\_db library “/lib64/libthread\_db.so.1”.  
Reading symbols from /lib64/libpthread.so.0…done.  
Loaded symbols for /lib64/libpthread.so.0  
Reading symbols from /lib64/libcrypt.so.1…done.  
Loaded symbols for /lib64/libcrypt.so.1  
Reading symbols from /usr/lib64/libpcre.so.0…done.  
Loaded symbols for /usr/lib64/libpcre.so.0  
Reading symbols from /usr/lib64/libssl.so.0.9.8…done.  
Loaded symbols for /usr/lib64/libssl.so.0.9.8  
Reading symbols from /usr/lib64/libcrypto.so.0.9.8…done.  
Loaded symbols for /usr/lib64/libcrypto.so.0.9.8  
Reading symbols from /lib64/libdl.so.2…done.  
Loaded symbols for /lib64/libdl.so.2  
Reading symbols from /lib64/libz.so.1…done.  
Loaded symbols for /lib64/libz.so.1  
Reading symbols from /usr/lib64/libGeoIP.so.1…done.  
Loaded symbols for /usr/lib64/libGeoIP.so.1  
Reading symbols from  
/usr/lib/perl5/5.8.8/x86\_64-linux-thread-multi/CORE/libperl.so…done.  
Loaded symbols for  
/usr/lib/perl5/5.8.8/x86\_64-linux-thread-multi/CORE/libperl.so  
Reading symbols from /lib64/libm.so.6…done.  
Loaded symbols for /lib64/libm.so.6  
Reading symbols from /lib64/libc.so.6…done.  
Loaded symbols for /lib64/libc.so.6  
Reading symbols from /lib64/ld-linux-x86-64.so.2…done.  
Loaded symbols for /lib64/ld-linux-x86-64.so.2  
Reading symbols from  
/usr/lib/perl5/site\_perl/5.8.8/x86\_64-linux-thread-multi/auto/nginx/nginx.so…done.  
Loaded symbols for  
/usr/lib/perl5/site\_perl/5.8.8/x86\_64-linux-thread-multi/auto/nginx/nginx.so  
Core was generated by `nginx: worker pr’.  
Program terminated with signal 11, Segmentation fault.  
#0 0x000000000042cbe8 in ngx\_readv\_chain (c=0x2b508a1d43d0,  
chain=0x602348) at src/os/unix/ngx\_readv\_chain.c:195  
195 src/os/unix/ngx\_readv\_chain.c: No such file or directory.  
in src/os/unix/ngx\_readv\_chain.c  
(gdb) bt  
#0 0x000000000042cbe8 in ngx\_readv\_chain (c=0x2b508a1d43d0,  
chain=0x602348) at src/os/unix/ngx\_readv\_chain.c:195  
#1 0x000000000042b133 in ngx\_event\_pipe (p=0x747170, do\_write=) at src/event/ngx\_event\_pipe.c:271  
#2 0x000000000045410e in ngx\_http\_upstream\_process\_upstream  
(r=0x69ef00, u=0x746db8) at src/http/ngx\_http\_upstream.c:2641  
#3 0x0000000000454269 in ngx\_http\_upstream\_handler (ev=0x2b50cc7d9218)  
at src/http/ngx\_http\_upstream.c:935  
#4 0x0000000000428f64 in ngx\_event\_process\_posted (cycle=0x5f6490,  
posted=0x5d14a0) at src/event/ngx\_event\_posted.c:40  
#5 0x000000000042fc36 in ngx\_worker\_process\_cycle (cycle=0x5f6490,  
data=) at src/os/unix/ngx\_process\_cycle.c:806  
#6 0x000000000042e14d in ngx\_spawn\_process (cycle=0x5f6490,  
proc=0x42fb70 \<ngx\_worker\_process\_cycle\>, data=0x0, name=0x49e05e  
“worker process”, respawn=5) at src/os/unix/ngx\_process.c:198  
#7 0x0000000000430c76 in ngx\_master\_process\_cycle (cycle=0x5f6490) at  
src/os/unix/ngx\_process\_cycle.c:622  
#8 0x000000000041235d in main (argc=1, argv=) at  
src/core/nginx.c:410  
(gdb) backtrace full  
#0 0x000000000042cbe8 in ngx\_readv\_chain (c=0x2b508a1d43d0,  
chain=0x602348) at src/os/unix/ngx\_readv\_chain.c:195  
prev = (u\_char \*) 0x0  
n =   
size = 0  
err =   
vec = {elts = 0x7fff79073f20, nelts = 0, size = 16, nalloc = 16,  
pool = 0x5f57a0}  
rev =   
iov = (struct iovec \*) 0x0  
iovs = {{iov\_base = 0x40000, iov\_len = 140735223906344},  
{iov\_base = 0x69ef00, iov\_len = 47624914552144}, {iov\_base = 0x6024c0,  
iov\_len = 262144}, {iov\_base = 0x1, iov\_len = 4566241}, {  
iov\_base = 0x40000, iov\_len = 6942464}, {iov\_base = 0x2b508a1d4550,  
iov\_len = 0}, {iov\_base = 0x0, iov\_len = 47624930312022}, {iov\_base =  
0x69b6e0, iov\_len = 6942464}, {  
iov\_base = 0x2b508a1d4550, iov\_len = 6300864}, {iov\_base = 0x40000,  
iov\_len = 1}, {iov\_base = 0x602460, iov\_len = 4367464}, {iov\_base = 0x1,  
iov\_len = 2}, {iov\_base = 0x0,  
iov\_len = 47624930399933}, {iov\_base = 0x40000, iov\_len = 4309842},  
{iov\_base = 0x2b50cc80e2e8, iov\_len = 4370439}, {iov\_base = 0x2b,  
iov\_len = 2}}  
#1 0x000000000042b133 in ngx\_event\_pipe (p=0x747170, do\_write=) at src/event/ngx\_event\_pipe.c:271  
rc =   
rev =   
wev =   
#2 0x000000000045410e in ngx\_http\_upstream\_process\_upstream  
(r=0x69ef00, u=0x746db8) at src/http/ngx\_http\_upstream.c:2641  
c = (ngx\_connection\_t \*) 0x2b508a1d43d0  
#3 0x0000000000454269 in ngx\_http\_upstream\_handler (ev=0x2b50cc7d9218)  
at src/http/ngx\_http\_upstream.c:935  
c = (ngx\_connection\_t \*) 0x2b508a1d4550  
r = (ngx\_http\_request\_t \*) 0x69ef00  
u = (ngx\_http\_upstream\_t \*) 0x746db8  
#4 0x0000000000428f64 in ngx\_event\_process\_posted (cycle=0x5f6490,  
posted=0x5d14a0) at src/event/ngx\_event\_posted.c:40  
ev = (ngx\_event\_t \*) 0x2b50cc7d9218  
#5 0x000000000042fc36 in ngx\_worker\_process\_cycle (cycle=0x5f6490,  
data=) at src/os/unix/ngx\_process\_cycle.c:806  
i = 0  
c =   
#6 0x000000000042e14d in ngx\_spawn\_process (cycle=0x5f6490,  
proc=0x42fb70 \<ngx\_worker\_process\_cycle\>, data=0x0, name=0x49e05e  
“worker process”, respawn=5) at src/os/unix/ngx\_process.c:198  
on = 1  
pid = 0  
s = 5  
#7 0x0000000000430c76 in ngx\_master\_process\_cycle (cycle=0x5f6490) at  
src/os/unix/ngx\_process\_cycle.c:622  
title =   
p =   
size = 9  
i =   
n = 432  
sigio = 0  
set = {\_\_val = {0 \<repeats 16 times\>}}  
itv = {it\_interval = {tv\_sec = 0, tv\_usec = 6251688}, it\_value =  
{tv\_sec = 0, tv\_usec = 47624916165168}}  
live = 1  
delay = 0  
ccf = (ngx\_core\_conf\_t \*) 0x5f74f0  
#8 0x000000000041235d in main (argc=1, argv=) at  
src/core/nginx.c:410  
i =   
log = (ngx\_log\_t \*) 0x5ce6e0  
cycle = (ngx\_cycle\_t \*) 0x5f6490  
init\_cycle = {conf\_ctx = 0x0, pool = 0x5f5440, log = 0x5ce6e0,  
new\_log = {log\_level = 0, file = 0x0, connection = 0, handler = 0, data  
= 0x0, action = 0x0}, files = 0x0,  
free\_connections = 0x0, free\_connection\_n = 0,  
reusable\_connections\_queue = {prev = 0x0, next = 0x0}, listening = {elts  
= 0x0, nelts = 0, size = 0, nalloc = 0, pool = 0x0}, pathes = {  
elts = 0x0, nelts = 0, size = 0, nalloc = 0, pool = 0x0}, open\_files  
= {last = 0x0, part = {elts = 0x0, nelts = 0, next = 0x0}, size = 0,  
nalloc = 0, pool = 0x0}, shared\_memory = {  
last = 0x0, part = {elts = 0x0, nelts = 0, next = 0x0}, size = 0,  
nalloc = 0, pool = 0x0}, connection\_n = 0, files\_n = 0, connections =  
0x0, read\_events = 0x0, write\_events = 0x0,  
old\_cycle = 0x0, conf\_file = {len = 21, data = 0x4998b3  
“/etc/nginx/nginx.conf”}, conf\_param = {len = 0, data = 0x0},  
conf\_prefix = {len = 11, data = 0x4998b3 “/etc/nginx/nginx.conf”},  
prefix = {len = 6, data = 0x4998ac “/usr//”}, lock\_file = {len = 0,  
data = 0x0}, hostname = {len = 0, data = 0x0}}  
ccf = (ngx\_core\_conf\_t \*) 0x5f74f0

Posted at Nginx Forum:

> **[worker process exited on signal 11](https://forum.nginx.org/read.php?2,225420,225420#msg-225420)**
>
> Dear all, error logs show this error (worker process exited on signal 11). The error appears after ~1 hour, then with the half of the worker processes. No problems with 1.0-versions of nginx. Problem existed in 1.1.18 already. I'm using a custom rpm...

---

<div class="post-metadata">

**Author:** ![mscheel](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/da6949/32.png) [@mscheel](https://www.ruby-forum.com/u/mscheel)\
**Post date:** [April 17, 2012, 4:20pm UTC](https://www.ruby-forum.com/t/worker-process-exited-on-signal-11/218966/2 "2012-04-17T16:20:45Z")

</div>

sorry, forgot uname -a:

Linux hostname 2.6.16.60-0.42.10-smp #1 SMP Tue Apr 27 05:11:27 UTC 2010  
x86\_64 x86\_64 x86\_64 GNU/Linux

nginx -V  
nginx version: nginx/1.1.19  
built by gcc 4.1.2 20070115 (SUSE Linux)  
TLS SNI support disabled  
configure arguments:  
–add-module=/root/rpm/SOURCES/agentzh-headers-more-nginx-module-3580526  
–add-module=/root/rpm/SOURCES/FRiCKLE-ngx\_cache\_purge-e34bd2c  
–add-module=/root/rpm/SOURCES/substitutions4nginx-read-only  
–with-debug --prefix=/usr/ --sbin-path=/usr/sbin/nginx  
–conf-path=/etc/nginx/nginx.conf  
–error-log-path=/var/log/nginx/error\_log  
–http-log-path=/var/log/nginx/access\_log --pid-path=/var/run/nginx.pid  
–lock-path=/var/run/nginx.lock  
–http-client-body-temp-path=/var/lib/nginx/tmp/  
–http-proxy-temp-path=/var/lib/nginx/proxy/  
–http-fastcgi-temp-path=/var/lib/nginx/fastcgi/  
–http-uwsgi-temp-path=/var/lib/nginx/uwsgi/  
–http-scgi-temp-path=/var/lib/nginx/scgi/ --user=wwwrun --group=www  
–with-rtsig\_module --with-select\_module --with-poll\_module --with-ipv6  
–with-http\_ssl\_module --with-http\_realip\_module  
–with-http\_addition\_module --with-http\_geoip\_module  
–with-http\_sub\_module --with-http\_dav\_module --with-http\_flv\_module  
–with-http\_gzip\_static\_module --with-http\_random\_index\_module  
–with-http\_secure\_link\_module --with-http\_degradation\_module  
–with-http\_stub\_status\_module --with-http\_perl\_module  
–with-perl=/usr/bin/perl --with-mail --with-mail\_ssl\_module --with-pcre  
–with-md5=/usr --with-sha1=/usr --with-cc-opt=‘-O2 -g  
-fmessage-length=0 -D\_FORTIFY\_SOURCE=2 -fstack-protector’

Posted at Nginx Forum:

> **[Re: worker process exited on signal 11](https://forum.nginx.org/read.php?2,225420,225433#msg-225433)**
>
> Dear all, error logs show this error (worker process exited on signal 11). The error appears after ~1 hour, then with the half of the worker processes. No problems with 1.0-versions of nginx. Problem existed in 1.1.18 already. I'm using a custom rpm...

---

<div class="post-metadata">

**Author:** ![mscheel](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/da6949/32.png) [@mscheel](https://www.ruby-forum.com/u/mscheel)\
**Post date:** [April 17, 2012, 7:21pm UTC](https://www.ruby-forum.com/t/worker-process-exited-on-signal-11/218966/3 "2012-04-17T19:21:58Z")

</div>

Hello!

On Tue, Apr 17, 2012 at 09:44:13AM -0400, mscheel wrote:

> –add-module=/root/rpm/SOURCES/FRiCKLE-ngx\_cache\_purge-e34bd2c   
> –add-module=/root/rpm/SOURCES/substitutions4nginx-read-only \

Are you able to reproduce this without 3rd party modules?

[…]

> (gdb) bt  
> #0 0x000000000042cbe8 in ngx\_readv\_chain (c=0x2b508a1d43d0,  
> chain=0x602348) at src/os/unix/ngx\_readv\_chain.c:195

Please provide gdb output of the following commands:

fr 0  
p chain  
p \*chain

> #1 0x000000000042b133 in ngx\_event\_pipe (p=0x747170, do\_write=) at src/event/ngx\_event\_pipe.c:271

fr 1  
p \*p

> #2 0x000000000045410e in ngx\_http\_upstream\_process\_upstream  
> (r=0x69ef00, u=0x746db8) at src/http/ngx\_http\_upstream.c:2641

fr 2  
p \*r  
p \*u

[…]

It might be also helpful to see full configuration used.

Maxim D.

---

<div class="post-metadata">

**Author:** ![mscheel](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/da6949/32.png) [@mscheel](https://www.ruby-forum.com/u/mscheel)\
**Post date:** [April 18, 2012, 1:04pm UTC](https://www.ruby-forum.com/t/worker-process-exited-on-signal-11/218966/4 "2012-04-18T13:04:17Z")

</div>

Hello!

On Wed, Apr 18, 2012 at 01:06:25AM -0400, mscheel wrote:

> Hello Maxim,
> 
> it appears only on two productive servers. Compiling without these  
> modules is not possible at the moment but i’ve tried several times with  
> each module deactivated (and without any of these three).
> 
> Full config is huge. I’m using a php-fpm backend with keepalive on and  
> one upstream server with nginx and php-fpm.

[…]

> proxy\_set\_header Connection “”; proxy\_http\_version 1.1;  
> include /etc/nginx/proxy.conf;

This part of the config shows proxy\_pass, while gdb prints  
provided say the request used fastcgi\_pass. Could you please:

1. Check processing flow of the  
“/nc/pdf-generation.html?pageID=776404” uri and provide relevant  
parts of the config. According to gdb it’s transformed to  
“/index.php?pageID=776404”.

2. Check if the problem is reproduceable with the uri in question.

3. Check if the problem persists with the upstream keepalive  
turned off.

It would be also cool to obtain debug log if possible, it should  
be ok to grab one from a particular location with fastcgi\_pass.

And see below from more gdb commands I would like to see output  
from.

> (gdb) p \*chain  
> $2 = {buf = 0x602460, next = 0x752628}

fr 0  
p \*chain-\>buf  
p \*chain-\>next  
p \*chain-\>next-\>buf  
p \*chain-\>next-\>next  
p \*chain-\>next-\>next-\>buf

(repeat last two prints with one more “-\>next” added till it fails)

> (gdb) fr 1  
> #1 0x000000000042b133 in ngx\_event\_pipe (p=0x747170, do\_write=) at src/event/ngx\_event\_pipe.c:271  
> 271 src/event/ngx\_event\_pipe.c: No such file or directory.  
> in src/event/ngx\_event\_pipe.c
> 
> (gdb) p \*p  
> $3 = {upstream = 0x2b508a1d43d0, downstream = 0x2b508a1d4550,  
> free\_raw\_bufs = 0x0, in = 0x6024d0, last\_in = 0x7523a0, out = 0x0, free  
> = 0x602450, busy = 0x0,

fr 1  
p \*p-\>in  
p \*p-\>in-\>buf  
p \*p-\>in-\>next  
p \*p-\>in-\>next-\>buf

(same here, repeat with “-\>next” added till it fails)

p \*p-\>free  
p \*p-\>free-\>buf  
p \*p-\>free-\>next  
p \*p-\>free-\>next-\>buf

(same here)

[…]

Maxim D.

---

<div class="post-metadata">

**Author:** ![mscheel](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/da6949/32.png) [@mscheel](https://www.ruby-forum.com/u/mscheel)\
**Post date:** [April 18, 2012, 7:07am UTC](https://www.ruby-forum.com/t/worker-process-exited-on-signal-11/218966/5 "2012-04-18T07:07:11Z")

</div>

Hello Maxim,

it appears only on two productive servers. Compiling without these  
modules is not possible at the moment but i’ve tried several times with  
each module deactivated (and without any of these three).

Full config is huge. I’m using a php-fpm backend with keepalive on and  
one upstream server with nginx and php-fpm.

http {  
include mime.types;  
include gzip.conf;  
default\_type application/octet-stream;  
keepalive\_timeout 240;  
keepalive\_requests 2000;  
open\_file\_cache max=10000 inactive=3m;  
open\_file\_cache\_valid 3m;  
open\_file\_cache\_min\_uses 2;  
open\_file\_cache\_errors off;  
index wartung.html index.php index.html index.htm;  
recursive\_error\_pages on;

large\_client\_header\_buffers 8 256k;  
client\_max\_body\_size 1024m;  
client\_body\_buffer\_size 128K;  
client\_header\_buffer\_size 10k;  
client\_header\_timeout 120;  
client\_body\_timeout 120;  
fastcgi\_read\_timeout 240;

set\_real\_ip\_from x.x.x.0/24;  
real\_ip\_header X-Forwarded-For;

#nginx 1.0  
#limit\_zone spiders $binary\_remote\_addr 10m;  
#nginx 1.1  
limit\_conn\_zone $binary\_remote\_addr zone=spiders:10m;

limit\_req\_zone $binary\_remote\_addr zone=spider\_req:10m rate=5r/s;  
limit\_conn\_log\_level info;

upstream be\_server\_nginx {  
server IP:80;  
keepalive 300;  
}  
}

…

location @be\_nginx {  
proxy\_pass http://be\_server\_nginx;  
proxy\_set\_header Connection “”; proxy\_http\_version 1.1;  
include /etc/nginx/proxy.conf;

output of gdb:

(gdb) fr 0  
#0 0x000000000042cbe8 in ngx\_readv\_chain (c=0x2b508a1d43d0,  
chain=0x602348) at src/os/unix/ngx\_readv\_chain.c:195  
195 in src/os/unix/ngx\_readv\_chain.c

(gdb) p chain  
$1 = (ngx\_chain\_t \*) 0x602348

(gdb) p \*chain  
$2 = {buf = 0x602460, next = 0x752628}

(gdb) fr 1  
#1 0x000000000042b133 in ngx\_event\_pipe (p=0x747170, do\_write=) at src/event/ngx\_event\_pipe.c:271  
271 src/event/ngx\_event\_pipe.c: No such file or directory.  
in src/event/ngx\_event\_pipe.c

(gdb) p \*p  
$3 = {upstream = 0x2b508a1d43d0, downstream = 0x2b508a1d4550,  
free\_raw\_bufs = 0x0, in = 0x6024d0, last\_in = 0x7523a0, out = 0x0, free  
= 0x602450, busy = 0x0,  
input\_filter = 0x47bc30 \<ngx\_http\_fastcgi\_input\_filter\>, input\_ctx =  
0x69ef00, output\_filter = 0x43ca40 \<ngx\_http\_output\_filter\>, output\_ctx  
= 0x69ef00, read = 0, cacheable = 0,  
single\_buf = 0, free\_bufs = 1, upstream\_done = 0, upstream\_error = 0,  
upstream\_eof = 0, upstream\_blocked = 0, downstream\_done = 0,  
downstream\_error = 0, cyclic\_temp\_file = 0, allocated = 2,  
bufs = {num = 10, size = 262144}, tag = 0x5c8480, busy\_size = 262144,  
read\_length = 378731, length = 1, max\_temp\_file\_size = 0,  
temp\_file\_write\_size = 524288, read\_timeout = 180000,  
send\_timeout = 60000, send\_lowat = 0, pool = 0x69b6e0, log = 0x6c5380,  
preread\_bufs = 0x0, preread\_size = 129547, buf\_to\_file = 0x0, temp\_file  
= 0x602358, num = 6}

(gdb) fr 2  
#2 0x000000000045410e in ngx\_http\_upstream\_process\_upstream  
(r=0x69ef00, u=0x746db8) at src/http/ngx\_http\_upstream.c:2641  
2641 src/http/ngx\_http\_upstream.c: No such file or directory.  
in src/http/ngx\_http\_upstream.c

(gdb) p \*r  
$4 = {signature = 1347703880, connection = 0x2b508a1d4550, ctx =  
0x69baf0, main\_conf = 0x5f7910, srv\_conf = 0x617f48, loc\_conf =  
0x640e00,  
read\_event\_handler = 0x454510  
\<ngx\_http\_upstream\_rd\_check\_broken\_connection\>, write\_event\_handler =  
0x454b50 \<ngx\_http\_upstream\_process\_downstream\>, cache = 0x0, upstream =  
0x746db8,  
upstream\_states = 0x747890, pool = 0x69b6e0, header\_in = 0x6c5450,  
headers\_in = {headers = {last = 0x69ef70, part = {elts = 0x69c008, nelts  
= 11, next = 0x0}, size = 48, nalloc = 20,  
pool = 0x69b6e0}, host = 0x69c0f8, connection = 0x0,  
if\_modified\_since = 0x0, if\_unmodified\_since = 0x0, user\_agent =  
0x69c098, referer = 0x69c038, content\_length = 0x0,  
content\_type = 0x0, range = 0x0, if\_range = 0x0, transfer\_encoding =  
0x0, expect = 0x0, accept\_encoding = 0x69c0c8, via = 0x69c188,  
authorization = 0x0, keep\_alive = 0x0,  
x\_forwarded\_for = 0x69c1b8, x\_real\_ip = 0x0, depth = 0x0,  
destination = 0x0, overwrite = 0x0, date = 0x0, user = {len = 0, data =  
0x0}, passwd = {len = 0, data = 0x0}, cookies = {  
elts = 0x69c3c8, nelts = 1, size = 8, nalloc = 2, pool =  
0x69b6e0}, server = {len = 22, data = 0x74eadb “[domain.com](http://domain.com)”},  
content\_length\_n = -1, keep\_alive\_n = -1,  
connection\_type = 2, msie = 1, msie6 = 0, opera = 0, gecko = 0,  
chrome = 0, safari = 0, konqueror = 0}, headers\_out = {headers = {last =  
0x69f0c8, part = {elts = 0x69b730, nelts = 5,  
next = 0x0}, size = 48, nalloc = 20, pool = 0x69b6e0}, status =  
200, status\_line = {len = 6, data = 0x4a2662 “200 OK”}, server =  
0x69b790, date = 0x0, content\_length = 0x0,  
content\_encoding = 0x69b7f0, location = 0x0, refresh = 0x0,  
last\_modified = 0x0, content\_range = 0x0, accept\_ranges = 0x0,  
www\_authenticate = 0x0, expires = 0x0, etag = 0x0,  
override\_charset = 0x0, content\_type\_len = 9, content\_type = {len =  
24, data = 0x747a0d “text/html; charset=utf-8”}, charset = {len = 5,  
data = 0x747a20 “utf-8”},  
content\_type\_lowcase = 0x602078 “text/html”, content\_type\_hash =  
101824002410317, cache\_control = {elts = 0x6016c8, nelts = 1, size = 8,  
nalloc = 2, pool = 0x69b6e0}, content\_length\_n = -1,  
date\_time = 0, last\_modified\_time = -1}, request\_body = 0x747270,  
lingering\_time = 0, start\_sec = 1334650957, start\_msec = 91, method = 2,  
http\_version = 1001, request\_line = {len = 50,  
data = 0x74e8c0 “GET /nc/pdf-generation.html?pageID=776404  
HTTP/1.1\r\nAccept”}, uri = {len = 10, data = 0x746d62 “/index.php”},  
args = {len = 13, data = 0x746a8f “pageID=776404”}, exten = {  
len = 3, data = 0x746d69 “php”}, unparsed\_uri = {len = 37, data =  
0x74e8c4 “/nc/pdf-generation.html?pageID=776404 HTTP/1.1\r\nAccept”},  
method\_name = {len = 3,  
data = 0x74e8c0 “GET /nc/pdf-generation.html?pageID=776404  
HTTP/1.1\r\nAccept”}, http\_protocol = {len = 8, data = 0x74e8ea  
“HTTP/1.1\r\nAccept”}, out = 0x7523b8, main = 0x69ef00,  
parent = 0x0, postponed = 0x0, post\_subrequest = 0x0, posted\_requests  
= 0x0, virtual\_names = 0x744098, phase\_handler = 13, content\_handler =  
0x47b940 \<ngx\_http\_fastcgi\_handler\>,  
access\_code = 0, variables = 0x69bc98, ncaptures = 0, captures =  
0x69c580, captures\_data = 0x746a84 “/index.php?pageID=776404”,  
limit\_rate = 0, header\_size = 339, request\_length = 785,  
err\_status = 0, http\_connection = 0x6c53d8, log\_handler = 0x442e60  
\<ngx\_http\_log\_error\_handler\>, cleanup = 0x7478f0, subrequests = 201,  
count = 1, blocked = 0, aio = 0, http\_state = 2,  
complex\_uri = 0, quoted\_uri = 0, plus\_in\_uri = 0, space\_in\_uri = 0,  
invalid\_header = 0, add\_uri\_to\_alias = 0, valid\_location = 1,  
valid\_unparsed\_uri = 0, uri\_changed = 0, uri\_changes = 10,  
request\_body\_in\_single\_buf = 0, request\_body\_in\_file\_only = 0,  
request\_body\_in\_persistent\_file = 0, request\_body\_in\_clean\_file = 0,  
request\_body\_file\_group\_access = 0,  
request\_body\_file\_log\_level = 5, subrequest\_in\_memory = 0, waited = 0,  
cached = 0, gzip\_tested = 1, gzip\_ok = 1, gzip\_vary = 1, proxy = 0,  
bypass\_cache = 0, no\_cache = 0, limit\_conn\_set = 1,  
limit\_req\_set = 1, pipeline = 0, plain\_http = 0, chunked = 1,  
header\_only = 0, keepalive = 1, lingering\_close = 0, discard\_body = 0,  
internal = 1, error\_page = 0, ignore\_content\_encoding = 0,  
filter\_finalize = 0, post\_action = 0, request\_complete = 0,  
request\_output = 1, header\_sent = 1, expect\_tested = 0, root\_tested = 0,  
done = 0, logged = 0, buffered = 0,  
main\_filter\_need\_in\_memory = 1, filter\_need\_in\_memory = 1,  
filter\_need\_temporary = 0, allow\_ranges = 0, stat\_reading = 0,  
stat\_writing = 1, state = 0, header\_hash = 6740241951503570450,  
lowcase\_index = 13, lowcase\_header = “cache-controlsitiont”, ‘\0’  
\<repeats 11 times\>,  
header\_name\_start = 0x2b50cc8430b3  
“^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×ÿÐßãߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^”…,  
header\_name\_end = 0x2b50cc8430a8  
“û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×ÿÐßãߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û”…,  
header\_start = 0x2b50cc8430aa  
“uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×ÿÐßãߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯u”…,  
header\_end = 0x2b50cc8430b3  
“^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×ÿÐßãߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^”…,  
uri\_start = 0x74e8c4 “/nc/pdf-generation.html?pageID=776404  
HTTP/1.1\r\nAccept”, uri\_end = 0x74e8e9 " HTTP/1.1\r\nAccept",  
uri\_ext = 0x74e8d7 “html?pageID=776404 HTTP/1.1\r\nAccept”, args\_start  
= 0x74e8dc “pageID=776404 HTTP/1.1\r\nAccept”,  
request\_start = 0x74e8c0 “GET /nc/pdf-generation.html?pageID=776404  
HTTP/1.1\r\nAccept”, request\_end = 0x74e8f2 “\r\nAccept”,  
method\_end = 0x74e8c2 “T /nc/pdf-generation.html?pageID=776404  
HTTP/1.1\r\nAccept”, schema\_start = 0x0, schema\_end = 0x0, host\_start =  
0x0, host\_end = 0x0, port\_start = 0x0, port\_end = 0x0,  
http\_minor = 1, http\_major = 1}

(gdb) p \*u  
$5 = {read\_event\_handler = 0x454090  
\<ngx\_http\_upstream\_process\_upstream\>, write\_event\_handler = 0x452980  
\<ngx\_http\_upstream\_dummy\_handler\>, peer = {connection = 0x2b508a1d43d0,  
sockaddr = 0x617ea8, socklen = 110, name = 0x68a6b0, tries = 1, get  
= 0x487370 \<ngx\_http\_upstream\_get\_keepalive\_peer\>, free = 0x4875b0  
\<ngx\_http\_upstream\_free\_keepalive\_peer\>,  
data = 0x747908, set\_session = 0x486e80  
\<ngx\_http\_upstream\_keepalive\_set\_session\>, save\_session = 0x486e90  
\<ngx\_http\_upstream\_keepalive\_save\_session\>, local = 0x0, rcvbuf = 0,  
log = 0x6c5380, cached = 1, log\_error = 1}, pipe = 0x747170,  
request\_bufs = 0x747880, output = {buf = 0x0, in = 0x0, free = 0x0, busy  
= 0x0, sendfile = 0, directio = 0, unaligned = 0,  
need\_in\_memory = 0, need\_in\_temp = 0, alignment = 512, pool =  
0x69b6e0, allocated = 0, bufs = {num = 1, size = 131072}, tag =  
0x5c8480, output\_filter = 0x414db0 \<ngx\_chain\_writer\>,  
filter\_ctx = 0x746eb0}, writer = {out = 0x0, last = 0x746eb0,  
connection = 0x2b508a1d43d0, pool = 0x69b6e0, limit = 0}, conf =  
0x6416a8, headers\_in = {headers = {last = 0x746ee8, part = {  
elts = 0x601520, nelts = 4, next = 0x0}, size = 48, nalloc = 8,  
pool = 0x69b6e0}, status\_n = 200, status\_line = {len = 6, data =  
0x4a2662 “200 OK”}, status = 0x0, date = 0x0,  
server = 0x0, connection = 0x0, expires = 0x0, etag = 0x0,  
x\_accel\_expires = 0x0, x\_accel\_redirect = 0x0, x\_accel\_limit\_rate = 0x0,  
content\_type = 0x601580, content\_length = 0x601520,  
last\_modified = 0x0, location = 0x0, accept\_ranges = 0x0,  
www\_authenticate = 0x0, transfer\_encoding = 0x0, content\_encoding = 0x0,  
content\_length\_n = 355230, cache\_control = {  
elts = 0x747a38, nelts = 1, size = 8, nalloc = 2, pool =  
0x69b6e0}, connection\_close = 0, chunked = 0}, resolved = 0x0, buffer =  
{  
pos = 0x2b50cc843010  
“¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î”…,  
last = 0x2b50cc843010  
“¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î”…,  
file\_pos = 0, file\_last = 0,  
start = 0x2b50cc843010  
“¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î”…,  
end = 0x2b50cc883010 “”, tag = 0x5c8480, file = 0x0, shadow = 0x0,  
temporary = 1, memory = 0, mmap = 0, recycled = 1, in\_file = 0, flush =  
0, sync = 0,  
last\_buf = 0, last\_in\_chain = 0, last\_shadow = 0, temp\_file = 0, num  
= 2}, length = 355230, out\_bufs = 0x0, busy\_bufs = 0x0, free\_bufs =  
0x0,  
input\_filter\_init = 0x47af80 \<ngx\_http\_fastcgi\_input\_filter\_init\>,  
input\_filter = 0, input\_filter\_ctx = 0x0, create\_key = 0x47c100  
\<ngx\_http\_fastcgi\_create\_key\>,  
create\_request = 0x47c3a0 \<ngx\_http\_fastcgi\_create\_request\>,  
reinit\_request = 0x47af50 \<ngx\_http\_fastcgi\_reinit\_request\>,  
process\_header = 0x47cf20 \<ngx\_http\_fastcgi\_process\_header\>,  
abort\_request = 0x47b560 \<ngx\_http\_fastcgi\_abort\_request\>,  
finalize\_request = 0x47b530 \<ngx\_http\_fastcgi\_finalize\_request\>,  
rewrite\_redirect = 0, rewrite\_cookie = 0, timeout = 0,  
state = 0x7478b8, method = {len = 0, data = 0x0}, schema = {len = 10,  
data = 0x4a8c69 “fastcgi://”}, uri = {len = 0, data = 0x0}, cleanup =  
0x7478f0, store = 0, cacheable = 0, accel = 0,  
ssl = 0, cache\_status = 0, buffering = 1, keepalive = 0, request\_sent  
= 1, header\_sent = 1}

Posted at Nginx Forum:

> **[Re: worker process exited on signal 11](https://forum.nginx.org/read.php?2,225420,225457#msg-225457)**
>
> Dear all, error logs show this error (worker process exited on signal 11). The error appears after ~1 hour, then with the half of the worker processes. No problems with 1.0-versions of nginx. Problem existed in 1.1.18 already. I'm using a custom rpm...

---

<div class="post-metadata">

**Author:** ![mscheel](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/da6949/32.png) [@mscheel](https://www.ruby-forum.com/u/mscheel)\
**Post date:** [April 18, 2012, 1:13pm UTC](https://www.ruby-forum.com/t/worker-process-exited-on-signal-11/218966/6 "2012-04-18T13:13:38Z")

</div>

Hello Maxim, thanks alot for your help. I’ll try to reproduce it.

(gdb) fr 0  
#0 0x000000000042cbe8 in ngx\_readv\_chain (c=0x2b508a1d43d0,  
chain=0x602348) at src/os/unix/ngx\_readv\_chain.c:195  
195 src/os/unix/ngx\_readv\_chain.c: No such file or directory.  
in src/os/unix/ngx\_readv\_chain.c

(gdb) p \*chain-\>buf  
$6 = {pos = 0x0, last = 0x0, file\_pos = 0, file\_last = 0, start = 0x0,  
end = 0x0, tag = 0x5c8480, file = 0x0, shadow = 0x0, temporary = 1,  
memory = 0, mmap = 0, recycled = 1, in\_file = 0,  
flush = 0, sync = 0, last\_buf = 0, last\_in\_chain = 0, last\_shadow = 0,  
temp\_file = 0, num = 0}

(gdb) p \*chain-\>next  
$7 = {buf = 0x7523d8, next = 0x752638}

(gdb) p \*chain-\>next-\>buf  
$8 = {pos = 0x0, last = 0x0, file\_pos = 0, file\_last = 0, start = 0x0,  
end = 0x0, tag = 0x5c8480, file = 0x0, shadow = 0x0, temporary = 1,  
memory = 0, mmap = 0, recycled = 1, in\_file = 0,  
flush = 0, sync = 0, last\_buf = 0, last\_in\_chain = 0, last\_shadow = 0,  
temp\_file = 0, num = 0}

(gdb) p \*chain-\>next-\>next  
$9 = {buf = 0x746ff8, next = 0x0}

(gdb) p \*chain-\>next-\>next-\>buf  
$10 = {  
pos = 0x2b50cc843010  
“¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î”…,  
last = 0x2b50cc843010  
“¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î”…,  
file\_pos = 0, file\_last = 0,  
start = 0x2b50cc843010  
“¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î”…,  
end = 0x2b50cc883010 “”, tag = 0x5c8480, file = 0x0, shadow = 0x0,  
temporary = 1, memory = 0, mmap = 0, recycled = 1, in\_file = 0, flush =  
0, sync = 0,  
last\_buf = 0, last\_in\_chain = 0, last\_shadow = 0, temp\_file = 0, num =  
2}

(gdb) p \*chain-\>next-\>next-\>next  
Cannot access memory at address 0x0  
(gdb) p \*chain-\>next-\>next-\>next-\>buf  
Cannot access memory at address 0x0

* * *

(gdb) fr 1  
#1 0x000000000042b133 in ngx\_event\_pipe (p=0x747170, do\_write=) at src/event/ngx\_event\_pipe.c:271  
271 src/event/ngx\_event\_pipe.c: No such file or directory.  
in src/event/ngx\_event\_pipe.c  
(gdb) p \*p-\>in  
$11 = {buf = 0x7524d8, next = 0x7523a8}  
(gdb) p \*p-\>in-\>buf  
$12 = {pos = 0x2b50cc908010 “½:÷ü\>Gò\223ÿ”, last = 0x2b50cc918008  
“\001\006”, file\_pos = 0, file\_last = 0, start = 0x2b50cc908010  
“½:÷ü\>Gò\223ÿ”, end = 0x2b50cc948010 “”, tag = 0x5c8480,  
file = 0x0, shadow = 0x752528, temporary = 1, memory = 0, mmap = 0,  
recycled = 1, in\_file = 0, flush = 0, sync = 0, last\_buf = 0,  
last\_in\_chain = 0, last\_shadow = 0, temp\_file = 0, num = 4}  
(gdb) p \*p-\>in-\>next  
$13 = {buf = 0x752528, next = 0x752398}  
(gdb) p \*p-\>in-\>next-\>buf  
$14 = {pos = 0x2b50cc918010 “tú|=\036oøG\_ý\227\027ÉïüU)ÿ”, last =  
0x2b50cc91be90 “”, file\_pos = 0, file\_last = 0, start = 0x2b50cc908010  
“½:÷ü\>Gò\223ÿ”, end = 0x2b50cc948010 “”, tag = 0x5c8480,  
file = 0x0, shadow = 0x752478, temporary = 1, memory = 0, mmap = 0,  
recycled = 1, in\_file = 0, flush = 0, sync = 0, last\_buf = 0,  
last\_in\_chain = 0, last\_shadow = 1, temp\_file = 0, num = 4}  
(gdb) p \*p-\>in-\>next-\>next  
$15 = {buf = 0x7525d8, next = 0x0}  
(gdb) p _p-\>in-\>next-\>next-\>buf  
$16 = {  
pos = 0x2b50cc949010  
"gƲdäz\211»\017\027ÜÈ\026\225h^3¦\217?}\221&NG\230N\033ÏÉ.\207¼\222ñ=R\215þí\037\224|É«N\224¬\206Þ\205)J\003%dY\2348\235¥5[UnJ+¶à6&¬pÒz.K\203¼ZG©YJvõ\025óbZM«\227,\016×\236’YrKFoqhN\223\022`L\213z{Ïi,\226?\b\t)\033\217Ó\225V÷\005*÷¥W¶^>ÞU«¢$­ÿ", last = 0x2b50cc9518f0 "", file_pos = 0, file_last = 0, start = 0x2b50cc949010 "gƲdäz\211»\017\027ÜÈ\026\225h^3¦\217?}\221&NG\230N\033ÏÉ.\207¼\222ñ=R\215þí\037\224|É«N\224¬\206Þ\205)J\003%dY\2348\235¥5[UnJ+¶à6&¬pÒz.K\203¼ZG©YJvõ\025óbZM«\227,\016×\236'YrKFoqhN\223\022`L\213z{Ïi,\226?\b\t)\033\217Ó\225V÷\005_÷¥W¶^\>ÞU«¢$­ÿ",  
end = 0x2b50cc989010 “”, tag = 0x5c8480, file = 0x0, shadow = 0x752578,  
temporary = 1, memory = 0, mmap = 0,  
recycled = 1, in\_file = 0, flush = 0, sync = 0, last\_buf = 0,  
last\_in\_chain = 0, last\_shadow = 1, temp\_file = 0, num = 5}  
(gdb) p \*p-\>in-\>next-\>next-\>next  
Cannot access memory at address 0x0

* * *

(gdb) p \*p-\>free  
$17 = {buf = 0x752428, next = 0x752290}  
(gdb) p \*p-\>free-\>buf  
$18 = {  
pos = 0x2b50cc843010  
“¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î”…,  
last = 0x2b50cc843010  
“¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î”…,  
file\_pos = 0, file\_last = 0,  
start = 0x2b50cc843010  
“¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î½×½û¯uï~ëÝ{ߺ÷^÷î”…,  
end = 0x2b50cc883010 “”, tag = 0x5c8480, file = 0x0, shadow = 0x0,  
temporary = 1, memory = 0, mmap = 0, recycled = 1, in\_file = 0, flush =  
0, sync = 0,  
last\_buf = 0, last\_in\_chain = 0, last\_shadow = 0, temp\_file = 0, num =  
2}  
(gdb) p \*p-\>free-\>next  
$19 = {buf = 0x7523d8, next = 0x6024b0}  
(gdb) p \*p-\>free-\>next-\>buf  
$20 = {pos = 0x0, last = 0x0, file\_pos = 0, file\_last = 0, start = 0x0,  
end = 0x0, tag = 0x5c8480, file = 0x0, shadow = 0x0, temporary = 1,  
memory = 0, mmap = 0, recycled = 1, in\_file = 0,  
flush = 0, sync = 0, last\_buf = 0, last\_in\_chain = 0, last\_shadow = 0,  
temp\_file = 0, num = 0}  
(gdb)  
$21 = {pos = 0x0, last = 0x0, file\_pos = 0, file\_last = 0, start = 0x0,  
end = 0x0, tag = 0x5c8480, file = 0x0, shadow = 0x0, temporary = 1,  
memory = 0, mmap = 0, recycled = 1, in\_file = 0,  
flush = 0, sync = 0, last\_buf = 0, last\_in\_chain = 0, last\_shadow = 0,  
temp\_file = 0, num = 0}  
(gdb) p \*p-\>free-\>next-\>next  
$22 = {buf = 0x602460, next = 0x6024c0}  
(gdb) p \*p-\>free-\>next-\>next-\>buf  
$23 = {pos = 0x0, last = 0x0, file\_pos = 0, file\_last = 0, start = 0x0,  
end = 0x0, tag = 0x5c8480, file = 0x0, shadow = 0x0, temporary = 1,  
memory = 0, mmap = 0, recycled = 1, in\_file = 0,  
flush = 0, sync = 0, last\_buf = 0, last\_in\_chain = 0, last\_shadow = 0,  
temp\_file = 0, num = 0}  
(gdb) p \*p-\>free-\>next-\>next-\>next  
$24 = {buf = 0x7510f0, next = 0x0}  
(gdb) p \*p-\>free-\>next-\>next-\>next-\>buf  
$25 = {pos = 0x0, last = 0x0, file\_pos = 0, file\_last = 0, start = 0x0,  
end = 0x0, tag = 0x5c8480, file = 0x0, shadow = 0x0, temporary = 1,  
memory = 0, mmap = 0, recycled = 1, in\_file = 0,  
flush = 0, sync = 0, last\_buf = 0, last\_in\_chain = 0, last\_shadow = 0,  
temp\_file = 0, num = 0}  
(gdb) p \*p-\>free-\>next-\>next-\>next-\>next  
Cannot access memory at address 0x0  
(gdb) p \*p-\>free-\>next-\>next-\>next-\>next-\>buf  
Cannot access memory at address 0x0

Posted at Nginx Forum:

> **[Re: worker process exited on signal 11](https://forum.nginx.org/read.php?2,225420,225474#msg-225474)**
>
> Dear all, error logs show this error (worker process exited on signal 11). The error appears after ~1 hour, then with the half of the worker processes. No problems with 1.0-versions of nginx. Problem existed in 1.1.18 already. I'm using a custom rpm...

---

<div class="post-metadata">

**Author:** ![mscheel](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/da6949/32.png) [@mscheel](https://www.ruby-forum.com/u/mscheel)\
**Post date:** [April 18, 2012, 4:22pm UTC](https://www.ruby-forum.com/t/worker-process-exited-on-signal-11/218966/7 "2012-04-18T16:22:36Z")

</div>

Hello!

On Wed, Apr 18, 2012 at 07:13:02AM -0400, mscheel wrote:

> end = 0x0, tag = 0x5c8480, file = 0x0, shadow = 0x0, temporary = 1,  
> memory = 0, mmap = 0, recycled = 1, in\_file = 0,  
> flush = 0, sync = 0, last\_buf = 0, last\_in\_chain = 0, last\_shadow = 0,  
> temp\_file = 0, num = 0}
> 
> (gdb) p \*chain-\>next-\>next  
> $9 = {buf = 0x746ff8, next = 0x0}

[…]

Ok, thanks. The buffer chain looks completely screwed up, and I  
suspect the culprit is substitutions4nginx module you use. Quick  
look over it’s sources suggests it at least changes the  
last\_shadow flag of the buffers it gets, and this may cause  
undefined behaviour.

Additionally, the response in question has Content-Type set to  
“text/html” (though it looks like the actual response is pdf), which  
is handled by substitutions4nginx by default.

Could you please try to reproduce the problem without the module  
in question?

Maxim D.

---

<div class="post-metadata">

**Author:** ![mscheel](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/da6949/32.png) [@mscheel](https://www.ruby-forum.com/u/mscheel)\
**Post date:** [April 18, 2012, 5:12pm UTC](https://www.ruby-forum.com/t/worker-process-exited-on-signal-11/218966/8 "2012-04-18T17:12:58Z")

</div>

Thanks. Do you think it’s necessary to create a new rpm without this  
module or will it be sufficient to turn it off in the configs?  
I’ll disable the config now.

Posted at Nginx Forum:

> **[Re: worker process exited on signal 11](https://forum.nginx.org/read.php?2,225420,225485#msg-225485)**
>
> Dear all, error logs show this error (worker process exited on signal 11). The error appears after ~1 hour, then with the half of the worker processes. No problems with 1.0-versions of nginx. Problem existed in 1.1.18 already. I'm using a custom rpm...

---

<div class="post-metadata">

**Author:** ![mscheel](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/da6949/32.png) [@mscheel](https://www.ruby-forum.com/u/mscheel)\
**Post date:** [April 18, 2012, 7:28pm UTC](https://www.ruby-forum.com/t/worker-process-exited-on-signal-11/218966/9 "2012-04-18T19:28:46Z")

</div>

Hello!

On Wed, Apr 18, 2012 at 11:12:11AM -0400, mscheel wrote:

> Thanks. Do you think it’s necessary to create a new rpm without this  
> module or will it be sufficient to turn it off in the configs?  
> I’ll disable the config now.

It should be sufficient to disable it in config (that is,  
remove/comment out all “subs\_filter” directives).

Maxim D.

---

<div class="post-metadata">

**Author:** ![mscheel](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/da6949/32.png) [@mscheel](https://www.ruby-forum.com/u/mscheel)\
**Post date:** [April 18, 2012, 1:29pm UTC](https://www.ruby-forum.com/t/worker-process-exited-on-signal-11/218966/10 "2012-04-18T13:29:13Z")

</div>

Interesting, i’m unable to reproduce it with the uri. The uri generates  
a pdf file (when it does not exist on the fs). I’ve deleted it on the fs  
and it’s created fine by our php application. I’ve also made some tests  
with keepalive off, no change.  
Maybe there is something missing in SLES 10 which is required by 1.1? A  
manual configure and make (without install) with my nginx -V runs  
without errors.

My fastcgi\_params:  
set $php\_root /docroot;  
root $php\_root;  
try\_files $uri $uri/ @404;  
fastcgi\_pass php-fpm\_backend;  
fastcgi\_keep\_conn on;  
fastcgi\_index index.php;  
fastcgi\_param SCRIPT\_FILENAME $php\_root$fastcgi\_script\_name;

fastcgi\_param QUERY\_STRING $query\_string;  
fastcgi\_param REQUEST\_METHOD $request\_method;  
fastcgi\_param CONTENT\_TYPE $content\_type;  
fastcgi\_param CONTENT\_LENGTH $content\_length;

fastcgi\_param SCRIPT\_NAME $fastcgi\_script\_name;  
fastcgi\_param REQUEST\_URI $request\_uri;  
fastcgi\_param DOCUMENT\_URI $document\_uri;  
fastcgi\_param DOCUMENT\_ROOT $document\_root;  
fastcgi\_param SERVER\_PROTOCOL $server\_protocol;

fastcgi\_param GATEWAY\_INTERFACE CGI/1.1;

fastcgi\_param REMOTE\_ADDR $HTTP\_X\_FORWARDED\_FOR;  
fastcgi\_param REMOTE\_PORT $remote\_port;  
fastcgi\_param SERVER\_ADDR $server\_addr;  
fastcgi\_param SERVER\_PORT $server\_port;  
#fastcgi\_param SERVER\_NAME $server\_name;  
#fastcgi\_param HOST\_LSG $http\_host;  
fastcgi\_param HTTP\_REFERER $http\_referer;  
fastcgi\_param HTTPS $https\_ms;

fastcgi\_pass\_header HTTP\_HOST;  
fastcgi\_pass\_header SERVER\_NAME;  
fastcgi\_pass\_header HOST;  
fastcgi\_pass\_header SERVER\_NAME;  
fastcgi\_pass\_header X-REAL-IP;  
fastcgi\_pass\_header X-FORWARDED-PROTO;  
fastcgi\_pass\_header X-FORWARDED-FOR;

Posted at Nginx Forum:

> **[Re: worker process exited on signal 11](https://forum.nginx.org/read.php?2,225420,225475#msg-225475)**
>
> Dear all, error logs show this error (worker process exited on signal 11). The error appears after ~1 hour, then with the half of the worker processes. No problems with 1.0-versions of nginx. Problem existed in 1.1.18 already. I'm using a custom rpm...

---

<div class="post-metadata">

**Author:** ![mscheel](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/da6949/32.png) [@mscheel](https://www.ruby-forum.com/u/mscheel)\
**Post date:** [April 19, 2012, 7:29am UTC](https://www.ruby-forum.com/t/worker-process-exited-on-signal-11/218966/11 "2012-04-19T07:29:11Z")

</div>

Hello Maxim, until now no signal 11 occured without substitutions4nginx.  
It’s interesting that such errors do not occur in 1.0. I’ll contact the  
developer of this module, maybe he can help or fix it. We’re parsing the  
content with regexp to rewrite links to another domain, mainly for  
external CDN domains.

Thank you very much for your help.

Posted at Nginx Forum:

> **[Re: worker process exited on signal 11](https://forum.nginx.org/read.php?2,225420,225498#msg-225498)**
>
> Dear all, error logs show this error (worker process exited on signal 11). The error appears after ~1 hour, then with the half of the worker processes. No problems with 1.0-versions of nginx. Problem existed in 1.1.18 already. I'm using a custom rpm...