We can’t reproduce it in test environment(without real load). It
appears on different servers with different CentOS versions on it.
In most cases error appears after HEAD requests(by google bot, for
example).
So, when there’s no file in cache path, and we made, for example, a
HEAD request, in some cases it generates an error.
Example with full debug in attachment.
So, when there’s no file in cache path, and we made, for example, a
HEAD request, in some cases it generates an error.
Example with full debug in attachment.
Thank you for report. From debug log it’s more or less clear what
goes on here, it indeed affects HEAD (as well as other header
only) requests while loading cache entry.
I’m able to reproduce it here with the following config:
Tried your patch, but after few minutes workers starts crashing.
GDB:
Core was generated by `nginx: worker process '.
Program terminated with signal 11, Segmentation fault. #0 0x000000000040ab19 in ngx_vslprintf (buf=0x7fff9e5f2785 <Address
0x7fff9e5f2785 out of bounds>,
last=0x7fff9e5f2f50 <Address 0x7fff9e5f2f50 out of bounds>,
fmt=, args=0x7fff9e5f2730) at
src/core/ngx_string.c:178
178 while (*fmt >= ‘0’ && *fmt <= ‘9’) {
On Wed, Oct 10, 2012 at 05:43:20PM +0400, Andrey F. wrote:
#0 0x000000000040ab19 in ngx_vslprintf (buf=0x7fff9e5f2785 <Address
0x7fff9e5f2785 out of bounds>,
last=0x7fff9e5f2f50 <Address 0x7fff9e5f2f50 out of bounds>,
fmt=, args=0x7fff9e5f2730) at
src/core/ngx_string.c:178
178 while (*fmt >= ‘0’ && *fmt <= ‘9’) {
Strange. What’s in backtrace?
Backtrace in attach.
Unfortunately, backtrace is completely unreadable. Not sure if
it’s a result of memory corruption, a result of optimization
used, or just a core vs. binary mismatch.
0x7fff9e5f2785 out of bounds>,
last=0x7fff9e5f2f50 <Address 0x7fff9e5f2f50 out of bounds>,
fmt=, args=0x7fff9e5f2730) at
src/core/ngx_string.c:178
178 while (*fmt >= ‘0’ && *fmt <= ‘9’) {
Unfortunately, backtrace is completely unreadable.
GooglePlus shows the following as attachment:
Core was generated by `nginx: worker process '.
Program terminated with signal 11, Segmentation fault. #0 0x000000000040ab19 in ngx_vslprintf (buf=0x7fffefda6605 “”,
last=0x7fffefda6dd0 “\020”, fmt=,
args=0x7fffefda65b0)
at src/core/ngx_string.c:178
178 while (*fmt >= ‘0’ && *fmt <= ‘9’) {
(gdb) bt #0 0x000000000040ab19 in ngx_vslprintf (buf=0x7fffefda6605 “”,
last=0x7fffefda6dd0 “\020”, fmt=,
args=0x7fffefda65b0)
at src/core/ngx_string.c:178 #1 0x000000000040692e in ngx_log_error_core (level=3, log=0x14143930,
err=14, fmt=0x473e4a “\307”, <incomplete sequence \320>)
at src/core/ngx_log.c:108 #2 0x000000000040dfb2 in ngx_conf_set_path_slot (cf=0x14224fe8,
cmd=, conf=)
at src/core/ngx_file.c:267 #3 0x00000000141439b8 in ?? () #4 0x0000000014225850 in ?? () #5 0x00002b87720e8250 in ?? () #6 0x0000000014154770 in ?? () #7 0x0000000000449e6f in ngx_http_upstream_send_response (r=0x0,
u=0x1) at src/http/ngx_http_upstream.c:2124 #8 ngx_http_upstream_process_header (r=0x0, u=0x1) at
src/http/ngx_http_upstream.c:1644 #9 0x00000000141439b8 in ?? () #10 0x0000000000000078 in ?? () #11 0x4a8da000006b3500 in ?? () #12 0x0000000050757962 in ?? () #13 0x00000000141439b8 in ?? () #14 0x00000000141439b8 in ?? () #15 0xfffffffffffffffe in ?? () #16 0x0000000000000078 in ?? () #17 0x00000000141d7828 in ?? () #18 0x0000000050757962 in ?? () #19 0x000000000045106a in ngx_http_file_cache_exists (cache=0x4742a1,
c=0x40aa1c) at src/http/ngx_http_file_cache.c:690 #20 0x0000000300000010 in ?? () #21 0x0000000014143930 in ?? () #22 0x000000000068c2c0 in ?? () #23 0x00000000141439b8 in ?? () #24 0x0000000014224c80 in ?? () #25 0x00000000141439b8 in ?? () #26 0x0000000014224a00 in ?? () #27 0x00002b87720e8250 in ?? () #28 0x00000000141d6798 in ?? () #29 0x0000000000000001 in ?? () #30 0x00000000004409d6 in ngx_http_variable_sent_keep_alive (r=, v=0x78, data=)
at src/http/ngx_http_variables.c:1654 #31 0x000000000000012e in ?? () #32 0x0000000014224c80 in ?? () #33 0x0000000014224c80 in ?? () #34 0x00000000141439b8 in ?? () #35 0x00000000141439b8 in ?? () #36 0x00002b87720e8250 in ?? () #37 0x00000000141d6798 in ?? () #38 0x0000000014224a00 in ?? () #39 0x0000000014224c80 in ?? () #40 0x00000000141439b8 in ?? () #41 0x00002b87720e8250 in ?? () #42 0x00000000141d6798 in ?? () #43 0x000000000044321c in ngx_http_script_regex_start_code (e=0x1) at
src/http/ngx_http_script.c:974 #44 0x00000000140610d0 in ?? () #45 0x00000000004762c9 in ngx_http_modern_browser (cf=0x7fffefda6605,
cmd=0x0, conf=0x7fffefda6e00)
at src/http/modules/ngx_http_browser_module.c:553 #46 0x0000000000000089 in ?? ()
—Type to continue, or q to quit— #47 0x00000000000019b0 in ?? () #48 0xfffffffffffffffc in ?? () #49 0x000000000041fee0 in ngx_event_pipe_read_upstream (p=0x0,
do_write=) at src/event/ngx_event_pipe.c:382 #50 ngx_event_pipe (p=0x0, do_write=) at
src/event/ngx_event_pipe.c:49 #51 0x0000000000001260 in ?? () #52 0x0000000000000062 in ?? () #53 0x00000000004220e9 in ngx_signal_handler (signo=25593) at
src/os/unix/ngx_process.c:316 #54 0x0000000000000000 in ?? ()
On Wed, Oct 10, 2012 at 11:05:13PM +0400, Andrey F. wrote:
I made a new build, will try it now and give you normal bt, i hope:)
#2 0x000000000040e562 in ngx_ext_rename_file (src=0x1bdb2540, to=0x1bdb1c98,
ext=0x7fffce909330) at src/core/ngx_file.c:557 #3 0x000000000045153a in ngx_http_file_cache_update (r=0x1bcbf9e8,
tf=0x1bdb2538) at src/http/ngx_http_file_cache.c:938 #4 0x0000000000446ed6 in ngx_http_upstream_process_request (r=0x1bcbf9e8) at
src/http/ngx_http_upstream.c:2686
Ok, thanx, I see what goes on. The patch is indeed wrong, it
broke small HEAD requests. Disregard it.
