i use nginx ver 1.2.5 (also tried 1.2.7) with my module that sends
subrequest to an upstream, waits untill response get back, then goes to
backend upstream and fetch the regular web page from it.
when i add to nginx conf “proxy_ignore_client_abort on;”, nginx crash
with
signal 11 (seg fault) when i do “ab” test and stop it in the middle of
the
(log: “client prematurely closed…” etc.).
when i cancel my subrequest - no crash
or: when i remove the proxy_ignore_client_abort (default off) - no
crash,
even with the subrequest.
here’s the core dump:
Program terminated with signal 11, Segmentation fault. #0 0x000000000045926b in ngx_http_terminate_request (r=0x1964360, rc=0)
at
src/http/ngx_http_request.c:2147
2147 ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
(gdb) bt #0 0x000000000045926b in ngx_http_terminate_request (r=0x1964360, rc=0)
at
src/http/ngx_http_request.c:2147 #1 0x0000000000458b98 in ngx_http_finalize_request (r=0x1964360, rc=0)
at
src/http/ngx_http_request.c:1977 #2 0x0000000000459d80 in ngx_http_test_reading (r=0x1964360) at
src/http/ngx_http_request.c:2443 #3 0x00000000004587fc in ngx_http_request_handler (ev=0x192b428) at
src/http/ngx_http_request.c:1866 #4 0x000000000043f1e0 in ngx_epoll_process_events (cycle=0x18f0470,
timer=59782, flags=1) at src/event/modules/ngx_epoll_module.c:683 #5 0x000000000042eea4 in ngx_process_events_and_timers
(cycle=0x18f0470) at
src/event/ngx_event.c:247 #6 0x000000000043ba36 in ngx_single_process_cycle (cycle=0x18f0470) at
src/os/unix/ngx_process_cycle.c:315 #7 0x000000000040a33a in main (argc=3, argv=0x7fff9e8fec18) at
src/core/nginx.c:409
the cause of the crash: it appears thet “r->connection->log” is null or
garbaged - its being freed before by nginx core !
i attache here my “debug_http” log - note that “http finalize request”
is
called twice (i think that one of them nulls the connection so nulls its
log
too), and thats NOT happening when NOT using proxy_ignore_client_abort:
On Thu, Mar 14, 2013 at 11:36:58AM -0400, gadh wrote:
i use nginx ver 1.2.5 (also tried 1.2.7) with my module that sends
subrequest to an upstream, waits untill response get back, then goes to
backend upstream and fetch the regular web page from it.
when i add to nginx conf “proxy_ignore_client_abort on;”, nginx crash with
signal 11 (seg fault) when i do “ab” test and stop it in the middle of the
(log: “client prematurely closed…” etc.).
when i cancel my subrequest - no crash
or: when i remove the proxy_ignore_client_abort (default off) - no crash,
even with the subrequest.
Description of the problem suggests there is something wrong with
request reference counting, likely caused by what your module
does. It’s very easy to screw it up, especially when trying to do
subrequests before the request body is received.
On Thu, Mar 14, 2013 at 12:46:43PM -0400, gadh wrote:
thanks
after i get the subrequest response in a handler function i registered, what
can i do in order to tell the ngin core the subrequest had finished ? in my
case i do only these actions:
ngx_http_core_run_phases(r->main);
return NGX_OK;
is this ok ?
No. This looks like completely wrong code, which may easily screw
up things.
thanks
after i get the subrequest response in a handler function i registered,
what
can i do in order to tell the ngin core the subrequest had finished ? in
my
case i do only these actions:
ngx_http_core_run_phases(r->main);
return NGX_OK;
is this ok ?
BTW, its not a case of a client body, i’m talking about GET requests
also
that get crashed, not POST.
more info: when i use “ignore client abort = on” , the crash happens
when
the client aborts the connection, BEFORE my subrequest handler is
called, so
its unlikely this code causes the crash.
also, i send the subrequest to a configured url named “aaa_post/” which
uses
the proxy module to send it to other server.
Ok, i’ll attach my calling to subrequest code, its working flawlessly
except
the case i reported here:
//------------------------------------------------------------------
/*
Note: the purspose of this code is to call a handler module (at rewrite
phase), send special POST subrequest to another server (independant of
the
main request), wait with the module untill subrequest finishes, process
its
data , then continue to backend or next handler module
*/
// the subrequest will call this handler after it finishes
ngx_int_t ngx_aaa_post_subrequest_handler (ngx_http_request_t *r, void
*data, ngx_int_t rc)
{
ngx_aaa_ctx_t ctx = (ngx_aaa_ctx_t)data;
ngx_chain_t *bufs;
ngx_uint_t status;
if (rc != NGX_OK)
{
NGX_aaa_LOG_ERROR(“bad response (nginx code %d)”,rc);
ctx->post.error = 1;
aaa_SUB_PROF_END
ngx_http_core_run_phases(r->main); // continue main request
return NGX_OK; //cannot return rc if != NGX_OK - see below
}
if (r->upstream) // when sending to another server, then subrequest is
passed on upstream module
{
bufs = r->upstream->out_bufs;
status = r->upstream->state->status;
}
else // runs on the same nginx, another port
{
NGX_aaa_LOG_ERROR(“response could not get by ‘upstream’ method.
aborting”);
ctx->post.error = 1;
aaa_SUB_PROF_END
ngx_http_core_run_phases(r->main);
return NGX_OK;
}
if (status != NGX_HTTP_OK) // == 200 OK
{
NGX_aaa_LOG_ERROR(“bad response status (%d)”,status);
ctx->post.error = 1;
aaa_SUB_PROF_END
ngx_http_core_run_phases(r->main);
return NGX_OK; // when returning error in subrequest, the nginx
loops over
it untill ok, or after 2 loops its stucks the main req.
}
psr = ngx_palloc(r->pool, sizeof(ngx_http_post_subrequest_t));
if (!psr)
return NGX_HTTP_INTERNAL_SERVER_ERROR;
ctx->done = 0;
ctx->post.done = 0;
ctx->post.start = 1;
if (is_ret) // return answer to caller, async
{
psr->handler = ngx_aaa_post_subrequest_handler; // register
callback
function for returning ans from the other end
psr->data = ctx;
}
else
psr = NULL;
// this func only registers the subreq in queue, but not activates it
yet
// note: sr->request_body is nulled during this func, alloc later
res = ngx_http_subrequest(r, &uri, NULL , &sr, psr, flags);
if (res)
return NGX_HTTP_INTERNAL_SERVER_ERROR;
// do not inherit rb from parent
sr->request_body = ngx_palloc(r->pool,
sizeof(ngx_http_request_body_t));
NGX_aaa_CHECK_ALLOC_AND_RETURN(sr->request_body)
// note: always alloc bufs even if ptr is lid - since its garbage from
former request ! (caused seg fault in mod_proxy !)
sr->request_body->bufs = ngx_alloc_chain_link(r->pool);
NGX_aaa_CHECK_ALLOC_AND_RETURN(sr->request_body->bufs)
// post body - re-populate , do not inherit from parent
sr->request_body->bufs->buf = buf;
sr->request_body->bufs->next = NULL;
sr->request_body->buf = buf;
// handler module main function - calls the subrequest, waits for it to
finish
ngx_int_t ngx_aaa_handler(ngx_http_request_t *r)
{
// pseudo code: alloc module ctx - only once
if (ctx->post.start)
{
// check if post subrequest has ended - then call next module
handler
if (ctx->post.done)
{
return NGX_DECLINED; // declined - if hdl is reg. in rewrite phase
}
else // wait for post subrequest to finish unless error
{
if (ctx->post.error)
{
return NGX_DECLINED; // subrequest finished - call next handler
module
}
else
{
return NGX_AGAIN; // wait untill finish response to our
subrequest
}
}
}
// prepare subrequest
// ngx_str - post body for the subrequest
ctx->post.response_handler = ngx_aaa_response_handler; // for
subrequest
response data parsing
i’de appreciate your help,
BTW, is there any “nginx subrequest coding guide” documentation
available ?
its very confusing and lacks much info on the web, i got it working only
thru alot of trial-and-error.
Tnx
Gad
thanks Maxim ! i very appreciate your help on this.
about the temp file - i protect from a response to be written to a file
by
knowing the max size that can be sent by the server and enlarging the
proxy
buffers accordingly.
i know i ruin the original request header - its the main purpose for my
code
! i want to issue an independant subrequest to another server, no to to
the
original. but the r->main->… is not ruined and acting ok afterwards.
in any case, i ask you to support this subrequest mechasnim, its
obviously
needed to send a subrequest to any server, not just to the original one,
and
also to control its response instead of just adding it to the start/end
of
page, its alot more flexible.
can i use another mechanism in order to achive those goals ? to create a
new
upstream module ?
tnx
Gad
On Mon, Mar 18, 2013 at 01:40:24AM -0400, gadh wrote:
thanks Maxim ! i very appreciate your help on this.
about the temp file - i protect from a response to be written to a file by
knowing the max size that can be sent by the server and enlarging the proxy
buffers accordingly.
You are not initializing subrequest’s request_body->temp_file
pointer (among other request_body members). It might point
anywhere, and will cause problems.
i know i ruin the original request header - its the main purpose for my code
! i want to issue an independant subrequest to another server, no to to the
original. but the r->main->… is not ruined and acting ok afterwards.
Yes, indeed.
Note though, that by changing headers_in structure you are
responsible for it’s consistency. It’s usually much better idea
to use upstream functionality to create needed request to an
upstream instead (proxy_set_body, proxy_pass_headers and so on).
in any case, i ask you to support this subrequest mechasnim, its obviously
needed to send a subrequest to any server, not just to the original one, and
also to control its response instead of just adding it to the start/end of
page, its alot more flexible.
can i use another mechanism in order to achive those goals ? to create a new
upstream module ?
What is supported is subrequest in memory functionality, which
allows you to get the response in memory instead of appending it
to the response. It only works with certain upstream protocols
though. And it wasn’t supposed to work at arbitrary request
processing phases, so it might be non-trivial to do things
properly, in particular - ensure subrequest consistency at early
phases of request processing and to rerun the main request once
subrequest is complete.
Note though, that by changing headers_in structure you are
responsible for it’s consistency. It’s usually much better idea
to use upstream functionality to create needed request to an
upstream instead (proxy_set_body, proxy_pass_headers and so on).
but can i wait for the upstream to return and delay the request from
passing
on to backend as i do in my subrequest ?
when i use your suggested proxy directives i have no control on that
Note: you allocate request body structure and only initialize some
of it’s members. E.g. sr->request_body->temp_file remains
uninitialized and will likely be dereferenced, resulting in
segmentation fault.
You have to at least change ngx_palloc() to ngx_pcalloc().
[…]
BTW, is there any “nginx subrequest coding guide” documentation available ?
its very confusing and lacks much info on the web, i got it working only
thru alot of trial-and-error.
Subrequests are dead simple in it’s supported form: you just call
ngx_http_subrequest() in a body filter, and the result is added to
the output at the appropriate point. Good sample is available in
ngx_http_addition_filter_module.c.
What you try to do with subrequests isn’t really supported (the
fact that it works - is actually a side effect of subrequests
processing rewrite in 0.7.25), hence no guides.
after a few addons to the code - in totally irrelevant places - the
error
returns so it did not help.
Now i try to create a new upstream handler so i can use it instead of
the
subrequest model.
i described the model i work in the first post above. let me add this:
in my first tests of the upstream - i cannot get to the backend server
at
all - i just get the upstream response and nginx pass it directly to my
filters and then to the client. my needs are different - after i recv
the
upstream response, i need it to go to the backend and then i’m going to
inject some of the data i recv from the upstream - to the backend
response
and only then to the client.
can you tell me if creating a new upstream module (my examples are
proxy/memcached) can suit my needs here ?
Tnx alot
Gad
Posted at Nginx Forum:
This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.