# Sub-request using Parent Request Body **URL:** \ **Category:** NGINX\ **Created:** [January 4, 2011, 3:57am UTC](https://www.ruby-forum.com/t/sub-request-using-parent-request-body/201270 "2011-01-04T03:57:54Z")\ **Posts on this page:** 15\ **Page:** 1
**Author:** ![luislavena](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/l/0ea827/32.png) [@luislavena](https://www.ruby-forum.com/u/luislavena)\ **Post date:** [January 4, 2011, 3:57am UTC](https://www.ruby-forum.com/t/sub-request-using-parent-request-body/201270/1 "2011-01-04T03:57:54Z")
Hello, I’m trying to make POST analyzer module. To have ability with SPAM detection for passing throw traffic, for this I need to have request\_body in subrequest, for that I took agentzh / nginx-eval-module > **[Re: Subrequest using Parent Request Body](https://forum.nginx.org/read.php?2,84026,84605)** > > Hi, I'm trying to send a POST Subrequest using the same request\_body as the Parent Request. The subrequest is sent first, and once it has been finalized the parent request goes through. (this is inspired on Maxim's ngx\_http\_auth\_request\_module) It wo and after reading this posts made small changes (please find it below) on small tests it working fine, but on something with big POSTs it loosing information, could somebody help me with understanding how better fix this. — ngx\_http\_eval\_module.c 2010-12-19 19:55:08.000000000 -0800 +++ ngx\_http\_eval\_module.c.new 2011-01-03 11:39:36.000000000 -0800 @@ -155,6 +155,11 @@ NGX\_MODULE\_V1\_PADDING }; +void +ngx\_http\_stub(ngx\_http\_request\_t \*r) +{ +} + static ngx\_int\_t ngx\_http\_eval\_handler(ngx\_http\_request\_t \*r) { @@ -261,7 +266,23 @@ return rc; } - sr-\>discard\_body = 1; - ``` rc = ngx_http_read_client_request_body(r, ngx_http_stub); ``` - - ``` if (rc >= NGX_HTTP_SPECIAL_RESPONSE) { ``` - ``` return rc; ``` - ``` } ``` - - ngx\_log\_error(NGX\_LOG\_WARN, r-\>connection-\>log, 0, - ``` "eval: request_body \"%p\"", r->request_body); ``` - ``` if (r->request_body) { ``` - ``` sr->request_body = ngx_palloc(r->pool, ``` sizeof(ngx\_http\_request\_body\_t)); - ``` if (sr->request_body == NULL) { ``` - ``` return NGX_ERROR; ``` - ``` } ``` - ``` ngx_memcpy(sr->request_body, r->request_body, ``` sizeof(ngx\_http\_request\_body\_t)); - ``` } else { ``` - ``` sr->discard_body = 1; ``` - ``` } ctx->in_progress = 1; ``` ---
**Author:** ![Roman\_V](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/bc8723/32.png) [@Roman\_V](https://www.ruby-forum.com/u/Roman_V)\ **Post date:** [January 4, 2011, 4:20am UTC](https://www.ruby-forum.com/t/sub-request-using-parent-request-body/201270/2 "2011-01-04T04:20:24Z")
On Tue, Jan 4, 2011 at 10:57 AM, Roman V. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > Hello, > > I’m trying to make POST analyzer module. To have ability with SPAM detection > for passing throw traffic, for this I need to have request\_body in > subrequest, for that I took agentzh / nginx-eval-module > > [http://forum.nginx.org/read.php?2,84026,84605](http://forum.nginx.org/read.php?2,84026,84605) I don’t think it’s the right place to look into 🙂 > and after reading this posts made small changes (please find it below) > on small tests it working fine, but on something with big POSTs it loosing > information, could somebody help me with understanding how better fix this. I am afraid you’re using ngx\_http\_read\_client\_request\_body in a wrong way. When the request body is big enough to exceed the preread part of the request headers or the network is bad enough, you’ll surely loose data in the request body. Nginx works in a streaming and non-blocking manner, so you have to _wait_ for the ngx\_http\_read\_client\_request\_body fully complete (not just when this function returns but when your ngx\_http\_stub is called) before continuing with your rewrite-phase handler. So it’s totally wrong to do nothing in your ngx\_http\_stub callback. It should be the key part for your logic. Please take a peek at the ngx\_form\_input module to have a live demo for this: ``` http://github.com/calio/form-input-nginx-module ``` It was originally written by one of my intern students and now maintained by me 🙂 Good luck! -agentzh ---
**Author:** ![Roman\_V](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/bc8723/32.png) [@Roman\_V](https://www.ruby-forum.com/u/Roman_V)\ **Post date:** [January 4, 2011, 7:24am UTC](https://www.ruby-forum.com/t/sub-request-using-parent-request-body/201270/3 "2011-01-04T07:24:24Z")
On Tue, Jan 4, 2011 at 11:51 AM, Roman V. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > This module was from which I begun. Originally I saved all input data into > variable and tried to pass it into fastcgi subrequest, unfortunately when > I’m passing this variable in subrequest request\_body is already empty, How did you pass this variable into your subrequest? It looks weird to me 🙂 > because subrequest have empty request\_body, that’s was the reason why I’m > trying to pass request\_body to subrequest. May be people here tried other > variants or have some ideas to have good quality of post analysis > internally? I don’t think it’s a good idea to use subrequests here just for post body analysis. A rewrite phase or post access phase handler seems rather sufficient to me. Passing request bodies to a subrequest is as simple as passing r-\>request\_body-\>bufs to sr-\>request\_body-\>bufs. Cheers, -agentzh ---
**Author:** ![Roman\_V](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/bc8723/32.png) [@Roman\_V](https://www.ruby-forum.com/u/Roman_V)\ **Post date:** [January 4, 2011, 4:52am UTC](https://www.ruby-forum.com/t/sub-request-using-parent-request-body/201270/4 "2011-01-04T04:52:17Z")
On 01/03/2011 07:19 PM, agentzh wrote: > > > > the request headers or the network is bad enough, you’ll surely loose > > ``` > http://github.com/calio/form-input-nginx-module > > ``` This module was from which I begun. Originally I saved all input data into variable and tried to pass it into fastcgi subrequest, unfortunately when I’m passing this variable in subrequest request\_body is already empty, because subrequest have empty request\_body, that’s was the reason why I’m trying to pass request\_body to subrequest. May be people here tried other variants or have some ideas to have good quality of post analysis internally? > It was originally written by one of my intern students and now > maintained by me 🙂 Thank you for eval extention 🙂 ---
**Author:** ![Roman\_V](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/bc8723/32.png) [@Roman\_V](https://www.ruby-forum.com/u/Roman_V)\ **Post date:** [January 6, 2011, 6:04am UTC](https://www.ruby-forum.com/t/sub-request-using-parent-request-body/201270/5 "2011-01-06T06:04:03Z")
On Wed, Jan 5, 2011 at 8:20 AM, Roman V. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > } > } > if ( $res = SPAM) { > rewrite ^ /terms-of-use.html redirect; > } > } Please try the following: ``` location = /check-spam { internal; include spam_fastcgi.conf; fastcgi_pass unix:/var/run/nginx/cgiwrap-dispatch.sock; } location / { client_max_body_size 100k; client_body_buffer_size 100k; lua_need_request_body on; rewrite_by_lua ' if ngx.var.request_method ~= "POST" then return end local res = ngx.location.capture( "/check-spam", { method = "POST", body = ngx.var.request_body } ) if res.body == "SPAM" then return ngx.redirect("/terms-of-use.html") end '; # ... your normal fastcgi_pass/proxy_pass/etc settings # go here ... } ``` Please note that you need the git HEAD of the “master” branch of the ngx\_lua git repos: ``` https://github.com/chaoslawful/lua-nginx-module ``` > I have some foreign spam analyzer which I’m running like fastcgi server > written in perl. > I tried to use $request\_body as fastsgi\_param arg, it was empty; > Wrote module which creates $post\_body var in rewrite stage and with echo > $post\_body it works, but on subrequest it gives empty body (I guess because > it not passing POST into subrequest) > maybe somebody have any ideas how to realize it other way, or may be how to > get POST in subrequest? If you have further issues, feel free to report here 😉 Happy nginx.conf hacking! -agentzh ---
**Author:** ![Roman\_V](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/bc8723/32.png) [@Roman\_V](https://www.ruby-forum.com/u/Roman_V)\ **Post date:** [January 5, 2011, 1:21am UTC](https://www.ruby-forum.com/t/sub-request-using-parent-request-body/201270/6 "2011-01-05T01:21:41Z")
On 01/03/2011 10:23 PM, agentzh wrote: > On Tue, Jan 4, 2011 at 11:51 AM, Roman V.[removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > > > This module was from which I begun. Originally I saved all input data into > > variable and tried to pass it into fastcgi subrequest, unfortunately when > > I’m passing this variable in subrequest request\_body is already empty, > > How did you pass this variable into your subrequest? It looks weird to me 🙂 Let me show one variant of nginx.conf’s: location / { include proxy.conf; eval\_subrequest\_in\_memory off; eval\_override\_content\_type text/plain; eval $res { include spam\_fastcgi.conf; if ( $request\_method = POST) { fastcgi\_pass unix:/var/run/nginx/cgiwrap-dispatch.sock; } } if ( $res = SPAM) { rewrite ^ /terms-of-use.html redirect; } } I have some foreign spam analyzer which I’m running like fastcgi server written in perl. I tried to use $request\_body as fastsgi\_param arg, it was empty; Wrote module which creates $post\_body var in rewrite stage and with echo $post\_body it works, but on subrequest it gives empty body (I guess because it not passing POST into subrequest) maybe somebody have any ideas how to realize it other way, or may be how to get POST in subrequest? Thank you any ideas. ---
**Author:** ![Roman\_V](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/bc8723/32.png) [@Roman\_V](https://www.ruby-forum.com/u/Roman_V)\ **Post date:** [January 6, 2011, 6:15am UTC](https://www.ruby-forum.com/t/sub-request-using-parent-request-body/201270/7 "2011-01-06T06:15:19Z")
On Thu, Jan 6, 2011 at 1:02 PM, agentzh [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > ``` > local res = ngx.location.capture( > "/check-spam", > { method = "POST", body = ngx.var.request_body } > ) > > ``` Sorry…these lines should have been ``` local res = ngx.location.capture( "/check-spam", { method = ngx.HTTP_POST, body = ngx.var.request_body } ) ``` That is, the “method” argument should be a constant specified by ngx.HTTP\_POST. See the documentation of ngx\_lua for more details 🙂 Cheers, -agentzh ---
**Author:** ![Roman\_V](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/bc8723/32.png) [@Roman\_V](https://www.ruby-forum.com/u/Roman_V)\ **Post date:** [January 7, 2011, 1:35am UTC](https://www.ruby-forum.com/t/sub-request-using-parent-request-body/201270/8 "2011-01-07T01:35:12Z")
On 01/05/2011 09:02 PM, agentzh wrote: > > ``` > > fastcgi_pass unix:/var/run/nginx/cgiwrap-dispatch.sock; > > > > ``` > > ``` > internal; > > rewrite_by_lua ' > > ``` Thank you for quick response, and I afraid of my stupid questions about embedded lua but 🙂 : 1. I’m using current development nginx-0.9.3 (my internals need huge table ~350k domain records for 2 different nginx parameters) and doable in current version. 2. when I’m just including _rewrite\_by\_lua_ with simple return, I’m loosing all headers from parent cite, please help me with best solution for that. 3. why you prefer NDK to clear nginx environment? ---
**Author:** ![Roman\_V](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/bc8723/32.png) [@Roman\_V](https://www.ruby-forum.com/u/Roman_V)\ **Post date:** [January 7, 2011, 1:36am UTC](https://www.ruby-forum.com/t/sub-request-using-parent-request-body/201270/9 "2011-01-07T01:36:27Z")
spierdalaj chamie 2011/1/7 Roman V. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) > > > eval\_override\_content\_type text/plain; > > > > ``` > > location / { > > > > ``` > > 1. I’m using current development nginx-0.9.3 (my internals need huge table > > > ``` > > local res = ngx.location.capture( > > } > > > > ``` > > > > > I tried to use $request\_body as fastsgi\_param arg, it was empty; > > > If you have further issues, feel free to report here 😉 > > > [mailman.nginx.org archive](http://nginx.org/mailman/listinfo/nginx) – Z wyrazami szacunku\* Adrian Janeczek - ---
**Author:** ![Roman\_V](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/bc8723/32.png) [@Roman\_V](https://www.ruby-forum.com/u/Roman_V)\ **Post date:** [January 7, 2011, 7:37am UTC](https://www.ruby-forum.com/t/sub-request-using-parent-request-body/201270/10 "2011-01-07T07:37:10Z")
On Fri, Jan 7, 2011 at 8:33 AM, Roman V. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > Thank you for quick response, and I afraid of my stupid questions about > embedded lua but 🙂 : > > 1. I’m using current development nginx-0.9.3 (my internals need huge table > ~350k domain records for 2 different nginx parameters) and doable in current > version. I’ve just confirmed that ngx\_lua’s test suite is passing completely with nginx 0.9.3 🙂 > 1. when I’m just including _rewrite\_by\_lua_ with simple return, I’m loosing > all headers from parent cite, please help me with best solution for that. I’m not sure what you’re talking about here. Can you provide a minimal config sample for this issue? > 1. why you prefer NDK to clear nginx environment? NDK just provides the “set\_var” mechanism for ngx\_lua. What do you mean by “clearing nginx environment”? Cheers, -agentzh ---
**Author:** ![Roman\_V](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/bc8723/32.png) [@Roman\_V](https://www.ruby-forum.com/u/Roman_V)\ **Post date:** [January 6, 2011, 6:05am UTC](https://www.ruby-forum.com/t/sub-request-using-parent-request-body/201270/11 "2011-01-06T06:05:56Z")
On Thu, Jan 6, 2011 at 1:02 PM, agentzh [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > Please note that you need the git HEAD of the “master” branch of the > ngx\_lua git repos: > > [GitHub - openresty/lua-nginx-module: Embed the Power of Lua into NGINX HTTP servers · GitHub](https://github.com/chaoslawful/lua-nginx-module) Please also note that due to a famous regression regarding rewrite phase handling in nginx 0.8.42 ~ 0.8.53, you need at least nginx 0.8.54+ to get rewrite\_by\_lua work 🙂 Good luck! -agentzh ---
**Author:** ![Roman\_V](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/bc8723/32.png) [@Roman\_V](https://www.ruby-forum.com/u/Roman_V)\ **Post date:** [January 7, 2011, 9:15pm UTC](https://www.ruby-forum.com/t/sub-request-using-parent-request-body/201270/12 "2011-01-07T21:15:56Z")
> > 1. when I’m just including _rewrite\_by\_lua_ with simple return, I’m loosing > > all headers from parent cite, please help me with best solution for that. > > I’m not sure what you’re talking about here. Can you provide a minimal > config sample for this issue? Showing 2 config which replacing whole headers from proxy to some internal staff: = _WRONG_ headers ======= HTTP/1.1 200 OK Server: nginx/0.9.3 Date: Fri, 07 Jan 2011 02:07:34 GMT Content-Type: text/plain Content-Length: 0 Connection: close 1. ================== location / { access\_by\_lua ’ '; include proxy.conf; } 2. ================== location / { rewrite\_by\_lua ’ '; include proxy.conf; } = _CORRECT_ headers ====== HTTP/1.1 200 OK Server: nginx/0.9.3 Date: Fri, 07 Jan 2011 20:10:38 GMT Content-Type: text/html Connection: close Accept-Ranges: bytes Via: 1.1 netcache-c (NetCache NetApp/5.6.2R1D19) ================== ``` location / { include proxy.conf; } ``` > > 1. why you prefer NDK to clear nginx environment? > > NDK just provides the “set\_var” mechanism for ngx\_lua. What do you > mean by “clearing nginx environment”? I mean if you using just this function may be better to include it into your module NDK patchnig nginx and it not clear, because you actually never need it. ---
**Author:** ![Roman\_V](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/bc8723/32.png) [@Roman\_V](https://www.ruby-forum.com/u/Roman_V)\ **Post date:** [January 10, 2011, 9:32pm UTC](https://www.ruby-forum.com/t/sub-request-using-parent-request-body/201270/13 "2011-01-10T21:32:58Z")
Sorry, everything is fine, my fault. ---
**Author:** ![Roman\_V](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/bc8723/32.png) [@Roman\_V](https://www.ruby-forum.com/u/Roman_V)\ **Post date:** [January 8, 2011, 8:23am UTC](https://www.ruby-forum.com/t/sub-request-using-parent-request-body/201270/14 "2011-01-08T08:23:23Z")
On Sat, Jan 8, 2011 at 4:14 AM, Roman V. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > Showing 2 config which replacing whole headers from proxy to some internal > staff: Nice catch 🙂 I’ve added this to the test suite and fixed it in the v0.1.2rc1 release candidate: ``` https://github.com/chaoslawful/lua-nginx-module/tarball/v0.1.2rc1 ``` Could you please try it out? > I mean if you using just this function may be better to include it into your > module > NDK patchnig nginx and it not clear, because you actually never need it. The latest NDK (pre-)release no longer patches the nginx core: ``` https://github.com/simpl/ngx_devel_kit/tarball/v0.2.14rc1 ``` Please confirm it on your side. If it still does, then it should be considered a bug and I’ll report it to Marcus C. 🙂 Thanks! -agentzh ---
**Author:** ![Roman\_V](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/bc8723/32.png) [@Roman\_V](https://www.ruby-forum.com/u/Roman_V)\ **Post date:** [January 10, 2011, 8:38pm UTC](https://www.ruby-forum.com/t/sub-request-using-parent-request-body/201270/15 "2011-01-10T20:38:55Z")
Hi, I’ve installed [https://github.com/chaoslawful/lua-nginx-module/tarball/v0.1.2rc3](https://github.com/chaoslawful/lua-nginx-module/tarball/v0.1.2rc3) with this config location / { rewrite\_by\_lua ’ return ngx.redirect(“/terms-of-use.html”) '; include proxy.conf; } and it doesnt rewrites anything, just working like without this line. is it correct or something which we need to fix next?