Forum: NGINX Parallel subrequests

2974d09ac2541e892966b762aad84943?d=identicon&s=25 honwel (Guest)
on 2013-12-19 08:59
(Received via mailing list)
Hi

   I want write a http hadler(using subrequest) to deal with combine
response from mutiple backend. but "Emiller's Advanced Topics In Nginx
Module Development - 2.3. Sequential subrequests" issue that "
Subrequests
might need to access the network, and if so, Nginx needs to return to
its
other work while it waits for a response. So we need to check the return
value of ngx_http_subrequest"
  How to write a Parallel subrequests which several subrequests by POST
in
parallel rather than one by one after received preivous response ?
thanks.



best regards
honwel

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,245707,245707#msg-245707
A8108a0961c6087c43cda32c8616dcba?d=identicon&s=25 Maxim Dounin (Guest)
on 2013-12-19 16:24
(Received via mailing list)
Hello!

On Thu, Dec 19, 2013 at 02:59:03AM -0500, honwel wrote:

> thanks.
Just call ngx_http_subrequest() multiple times.  Take a look at
the source to understand how it works.

--
Maxim Dounin
http://nginx.org/
2974d09ac2541e892966b762aad84943?d=identicon&s=25 honwel (Guest)
on 2013-12-19 17:02
(Received via mailing list)
Ok,thanks a lot,I will try.

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,245707,245726#msg-245726
2974d09ac2541e892966b762aad84943?d=identicon&s=25 honwel (Guest)
on 2013-12-25 08:15
(Received via mailing list)
hi, Maxim

      It is seem like that ngx_http_subrequest() supports only GET
method:

    sr->method = NGX_HTTP_GET;
    sr->http_version = r->http_version;
    ...............

    sr->method_name = ngx_http_core_get_method;

    In my application currently, NGINX received a POST request and issue
mutiple subrequests(Parallel) to get combine content by set proxy_pass
location,     but ngx_http_subrequest() not supports POST method, so,
What
do I need to pay attention, if i just modify to :
     sr->method = NGX_HTTP_POST
    sr->http_version = r->http_version;
    ...............

    sr->method_name = r->method_name;
    ???

    thanks a lot!


   best regards
   honwel

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,245707,245819#msg-245819
37f3ea777f96500b332a1a89d6027897?d=identicon&s=25 Yichun Zhang (agentzh) (Guest)
on 2013-12-25 21:22
(Received via mailing list)
Hello!

On Tue, Dec 24, 2013 at 11:14 PM, honwel wrote:
> but ngx_http_subrequest() not supports POST method, so, What
> do I need to pay attention, if i just modify to :
>      sr->method = NGX_HTTP_POST
>     sr->http_version = r->http_version;
>     ...............
>
>     sr->method_name = r->method_name;
>     ???
>

Yeah, you can do something like that.

My Nginx modules ngx_echo, ngx_srcache, and ngx_lua all support custom
method names in their subrequests. See

  https://github.com/agentzh/echo-nginx-module#echo_...

  https://github.com/agentzh/srcache-nginx-module#sr...

  https://github.com/chaoslawful/lua-nginx-module#ng...

Merry Christmas!

Best regards,
-agentzh
2974d09ac2541e892966b762aad84943?d=identicon&s=25 honwel (Guest)
on 2013-12-26 04:50
(Received via mailing list)
2974d09ac2541e892966b762aad84943?d=identicon&s=25 honwel (Guest)
on 2013-12-26 13:50
(Received via mailing list)
hi, agentzh

    There is also a problem that how can hold the one subrequest's
response
meanwhile waiting others(subrequest) comes, then combine the all
response(data) and send it to client?  is it possible?
    thanks a lot!



Best regards,
honwel

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,245707,245849#msg-245849
A8108a0961c6087c43cda32c8616dcba?d=identicon&s=25 Maxim Dounin (Guest)
on 2013-12-26 14:15
(Received via mailing list)
Hello!

On Thu, Dec 26, 2013 at 07:50:01AM -0500, honwel wrote:

> hi, agentzh
>
>     There is also a problem that how can hold the one subrequest's response
> meanwhile waiting others(subrequest) comes, then combine the all
> response(data) and send it to client?  is it possible?
>     thanks a lot!

By default, responses to subrequests are serialized in correct
order by postpone filter module, see
src/http/ngx_http_postpone_filter_module.c.

That is, if you want to just return responses in order there is no
need to do anything, it will happen automatically.

--
Maxim Dounin
http://nginx.org/
2974d09ac2541e892966b762aad84943?d=identicon&s=25 honwel (Guest)
on 2013-12-26 16:38
(Received via mailing list)
Yes,you are right. But I want combine the data like:
Subrequst1's response: "first name: John"
Subrequst2's response: "last name: abc"

Excepting response:
"John abc"

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,245707,245860#msg-245860
A8108a0961c6087c43cda32c8616dcba?d=identicon&s=25 Maxim Dounin (Guest)
on 2013-12-26 16:47
(Received via mailing list)
Hello!

On Thu, Dec 26, 2013 at 10:38:20AM -0500, honwel wrote:

> Yes,you are right. But I want combine the data like:
> Subrequst1's response: "first name: John"
> Subrequst2's response: "last name: abc"
>
> Excepting response:
> "John abc"

In this particular example, using a filter module after the
postpone filter should be a good solution.  It will see full
response concatenated by the postpone filter, and will be able
to transform it appropriately.

--
Maxim Dounin
http://nginx.org/
2974d09ac2541e892966b762aad84943?d=identicon&s=25 honwel (Guest)
on 2013-12-26 17:22
(Received via mailing list)
Excepting response:
"John abc"

Not
Response1:"John"
Response2:"abc"

Thanks for your patience!

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,245707,245864#msg-245864
A8108a0961c6087c43cda32c8616dcba?d=identicon&s=25 Maxim Dounin (Guest)
on 2013-12-26 18:12
(Received via mailing list)
Hello!

On Thu, Dec 26, 2013 at 11:21:42AM -0500, honwel wrote:

> Excepting response:
> "John abc"
>
> Not
> Response1:"John"
> Response2:"abc"
>
> Thanks for your patience!

Yes.  What I suggested is to write a filter which will transform

Response1:"John"
Response2:"abc"

as available after the pospone filter to

"John abc"

you want to have as a result.  This should be easier than writing
a filter which will also wait for all subrequests.  If it doesn't
work for you (e.g., if concatenation of subrequest responses isn't
parsable), you may do subrequest serialization yourself, before
the postpone filter.

--
Maxim Dounin
http://nginx.org/
2974d09ac2541e892966b762aad84943?d=identicon&s=25 honwel (Guest)
on 2014-04-10 08:28
(Received via mailing list)
how to write a filter module after the postpone filter .

if i change module's config file or complie file (auto/) ? any example?

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,245707,249183#msg-249183
Please log in before posting. Registration is free and takes only a minute.
Existing account

NEW: Do you have a Google/GoogleMail, Yahoo or Facebook account? No registration required!
Log in with Google account | Log in with Yahoo account | Log in with Facebook account
No account? Register here.