# Communications Sequence with Upstream

**URL:** <https://www.ruby-forum.com/t/communications-sequence-with-upstream/187545>\
**Category:** NGINX\
**Created:** [April 25, 2010, 10:59am UTC](https://www.ruby-forum.com/t/communications-sequence-with-upstream/187545 "2010-04-25T10:59:19Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Magnus\_L](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/db5fbb/32.png) [@Magnus\_L](https://www.ruby-forum.com/u/Magnus_L)\
**Post date:** [April 25, 2010, 10:59am UTC](https://www.ruby-forum.com/t/communications-sequence-with-upstream/187545/1 "2010-04-25T10:59:19Z")

</div>

I have been studying the nginx source code and read through Emiller’s  
guides and I’m interested in doing something more general than I have  
learned about so far.

Say I have a location /url that I want to construct responses to based  
on a sequence of communications with an upstream, host:port. To be  
specific, the upstream would be a database, and the sequence of  
communications would comprise commands and responses. A simple example  
of what I’d like to do would be:

1. client communicates message to /url meaning “if key foo exists,  
add 1 to bar”
2. module handling /url sends first database command to check if foo  
exists, if not reply “failed” to client, or it does and
3. module sends database command to increment bar, replies “succeeded”.

The details are arbitrary, but I would think there are myriad examples  
where you’d want a module to construct a reply based on back and forth  
communications.

I’m hoping there’s some nice way of handling this case, but from I’ve  
gathered looking at the code it seems oriented to “1 message to  
module, filter as desired, send to upstream, get response, filter  
again, then response back to client.”

Thanks

---

<div class="post-metadata">

**Author:** ![Magnus\_L](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/db5fbb/32.png) [@Magnus\_L](https://www.ruby-forum.com/u/Magnus_L)\
**Post date:** [April 25, 2010, 11:36am UTC](https://www.ruby-forum.com/t/communications-sequence-with-upstream/187545/2 "2010-04-25T11:36:12Z")

</div>

On Sun, Apr 25, 2010 at 4:58 PM, Magnus L. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> Say I have a location /url that I want to construct responses to based  
> on a sequence of communications with an upstream, host:port. To be  
> specific, the upstream would be a database, and the sequence of  
> communications would comprise commands and responses.

We’ve been doing this kind of things via ngx\_echo’s echo\_location  
directive in our ngx\_memc and ngx\_drizzle’s tests suite. Here’s some  
an example to do sequential memcached command execution against the  
ngx\_memc upstream module in a single location /main:

```
location /main {
    echo_location '/memc?cmd=flush_all';
    echo_location '/memc?key=foo&cmd=set&val=';
    echo_location '/memc?key=foo&cmd=get';
}
location /memc {
    set $memc_cmd $arg_cmd;
    set $memc_key $arg_key;
    set $memc_value $arg_val;
    memc_pass 127.0.0.1:11211;
}

```

You can find more examples here:  
[http://github.com/agentzh/memc-nginx-module/blob/master/test/t/](http://github.com/agentzh/memc-nginx-module/blob/master/test/t/)

And here’s an example for sequential SQL queries against a mysql  
backend:

```
location /test {
    echo_location /mysql "drop table if exists foo";
    echo;
    echo_location /mysql "create table foo (id serial, name

```

char(10), age integer);";  
echo;  
echo\_location /mysql “insert into foo (name, age) values (‘’,  
null);”;  
echo;  
echo\_location /mysql “insert into foo (name, age) values (null,  
0);”;  
echo;  
echo\_location /mysql “select \* from foo order by id”;  
echo;  
}  
location /mysql {  
drizzle\_pass backend;  
drizzle\_module\_header off;  
drizzle\_query $query\_string;  
rds\_json on;  
}

Responses look like this

```
{"errcode":0}
{"errcode":0}
{"errcode":0,"insert_id":1,"affected_rows":1}
{"errcode":0,"insert_id":2,"affected_rows":1}
[{"id":1,"name":"","age":null},{"id":2,"name":null,"age":0}]

```

More examples can be seen here:  
[http://github.com/agentzh/rds-json-nginx-module/blob/master/test/t/sanity.t](http://github.com/agentzh/rds-json-nginx-module/blob/master/test/t/sanity.t)

> A simple example  
> of what I’d like to do would be:
> 
> 1. client communicates message to /url meaning “if key foo exists,  
> add 1 to bar”
> 2. module handling /url sends first database command to check if foo  
> exists, if not reply “failed” to client, or it does and
> 3. module sends database command to increment bar, replies “succeeded”.

It reminds me of the “safe incr” sample in my slides for my talk on  
nginx.conf scripting:

```
http://agentzh.org/misc/slides/nginx-conf-scripting/nginx-conf-scripting.html

```

(Use the arrow keys on your keyboard to switch pages there.)

> The details are arbitrary, but I would think there are myriad examples  
> where you’d want a module to construct a reply based on back and forth  
> communications.

Check out ngx\_echo module to see if it fits your needs:

```
http://wiki.nginx.org/NginxHttpEchoModule

```

> I’m hoping there’s some nice way of handling this case, but from I’ve  
> gathered looking at the code it seems oriented to “1 message to  
> module, filter as desired, send to upstream, get response, filter  
> again, then response back to client.”

IMHO, Evan M.'s Guide is very limited in expressing nginx’s full  
power 😉 It’s just a good old introductory guide anyway 🙂 Still many  
thanks to Evan M. because the guide has helped so many people  
including me 🙂

I must add that the echo\_location and echo\_subrequest thingies in  
ngx\_echo are still limited in power 🙂 We’re currently releasing the  
full power of nginx’s subrequests in our ngx\_lua module. Soon we’ll be  
able to do things like this on the Lua land:

```
 res = ngx.location.capture('/sub1?id=32');
 if (res ...) {
       res2 = ngx.location.capture('/sub2?id=56');
 } else {
       res2 = ngx.subrequest.capture(ngx.HTTP_POST, '/sub2?id=56',

```

{ body: ‘hello’ });  
}

And all the capture operations are transparent non-blocking I/O ones  
that are based on nginx’s subrequests. Thanks to coco lua’s C-level  
coroutine support!

There will also be equivalences to the echo\_location and  
echo\_location\_async directives on the Lua land, i.e.,  
ngx.location.echo and ngx.location.echo\_async 🙂

Stay tuned!  
-agentzh

---

<div class="post-metadata">

**Author:** ![Magnus\_L](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/db5fbb/32.png) [@Magnus\_L](https://www.ruby-forum.com/u/Magnus_L)\
**Post date:** [April 25, 2010, 12:30pm UTC](https://www.ruby-forum.com/t/communications-sequence-with-upstream/187545/3 "2010-04-25T12:30:02Z")

</div>

> IMHO, Evan M.'s Guide is very limited in  
> expressing nginx’s full  
> power 😉 It’s just a good old introductory guide  
> anyway 🙂 Still many  
> thanks to Evan M. because the guide has helped  
> so many people  
> including me 🙂

me too 🙂

And I check out the ngx\_echo module’s source code.  
It’s very cool for newbie 🙂

Posted at Nginx Forum:  
[http://forum.nginx.org/read.php?2,79053,79064#msg-79064](http://forum.nginx.org/read.php?2,79053,79064#msg-79064)

---

<div class="post-metadata">

**Author:** ![Magnus\_L](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/db5fbb/32.png) [@Magnus\_L](https://www.ruby-forum.com/u/Magnus_L)\
**Post date:** [April 25, 2010, 12:44pm UTC](https://www.ruby-forum.com/t/communications-sequence-with-upstream/187545/4 "2010-04-25T12:44:40Z")

</div>

On Sun, Apr 25, 2010 at 6:29 PM, zhicheng [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> And I check out the ngx\_echo module’s source code.  
> It’s very cool for newbie 🙂

Personally I dislike the coding style in that module 😉 It’s my first  
non-trivial nginx module anyway. I do have plans to update it with my  
latest nginx knowledge. Oh, well…

ngx\_lua will reuse a good part of the ngx\_lua codebase 🙂

Cheers,  
-agentzh

P.S. The current ngx\_lua repository can be found here:  
[GitHub - openresty/lua-nginx-module: Embed the Power of Lua into NGINX HTTP servers · GitHub](http://github.com/chaoslawful/lua-nginx-module) and we’ve merely  
checked in the set\_by\_lua and set\_by\_lua\_file directives so far. But  
these directives are already awesome and fast enough to do lots of  
funny things that are impossible before 😉 And…it’s not releasable  
yet and still under active development.

---

<div class="post-metadata">

**Author:** ![Magnus\_L](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/db5fbb/32.png) [@Magnus\_L](https://www.ruby-forum.com/u/Magnus_L)\
**Post date:** [April 25, 2010, 5:37pm UTC](https://www.ruby-forum.com/t/communications-sequence-with-upstream/187545/5 "2010-04-25T17:37:19Z")

</div>

## agentzh Wrote:

> to update it with my  
> latest nginx knowledge. Oh, well…
> 
> ngx\_lua will reuse a good part of the ngx\_lua  
> codebase 🙂

Yeah,Maybe, but I just want know some nginx module’s rule.  
and I’m not very professor lua.  
I planning do some work with lua,but not now 🙂

> these directives are already awesome and fast  
> enough to do lots of  
> funny things that are impossible before 😉  
> And…it’s not releasable  
> yet and still under active development.
> 
> * * *
> 
> nginx mailing list  
> [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
> [mailman.nginx.org archive](http://nginx.org/mailman/listinfo/nginx)

Posted at Nginx Forum:  
[http://forum.nginx.org/read.php?2,79053,79164#msg-79164](http://forum.nginx.org/read.php?2,79053,79164#msg-79164)

---

<div class="post-metadata">

**Author:** ![Magnus\_L](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/db5fbb/32.png) [@Magnus\_L](https://www.ruby-forum.com/u/Magnus_L)\
**Post date:** [April 25, 2010, 12:48pm UTC](https://www.ruby-forum.com/t/communications-sequence-with-upstream/187545/6 "2010-04-25T12:48:46Z")

</div>

On Sun, Apr 25, 2010 at 6:43 PM, agentzh [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> ngx\_lua will reuse a good part of the ngx\_lua codebase 🙂

Sorry, I mean the ngx\_echo codebase 😛

Cheers,  
-agentzh

---

<div class="post-metadata">

**Author:** ![Magnus\_L](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/db5fbb/32.png) [@Magnus\_L](https://www.ruby-forum.com/u/Magnus_L)\
**Post date:** [April 26, 2010, 12:33pm UTC](https://www.ruby-forum.com/t/communications-sequence-with-upstream/187545/7 "2010-04-26T12:33:56Z")

</div>

On Sun, Apr 25, 2010 at 2:35 AM, agentzh [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> }
> 
> You can find more examples here:  
> [http://github.com/agentzh/memc-nginx-module/blob/master/test/t/](http://github.com/agentzh/memc-nginx-module/blob/master/test/t/)

Thanks. I looked over echo\_location. It uses ngx\_http\_subrequest,  
which I had gathered to be limited to concatenating responses  
together, and it still appears that way to me. It appears there’s no  
hook for you to deal with the data the subrequest produces, as the  
various filter hooks the upstream support provides.

This the flow it appears to me you’re limited to:  
handler does HTTP GET x\_1, response y\_1 comes back, …, HTTP GET x\_n,  
response y\_n. and what message the handler ultimately produces is  
y\_1…y\_n.

I’m interested in a more general ability to:  
handler sends arbitrary message x\_1, response y\_1 comes back, …  
repeat n times. handler sends response f(y\_1, …, y\_n) to original  
query, f an arbitrary function–which is to say I want to have the  
responses passed through a filter of mine as they’re generated, not  
sent straight to the client.

Maybe I’ll have to write my own code from scratch for dealing with this.

---

<div class="post-metadata">

**Author:** ![Magnus\_L](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/db5fbb/32.png) [@Magnus\_L](https://www.ruby-forum.com/u/Magnus_L)\
**Post date:** [April 26, 2010, 1:44pm UTC](https://www.ruby-forum.com/t/communications-sequence-with-upstream/187545/8 "2010-04-26T13:44:11Z")

</div>

On Mon, Apr 26, 2010 at 6:33 PM, Magnus L. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> On Sun, Apr 25, 2010 at 2:35 AM, agentzh [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:
> 
> Thanks. I looked over echo\_location. It uses ngx\_http\_subrequest,  
> which I had gathered to be limited to concatenating responses  
> together, and it still appears that way to me. It appears there’s no  
> hook for you to deal with the data the subrequest produces, as the  
> various filter hooks the upstream support provides.

And that’s way I said “echo\_location” does not expose the full power  
of nginx subrequests. We can use filters to capture all the outputs of  
subrequests, as be done in my fork of ngx\_eval module:

```
http://github.com/agentzh/nginx-eval-module

```

We’ll use this trick again in our ngx\_lua’s ngx.subrequest.capture  
implementation.

> I’m interested in a more general ability to:  
> handler sends arbitrary message x\_1, response y\_1 comes back, …  
> repeat n times. handler sends response f(y\_1, …, y\_n) to original  
> query, f an arbitrary function–which is to say I want to have the  
> responses passed through a filter of mine as they’re generated, not  
> sent straight to the client.

Sure you can do that. That’s the way to go 😉

> Maybe I’ll have to write my own code from scratch for dealing with this.

Indeed 🙂

BTW, I looked over the related code in ngx\_echo which was written 5  
months ago and found that it does not get r-\>main-\>count right for  
nginx 0.8.x \>= 0.8.11. I’ll rewrite the main dispatcher and the  
related parts for echo\_location, echo\_subrequest, and echo\_sleep in  
the next day or two and make a new release.

I apologize for not updating the ngx\_echo codebase in time. People who  
have copied the current implementation of  
echo\_subrequest/echo\_location should update their code according to  
the next release of ngx\_echo. I’m sorry 😛

Cheers,  
-agentzh

---

<div class="post-metadata">

**Author:** ![Magnus\_L](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/db5fbb/32.png) [@Magnus\_L](https://www.ruby-forum.com/u/Magnus_L)\
**Post date:** [May 5, 2010, 4:12am UTC](https://www.ruby-forum.com/t/communications-sequence-with-upstream/187545/9 "2010-05-05T04:12:38Z")

</div>

On Mon, Apr 26, 2010 at 7:43 PM, agentzh [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> the next release of ngx\_echo. I’m sorry 😛

I’ve completed the Big Refactoring of the ngx\_echo core in the git HEAD:

```
http://github.com/agentzh/echo-nginx-module/

```

I think the new implementation of echo\_location, echo\_subrequest,  
echo\_sleep, and echo\_read\_request\_body finally fit reasonably well  
with the nginx event model as well as Igor’s way of thinking 🙂

I’ll make a new release (v0.29) to include this major update once  
we’ve done more extensive testing.

This refactoring also paves a way to implementing the sequential  
subrequest support in ngx\_lua and “subrequests in subrequests” support  
in ngx\_srcache 🙂

If anybody experiences any regressions with the git HEAD of the  
ngx\_echo module, please let me know 🙂

Thanks!  
-agentzh