Proxy_next_upstream and POSTs

Server NGINX
1

Hi!
Nginx is great and the way it handles upstream and failed server as
well! I
would just like to avoid retrying request when they are POSTs, as POSTs
are
no idempotent and my backend can’t handle duplicated POSTs right now.
So I started to mess around with the code. What I have in mind is to
create
a module that will change the behavior of proxy_next_upstream by just
checking that http method != POST.
I believe it supposed to be really simple.
So a couple of questions for those who are into nginx source code:

  1. Is there any good article, blog post, etc, that explain how to create
    and
    test a module?
  2. Any good advice on how I should do it? I was thinking on adding the
    validation somewhere in ngx_http_upstream_test_next
    (ngx_http_upstream.c),
    something like:
    if (r->method == NGX_HTTP_POST) {
    return NGX_DECLINED;
    }

If this is OK, how could I do this from a module? So when someone
compiles
nginx with this module, he’ll have this behavior instead of the standard
one?

Sorry if my questions are dumb.

Jonathan

2

I read some modules out there and could manage to write a custom module
that
will do the job. I have some doubts though.
How can I call the original upstream module from my own upstream module?
I
don’t want to replace it.

I think the magic might go here:
ngx_http_upstream_init__peer method. Is that right? Is
there
any example that I can use?

Thanks!

Jonathan

3

Hello,

You might want to have a look at Evan’s Miller section on proxying
handlers: Emiller’s Guide to Nginx Module Development – Evan Miller
This page is pretty much considered the reference for module
development.

Enjoy,

Antoine.

Low fare on trips to nginx land: visit http://www.nginx-discovery.com

4

Hi,

  1. Any good advice on how I should do it? I was thinking on adding the
    validation somewhere in ngx_http_upstream_test_next (ngx_http_upstream.c),
    something like:
    if (r->method == NGX_HTTP_POST) {
    return NGX_DECLINED;
    }

Keep in mind that you should decline sending request to another backend,
only if you already sent it upstream. There is a bunch of errors (bad
gateway, connection timeout, etc) which could lead to the code path in
question, for which you should still send request to another backend.

Best regards,
Piotr S. < [email protected] >

5

Thanks! That guide was really useful.
I almost have it done. What is missing there is how to test and maybe
something even more generic.
Once I finish my module I will try to write an even more basic guide
to help people :slight_smile:

Jonathan

6

Ok. Finally I could work it out, so thanks anyway!

Here is my small contribution to nginx, it is not much, but it is
something.

Thanks everyone for your help, and if you have time and find something
weired in the code, let me know!

Jonathan

7

Right, that is why after digging for a while I realized I wan’t doing it
the
right way.
Now I wrote a module that will hook into the upstream logic and will set
all
the callbacks to the round-robin callbacks. Only the the peer.free
callback
will call the original round-robin callback and after that will do
something
like:

if (r->method == NGX_HTTP_POST) {
pc->retries = 0;
}

by setting retires = 0 I think nginx won’t retry and will fail and I
will be
using round-robin logic for all the rest.

My only problem is that in the peer.free callback I don’t have access to
the
request struct, so I don’t know is the request’s method. So now I am not
sure if the approach is the right one.

Any thoughts on this?

8

Glad to hear you made it.
As far as testing goes, you might find interesting (or just fun) to
read my own experiences with the subject:

nginx discovery journey: Day 33 - Test::Nginx, pipelined_requests, raw_request and other testing tricks

As far as documentation of Test::Nginx goes, you might find the devel
tree (GitHub - openresty/test-nginx at devel) which includes
my commits a bit more verbose than the “master” branch.

Enjoy testing.

A.

–
Round-trip fares to nginx land as low as $0: visit

9

On Wed, May 11, 2011 at 4:03 PM, Antoine B.
[email protected] wrote:

As far as documentation of Test::Nginx goes, you might find the devel
tree (GitHub - openresty/test-nginx at devel) which includes
my commits a bit more verbose than the “master” branch.

I’ve just merged test-nginx’s devel branch back to its master after
massive regression testing on my side. And I’ll do a new CPAN release
of Test::Nginx tomorrow or so :slight_smile:

Happy hacking!
-agentzh

10

On Wed, May 11, 2011 at 4:03 PM, Antoine B.
[email protected] wrote:

As far as documentation of Test::Nginx goes, you might find the devel
tree (GitHub - openresty/test-nginx at devel) which includes
my commits a bit more verbose than the “master” branch.

I’ve just merged test-nginx’s devel branch back to its master after
massive regression testing on my side. And I’ll do a new CPAN release
of Test::Nginx tomorrow or so :slight_smile:

Happy hacking!
-agentzh