I am using Amazon EC2 with nginx primarily for load balancing and
proxying requests using proxy_pass. In this EC2 environment upstream
servers can fail without warning.
It would be a fantastic edition to nginx to have it run a backend shell
script when it detects that it needs to skip over an upstream server
because it is not responding in accordance with proxy_next_upstream.
For instance, if I’ve set this:
proxy_next_upstream error timeout http_500 http_503;
Than at any point when nginx decides it must skip a particular upstream
server (based on the above config), then I’d want it to run my bash
script /etc/scripts/upstream_down. Likewise anytime it detects that an
upstream server previously being skipped is now back and available, I’d
want it to call another script /etc/scripts/upstream_isback. I would
only want it to call my upstream_down the first time it goes down and
not each time it retries (unless nginx detects it has come back up, and
then later goes down again).
Or perhaps it can be simplified so that there is one command it will
execute when an upstream server changes from down to up or vice-versa,
and nginix then passes to it parameters to indicate whether the status
has changed to down or up.
In either case I’d like to get parameters that tell me what the server
name or IP is (as defined in the upstream config following the “server”
statement) and perhaps what the reason is (i.e. if the upstream server
was down due to “http_500” vs “timeout” etc).
With this feature added, I would set up my script so that it would:
a) send me an email notification warning me that upstream server xyz
just went down (or that xyz just came back up)
b) and, in the case where a box went down, my script may then
automatically use the appropriate EC2 command to immediately launch a
new ec2 replacement instance!
I am interested to hear from others if they would find it useful to call
a script upon status change of upstream servers. Likewise I am curious
whether this would be a relatively easy feature to implement. At first
I was going to suggest having the email notification built into nginx as
a feature, but then thought it would be much more useful just to have it
kick off a script where we could send the email ourselves and do other
actions as well (such as launch replacement instances).
Be a better friend, newshound, and
know-it-all with Yahoo! Mobile. Try it now.