NGINX Team,
Is there a way to capture the User-Agent from an HTTP Header, then query
a
database for that User-Agent, then based on the result of the database
query, transparently route the request to an appropriate server?
Thanks,
Sean
On Apr 7, 2010, at 12:09 PM, Harmer, Sean wrote:
NGINX Team,
Is there a way to capture the User-Agent from an HTTP Header, then query a database for that User-Agent, then based on the result of the database query, transparently route the request to an appropriate server?
Thanks,
Sean
Sean-
Here is one way to do what you want, you will need to use redis as the
database to lookup your backend with. You will need to compile nginx
with two additional modules:
http://wiki.nginx.org/NginxHttpRedis
http://www.grid.net.ru/nginx/eval.en.html
Once you have both of those module compiled into your nginx you can
use a config file like this:
worker_processes 1;
user root root;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
log_format main '$remote_addr - $remote_user [$time_local] ’
'“$request” $status $body_bytes_sent “$http_referer”
’
‘“$http_user_agent” “$http_x_forwarded_for”’;
keepalive_timeout 65;
server {
listen 80;
server_name _;
access_log /var/log/nginx/redis.access.log main;
error_log /var/log/nginx/redis.error.log notice;
location / {
eval_escalate on;
eval $answer {
set $redis_key "$http_user_agent";
set $redis_db "0";
redis_pass 127.0.0.1:6379;
}
proxy_pass $answer;
}
}
}
Then in redis you will need to store key value pairs mapping the
$http_user_agent to the server:port combo that should serve that user
agent type:
redis.set “some-user-agent”, “http://192.168.0.1:80”
Then when a request comes into the nginx server and has a
$http_user_agent value of “some-user-agent”, redis will return
http://192.168.0.1:80 and then nginx will proxy_pass to the particular
server.
Hope this helps point you in the right direction.
Cheers-
Ezra Z.
[email protected]
Ezra - Thank you. That is very helpful.
It brings up another question: Once the match has been made in redis,
is
there a way to modify or append data to the User-Agent or HTTP HEADER?
Thanks again,
Sean
Harmer, Sean ha scritto:
NGINX Team,
Is there a way to capture the User-Agent from an HTTP Header, then query
a database for that User-Agent, then based on the result of the database
query, transparently route the request to an appropriate server?
The User-Agent is in $http_user_agent variable.
If you want to extract a more useful information, you need to use the
perl module, or a custom module (but, as far as I know, there is no
module that allow you to extract useful data from User-Agent header).
For database lookup, you can use the builtin map module:
http://wiki.nginx.org/NginxHttpMapModule
Manlio
By transparent, I mean passing the http header through nginx to other
servers. I’m just hoping nginx can help direct the request to the
correct server.
Thx
Depends on what you mean by ‘transparently routing’ - is it choosing a
different upstream (in case nginx is just a proxy) or redirecting to
diferent host/server?
While you can prolly try use this
GitHub - openresty/drizzle-nginx-module: an nginx upstream module that talks to mysql and drizzle by libdrizzle (although iirc the
possibility to get a single return value as variable back is still in
TODO)
I would imagine that using some dynamic language (like php or even the
embeded perl (if you really want to do this on server side)) is more
simple
and while client is being redirected its still more or less transparent
to
him.
With the nginx inbuilt features (without the DB backend) its like this -
a
simple rewrite:
if ($http_user_agent ~ MSIE) {
rewrite ^(.*)$ /msie/$1 break;
}
http://wiki.nginx.org/NginxHttpRewriteModule#if
rr
From: “Harmer, Sean” [email protected]
Sent: Wednesday, April 07, 2010 10:09 PM
To: [email protected]
Subject: Dynamic Redirects
NGINX Team,
Is there a way to capture the User-Agent from an HTTP Header, then query a
database for that User-Agent, then based on the result of the database
query, transparently route the request to an appropriate server?Thanks,
Sean
Ezra,
Tried following the steps and configuration you provided, but I keep
getting an empty reply from the server when I hit it. I can that the
variable $answer is properly set to the expected value (something like
http://some-ip), but for some reason it doesn’t work. It hangs for as
long as the keepalive_timeout is set to.
Any pointers as to why this doesn’t work would be highly appreciated.
Really like the idea of using redis for this purpose.
Thanks
Posted at Nginx Forum:
On Thu, Apr 8, 2010 at 3:09 AM, Harmer, Sean [email protected]
wrote:
NGINX Team,
Is there a way to capture the User-Agent from an HTTP Header, then query a
database for that User-Agent, then based on the result of the database
query, transparently route the request to an appropriate server?
I’ve got a sample app running to demonstrate this, using our
opensource modules ngx_redis2, ngx_lua, and ngx_set_misc:
https://github.com/agentzh/redis2-nginx-module
http://github.com/chaoslawful/lua-nginx-module
https://github.com/agentzh/set-misc-nginx-module
Here’s the complete code listing for my nginx.conf:
worker_processes 1;
error_log logs/error.log info;
events {
worker_connections 1024;
}
http {
upstream apache.org {
server 140.211.11.131;
}
upstream nginx.org {
server 206.251.255.63;
}
server {
listen 8080;
location = /redis {
internal;
set_unescape_uri $key $arg_key;
redis2_query get $key;
redis2_pass 127.0.0.1:6379;
}
location / {
set $target '';
access_by_lua '
local key = ngx.var.http_user_agent
local res = ngx.location.capture(
"/redis", { args = { key = key } }
)
print("key: ", key)
if res.status ~= 200 then
ngx.log(ngx.ERR, "redis server returns bad
status: ",
res.status)
ngx.exit(res.status)
end
if not res.body then
ngx.log(ngx.ERR, "redis returned empty body")
ngx.exit(500)
end
-- we can use the lua-redis-parser library to
parse the reply
– here instead
local server = string.match(res.body,
“^$%d+\r\n(%S+)\r\n”)
if not server then
ngx.log(ngx.ERR, "bad redis response: ",
res.body)
ngx.exit(500)
end
print("server: ", server)
ngx.var.target = server
';
proxy_pass http://$target;
}
}
}
This version is longer than the one using ngx_eval, but with much
saner error handling.
And then let’s start the redis server on the localhost:6379:
./redis-server # default port is 6379
and feed some keys into this using the redis-cli utility:
$ ./redis-cli
redis> set foo apache.org
OK
redis> set bar nginx.org
OK
And then let’s test our nginx app!
$ curl --user-agent foo localhost:8080
<apache.org home page goes here>
$ curl --user-agent bar localhost:8080
<nginx.org home page goes here>
To further tune the performance, one could enable the connection pool
for the redis connections, as documented in ngx_redis2’s README.
You can consider using the ngx_openresty bundle to run this sample to
eliminate all the pain involved in downloading and enabling 3rd-party
modules by yourself 
http://openresty.org/
Regards,
-agentzh
On Thu, Apr 8, 2010 at 5:27 AM, Reinis R. [email protected] wrote:
While you can prolly try use this
GitHub - openresty/drizzle-nginx-module: an nginx upstream module that talks to mysql and drizzle by libdrizzle (although iirc the
possibility to get a single return value as variable back is still in TODO)
You can use the ngx_rds_json module to emit JSON from the RDS reply
generated by ngx_drizzle (and ngx_postgres), and use a decent lua JSON
library like lua-cjson or lua-yajl to parse it into Lua data
structures and proceed processing.
I would imagine that using some dynamic language (like php or even the
embeded perl (if you really want to do this on server side)) is more simple
and while client is being redirected its still more or less transparent to
him.
PHP and the Embedded Perl addon are all blocking on backend requests
to mysql/postgres/redis/whatever. The penalty and impact of the latter
is more serious if one does not start many nginx worker processes.
Well, anyway, if one doesn’t care about performance, especially
concurrency, then whatever methods can be used 
Regards,
-agentzh