Replacing apache with nginx

I have apache 2.2 load-balancing a pool of 3 tomcat servers. I want
to replace it with nginx. I have the following very simple setup to
test the concept:

user nginx;
worker_processes 2;

error_log logs/error.log debug;

pid logs/nginx.pid;

events {
worker_connections 1024;
}

http {
default_type application/octet-stream;

log_format  main  '$remote_addr - $remote_user [$time_local] $status 


'“$request” $body_bytes_sent “$http_referer” ’
‘“$http_user_agent” “http_x_forwarded_for”’;

access_log  logs/access.log  main;

sendfile        on;
tcp_nopush     on;
tcp_nodelay    on;

server {
    listen       80;
    server_name  www.mysite.dmz;
    large_client_header_buffers 4 16k;

    access_log  logs/host.access.log  main;

    # Main location
    location / {
        proxy_pass         http://10.1.1.241:8080/;
        proxy_redirect     off;

        proxy_set_header   Host             $host;
        proxy_set_header   X-Real-IP        $remote_addr;
        proxy_set_header   X-Forwarded-For 

$proxy_add_x_forwarded_for;

        client_max_body_size       10m;
        client_body_buffer_size    128k;

        proxy_connect_timeout      90;
        proxy_send_timeout         90;
        proxy_read_timeout         90;

        proxy_buffer_size          4k;
        proxy_buffers              4 32k;
        proxy_busy_buffers_size    64k;
        proxy_temp_file_write_size 64k;
    }

}

}

This seems to work ok, although I’ve not yet looked up what each thing
does. What I need it to do, which I had some problems doing with
apache is the following:

  1. Needs to loadbalance over 3 servers
  2. Sessions need to be sticky - so the user goes to the same server
    all the time, after the session is started.
  3. All http traffic needs to be coerced to https apart from
    www.mysite.com/xml

Any pointers to example configs or bits of the documentation would be
most welcome. I’m just diving into the wiki now, but and handy
pointers would be a great help.

Thanks,

S.

On Tue, Sep 30, 2008 at 11:13 AM, Stephen Nelson-Smith
[email protected] wrote:

  1. Needs to loadbalance over 3 servers

create an upstream: http://wiki.codemongers.com/NginxHttpUpstreamModule

  1. Sessions need to be sticky - so the user goes to the same server
    all the time, after the session is started.

i’m always curious why people require sticky sessions. with
database-backed or some other central storage (hell even files over
NFS) i don’t see the need for this at all. i’ve never used session
affinity and i’ve never had a single issue…

  1. All http traffic needs to be coerced to https apart from www.mysite.com/xml

use a rewrite. that should work. or if it’s a unique hostname you could
do a

server {
listen 80;
server_name foo.com;
rewrite ^(.*) https://foo.com/$1 last;
}

depends on your setup though

Hi Mike,

  1. Sessions need to be sticky - so the user goes to the same server
    all the time, after the session is started.

i’m always curious why people require sticky sessions. with
database-backed or some other central storage (hell even files over
NFS) i don’t see the need for this at all. i’ve never used session
affinity and i’ve never had a single issue…

In my case, because of the way the app is written. If it detects a
different session cookie, it logs the user out.

S.

server {
listen 80;
server_name foo.com;
rewrite ^(.*) https://foo.com/$1 last;
}

In apache I do:

RewriteCond %{REQUEST_URI} !^/xml

Can I do something similar here?

S.

Hi,

Thanks for your help!

unique location quite enough:

server {
listen 80;
server_name example.com;
location /xml {
rewrite ^ https://example.com$request_uri permanent;
}
}

Please would you explain how this works - I can’t see how this
redirects -everything- apart from http://www.example.com/xml, which
should be allowed to proceed as http only.

S.

On Tuesday, September 30, 2008 at 21:30:53, mike wrote:

Sessions need to be sticky - so the user goes to the same server
all the time, after the session is started.

yes, it is possible: use ip_hash directive of ngx_http_upstream module.

m> i’m always curious why people require sticky sessions.
m> with database-backed or some other central storage
m> (hell even files over NFS) i don’t see the need for this at all.
m> i’ve never used session affinity and i’ve never had a single issue…

there is one good reason: shared session storage is single point of
failure.

  1. All http traffic needs to be coerced to https apart from www.mysite.com/xml

m> use a rewrite. that should work. or if it’s a unique hostname you
could do a

m> server {
m> listen 80;
m> server_name foo.com;
m> rewrite ^(.*) https://foo.com/$1 last;
m> }

unique location quite enough:

server {
listen 80;
server_name example.com;
location /xml {
rewrite ^ https://example.com$request_uri permanent;
}
}

On Tuesday, September 30, 2008 at 22:41:03, Stephen Nelson-Smith wrote:

SNS> redirects -everything- apart from http://www.example.com/xml,
SNS> which should be allowed to proceed as http only.

server {
listen 443;
server_name www.example.com;

location = /xml {
    return 403;
}

location / {
    proxy_pass http://backend;
}

}

server {
listen 80;
server_name www.example.com;

location = /xml {
    proxy_pass http://backend;
}

location / {
    rewrite ^ https://www.example.com$request_uri permanent;
}

}

On 30.09.2008 23:20, Gena M. wrote:

there is one good reason: shared session storage is single point of failure.

Shared session store can be distributed.

On Tuesday, September 30, 2008 at 22:50:16, mike wrote:

Sessions need to be sticky - so the user goes
to the same server all the time, after the session is started.

m> i’m always curious why people require sticky sessions.
m> with database-backed or some other central storage
m> (hell even files over NFS) i don’t see the need for this at all.

there is one good reason: shared session storage is single point of failure.

if the shared session storage are not redundant - we must
use sticky sessions if want avoid single point of failure

m> And per-server session affinity is no different.
m> Still a single point of failure.

if one of backend servers fail - it will be marked
as failed, but rest backend servers will works fine.

the future clients requests will not be
routed by nginx to failed backend server.

but if dedicated session storage server fail -
all backend servers can`t work with sessions…

this is reason why session affinity is better,
if the session storage server is not redundant.

m> Also if sessions are that important you can
m> easily make central session management redundant.

it is not so easy to do if SQL Server is not redundant
in multi-master mode ( for read and write operations )

PS sorry for my bad english,
it is not my native language.

On Tue, Sep 30, 2008 at 2:15 PM, Gena M. [email protected] wrote:

if the shared session storage are not redundant - we must
use sticky sessions if want avoid single point of failure

if i am user A going to server A, when server A dies, my session needs
to be migrated to server B. if server A is dead, it can’t migrate.

if one of backend servers fail - it will be marked
as failed, but rest backend servers will works fine.

the future clients requests will not be
routed by nginx to failed backend server.

but if dedicated session storage server fail -
all backend servers can`t work with sessions…

understood; but I think you might be missing my point.

this is reason why session affinity is better,
if the session storage server is not redundant.

single server sessions are not redundant either. it requires migrating
session data or keeping it in two places, which is redundant - but why
not do shared then and make that redundant and simplify your load
balancing/code?

it is not so easy to do if SQL Server is not redundant
in multi-master mode ( for read and write operations )

depending on the size of your site you can use a two-phase plan
involving memcached and mysql, use mysql cluster, or just mysql
master/slave (not multi-master)

Hello!

On Tue, Sep 30, 2008 at 11:36:06PM +0300, Gena M. wrote:

   return 403;

location = /xml {

I believe the original regex from apache’s config was
prefix-match, so ‘=’ should be removed.

   proxy_pass http://backend;

}

location / {

  •    rewrite ^ https://www.example.com$request_uri permanent;
    
  •    rewrite ^ https://www.example.com$request_uri? permanent;
    

Since $request_uri already includes query string, it’s important
to add ‘?’ to prevent duplication of query string.

}
}

Maxim D.

On Sep 30, 2008, at 12:20 PM, Gena M. [email protected] wrote:

m> (hell even files over NFS) i don’t see the need for this at all.
m> i’ve never used session affinity and i’ve never had a single
issue…

there is one good reason: shared session storage is single point of
failure.

And per-server session affinity is no different. Still a single point
of failure. Also if sessions are that important you can easily make
central session management redundant. Not to mention to make session
affinity based approaches redundant you have to add in some sort of
central or redundant facility built in… Would be a waste of time at
that point to not just simplify life with a central storage manager.

To OP: sorry I misunderstood your /xml comment. I thought it was just
an example uri.

On Wednesday, October 1, 2008 at 0:48:26, mike wrote:

if the shared session storage are not redundant - we must
use sticky sessions if want avoid single point of failure

m> if i am user A going to server A, when server A dies, my session
needs
m> to be migrated to server B. if server A is dead, it can’t migrate.

yes, 1/N of all sessions will be lost, but failed server will be
marked as failed and for all new clients site will works normally.

if fail dedicated session storage server - all session will not work.
may be in hours, before admins configure new server instead of failed.

this is simple tradeoff: “reliability” vs “performance” vs “complexity”
“complexity” - in case of “failsafe (redundant) shared session storage”

m> depending on the size of your site you can use
m> a two-phase plan involving memcached and mysql,

this is may be too complex and/or too slow in use.

m> use mysql cluster,

may be, this is the best solution. thanks for idea!

m> or just mysql master/slave (not multi-master)

if master server fails - sessions will be “read-only”.

On Tue, Sep 30, 2008 at 6:36 PM, Gena M. [email protected] wrote:

if fail dedicated session storage server - all session will not work.
may be in hours, before admins configure new server instead of failed.

the assumption is you have relatively decent reliability and
redundancy setup if it is important to you. :slight_smile:

this is simple tradeoff: “reliability” vs “performance” vs “complexity”
“complexity” - in case of “failsafe (redundant) shared session storage”

i use shared session storage and have for years. i don’t lose
sessions. also, if the database is down, the site is down anyway; so
the user’s loss of a session is not the biggest deal. perhaps that is
one of my reasons behind asking this - if the site is using the
database, you can use the same connection to the database for session
handling as well; and if the db is down, who cares if sessions are
down - the whole site is.

m> depending on the size of your site you can use
m> a two-phase plan involving memcached and mysql,

this is may be too complex and/or too slow in use.

not really, memcached would be faster than mysql most likely, but it
is not persistent. which is why you would need database-backing
(persistence) behind it.

m> use mysql cluster,

may be, this is the best solution. thanks for idea!

this is much more complex than the memcached solution IMHO :slight_smile:

Have you considered other options, both of which have no single point
of failure.

  1. Cookie sessions, as implemented by Rails 2.0

  2. Having no sessions at all, the cookie is a token into the user
    record, which is used to construct the rest of the object graph on the
    fly, as used by ebay

Cheers

Dave

On Wed, Oct 1, 2008 at 5:56 AM, Dave C. [email protected] wrote:

Have you considered other options, both of which have no single point of
failure.

  1. Cookie sessions, as implemented by Rails 2.0

  2. Having no sessions at all, the cookie is a token into the user record,
    which is used to construct the rest of the object graph on the fly, as used
    by ebay

what is the difference of “cookie sessions” vs normal?

is the payload completely stored on the user’s machine then?

essentially sessions -are- a cookie-based token that has the data
stored on the server somewhere… sounds like they’re just using a
persistent cookie with the user info instead of a randomly generated
session ID per session and retrieving the information based on user ID
(which actually requires a central database/user management then, as
it would be horrible to have -no- information on the user by sharding
up the data and pairing a webserver/webservers to a specific
database/pool of databases, unless it was redundant)

On Wednesday, October 1, 2008 at 5:35:29, mike wrote:

m> if the database is down, the site is down anyway;
m> so the user’s loss of a session is not the biggest deal.

database may be protected by master/slave replication -
master for write, slaves for read only - site will work
with read-only access to database and read/write sessions.

m> perhaps that is one of my reasons behind asking this -
m> if the site is using the database, you can use the same connection
m> to the database for session handling as well; and if the db is down,
m> who cares if sessions are down - the whole site is.

using SQL database for session storage may be not useful
for performance reasons - session data updated too often,
and this will generate too many write requests for database.

m> depending on the size of your site you can use
m> a two-phase plan involving memcached and mysql,

this is may be too complex and/or too slow in use.

m> not really, memcached would be faster than mysql most likely,
m> but it is not persistent. which is why you would need
m> database-backing (persistence) behind it.

“two-phase plan involving memcached and mysql” is:

==========================================

for write:

  1. write session data to memcached
  2. write session data to mysql

for read:

  1. read session data from memcached
  2. if all ok - finish read operation
  3. else - read session data from mysql

==========================================

I am right understood your proposition?

this will slowdown sql server by many write operations.
also this is more complex access protocol for backends.

m> use mysql cluster,

may be, this is the best solution. thanks for idea!

m> this is much more complex than the memcached solution IMHO :slight_smile:

Mysql Cluster has no “Single Point of Failure”.
complex may be only setup of cluster, not usage.

“memcached solution” is the best choice if memcached
located on fronted server and frontend is not redundant -
there is no additional “Single Points of Failure”, in this
case if frontend server is down - the whole site is down too.

On Wed, Oct 1, 2008 at 5:00 AM, Gena M. [email protected] wrote:

Mysql Cluster has no “Single Point of Failure”.
complex may be only setup of cluster, not usage.

“memcached solution” is the best choice if memcached
located on fronted server and frontend is not redundant -
there is no additional “Single Points of Failure”, in this
case if frontend server is down - the whole site is down too.

we’re going in circles here. central session management can be done
and scaled appropriately, just like -any- other database-centric
workload, and a two-phase memcached can be done probably a few ways as
well, including syncing the data in batches instead of per request,
etc.

mysql cluster still seems to be a bit complex to administer, i think
it can fail and is hard to recover right now (it is designed to be
redundant, but i think it can still break) and some apps may need to
change how they use the database to get proper performance and such
out of it. not to mention depending on the version in use, you need a
ton of RAM. it’s not a 100% drop-in for standard mysql engines.