Session state management in Lighttpd-mongrel cluster

Hi All

I am using file based session store the file is written to
<Rails_Root>\tmp\sessions

Naturally this setup won’t work in production/test in a clustered
environment. For my clustering I am using 1 Lighty and 4 Mongrels, all
these 4 mongrels are running on separate machines.

The question is how do I synchronize the session data across all 4
mongrels, without which a transparent failover won’t be possible.

Please help me.

thanks
-daya

On Aug 22, 2006, at 10:53 AM, Daya S. wrote:

I am using file based session store the file is written to
<Rails_Root>\tmp\sessions

Naturally this setup won’t work in production/test in a clustered
environment. For my clustering I am using 1 Lighty and 4 Mongrels, all
these 4 mongrels are running on separate machines.

You’re right, it won’t work.

The question is how do I synchronize the session data across all 4
mongrels, without which a transparent failover won’t be possible.

2 possibilities come to mind:

  1. Switch to ActiveRecordStore (DB backed sessions)
  2. Share session data via NFS or something similar. I don’t know if
    the filesystem session system uses cluster safe unique filenames,
    though. I’d guess offhand that it doesn’t as I’d be surprised if
    anyone running a cluster would opt for filesystem based session storage.


– Tom M.

On 8/22/06, Daya S. [email protected] wrote:

The question is how do I synchronize the session data across all 4
mongrels, without which a transparent failover won’t be possible.

You can’t. You need some type of load balancer that can route
requests from the same ip addresses to the same mongrel on subsequent
requests. Yes it’s a pain, but if you want to have clusters on
different servers and use the local filesystem for session storage you
don’t really have another option. I think you can do this with
something like squid. I’ve always used something like the serverirons
for this type of stuff, but I know there are open source solutions for
it.

A second best approach would probably be mysql. It should be pretty
fast for session storage if configured correctly.

Pound appears to be able to maintain session state, you might try it
instead of lighttpd.

snacktime wrote:

Pound appears to be able to maintain session state, you might try it
instead of lighttpd.

I will investigate Pound, but I can’t find a Windows executable for
Pound, I need win32 binaries for Pound so that I can demo how cool RoR
is at a fortune 100 company I work for. And yes they want transparent
failover cluster configuration.

Anyother ideas?

thanks
-daya

Excuse me for my ignorance but I am not sure if ActiveRecordStore for
session would be faster than file system session store?

It’s most likely not.

BTW its really surprising that there is no rails setup for a transparent
failover, the idea of which can be sold to large corporations like mine.
I think this is a common requirement nowadays when most web applications
require 99.95% uptime. I am not so highly qualified to take on such a
task, but unless issues like this are tackled RoR adoption rate in
corporations will be extrememly low at best.

What you want to do isn’t a function of the application server,
whether it’s rails, mod perl, jboss, whatever… Load balancing and
failover is handled by other layers. Just because you aren’t familiar
with the different tools available to you dont’ assume they don’t
exist. I already pointed three that would probably do the job.
Squid, Pound, or something like a ServerIron, and there are more if
you look around.

snacktime wrote:

On 8/22/06, Daya S. [email protected] wrote:

The question is how do I synchronize the session data across all 4
mongrels, without which a transparent failover won’t be possible.

You can’t. You need some type of load balancer that can route
requests from the same ip addresses to the same mongrel on subsequent
requests. Yes it’s a pain, but if you want to have clusters on
different servers and use the local filesystem for session storage you
don’t really have another option. I think you can do this with
something like squid. I’ve always used something like the serverirons
for this type of stuff, but I know there are open source solutions for
it.

A second best approach would probably be mysql. It should be pretty
fast for session storage if configured correctly.

Excuse me for my ignorance but I am not sure if ActiveRecordStore for
session would be faster than file system session store?

BTW its really surprising that there is no rails setup for a transparent
failover, the idea of which can be sold to large corporations like mine.
I think this is a common requirement nowadays when most web applications
require 99.95% uptime. I am not so highly qualified to take on such a
task, but unless issues like this are tackled RoR adoption rate in
corporations will be extrememly low at best.

-daya

On Aug 22, 2006, at 1:13 PM, Daya S. wrote:

you
Excuse me for my ignorance but I am not sure if ActiveRecordStore for
corporations will be extrememly low at best.

-daya

We are working on high availability redundant rails clustered

hosting. http://engineyard.com . It will be ready for beta in early
October.

Cheers-
-Ezra

On Tue, Aug 22, 2006 at 01:45:22PM -0700, snacktime wrote:

Excuse me for my ignorance but I am not sure if ActiveRecordStore for
session would be faster than file system session store?

It’s most likely not.

And, to speak with someone elses words: that’s not a problem until it’s
a problem.

Once the AR session store proves to be a bottleneck, you can always
replace it with a faster, still MySQL-based solution (afair the Rails
performance blog had a post about this some time ago). And if that still
doesn’t work out, you can go back to solve the problem by choosing
another session store, or even by implementing your own
shared-fs-storage-safe one.

failover is handled by other layers. Just because you aren’t familiar
with the different tools available to you dont’ assume they don’t
exist. I already pointed three that would probably do the job.
Squid, Pound, or something like a ServerIron, and there are more if
you look around.

Pen can do session aware load balancing, too. In fact it’s
not really session-aware, it simply assigns same clients (IP-based, I
suppose), the same Mongrel instance with each request.

Don’t know about Windows support, though.

regards,
Jens


Jens Krämer
[email protected]

    We are working on high availability redundant rails clustered

hosting. http://engineyard.com . It will be ready for beta in early
October.

Will be looking forward to seeing that when it’s out, definitely seems
like a void that needs filling.

Jens K. wrote:

On Tue, Aug 22, 2006 at 01:45:22PM -0700, snacktime wrote:

Excuse me for my ignorance but I am not sure if ActiveRecordStore for
session would be faster than file system session store?

It’s most likely not.

And, to speak with someone elses words: that’s not a problem until it’s
a problem.

Once the AR session store proves to be a bottleneck, you can always
replace it with a faster, still MySQL-based solution (afair the Rails
performance blog had a post about this some time ago). And if that still
doesn’t work out, you can go back to solve the problem by choosing
another session store, or even by implementing your own
shared-fs-storage-safe one.

I wonder if I can get the same performance using Oracle based
ActiveRecordStore for session.

failover is handled by other layers. Just because you aren’t familiar
with the different tools available to you dont’ assume they don’t
exist. I already pointed three that would probably do the job.
Squid, Pound, or something like a ServerIron, and there are more if
you look around.

Pen can do session aware load balancing, too. In fact it’s
not really session-aware, it simply assigns same clients (IP-based, I
suppose), the same Mongrel instance with each request.

What you are referring to is same/similar to proxy.balance=“hash” option
of lighttpd, but I still can’t understand how it will help with session
state management in case of failover. Lets say Pen/Lighty assigs the a
mongrel instance to all requests from the same IP, if this mongrel
instance goes down, what happens to the session data and where will the
next request from the same IP go?

Don’t know about Windows support, though.

Unfortunately I only have access to windows machines, f***king
microsoft.

-daya

regards,
Jens


Jens Kr�mer
[email protected]

What you are referring to is same/similar to proxy.balance=“hash” option
of lighttpd, but I still can’t understand how it will help with session
state management in case of failover. Lets say Pen/Lighty assigs the a
mongrel instance to all requests from the same IP, if this mongrel
instance goes down, what happens to the session data and where will the
next request from the same IP go?

To another server. If you use sessions wisely and don’t just store
the whole kitchen sink in them, then losing a few now and then is not
going to be that big of a deal. So once a year a server dies and 10%
of your users find themselves logged out or their shopping carts gone.
Some things are an acceptable risk.

Use mysql/memcached based session store ?

On 8/22/06, Daya S. [email protected] wrote:


rm -rf / 2>/dev/null - http://null.in

Dont judge those who try and fail, judge those who fail to try…

I agree that some risks are acceptable and this may be the case for
80-90% of apps, but that was not what I as looking for.

Risks involved in loosing the session state can be very high depending
on the critical nature of the application, for instance loosing
transaction data due to session failure if this happens in a e-commerce
application will be disastrous.

If you store that kind of information in sessions you are nuts.
That’s not what sessions are for. If you insist on doing that, I’m
not sure what to say other than you need to seriously rethink your
design.

Coming back to our original subject, when we talk about load balancer
like lighty, Pen or Pound I don’t see any of them managing the session
synchronization between various nodes in the cluster. So basically none
of the currenty existing solutions provide transparent failover
capabilities like BEA Weblogic or IBM’s Websphere. This is one area
where RoR so far is behind the technology curve, but hopefully somebody
is already working on it.

Your requirements were for file based sessions. I highly doubt that
IBM or BEA can provide truly transparent failover under those
conditions. And like I said before, there are other solutions such as
squid, mod backhand, or hardware based solutions like BigIP or
ServerIrons. BEA or Websphere don’t do anything you couldn’t do with
rails, mod perl, zope, or other platforms. What you dont’ realize is
that those commercial solutions simply incorporate the other layers
into one single solution, but it’s still all the same. With the open
source solution you just need to put the pieces together on your own.
The open source approach takes more knowledge. You do need to have a
good grasp of what tools are available and how to use them.

On Aug 22, 2006, at 9:51 PM, Daya S. wrote:

Coming back to our original subject, when we talk about load balancer
like lighty, Pen or Pound I don’t see any of them managing the session
synchronization between various nodes in the cluster. So basically
none
of the currenty existing solutions provide transparent failover
capabilities like BEA Weblogic or IBM’s Websphere. This is one area
where RoR so far is behind the technology curve, but hopefully
somebody
is already working on it.

Wait, this isn’t correct at all.

Rails currently provides, behind all web servers including Apache,
Lighty,
Pen, Pound, Mongrel, Litespeed, hell even WEBrick(!) several
solutions for
handling session in failover conditions with 100% alacrity.

  1. Shared filesystem
  2. DB (ActiveRecordStore and SQLSessionStore)
  3. Memcached

Where do you think Weblogic and Websphere store the session data?


– Tom M.

snacktime wrote:

What you are referring to is same/similar to proxy.balance=“hash” option
of lighttpd, but I still can’t understand how it will help with session
state management in case of failover. Lets say Pen/Lighty assigs the a
mongrel instance to all requests from the same IP, if this mongrel
instance goes down, what happens to the session data and where will the
next request from the same IP go?

To another server. If you use sessions wisely and don’t just store
the whole kitchen sink in them, then losing a few now and then is not
going to be that big of a deal. So once a year a server dies and 10%
of your users find themselves logged out or their shopping carts gone.
Some things are an acceptable risk.

I agree that some risks are acceptable and this may be the case for
80-90% of apps, but that was not what I as looking for.

Risks involved in loosing the session state can be very high depending
on the critical nature of the application, for instance loosing
transaction data due to session failure if this happens in a e-commerce
application will be disastrous.

Coming back to our original subject, when we talk about load balancer
like lighty, Pen or Pound I don’t see any of them managing the session
synchronization between various nodes in the cluster. So basically none
of the currenty existing solutions provide transparent failover
capabilities like BEA Weblogic or IBM’s Websphere. This is one area
where RoR so far is behind the technology curve, but hopefully somebody
is already working on it.

-daya