Simple solution to the 'twitter' scaling problem

By now i’m sure we’ve all read the comments bouncing around in blogs
about the twitter scaling problem: namely they were worried about how to
use rails against multiple cloned database backends.

Well, whats wrong with having a pair (or more) of cloned database hosts,
and editing the tcpip ‘hosts’ file on your different rails servers so
that half your rails servers are pointing at one database, and half are
pointing at the other?

Wouldn’t that be pretty simple and do the job? Or am i missing
something?

On 4/17/07, chris hulbert [email protected] wrote:

Wouldn’t that be pretty simple and do the job? Or am i missing
something?

You couldn’t do this, because you would get ID conflicts from
auto-increment. Two inserts on the same table on each DB would
conflict.

However, there are facilities for multi-master replication in some
databases, such as Mysql 5 [1] You need to have some sort of code to
handle connections, at least to watch for dead servers and failover.
Also, you’d want to designate some read-only servers pretty early on
because writing across many masters would get expensive quickly. So
then you are back to square one, as need to handle writing to certain
databases and reading from others. Nic’s plugin would handle that, I
believe.

In other words, scaling can start simple but it gets complicated quick
:).

1 -

  • Rob

I think you got it right here:

Rob S. wrote:

because writing across many masters would get expensive quickly.