Enterprise Database Tier (XA transactions, COBRA and MIDAS)

I am developing a large scale financial application which is going to be
very intensive write-wise, and use a lot of transactions.

I can see how the the web and application servers can be scaled and load
balanced quite easily, but not the database tier.

The setups I have looked at so far seem to deal with coping with a lot
of reads, and minimal writes - this maybe because they are largely
content sites not web applications as such. So they use one Master
database which is replicated to fast memcached read-only databases and
loads
of static content caching.

I have looked at Multi Master Replicated setups but am worried about the
lag time between replication. The integrity of the data is critical.

The only real (and deal breaking in this case) downside to Rails is the
scalability of the database for this specific appliction type. Support
for XA transactions which would allow your tables to be spread over
multiple databases and solve this. Apart from that is seems any one
database vendor can be choosen for its particular features and
approprietness for an install size/needs. In the past we have used COBRA
and Borlands MIDAS to connect our applications to the database.

I know some people are touchy about saying Rails is not Enterprise ready
:wink: Its only version 1, you wouldn’t expect it to be.

Is is likly Rails will develop in these areas? Being at only version 1
there are a number of ways Rails could go. Rails is highly suited to a
certain type of applications such as Odeo, Basecamp and CMS’s. Is this
the type of application Rails will remain suited to or will it be
developed to encompass what I call enterprise features as needed for
financial, mission critical applications.

I look forward to your thoughts.

On May 4, 2006, at 3:25 AM, Kris wrote:

I am developing a large scale financial application which is going
to be
very intensive write-wise, and use a lot of transactions.

I can see how the the web and application servers can be scaled and
load
balanced quite easily, but not the database tier.

There’s an Oracle adapter. Isn’t Oracle scalable enough?


– Tom M.

Maybe, if it was not so expensive.

I have to be honest and say I have little experience with Oracle, but it
seems there are certain features that Rails needs to have to gain
certain Enterprise scaleability - like multiple database transactions
(XA), they would make me feel happy to deploy using Rails - knowing I
can scale by putting each table on its own database.

The other big problem is you cant distribute code without it being
opensource. I can’t sell me application commerically as a product only
as a hosted service.

From ActiveRecords point of view how does it deal with paritioned
tables?
If I was partioning by say user, like LiveJournel does, and I had the
first 1000 users on users1 table and the second 1000 on users2 table,
would AR be able to do a seemless query across both tables? Or would I
need to write a plugin?

I am actually considering writing an acts_as which overrides the create
(in AR base) or insert (in AbstractAdaptor) method to save the object or
SQL statement to file (Pstore maybe). The file acts as a que/stack and
one by one a process takes each entry and does the actual save. That way
if the database has to large a load it is buffered and if the database
goes down it will not matter, the application servers will continue to
write to file and as soon as the db comes back up it will get the
missing stuff. Id’s would need to be generated by the application
instead of the database.
The whole thing could actually be abstracted to a new layer which AR
interacts with it over TCP. AR could send objects using dRuby.

Thoughts?

Tom M. wrote:

On May 4, 2006, at 3:25 AM, Kris wrote:

I am developing a large scale financial application which is going
to be
very intensive write-wise, and use a lot of transactions.

I can see how the the web and application servers can be scaled and
load
balanced quite easily, but not the database tier.

There’s an Oracle adapter. Isn’t Oracle scalable enough?


– Tom M.