Forum: Ruby on Rails Updating multiple databases at the same time

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
E884d559ed311e48ec9831b895719b2a?d=identicon&s=25 Yanni Mac (yannimac)
on 2009-01-13 21:45
I have an application that is load balanced.  I have a master database
which I update once a day.  Then I push the raw mysql files to all other
servers so they are the same as the master.  This works fine, but there
are a few situations where I need all databases to update in real-time.
What would be the best way to achieve this in rails?

Here is an example of what I am trying to do.  I want to update the name
of a product on my website.  Here is the controller:

def update_product_name
  product = Product.find(params[:id])
  product.name = params[:name]
  product.save
  #Say I want to save this to 3 other databases
  #can I do this with activerecord?
  #PSUEDO CODE... I know it cant do this
  product.save(mysql_host2)
  product.save(mysql_host3)
  product.save(mysql_host4)
end

Any ideas?  Alternative ways to do this?
E884d559ed311e48ec9831b895719b2a?d=identicon&s=25 Yanni Mac (yannimac)
on 2009-01-13 21:54
Another thing.. I don't want to do this per model.  This would happen in
very few situations, so I would like to code this in the controller if
possible (I will probably add a method to application.rb and call it in
the controller when appropriate).

Yanni Mac wrote:
> I have an application that is load balanced.  I have a master database
> which I update once a day.  Then I push the raw mysql files to all other
> servers so they are the same as the master.  This works fine, but there
> are a few situations where I need all databases to update in real-time.
> What would be the best way to achieve this in rails?
>
> Here is an example of what I am trying to do.  I want to update the name
> of a product on my website.  Here is the controller:
>
> def update_product_name
>   product = Product.find(params[:id])
>   product.name = params[:name]
>   product.save
>   #Say I want to save this to 3 other databases
>   #can I do this with activerecord?
>   #PSUEDO CODE... I know it cant do this
>   product.save(mysql_host2)
>   product.save(mysql_host3)
>   product.save(mysql_host4)
> end
>
> Any ideas?  Alternative ways to do this?
89441a6c74647d292f5cc951eae24cfa?d=identicon&s=25 Jack Christensen (Guest)
on 2009-01-13 22:47
(Received via mailing list)
Yanni Mac wrote:
>   product = Product.find(params[:id])
> Any ideas?  Alternative ways to do this?
>
You could use MySQL's built-in replication support.

--
Jack Christensen
jackc@hylesanderson.edu
4cd3615da7c39beacaaab174267ad8b2?d=identicon&s=25 Davi Vidal (Guest)
on 2009-01-13 22:51
(Received via mailing list)
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Jack Christensen wrote:
>> def update_product_name
>>
>> Any ideas?  Alternative ways to do this?
>>
> You could use MySQL's built-in replication support.
>


  What if I want it on other SQL server?

davi
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkltDOUACgkQ76Bs0E5RGKO3twCePRLi1tyfJM1FTGgQX3jiT7Co
BDgAnilihvAKaFQE+17ifR4RaASgLqjV
=a/a+
-----END PGP SIGNATURE-----
A91bd6cef23eb3516245a092e196c4da?d=identicon&s=25 Maurício Linhares (mauricio)
on 2009-01-14 00:09
(Received via mailing list)
A "not so easy" way to solve this is to write your own database
adapter, that would send the real SQL code to many different databases
(your adapter would contain an array of other adapters that would
receive the real SQL). That's how tools like C-JDBC do it.

But if you need this for MySQL only, you should try to use the
database replication mechanism.

-
Maurício Linhares
http://alinhavado.wordpress.com/ (pt-br) | http://blog.codevader.com/
(en)
81b61875e41eaa58887543635d556fca?d=identicon&s=25 Frederick Cheung (Guest)
on 2009-01-14 01:28
(Received via mailing list)
On 13 Jan 2009, at 23:08, Maurício Linhares wrote:

>
> A "not so easy" way to solve this is to write your own database
> adapter, that would send the real SQL code to many different databases
> (your adapter would contain an array of other adapters that would
> receive the real SQL). That's how tools like C-JDBC do it.

Doesn't that boil down to reinventing replication ?

Fred
E884d559ed311e48ec9831b895719b2a?d=identicon&s=25 Yanni Mac (yannimac)
on 2009-01-14 02:05
The problem is that I don't want to replicate everything.  The way I
understand it, and correct me if I am wrong b/c I have never set up
master/slave replication in MySQL, every time there is a
insert/update/delete to the database, it will replicate to the others
automatically in real-time.  I don't want this to happen, because I am
running a script that updates 16 million rows and takes 20 hours to
complete... replication would affect performance on the live slaves.
In parallel to this script running I am making live changes to the data
in my app via a web interface.  These changes (small portion compared to
the script running) will need to write to all 4 databases at the same
time.  I am assuming mysql replication is all or none and can't do
this... meaning replicate some times and not others??
2b1df44ef63bbf410976ef082d0d5d8b?d=identicon&s=25 Yacobus Reinhart (malioboro)
on 2009-01-14 02:40
> def update_product_name
>   product = Product.find(params[:id])
>   product.name = params[:name]
>   product.save
>   #Say I want to save this to 3 other databases
>   #can I do this with activerecord?
>   #PSUEDO CODE... I know it cant do this
>   product.save(mysql_host2)
>   product.save(mysql_host3)
>   product.save(mysql_host4)
> end
>
> Any ideas?  Alternative ways to do this?

I would like to know what their connection are already configured to
your applications? because to connect multiple databases are different
configuration or you can do manually connection in your AR (if this is a
false solution, please give me issues and resolution ):

require "activerecord"

def update_product_name


// i am not sure about this script is put in Class AR
// Maybe it is suitable to put out of class AR
ActiveRecord::Base.establish_connection(
  :adapter=>"mysql",
  :database=>"theSame",
  :username=>"theSame_theSame",
  :password=>"1234567890"
  :host=>"theSame-2.com")

   saving_name(params[:id],params[:name])
end

// you can make array to descript your multiple login then loop it.

end


def saving_name(id,name)
   product = Product.find(id)
   product.name = name
   product.save
   ActiveRecord::Base.remove_connection
end


- Ruby Servant -
90e7e04c730e1b747288051af34c3bb0?d=identicon&s=25 rp8 -_- (rp8)
on 2009-01-14 02:40
Yanni Mac wrote:
> I have an application that is load balanced.  I have a master database
> which I update once a day.  Then I push the raw mysql files to all other
> servers so they are the same as the master.  This works fine, but there
> are a few situations where I need all databases to update in real-time.
> What would be the best way to achieve this in rails?
>
> Here is an example of what I am trying to do.  I want to update the name
> of a product on my website.  Here is the controller:
>
> def update_product_name
>   product = Product.find(params[:id])
>   product.name = params[:name]
>   product.save
>   #Say I want to save this to 3 other databases
>   #can I do this with activerecord?
>   #PSUEDO CODE... I know it cant do this
>   product.save(mysql_host2)
>   product.save(mysql_host3)
>   product.save(mysql_host4)
> end
>
> Any ideas?  Alternative ways to do this?

Before we have an adapter capable of supporting two-phase commit
distributed transaction, it will be difficult to have a single
transaction to commit changes to more than one database.

One alternative is to use message queuing. Essentially you generate
messages destined to each DB and each DB will process its queue. Of
course, you might have to devise some additional messages to compensate
any actions rolled back in the master/slave due to some reasons like
errors etc.

Regards,

rp8
=======================
http://competo.com
E884d559ed311e48ec9831b895719b2a?d=identicon&s=25 Yanni Mac (yannimac)
on 2009-01-14 03:26
Rails Terrorist wrote:
>
>> def update_product_name
>>   product = Product.find(params[:id])
>>   product.name = params[:name]
>>   product.save
>>   #Say I want to save this to 3 other databases
>>   #can I do this with activerecord?
>>   #PSUEDO CODE... I know it cant do this
>>   product.save(mysql_host2)
>>   product.save(mysql_host3)
>>   product.save(mysql_host4)
>> end
>>
>> Any ideas?  Alternative ways to do this?
>
> I would like to know what their connection are already configured to
> your applications? because to connect multiple databases are different
> configuration or you can do manually connection in your AR (if this is a
> false solution, please give me issues and resolution ):
>
> require "activerecord"
>
> def update_product_name
>
>
> // i am not sure about this script is put in Class AR
> // Maybe it is suitable to put out of class AR
> ActiveRecord::Base.establish_connection(
>   :adapter=>"mysql",
>   :database=>"theSame",
>   :username=>"theSame_theSame",
>   :password=>"1234567890"
>   :host=>"theSame-2.com")
>
>    saving_name(params[:id],params[:name])
> end
>
> // you can make array to descript your multiple login then loop it.
>
> end
>
>
> def saving_name(id,name)
>    product = Product.find(id)
>    product.name = name
>    product.save
>    ActiveRecord::Base.remove_connection
> end
>
>
> - Ruby Servant -

Yeah.. I think this is sort of what I am looking for.  I was playing
around with something like this (still tweaking it after getting some
errors)

slaves = ["host2","host3","host4"]
for slave in slaves
 ActiveRecord::Base.establish_connection(ActiveRecord::Base.configurations[RAILS_ENV].merge('host'
=> slave))
 unique_product.save
end
#set back to original host
ActiveRecord::Base.establish_connection(ActiveRecord::Base.configurations[RAILS_ENV].merge('host'
=> original_host))
This topic is locked and can not be replied to.