Forum: Ruby on Rails Best way to convert a legacy database that's full of data?

Announcement (2017-05-07): is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see and for other Rails- und Ruby-related community platforms.
8749270955fc7c361eac3d6073a8ffcf?d=identicon&s=25 Maurice Gladwell (maurice-gladwell)
on 2007-05-30 14:17
I need to convert a legacy database - the usual non-Railish schema with
several million rows - to a Rails-compatible schema.

Are there any tools / tutorials / books / blog-posts that would help me
with that?

I need to solve problems like connecting to two databases at the same
time, querying the old non-Railish schema and using the data to create
Rails objects, etc.

Thanks in advance,
Maurice B. Gladwell
A recent Rails convert
4e69f7adb2b29a7778da7a52aabe74a0?d=identicon&s=25 Philip Nelson (Guest)
on 2007-05-30 14:36
(Received via mailing list)

Which DBMS ?

Each DBMS has different facilities available and different capabilities,
especially when it comes to linking two databases together.   If you are
migrating everything, you'd probably be a lot easier putting it all into
database which will save you have to use "federated" support to join

You also have a chance to think whether you want to change DBMS.

Are you talking about modifying the structures ?

If so, the book "Refactoring Databases : Evolutionary Database Design"
Scott Ambler and Pramodkumar J Sadalage (Addison-Wesley, ISBN-10:
ISBN-13: 978-0321293534) is a great primer for the types of changes you
have to make.

The biggest challenge you are going to face is making modifications
while not
breaking existing applications (I assume you have existing apps).

If I can be any help then drop me a line : especially if you wish to
DB2 as the DBMS of choice going forward.   DB2 has good Rails support,
getting better by the day, and the free DB2 Express-C has no limit on
database size, only on memory (4 gig) and processors (2 sockets, not


106bb8b6849d725ceea39a6a7165780b?d=identicon&s=25 Mr Dougal (Guest)
on 2007-05-30 14:42
(Received via mailing list)
The Rails recipes from the Pragmatic Programmers by Chad Flower, has
some tips on connecting to legacy databases.
That should point you in the right direction, or at least give you
some idea of what you're up against
8749270955fc7c361eac3d6073a8ffcf?d=identicon&s=25 Maurice Gladwell (maurice-gladwell)
on 2007-05-30 15:17
Hi Phil,

The DBMS is MySQL 5. It is serving a single, high-traffic web

My naive approach was much simpler than what you outlined - I was
thinking about writing a conversion script, that will:

1) go over the legacy database, pulling data from it

2) use data pulled in #1 to instantiate ActiveRecord objects, which will
be compatible with the new schema of the new database

3) save those ActiveRecord objects to the new database

Does that make sense?  I thought it would be the most conventional
approach, so there must be material describing such a procedure.

Bc6c0fd7296921fb7eae416f7c9e86a1?d=identicon&s=25 David Richards (Guest)
on 2007-05-30 15:18
(Received via mailing list)
You might also want to look at the ETL (Extract, Transform, Load)
approach found in active_warehouse.  If you can reduce your dependency
to cyclical connectivity (say daily or hourly), you can simplify the
process quite a bit.  Or, if you could ETL the legacy data and just
maintain connectivity for the sake of a few tables, you've reduced the
runtime load and complexity of your system quite a bit.
E91f664e02b73c99a508dacd6daecf37?d=identicon&s=25 JS (Guest)
on 2007-05-31 11:36
(Received via mailing list)
Hi Maurice,

On May 30, 2:17 pm, Maurice Gladwell <rails-mailing-l...@andreas-> wrote:
> Does that make sense?  I thought it would be the most conventional
> approach, so there must be material describing such a procedure.

This is exactly how we're doing it for one of our clients. We've
created AR objects for all the old tables, and then we're doing a find
on the old AR object, and looping over the result, to stick it into
the new AR models.
Something like this:
class ImportBibData < ActiveRecord::Base
  establish_connection :import
  set_table_name "BibData"
Then doing something like this...
  i_bib_items = ImportBibItem.find_all_by_doctype("M")
  i_bib_items.each do |i_bib_item|
    new_object =
It's not DRY at all, and it's quite a bit of a mess - but it's going
to be more or less a one-off import once the client freezes their old
(Access!!) DB and we start the import.

Hope that's of any help.



PS, my first post, so have mercy with me if I'm doing something wrong!
PPS, Hi all!
4e9b34b410463c486f8d1ca11f5ce892?d=identicon&s=25 Ron Phillips (paron)
on 2007-05-31 12:58
(Received via mailing list)
How are you connecting to Access? I hope this isn't too OT, but there
are a lot of threads in this and other forums saying it's a mess/not
worth it.

33a61362b1a372f38731719cda90c101?d=identicon&s=25 JJ_RoR (Guest)
on 2007-05-31 14:28
(Received via mailing list)
Re connecting to multiple databases. Bookmarked this 'how to' earlier
today anticipating a similar need:

On May 30, 8:17 am, Maurice Gladwell <rails-mailing-l...@andreas-
E91f664e02b73c99a508dacd6daecf37?d=identicon&s=25 JS (Guest)
on 2007-06-01 10:35
(Received via mailing list)
On May 31, 11:57 am, paron <> wrote:
> How are you connecting to Access? I hope this isn't too OT, but there
> are a lot of threads in this and other forums saying it's a mess/not
> worth it.

It is a mess - so we don't! :D We use the mdb-tools package to extract
the data as CSV, and the schema as SQL. We then convert the schema to
MySQL compatible SQL, and dump it into MySQL. From there we run a
mysqlimport on the CSV files to populate the DB with the old data.

It's a mess for sure, but once it's in the MySQL DB it's much nicer to
work with, and once it's in the new structure we should be all happy
smiling Railers...

This topic is locked and can not be replied to.