Migrations and production databases

Rails
1

I was searching around to see how other people handle the issue of
updating a production database during a release of new version of an
application, and all I could find is that (according to one post) it
was typically done via migrations.

Migrations have a problem, though. After a while, the code changes to
a point where earlier migrations stop working. If you are doing
simple schema changes in your migrations, it will probably keep
working, but if you are modifying application data in your migrations,
then sooner or later someone will do something like add a new column
and write validation code for it, which causes earlier migrations that
worked with that table to fail. Usually this is only a problem if a
few weeks have elapsed since the last update.

Have other people run into this issue? How are you working around it
(or avoiding it)?

Thanks,
–Paul

2

Paul L. wrote:
[…]

Migrations have a problem, though. After a while, the code changes to
a point where earlier migrations stop working.
[…]
Usually this is only a problem if a
few weeks have elapsed since the last update.

Have other people run into this issue? How are you working around it
(or avoiding it)?

You shouldn’t be running lots of old migrations. In general, new
installations should be done with rake db:schema:load, not by running
every migration since the beginning.

Use migrations properly and this problem will go away.

Thanks,
–Paul

Best,

Marnen Laibow-Koser
http://www.marnen.org
[email protected]

3

I was actually talking about updating an existing production
database. In such a case, the migrations which would be run would be
just the ones between the previous release and the current one.

I have not tried your suggestion of rake db:schema:load but, judging
by the contents of schema.rb, this would drop all of the tables and
recreate them. We would lose both our application data and our user
data. Perhaps I misunderstood how you were thinking to use that?

On Jun 12, 2:28 pm, Marnen Laibow-Koser <rails-mailing-l…@andreas-

4

Thanks, Marnen, for mentioning seed_fu-- that is an intriguing plugin,
and I have been thinking about whether we could use it. I think we
have a situation in which it would not work well for us-- for at least
some of our tables.

As a concrete example, suppose you have a large table of product
information. If the table is large, loading it from a fixture (or
seed_fu file) could be slow, and anyway some of that information will
probably be edited through a separate administrative web application,
which would not be updating seed/fixture files.

Also, you might want occasionally to add a new field to the table
(another product code field) and compute initial values via a
migration. In this case it is convenient to use a migration because:

  • you want to try out the change with the code that uses it in your
    development area before checking in
  • computing initial values for the new field might be very quick and
    easy with ruby code
  • you want the other developers to get the data change when they do
    an update of the project files

If it is very important that the new product code field have a value
for all records, you might put some validation code into the model
class to prevent new records from being created without it. Doing so
would break any earlier migration that created product records.

I don’t really see any solution to this problem except identifying
which tables are like this and directly copying them (from a dump
file) into the production database. The migrations that modify them
can be coded so that they don’t do anything when run on the production
system, and that way the migrations can still be run to update other
tables (e.g. user data tables) which cannot be copied.

–Paul

On Jun 12, 3:26 pm, Marnen Laibow-Koser <rails-mailing-l…@andreas-

5

Paul L. wrote:
[…]

As a concrete example, suppose you have a large table of product
information. If the table is large, loading it from a fixture (or
seed_fu file) could be slow, and anyway some of that information will
probably be edited through a separate administrative web application,
which would not be updating seed/fixture files.

Right, although I suppose you could always dump the data to YAML or
something if you needed to.

Also, you might want occasionally to add a new field to the table
(another product code field) and compute initial values via a
migration.

Yes. This is entirely appropriate (although for myself, I like to set a
default value on the DB side if things lend themselves to that).
Migrations are about state transitions, and this recalculation is part
of getting the DB to the new consistent state.

[…]

If it is very important that the new product code field have a value
for all records, you might put some validation code into the model
class to prevent new records from being created without it. Doing so
would break any earlier migration that created product records.

Generally, you wouldn’t be running the earlier migration with the later
codebase. I confess that I’m having trouble coming up with a scenario
in which you’d ever do this.

I don’t really see any solution to this problem except identifying
which tables are like this and directly copying them (from a dump
file) into the production database.

No need. As I implied above, just don’t run old migrations with new
model files. In general, you should be releasing often enough that this
won’t happen, and in any case, if problems arise, you can always apply
one commit, run migrations, and then apply the next commit (assuming
your developers are committing as frequently as they should be).

The migrations that modify them
can be coded so that they don’t do anything when run on the production
system, and that way the migrations can still be run to update other
tables (e.g. user data tables) which cannot be copied.

No! Migrations are a fabulous tool for managing DB state. Work with
them in the ways I’ve been describing, not against them. Disabling
migrations in the way you are proposing suggests that something is wrong
with your release process. Fix the problem rather than breaking your
tools. :slight_smile:

–Paul

Best,

Marnen Laibow-Koser
http://www.marnen.org
[email protected]

6

Paul L. wrote:

I was actually talking about updating an existing production
database. In such a case, the migrations which would be run would be
just the ones between the previous release and the current one.

This should not cause problems in most cases. You mentioned that some
of the problems involved application data; either that sort of thing
should be removed from migrations or the migrations should provide for
it. (If you have more concrete examples, I’ll try to see how I’d handle
them.)

If by “application data” you mean things like tables of country codes or
shipping rates, these absolutely do not belong in migrations. Try
something like seed_fu.

Basically, migrations should provide all the changes necessary to get
the DB from one consistent state to the next. If you are having
problems with migrations, then they probably do not adhere to this
guideline.

I have not tried your suggestion of rake db:schema:load but, judging
by the contents of schema.rb, this would drop all of the tables and
recreate them. We would lose both our application data and our user
data.

Right.

Perhaps I misunderstood how you were thinking to use that?

You didn’t misunderstand; I thought you were talking about a different
situation.

Best,

Marnen Laibow-Koser
http://www.marnen.org
[email protected]