TDD and migrations?

TDD, SVN and migrations seem to be a natural combination for model
development: create a test, add the column with a migration, write the
code, and check it all in when you’re done.

The trouble is that this ends up creating one migration per field. I
guess
it’s not horrible in and of itself, but it does have a code smell to it.

Has anyone come up with a different way of doing TDD? It would seem
that
creating multiple columns per migration would break TDD, since you’d
then
end up writing tests that passed instantly.

Jay L.

On 6/28/06, Jay L. [email protected] wrote:

Jay L.

Just refactor:

  • write a failing test

  • add a migration, adding the required field

  • code to make the test pass

  • write another failing test

  • add any other migrations needed for other business stuff

  • code to make tests pass

  • refactor migrations to consolidate

  • refactor other code

  • THEN checkin to SVN, once all the tests pass

  • repeat as needed…

  • Rob

    http://www.robsanheim.com
    http://www.seekingalpha.com
    http://www.ajaxian.com

On 6/28/06, Jay L. [email protected] wrote:

TDD, SVN and migrations seem to be a natural combination for model
development: create a test, add the column with a migration, write the
code, and check it all in when you’re done.

The trouble is that this ends up creating one migration per field. I guess
it’s not horrible in and of itself, but it does have a code smell to it.

Has anyone come up with a different way of doing TDD? It would seem that
creating multiple columns per migration would break TDD, since you’d then
end up writing tests that passed instantly.

  1. Write test.
  2. Create migration.
  3. Write code that passes test.
  4. Write another test.
  5. Modify migration created in step 2.
  6. Write code that passes test.
  7. Checkin.

I don’t see the problem here.

Joe

On Wed, 28 Jun 2006 20:06:58 -0700, Joe Van D. wrote:

  1. Modify migration created in step 2.

I didn’t know it was “legit” to modify a migration! Do you migrate back
down and up to re-execute? (That would actually seem to be a nice
robust
technique…)

Jay

I didn’t know it was “legit” to modify a migration! Do you migrate back
down and up to re-execute? (That would actually seem to be a nice robust
technique…)

When working on a new migration (not yet checked in), I just migrate
back /
modify / migrate to keep things consistent.

I don’t modify already checked in migrations (especially if already used
in
production), I prefer to add a new one to avoid messing things up.

hth

Thibaut

Thibaut Barrère wrote:

I didn’t know it was “legit” to modify a migration! Do you migrate back
down and up to re-execute? (That would actually seem to be a nice robust
technique…)

When working on a new migration (not yet checked in), I just migrate
back /
modify / migrate to keep things consistent.

I don’t modify already checked in migrations (especially if already used
in
production), I prefer to add a new one to avoid messing things up.

hth

Thibaut

I prefer a similar but slightly different way of protecting production
deployments. I prefer to have a /trunk, /tags, and /stable directory in
my repositories.

With this steup…

Trunk is unstable and only for use by the development team. Anything in
trunk may change (including migrations newer than the last stable
release). Any time a developer updates they can assume they will have
to re-migrate their development database. When there is a release it is
copied to /tags. The most current release also has a copy in stable.

n 6/29/06, Jay L. [email protected] wrote:

On Wed, 28 Jun 2006 20:06:58 -0700, Joe Van D. wrote:

  1. Modify migration created in step 2.

I didn’t know it was “legit” to modify a migration! Do you migrate back
down and up to re-execute? (That would actually seem to be a nice robust
technique…)

Jay

Yeah, its fine if its an “in progress” migration that hasn’t been
checked in yet. Just keep migrating to that version to pick up your
chances, then when its solid check it in and go.

We’ve had to change checked in migrations before, but only because of
model name changes that made earlier migrations invalid…

  • rob

On 6/29/06, Jay L. [email protected] wrote:

On Wed, 28 Jun 2006 20:06:58 -0700, Joe Van D. wrote:

  1. Modify migration created in step 2.

I didn’t know it was “legit” to modify a migration! Do you migrate back
down and up to re-execute? (That would actually seem to be a nice robust
technique…)

Actually, I have a script (scripts/reload_everything.sh) that:

  1. Stops mongrel
  2. Deletes the test and development databases
  3. Creates the test and development databases
  4. Runs the migrations
  5. Populates the development database with data from the test fixtures
  6. Runs the unit and functional tests.
  7. Starts mongrel

So, I’d make a change to the migration script, and then run the
reload_everything.sh script.

Also, you can create a script or rake task called “commit”. You’d run
this every time you check something in. This task or script would:

  1. Do a svn update
  2. Check to see if there are any unadded, conflicting, or deleted
    files (and fail if there are)
  3. Run the reload_everything script above
  4. If everything passed ok, then it runs svn commit, which asks you
    for the commit message.

So, after i make a change, I run the script/reload_everything program.
When I want to commit, I run ‘rake commit’. Makes it pretty easy.

I highly recommend it.

On 6/30/06, Alan F. [email protected] wrote:

That sounds like a good thing to have, The only problem with using this
approach all the time is you never get to test your migrations. If you
always start from zero, how can you be sure when you incrementally
migrate production you haven’t messed something up ?

On the TDD questions, I think adding a single field at a time is
overkill, and I’ve been a vocal XP practitioner and coach since 1999.
You don’t need to drive in first-gear all the time.

When you reload everything, the migrations are done incrementally.
Under what circumstance would I mess something up?

Excellent. Mind posting a link to the scripts?

Joe Van D. wrote:

Actually, I have a script (scripts/reload_everything.sh) that:

  1. Stops mongrel
  2. Deletes the test and development databases
  3. Creates the test and development databases
  4. Runs the migrations
  5. Populates the development database with data from the test fixtures
  6. Runs the unit and functional tests.
  7. Starts mongrel

So, I’d make a change to the migration script, and then run the
reload_everything.sh script.

Also, you can create a script or rake task called “commit”. You’d run
this every time you check something in. This task or script would:
So, after i make a change, I run the script/reload_everything program.
When I want to commit, I run ‘rake commit’. Makes it pretty easy.

I highly recommend it.

Joe Van D. wrote:

When you reload everything, the migrations are done incrementally.
Under what circumstance would I mess something up?

When you reload everything from scratch, that starts with an empty
database, when you migrate production, you have existing data to worry
about. It might all be fine, but it pays to run your new stuff against
a production-like database once in a while.

Outside of rails, we work this way at my current Java client, and we can
run our migrations against yesterdays production data each time, so we
know we won’t lose prod data.

A.

Joe Van D. wrote:

Actually, I have a script (scripts/reload_everything.sh) that:

  1. Stops mongrel
  2. Deletes the test and development databases
  3. Creates the test and development databases
  4. Runs the migrations
  5. Populates the development database with data from the test fixtures
  6. Runs the unit and functional tests.
  7. Starts mongrel

That sounds like a good thing to have, The only problem with using this
approach all the time is you never get to test your migrations. If you
always start from zero, how can you be sure when you incrementally
migrate production you haven’t messed something up ?

On the TDD questions, I think adding a single field at a time is
overkill, and I’ve been a vocal XP practitioner and coach since 1999.
You don’t need to drive in first-gear all the time.

Alan

On 6/30/06, Robert J. [email protected] wrote:

  1. Runs the unit and functional tests.
  2. Starts mongrel

So, I’d make a change to the migration script, and then run the
reload_everything.sh script.

Also, you can create a script or rake task called “commit”. You’d run
this every time you check something in. This task or script would:
So, after i make a change, I run the script/reload_everything program.
When I want to commit, I run ‘rake commit’. Makes it pretty easy.

Sure, it’s pretty simple. (using postgres)

===============================================
script/reload_everything.sh.sample
(script/reload_everything.sh is ignored by subversion, so on a fresh
checkout, a developer would copy the sample file)

#!/bin/sh

mongrel_rails stop

echo
echo " *** Dropping the development and test databases…"
dropdb tanga_test
dropdb tanga_development

echo
echo " *** Creating the development and test databases… "
createdb tanga_test
createdb tanga_development

echo &&
echo " *** Running the migration scripts…" &&
rake migrate &&

echo &&
echo " *** Loading the test fixtures… " &&
rake load_fixtures &&

echo &&
echo " *** Running the tests" &&
rake default

mongrel_rails start -d

==================================================
lib/tasks/svn_commit.rake
(run with ‘rake commit’)

desc “Tests the files, then commits”
task :commit do |t|

check to see if there’s unadded files. If there are, I’m not

saving anything, darn it.
if (unadded_files = system “svn status | grep -v ‘.swp’ | grep
[C?!~]”)
puts “There were unadded, removed, screwed up, or conflicting
files!”
else
system “script/reload_everything.sh && svn commit”
end
end

On Fri, 30 Jun 2006 14:21:18 +0200, Alan F. wrote:

On the TDD questions, I think adding a single field at a time is
overkill, and I’ve been a vocal XP practitioner and coach since 1999.
You don’t need to drive in first-gear all the time.

Well, my (novice) thoughts were that, although you could pretty safely
add
multiple fields to the database at once, you’d end up on various side
trips
setting up the validations, edge cases for testing, etc., which would
result in multiple test failures. And some of my best test cases were
the
result of other tests that I thought would fail (before doing any
coding),
and actually passed; those are things I’d be likely to miss if I coded
the
whole object at once.

What does second or third gear look like to you? I’d imagine that too
much coding in a row gets away from TDD, but I’m still learning the
ropes.

Jay L.

On 6/30/06, Joe Van D. [email protected] wrote:

  1. Populates the development database with data from the test fixtures

echo &&

check to see if there’s unadded files. If there are, I’m not

saving anything, darn it.
if (unadded_files = system “svn status | grep -v ‘.swp’ | grep [C?!~]”)
puts “There were unadded, removed, screwed up, or conflicting files!”
else
system “script/reload_everything.sh && svn commit”
end
end

Oops, had a little problems with copy and pasting, imagine that.

desc “Tests the files, then commits”
task :commit do |t|

check to see if there’s unadded files. If there are, I’m not

saving anything, darn it.
if (unadded_files = system “svn status | grep -v ‘.swp’ | grep
^[C?!~]”)
puts “There were unadded or removed files!”
else
system “script/reload_everything.sh && svn commit”
end
end