Reccommended work flow for unit tests and databases in ruby

Ruby
1

How does one go around creating unit tests for database tables during
developement. Is there a rock solid work flow plan that everyone uses.
Im new to dbs and relatively so with unit tests. I have my project and
it usually starts with a rough idea of what fields need to be in the
table. As i code away items in the db change so it makes it difficult to
write unit tests that will withstand these changes.

I know i need a test version of a database with static items but what id
like to do is for this test version to always reflect whats happening in
the development one. So if I add a field in the development one, the
test one gets one too.

I could just go and code this and probably get something that works but
is probably ugly. I dont like to reinvent the wheel, a bad one at that
so if anyone has some tips or pointers in the right direction id really
appreciate it.

This is for small projects so it doesnt have to industrial strength
bullet proof ninja stuff, small but elegant will be good enough

2

Adam A. wrote:

How does one go around creating unit tests for database tables during
developement. Is there a rock solid work flow plan that everyone uses.

A myth of software development is you can plan out and invent your
database
schema before writing your code. This myth is so pervasive that some
communities
(>cough< Oracle) actively prevent programmers from changing their
schemas.

Refactoring databases is a Best Practice, and all projects should
include
support scripts that overcome our database platform limitations.

Im new to dbs and relatively so with unit tests. I have my project and
it usually starts with a rough idea of what fields need to be in the
table. As i code away items in the db change so it makes it difficult to
write unit tests that will withstand these changes.

Investigate (and use!) the ActiveRecord system exemplified in Rails
fixtures and
migrations. When your program starts, only only only put in the
database the
very few fields you are actually using. Rails’s “script/generate model”
will
write a migration which adds that table to the db. Write tests for the
model’s
behavior, and these will require fields in the table. Edit the model’s
migration
file to add them. Repeat in tiny cycles, testing and integrating as
often as
possible. Never integrate if the tests break - revert if necessary.
(This is
more efficient than debugging!)

I know i need a test version of a database with static items but what id
like to do is for this test version to always reflect whats happening in
the development one. So if I add a field in the development one, the
test one gets one too.

Put sample data into the fixture files, and use it in the tests. Your
scripts
should then be able to migrate your test, development, and production
databases
in lockstep.

If you are not writing a Rails project, research how to get all these
benefits
without its framework. Rails can import fixture data into the
development
database with ‘rake db:migrate’.

When you deploy to a live site, at this point stop editing your initial
migration files, and only add new files that add and adjust your fields.

I could just go and code this and probably get something that works but
is probably ugly. I dont like to reinvent the wheel, a bad one at that
so if anyone has some tips or pointers in the right direction id really
appreciate it.

This is for small projects so it doesnt have to industrial strength
bullet proof ninja stuff, small but elegant will be good enough

These techniques - after you spend a little time implementing them -
keep
potentially big projects very small and manageable. I would use them on
projects
of any size!

3

Hi –

On Sun, 11 Jan 2009, Adam A. wrote:

test one gets one too.

I could just go and code this and probably get something that works but
is probably ugly. I dont like to reinvent the wheel, a bad one at that
so if anyone has some tips or pointers in the right direction id really
appreciate it.

This is for small projects so it doesnt have to industrial strength
bullet proof ninja stuff, small but elegant will be good enough

Rails has a bunch of Rake tasks that copy the schema from one database
to another. I don’t know whether your project is a Rails application,
but even if it isn’t, you might be able to integrate one of those
fairly easily, so that when you run your tests, the test database will
pick up any changes in the development database.

David

–
David A. Black / Ruby Power and Light, LLC
Ruby/Rails consulting & training: http://www.rubypal.com
Coming in 2009: The Well-Grounded Rubyist (The Well-Grounded Rubyist)

http://www.wishsight.com => Independent, social wishlist management!

4

David A. Black wrote:

Rails has a bunch of Rake tasks that copy the schema from one database
to another. I don’t know whether your project is a Rails application,
but even if it isn’t, you might be able to integrate one of those
fairly easily, so that when you run your tests, the test database will
pick up any changes in the development database.

However…

Every once in a while I feel the urge to think outside those
ActiveRecord
scripts’ box. For example, I want to write a method
nab_fixtures(‘models.yml’),
and it will reach out to a yml file and load it, on command, without
clawing its
way thru the uber-kewt fixtures() directive at the top of my suite.

When I feel that urge, I often have to repress it, because when I try to
re-use
the code inside fixtures(), it’s all “refactored” into a maze of fragile
and
un-reusable methods. When I rewrite the system that loads a yml file
into a
table myself, I replace fixtures() bugs with my own!

How the heck does anyone use ActiveRecord, with fixtures and migrations,
outside
Rails? How, for example, can that magic luvvable plugable Merb do it??

5

Hi –

On Mon, 12 Jan 2009, Phlip wrote:

Every once in a while I feel the urge to think outside those ActiveRecord
scripts’ box.

Rake will let you do that :slight_smile:

For example, I want to write a method
nab_fixtures(‘models.yml’), and it will reach out to a yml file and load it,
on command, without clawing its way thru the uber-kewt fixtures() directive
at the top of my suite.

When I feel that urge, I often have to repress it, because when I try to
re-use the code inside fixtures(), it’s all “refactored” into a maze of
fragile and un-reusable methods. When I rewrite the system that loads a yml
file into a table myself, I replace fixtures() bugs with my own!

I’d have to see the code to know exactly what’s going on. Is it a
problem reconciling foreign key references?

David

–
David A. Black / Ruby Power and Light, LLC
Ruby/Rails consulting & training: http://www.rubypal.com
Coming in 2009: The Well-Grounded Rubyist (The Well-Grounded Rubyist)

http://www.wishsight.com => Independent, social wishlist management!

6

David A. Black wrote:

I’d have to see the code to know exactly what’s going on. Is it a
problem reconciling foreign key references?

It’s a general, structural question, so I will ask it a different way.

This thread seems to propose one can borrow ActiveRecord, plus fixtures
and
migrations, and use them in a Ruby project that is (shocked gasp) not
Rails.

If so, then doing this, in Rails, should be easy, too!

class MyUnitTestSuite < ActiveSupport::TestCase
fixtures :foos

 def setup
    fixtures :bars
 end
 ...

end

For whatever reason, we want to load some fixtures the old-fashioned
way, and
some fixtures optionally. Maybe in the top of a test case, or maybe
conditionally in a setup. Or whatever.

If we promise the OP they can borrow ActiveRecord facilities, then
borrowing
them should indeed be easy. But whenever I try to call or re-use or
re-wrap the
methods inside fixtures, they screw up in horrible ways. This might be
a sign
of coupling!

7

Matt H. wrote:

This sounds great, but I’ve read in several places (sorry can’t find
references) that you should create a new migration to make changes to an
existing table.

If you have not yet deployed a migration to your production site, you
can go
back into it and tweak it. This is still within the inner TDD cycle,
because
your tests will cover the new versions of the tables.

I’ve tried before to modify an existing migration but unless I totally
wipe the DB, it doesn’t take notice, as the schema version isn’t
incremented.

Right - rake db:reset, and rake db:migrate:reset.

Tweaking existing migrations before deploying them (such as adding each
field in
a new table, as you TDD it) is a convenience that prevents your migrate
folder
from filling up with many trivial variations.

8

Phlip wrote:

Investigate (and use!) the ActiveRecord system exemplified in Rails
fixtures and migrations. When your program starts, only only only put
in the database the very few fields you are actually using. Rails’s
“script/generate model” will write a migration which adds that table to
the db. Write tests for the model’s behavior, and these will require
fields in the table. Edit the model’s migration file to add them. Repeat

This sounds great, but I’ve read in several places (sorry can’t find
references) that you should create a new migration to make changes to an
existing table.

I’ve tried before to modify an existing migration but unless I totally
wipe the DB, it doesn’t take notice, as the schema version isn’t
incremented.

Is this expected behavior or am I doing something wrong?

9

Well I’m just getting into TDD and thats going well, but I did notice
that when running rake:test, that it complains there are migrations that
need to be run before the tests will be performed, even though the tests
are running on a different database.

That’s because the migrations (should!) happen on the Development
database, and
this gets cloned into the Test database just before each test run…

10

Phlip wrote:

I’ve tried before to modify an existing migration but unless I totally
wipe the DB, it doesn’t take notice, as the schema version isn’t
incremented.

Right - rake db:reset, and rake db:migrate:reset.

Tweaking existing migrations before deploying them (such as adding each
field in a new table, as you TDD it) is a convenience that prevents your
migrate folder from filling up with many trivial variations.

Well I’m just getting into TDD and thats going well, but I did notice
that when running rake:test, that it complains there are migrations that
need to be run before the tests will be performed, even though the tests
are running on a different database.

Matt

11

Matt H. wrote:

So on the development database, it’s safe to add the columns or whatever
manually, update the migration to reflect the changes and test/code as
usual. Then when the production deployment happens it uses the
migrations from scratch?

Not safe to add columns “manually”. Add them to your migrations, rake
db:migrate:reset to rebuild your databases, manually test, deploy, and
lock down
those migrations.

The deploy script will migrate on the server, so all your existing data
don’t
get erased!

12

Phlip wrote:

Well I’m just getting into TDD and thats going well, but I did notice
that when running rake:test, that it complains there are migrations
that need to be run before the tests will be performed, even though
the tests are running on a different database.

That’s because the migrations (should!) happen on the Development
database, and this gets cloned into the Test database just before each
test run…

So on the development database, it’s safe to add the columns or whatever
manually, update the migration to reflect the changes and test/code as
usual. Then when the production deployment happens it uses the
migrations from scratch?

Thanks

Matt

13

Phlip wrote:

The deploy script will migrate on the server, so all your existing data
don’t get erased!

Ok, I’ll have a go at that, thanks for clearing that up.

Matt

14

hi guys, sorry this is actually a railess app. Just ruby and the
console. Ive found a few links though that show how to use active record
without rails though here including setting up migrations and fixutres.

http://rails.aizatto.com/category/activerecord/activerecord-without-rails/

and there is also this piece here

http://wiki.rubyonrails.org/rails/pages/HowToUseActiveRecordOutsideRails

does anyone know of anyother links?

From what i can tell those links dont discuss how to implement the
automatic generation of a test version of a development db that rails
does everytime you run a test. If anyone has a link etc id really
appreciate it!.

Thanks for the input so far

15

Hi –

On Mon, 12 Jan 2009, Adam A. wrote:

does anyone know of anyother links?

From what i can tell those links dont discuss how to implement the
automatic generation of a test version of a development db that rails
does everytime you run a test. If anyone has a link etc id really
appreciate it!.

I don’t have a link to a complete setup but what I’d recommend is have
a look at the Rake tasks that come with Rails, and adapt them. It’s
non-trivial enough that I can’t offer to do it on the spot, but
reasonably straightforward and will give you some Rake practice
anyway.

David

–
David A. Black / Ruby Power and Light, LLC
Ruby/Rails consulting & training: http://www.rubypal.com
Coming in 2009: The Well-Grounded Rubyist (The Well-Grounded Rubyist)

http://www.wishsight.com => Independent, social wishlist management!