Forum: Ruby reccommended work flow for unit tests and databases in ruby

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
Adam A. (Guest)
on 2009-01-11 15:16
How does one go around creating unit tests for database tables during
developement. Is there a rock solid work flow plan that everyone uses.
Im new to dbs and relatively so with unit tests. I have my project and
it usually starts with a rough idea of what fields need to be in the
table. As i code away items in the db change so it makes it difficult to
write unit tests that will withstand these changes.

I know i need a test version of a database with static items but what id
like to do is for this test version to always reflect whats happening in
the development one. So if I add a field in the development one, the
test one gets one too.

I could just go and code this and probably get something that works but
is probably ugly. I dont like to reinvent the wheel, a bad one at that
so if anyone has some tips or pointers in the right direction id really
appreciate it.

This is for small projects so it doesnt have to industrial strength
bullet proof ninja stuff, small but elegant will be good enough
Phlip (Guest)
on 2009-01-11 17:20
(Received via mailing list)
Adam A. wrote:

> How does one go around creating unit tests for database tables during
> developement. Is there a rock solid work flow plan that everyone uses.

A myth of software development is you can plan out and invent your
database
schema before writing your code. This myth is so pervasive that some
communities
(>cough< Oracle) actively prevent programmers from changing their
schemas.

Refactoring databases is a Best Practice, and all projects should
include
support scripts that overcome our database platform limitations.

> Im new to dbs and relatively so with unit tests. I have my project and
> it usually starts with a rough idea of what fields need to be in the
> table. As i code away items in the db change so it makes it difficult to
> write unit tests that will withstand these changes.

Investigate (and use!) the ActiveRecord system exemplified in Rails
fixtures and
migrations. When your program starts, only only _only_ put in the
database the
very few fields you are actually using. Rails's "script/generate model"
will
write a migration which adds that table to the db. Write tests for the
model's
behavior, and these will require fields in the table. Edit the model's
migration
file to add them. Repeat in tiny cycles, testing and integrating as
often as
possible. Never integrate if the tests break - revert if necessary.
(This is
more efficient than debugging!)

> I know i need a test version of a database with static items but what id
> like to do is for this test version to always reflect whats happening in
> the development one. So if I add a field in the development one, the
> test one gets one too.

Put sample data into the fixture files, and use it in the tests. Your
scripts
should then be able to migrate your test, development, and production
databases
in lockstep.

If you are not writing a Rails project, research how to get all these
benefits
without its framework. Rails can import fixture data into the
development
database with 'rake db:migrate'.

When you deploy to a live site, at this point stop editing your initial
migration files, and only add new files that add and adjust your fields.

> I could just go and code this and probably get something that works but
> is probably ugly. I dont like to reinvent the wheel, a bad one at that
> so if anyone has some tips or pointers in the right direction id really
> appreciate it.
>
> This is for small projects so it doesnt have to industrial strength
> bullet proof ninja stuff, small but elegant will be good enough

These techniques - after you spend a little time implementing them -
keep
potentially big projects very small and manageable. I would use them on
projects
of any size!
David A. Black (Guest)
on 2009-01-11 17:53
(Received via mailing list)
Hi --

On Sun, 11 Jan 2009, Adam A. wrote:

> test one gets one too.
>
> I could just go and code this and probably get something that works but
> is probably ugly. I dont like to reinvent the wheel, a bad one at that
> so if anyone has some tips or pointers in the right direction id really
> appreciate it.
>
> This is for small projects so it doesnt have to industrial strength
> bullet proof ninja stuff, small but elegant will be good enough

Rails has a bunch of Rake tasks that copy the schema from one database
to another. I don't know whether your project is a Rails application,
but even if it isn't, you might be able to integrate one of those
fairly easily, so that when you run your tests, the test database will
pick up any changes in the development database.


David

--
David A. Black / Ruby Power and Light, LLC
Ruby/Rails consulting & training: http://www.rubypal.com
Coming in 2009: The Well-Grounded Rubyist (http://manning.com/black2)

http://www.wishsight.com => Independent, social wishlist management!
Phlip (Guest)
on 2009-01-11 18:05
(Received via mailing list)
David A. Black wrote:

> Rails has a bunch of Rake tasks that copy the schema from one database
> to another. I don't know whether your project is a Rails application,
> but even if it isn't, you might be able to integrate one of those
> fairly easily, so that when you run your tests, the test database will
> pick up any changes in the development database.

However...

Every once in a while I feel the urge to think outside those
ActiveRecord
scripts' box. For example, I want to write a method
nab_fixtures('models.yml'),
and it will reach out to a yml file and load it, on command, without
clawing its
way thru the uber-kewt fixtures() directive at the top of my suite.

When I feel that urge, I often have to repress it, because when I try to
re-use
the code inside fixtures(), it's all "refactored" into a maze of fragile
and
un-reusable methods. When I rewrite the system that loads a yml file
into a
table myself, I replace fixtures() bugs with my own!

How the heck does anyone use ActiveRecord, with fixtures and migrations,
outside
Rails? How, for example, can that magic luvvable plugable Merb do it??
David A. Black (Guest)
on 2009-01-11 18:32
(Received via mailing list)
Hi --

On Mon, 12 Jan 2009, Phlip wrote:

> Every once in a while I feel the urge to think outside those ActiveRecord
> scripts' box.

Rake will let you do that :-)

> For example, I want to write a method
> nab_fixtures('models.yml'), and it will reach out to a yml file and load it,
> on command, without clawing its way thru the uber-kewt fixtures() directive
> at the top of my suite.
>
> When I feel that urge, I often have to repress it, because when I try to
> re-use the code inside fixtures(), it's all "refactored" into a maze of
> fragile and un-reusable methods. When I rewrite the system that loads a yml
> file into a table myself, I replace fixtures() bugs with my own!

I'd have to see the code to know exactly what's going on. Is it a
problem reconciling foreign key references?


David

--
David A. Black / Ruby Power and Light, LLC
Ruby/Rails consulting & training: http://www.rubypal.com
Coming in 2009: The Well-Grounded Rubyist (http://manning.com/black2)

http://www.wishsight.com => Independent, social wishlist management!
Phlip (Guest)
on 2009-01-11 19:45
(Received via mailing list)
David A. Black wrote:

> I'd have to see the code to know exactly what's going on. Is it a
> problem reconciling foreign key references?

It's a general, structural question, so I will ask it a different way.

This thread seems to propose one can borrow ActiveRecord, plus fixtures
and
migrations, and use them in a Ruby project that is (shocked gasp) not
Rails.

If so, then doing this, in Rails, should be easy, too!

   class MyUnitTestSuite < ActiveSupport::TestCase
     fixtures :foos

     def setup
        fixtures :bars
     end
     ...
   end

For whatever reason, we want to load some fixtures the old-fashioned
way, and
some fixtures optionally. Maybe in the top of a test case, or maybe
conditionally in a setup. Or whatever.

If we promise the OP they can borrow ActiveRecord facilities, then
borrowing
them should indeed be easy. But whenever I try to call or re-use or
re-wrap the
methods inside fixtures, they screw up in horrible ways. This _might_ be
a sign
of coupling!
Matt H. (Guest)
on 2009-01-11 20:07
(Received via mailing list)
Phlip wrote:
> Investigate (and use!) the ActiveRecord system exemplified in Rails
> fixtures and migrations. When your program starts, only only _only_ put
> in the database the very few fields you are actually using. Rails's
> "script/generate model" will write a migration which adds that table to
> the db. Write tests for the model's behavior, and these will require
> fields in the table. Edit the model's migration file to add them. Repeat

This sounds great, but I've read in several places (sorry can't find
references) that you should create a new migration to make changes to an
existing table.

I've tried before to modify an existing migration but unless I totally
wipe the DB, it doesn't take notice, as the schema version isn't
incremented.

Is this expected behavior or am I doing something wrong?
Phlip (Guest)
on 2009-01-11 20:40
(Received via mailing list)
Matt H. wrote:

> This sounds great, but I've read in several places (sorry can't find
> references) that you should create a new migration to make changes to an
> existing table.

If you have not yet deployed a migration to your production site, you
can go
back into it and tweak it. This is still within the inner TDD cycle,
because
your tests will cover the new versions of the tables.

> I've tried before to modify an existing migration but unless I totally
> wipe the DB, it doesn't take notice, as the schema version isn't
> incremented.

Right - rake db:reset, and rake db:migrate:reset.

Tweaking existing migrations before deploying them (such as adding each
field in
a new table, as you TDD it) is a convenience that prevents your migrate
folder
from filling up with many trivial variations.
Matt H. (Guest)
on 2009-01-11 20:57
(Received via mailing list)
Phlip wrote:
>> I've tried before to modify an existing migration but unless I totally
>> wipe the DB, it doesn't take notice, as the schema version isn't
>> incremented.
>
> Right - rake db:reset, and rake db:migrate:reset.
>
> Tweaking existing migrations before deploying them (such as adding each
> field in a new table, as you TDD it) is a convenience that prevents your
> migrate folder from filling up with many trivial variations.
>

Well I'm just getting into TDD and thats going well, but I did notice
that when running rake:test, that it complains there are migrations that
need to be run before the tests will be performed, even though the tests
are running on a different database.

Matt
Phlip (Guest)
on 2009-01-11 22:45
(Received via mailing list)
> Well I'm just getting into TDD and thats going well, but I did notice
> that when running rake:test, that it complains there are migrations that
> need to be run before the tests will be performed, even though the tests
> are running on a different database.

That's because the migrations (should!) happen on the Development
database, and
this gets cloned into the Test database just before each test run...
Matt H. (Guest)
on 2009-01-11 23:16
(Received via mailing list)
Phlip wrote:
>> Well I'm just getting into TDD and thats going well, but I did notice
>> that when running rake:test, that it complains there are migrations
>> that need to be run before the tests will be performed, even though
>> the tests are running on a different database.
>
> That's because the migrations (should!) happen on the Development
> database, and this gets cloned into the Test database just before each
> test run...
>

So on the development database, it's safe to add the columns or whatever
manually, update the migration to reflect the changes and test/code as
usual. Then when the production deployment happens it uses the
migrations from scratch?

Thanks

Matt
Phlip (Guest)
on 2009-01-11 23:41
(Received via mailing list)
Matt H. wrote:

> So on the development database, it's safe to add the columns or whatever
> manually, update the migration to reflect the changes and test/code as
> usual. Then when the production deployment happens it uses the
> migrations from scratch?

Not safe to add columns "manually". Add them to your migrations, rake
db:migrate:reset to rebuild your databases, manually test, deploy, and
lock down
those migrations.

The deploy script will migrate on the server, so all your existing data
don't
get erased!
Matt H. (Guest)
on 2009-01-12 00:10
(Received via mailing list)
Phlip wrote:
>
> The deploy script will migrate on the server, so all your existing data
> don't get erased!
>

Ok, I'll have a go at that, thanks for clearing that up.

Matt
Adam A. (Guest)
on 2009-01-12 07:03
hi guys, sorry this is actually a railess app. Just ruby and the
console. Ive found a few links though that show how to use active record
without rails though here including setting up migrations and fixutres.

http://rails.aizatto.com/category/activerecord/act...


and there is also this piece here

http://wiki.rubyonrails.org/rails/pages/HowToUseAc...

does anyone know of anyother links?

From what i can tell those links dont discuss how to implement the
automatic generation of a test version of a development db that rails
does everytime you run a test. If anyone has a link etc id really
appreciate it!.

Thanks for the input so far
David A. Black (Guest)
on 2009-01-12 15:21
(Received via mailing list)
Hi --

On Mon, 12 Jan 2009, Adam A. wrote:

>
> does anyone know of anyother links?
>
>> From what i can tell those links dont discuss how to implement the
> automatic generation of a test version of a development db that rails
> does everytime you run a test. If anyone has a link etc id really
> appreciate it!.

I don't have a link to a complete setup but what I'd recommend is have
a look at the Rake tasks that come with Rails, and adapt them. It's
non-trivial enough that I can't offer to do it on the spot, but
reasonably straightforward and will give you some Rake practice
anyway.


David

--
David A. Black / Ruby Power and Light, LLC
Ruby/Rails consulting & training: http://www.rubypal.com
Coming in 2009: The Well-Grounded Rubyist (http://manning.com/black2)

http://www.wishsight.com => Independent, social wishlist management!
This topic is locked and can not be replied to.