Forum: Rails-core (closed, excessive spam) rake db:test:prepare

Accaf690f3d97aab6e6e02adac8150be?d=identicon&s=25 Student (Guest)
on 2008-02-20 21:54
(Received via mailing list)
I'm having fun with a legacy DB (SQL Server), and in the process found
some code in rails/lib/tasks/databases.rake which feels wrong.

0) Errors which occur during this phase of a rake:test or rake:spec
tend to be ignored, although they are logically fatal.  In the best
case, this means that the stack trace when an error finally does
manage to stop the process is worthless.  Best case.

1) Prepare's strategy is to dump the current schema, blow away the
test db, and then rebuild the test schema from the dump.  This strikes
me as dangerous in the event that someone has mucked with the db
outside of a migration--that is, made a change that won't get
reflected in source control.  Wouldn't it be safer to just blow away
the test db & do the migrations to rebuild?

2) db:structure:dump for sqlserver requires the scptxfr executable,
which is part of the SQLServer package--not the client.  While it is
likely that developers can get access to this file (and its supporting
files), there is no a priori reason to believe that a developer is
going to have access to this expensive bit of software.  More
generally, should we not restrict dependencies to client code?  (The
scptxfr script went away with SQLServer 2005, so this particular code
is also aging.)

3) Several of the tasks in the db namespace have code of the form
case abcs[blah]["adapter"]
  when "mysql", "oci", "oracle"
...
  when "postgresql"
...
  when "sqlserver"
...

...
end

This is generally considered bad practice ("hypnotic" coding).  It is
not extensible, and it breaks the model of separating out code for the
commercial databases.  Does rake support calls to computed tasks?
Efa76b164a7de4a5730e4fa397cc4425?d=identicon&s=25 Michael Koziarski (Guest)
on 2008-02-22 00:11
(Received via mailing list)
On Thu, Feb 21, 2008 at 9:53 AM, Student <blogger@pierian-spring.net>
wrote:
>
>  I'm having fun with a legacy DB (SQL Server), and in the process found
>  some code in rails/lib/tasks/databases.rake which feels wrong.
>
>  0) Errors which occur during this phase of a rake:test or rake:spec
>  tend to be ignored, although they are logically fatal.  In the best
>  case, this means that the stack trace when an error finally does
>  manage to stop the process is worthless.  Best case.

Yeah, I've actually been bitten by this in the past, please do
investigate a patch  to make these errors get caught and reported in a
useful manner.

>  1) Prepare's strategy is to dump the current schema, blow away the
>  test db, and then rebuild the test schema from the dump.  This strikes
>  me as dangerous in the event that someone has mucked with the db
>  outside of a migration--that is, made a change that won't get
>  reflected in source control.  Wouldn't it be safer to just blow away
>  the test db & do the migrations to rebuild?

This has been discussed in the past, cloning development (by importing
schema.rb) is the preferred way of doing this.  The short reasons are:

* Most migrations in a large application are for munging production data
* it takes far too
* and there's no guarantee that it will continue to work (referencing
old models etc)

The question of when the schema gets dumped is an open one.  The
reason it's done in that rake task is that prior to migrations the way
you made changes to your  database was to issue DDL directly against
the dev db, and you needed to have your tests dump the database if
they were to have any chance of working.

Now we could *probably* just dump schema.rb after running a migration,
and load it before test runs.  This will reduce the changes of it
containing 'bogus' information, without making you run all 400
migrations for every test run.


>  case abcs[blah]["adapter"]
>  This is generally considered bad practice ("hypnotic" coding).  It is
>  not extensible, and it breaks the model of separating out code for the
>  commercial databases.  Does rake support calls to computed tasks?

Even if rake doesn't support this we could push that behaviour into
the adapters.  Something along the lines of

task db:structure:dump do
  current_adapter.dump!(any, arguments, needed)
end

Again, if you feel like working on this, we'd be happy to help you get
a patch ready to apply.

--
Cheers

Koz
9f0f89bbd9e1ecfbaab6584e429b7a2f?d=identicon&s=25 Josh Susser (jsusser)
on 2008-02-22 00:27
(Received via mailing list)
On Feb 21, 2008, at 3:10 PM, Michael Koziarski wrote:
>
> they were to have any chance of working.
>
> Now we could *probably* just dump schema.rb after running a migration,
> and load it before test runs.  This will reduce the changes of it
> containing 'bogus' information, without making you run all 400
> migrations for every test run.


How timely.  I've been working on just this stuff this week.  I'm
about to dive into patching db:test:prepare so all it does it load the
schema.rb instead of dumping it from the dev db every time.  That
should "just work" ok, but if you want to be confident that schema.rb
reflects the state of the dev db, you need to know that migrations
have been run correctly.  I'm working on a plugin that helps detect
when you need to run migrations, even detecting when someone may have
modified an old migration (by saving a snapshot of SHA1 hashes of all
migrations files).  The two changes should play well together.

--
Josh Susser
http://blog.hasmanythrough.com
5a0d7f0cb2fac7858d234de7f7f01491?d=identicon&s=25 Alex Chaffee (Guest)
on 2008-02-22 01:45
(Received via mailing list)
For an alternative see my article at
http://pivots.pivotallabs.com/users/alex/blog/arti...

If you make it part of your process to regularly collapse migrations
(like after every deploy), you avoid both of the problems mentioned
above (referencing old class definitions and taking a long time) but
you also avoid the problems with relying on schema.rb, namely :

 * your development and test environments will be using the same
technique (migrations) for creating/altering the DB
 * your migrations will be exercised by developers and test builds
before being deployed
 * you won't get burned by forgetting to export and check in schema.rb
after adding a migration

Anyway, I'm not trying to change anyone's mind here, but this
technique has worked really well for us on several projects.

Cheers -

 - Alex
This topic is locked and can not be replied to.