[ANN] DevTower Beta3 Release!

Rails
1

DevTower Beta3! (0.3)

I’m proud to announce some major changes and improvements to DevTower in
this beta release! First and foremost, DevTower is now a full plugin.
Also,
DevTower works (only) with Rails 1.1.

==== ABOUT =====

DevTower used to synchronize the development of Ruby On
Rails applications with developers working simultaniously
on multiple systems with Subversion. DevTower is built
around the philosophy of responsible group-development
and enforces certain development practices which you may
or may not like. However, it is my belief that all are good
practices for any collaborative development system.

It is comprised of a few extensions to ActiveRecord::Base
(via a modified ar_fixtures) and several Rake tasks to speed
group development.

This was one of those things that was made for use in-house.
Many of you probably have scripts around that do similar things
to synchronize your development (especially databases are tricky).
I just happen to be making this one public.

STABILITY: This software is in development… so don’t use it
on huge projects with only one copy in existance. Aka, don’t
be stupid. However, for a new project, you can feel fairly secure
in its functionality.

==== INSTALLATION ======

To get the new plugin download from
http://rubyforge.org/frs/?group_id=1438&release_id=4838

Unzip so that… RAILS_PATH/vendor/plugins/devtower/

===== USE ======

A developer will run ‘rake commit’ and DevTower will write the database
schema and data to file and clear out all the logs and then run ‘svn
ci’.

Then, the next developer, to synchronize, will run ‘rake checkout’ and
it
will update his working directory and load up the new schema and new
data fresh from the development server.

===== DOCUMENTATION ======

DevTower gives you the following commands:

rake db:hold:dump
-writes the current database down to yaml files in
RAILS_APP/db/hold/*.yml Think of this as your data-dump to file.

rake db:hold:load
-loads the yaml model files from db/hold/*.yml into the current
database. CHECK /db/hold BEFORE YOU DO THIS, because it will dump your
current db and load it. So, just make sure that the yaml files have
something in them.

rake commit:prepare
-dumps the database down to the hold
-dumps the schema to schema.rb
-clears the /tmp folder
-clears the /log files

rake commit:force
-simply runs ‘svn ci -m “#{diff of CHANGELOG}”’
-uses whatever has been added to RAILS_APP/CHANGELOG since the last
commit as the commit message.

rake commit
-runs commit:prepare
-runs test … your tests should pass before you commit anything
ever!
-runs commit:force to do the actual commit
-uses whatever has been added to RAILS_APP/CHANGELOG since the last
commit as the commit message.

rake commit:skip_tests
-if your a bad-boy and don’t have working tests, then prepare and
commit
anyway

rake checkout:after
-WIPES current db
-reads in schema from /db/schema.rb
-reads in data from /db/hold/*.yml

rake checkout
-performs ‘svn up’
-yes, i know its an update and not a checkout
-runs checkout:after to do everything needed to start coding!

==== NOTES =====

Here are a few things that DevTower is NOT:

  • IT IS NOT a deployment system like Capistrano.
  • IT IS NOT the solution for everyone.
  • IT IS NOT built to work with CVS.
  • IT IS NOT completely stable.

All of those things being said. Here is what it does do.

  • Works with SVN.
  • Forces tests to pass before committing to the repository.
  • Writes database schema and data to /db/hold/*.yml before committing.
  • Reads them back after checkout.

===== BIG QUESTION =====

What about migrations?
migrations are great when you are working with a stable/production
database
and have muiltiple branches. but, we use schema.rb to keep things in
line early in
development before branching and before stability… really good for
when the database is
changingoften and radically. migrations are great for small
changes,
painful for large
ones very often.

Its not working right!
That’s not a question. Stop yelling and go to
http://rubyforge.org/projects/devtower/
and file a bug report.

It destroyed my precious, high-risk data! Why?
I told you not to do it. Only use development data in the holds.

Does it work on windows?
I don’t see why not. I didn’t test it. In fact, I only tested it on
Ubuntu Dapper.

Does it work with Rails 1.0?
No.

======

Special thanks to Geoffrey Grosenbach for his ar_fixtures plugin.

-hampton catlin.

2

Just for the record: Thats absolutely not my experience. Migrations
work very well during all phases of the development if you

a) Check in often
b) Make lots of small migrations instead of one big.

tip: Lock a file in the repository when creating a migration until its
checked in. This will give your fellow developers a heads up that they
should wait for you.

3

Is there any reason that some migrations couldn’t be generated
automatically? Starting with a schema.rb of the current state of the
database on disk, you could then modify your database directly, and
run a script to dump out the schema and compare it to the one already
on disk, and generate a migration that describes the difference. I
think this would only be useful for adding tables and columns
(renaming or modifying really wouldn’t work), but I’d imagine that is
the most common type of migration anyway.

I like all the things that migrations give me, but I miss being able
to dig around in the db and make quick changes on the fly, in the
context of the rest of of the tables and columns that already exist.
This had been a big selling point of rails.

  • Isaac
4

Tobias,

The main issue here is… when I add a row called “created_on” to my
user
table and I make a migration for it. Then, my fellow developer grabs the
new
source, and runs the migration. Then, he has a broken data-set. None of
the
created_ons in his user are filled out!

There is only merging of structure, not data.

So, tell me how doing small migrations would keep that from happening?

Later on when we have a production server, we’d write stuff into our
migration to generate the correct data, because its worth taking the
time to
do that to migrate an existing stable database.

The other issue here is when you have a large set of data. For instance,
I
was working on a system for keeping events data. Well, I would enter 5
events as I was developing one day, then my partner-in-crime would enter
4
on hers… We both wanted that shared information to be merged.

So, this provides a serialized database in its entirety. Also, what if I
do
early development on sqlite and want to move all of my existing data
over
to some other database type. This system would use database
serialization to
facilitate that.

Last comment, lots of people will find this horrible un-useful to their
work-habits. However, using migrations early on often left me and my
other
developers with odd looking database datasets that would break and were
a
pain in the neck to keep up with all the changes that were happening.
So,
this certainly isn’t for everyone, but it is something that others who
develop in a style similar to mine might find useful.

-hampton.

5

Maybe its just me but… 146 migrations doesn’t exactly sound efficient
or
fun to do a checkout with.

The other advantage to using the “hold” with YAML files is that you can
always take these YAML files and put them in your fixtures and run your
tests on your actual data instead of writing it by hand.

6

Migrations are ruby code which runs in your rails environment. Data is
meant to be converted and adjusted by them. In fact of Shopify’s 146
migrations only about 100 actually change the schema.rb. Most deal
with data transformation. Shopify is not in production mode yet ( well
it’s going to be later today )

On 4/13/06, Hampton [email protected] wrote:

Tobias,

The main issue here is… when I add a row called “created_on” to my user
table and I make a migration for it. Then, my fellow developer grabs the new
source, and runs the migration. Then, he has a broken data-set. None of the
created_ons in his user are filled out!

User.find(:all).each { |u| u.update_attribute(:created_on,
Time.parse(“#{rand(27)+1}-#{rand(11)+1}-2005”)) }

cheers.

was working on a system for keeping events data. Well, I would enter 5
developers with odd looking database datasets that would break and were a
work very well during all phases of the development if you

What about migrations?
Rails mailing list

–
Tobi
http://shopify.com - modern e-commerce software
http://typo.leetsoft.com - Open source weblog engine
http://blog.leetsoft.com - Technical weblog

7

Hampton wrote:

Maybe its just me but… 146 migrations doesn’t exactly sound efficient
or
fun to do a checkout with.

Actually, sounds a lot better to me. If they weren’t so stinking easy
to do, the sort of stuff that gets done in a migration ends up piling up
into a bigger database change. That big database change then happens at
some coordinated time when everyone can freeze what they’re doing and
get an update.

Instead of 146 small changes, you have (say) 12 bigger changes. Lots
more can go wrong with big changes than small changes.

Ease of migrations allows them to become more atomic and cause smaller
ripples throughout the design.

Jake

8

Hampton-

Pretty cool stuff. I will play with this a bit ans see if it fits in

with our dev methodologies.

Thanks
-Ezra

9

On 4/13/06, Hampton [email protected] wrote:

I’m proud to announce some major changes and improvements to DevTower in
this beta release!

Hampton,

This is really cool! =) Working great for me so far.

Noticed a couple of small bugs (missing slash in hold path, quotes not
escaped in commit comments, and /n being inserted instead of \n (new
line)), and added code to not copy contents of the session table if
you are storing sessions in the database. Diff attached.

Thanks!

  • Isaac

— vendor/plugins/devtower/lib/thehold.rb Fri Apr 21 16:44:04 2006
+++ /src/devtower/lib/thehold.rb Sat Apr 8 11:33:18 2006
@@ -11,7 +11,7 @@
require ‘active_record/fixtures’
ActiveRecord::Base.establish_connection(RAILS_ENV.to_sym)
Dir.glob(File.join(RAILS_ROOT, ‘db’, ‘hold’,
‘*.{yml,csv}’)).each do |fixture_file|

  •    Fixtures.create_fixtures('db/hold', File.basename(fixture_file,

‘.*’))

  •    Fixtures.create_fixtures('db/hold/',

File.basename(fixture_file, ‘.*’))
end
end

@@ -34,7 +34,7 @@
connection.select_all(“SELECT * FROM
#{join.options[:join_table]}”).each_with_index { |record, i|
hsh[“join_#{i}”] = record
}

write_file(File.expand_path(“db/hold#{join.options[:join_table]}.yml”,
RAILS_ROOT), hsh.to_yaml)
+
write_file(File.expand_path(“db/hold/#{join.options[:join_table]}.yml”,
RAILS_ROOT), hsh.to_yaml)
end
end

— vendor/plugins/devtower/tasks/devtower.rake Sat Apr 22 03:40:21 2006
+++ /src/devtower/tasks/devtower.rake Sat Apr 8 10:41:49 2006
@@ -8,6 +8,7 @@
def models
Dir.glob(RAILS_ROOT + ‘/app/models/*.rb’).each { |file| require
file }
models = Object.subclasses_of(ActiveRecord::Base)

  • models.delete(CGI::Session::ActiveRecordStore::Session) # no
    sessions for us!
    puts models.inspect
    models
    end
    @@ -79,12 +80,12 @@
    #generate a commit message from the lines added in the file
    CHANGELOG
    result = svn diff CHANGELOG.scan(/^[+].*$/).map { |str|
    unless str[1]==43
  •           str[1,str.length-1] + "/n"

  •           str[1,str.length-1] + "\n"
  end
    }
    message = result.join()
  • svn ci -m "#{message}"
  • svn ci -m "#{message.gsub('"','\"')}"
    end

desc “Dumps the schema, its data to hold, and clears the logs.”