# How do you migrate content between environments?

**URL:** <https://www.ruby-forum.com/t/how-do-you-migrate-content-between-environments/118783>\
**Category:** Radiant CMS\
**Created:** [October 30, 2007, 9:54pm UTC](https://www.ruby-forum.com/t/how-do-you-migrate-content-between-environments/118783 "2007-10-30T21:54:58Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![richardchaven](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/e480ec/32.png) [@richardchaven](https://www.ruby-forum.com/u/richardchaven)\
**Post date:** [October 30, 2007, 9:54pm UTC](https://www.ruby-forum.com/t/how-do-you-migrate-content-between-environments/118783/1 "2007-10-30T21:54:58Z")

</div>

Let’s say I have a Radiant site in production and I like it and it’s  
working  
fine. Now, the boss comes along and outlines a bunch of content changes  
(images, copy, etc.) but no code changes (extensions, etc.) I obviously  
cant build the new content in-place on the production system so I fire  
up a  
development environment and get it all worked out. Now my problem is  
how do  
I get all that stuff from DEV -\> PROD?

My first thought was to dump the DEV database and import it into the  
PROD  
database. The problem with that is I risk overwriting production data  
(!!)  
and that doesn’t help me move non-database resources (i.e.  
/public/images/\*). Is there some sort of freeze/thaw that I can use to  
migrate this data? Am I completely missing the mark and everyone else  
just  
develops in production? 🙂 How do you move content around?

Thanx!  
Richard

P.S. I apologize if I’ve covered this ground before. I looked through  
the  
archives but couldn’t find an answer.

---

<div class="post-metadata">

**Author:** ![richardchaven](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/e480ec/32.png) [@richardchaven](https://www.ruby-forum.com/u/richardchaven)\
**Post date:** [October 30, 2007, 10:06pm UTC](https://www.ruby-forum.com/t/how-do-you-migrate-content-between-environments/118783/2 "2007-10-30T22:06:08Z")

</div>

Richard,

Sorry, other than what you’ve described (dump/reload), there’s no good  
solution. On the Digital Pulp project, we’re keeping our complete  
instance (extensions and Radiant source included) in source control.  
You might consider something like that going forward, at least for  
managing your non-database stuff. Unless you can come up with a script  
to do it, any database content will have to be manually copied/merged.

Sean

---

<div class="post-metadata">

**Author:** ![richardchaven](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/e480ec/32.png) [@richardchaven](https://www.ruby-forum.com/u/richardchaven)\
**Post date:** [October 30, 2007, 10:30pm UTC](https://www.ruby-forum.com/t/how-do-you-migrate-content-between-environments/118783/3 "2007-10-30T22:30:41Z")

</div>

Sean C. wrote:

> Unless you can come up with a script to do it, any database content will have to be manually copied/merged

Ewww.

Has anyone made any progress out there on import/export? (I know it  
used to be in the list of core items to add to radiant).

Having this basic functionality would allow someone to add on some sort  
of diff/sync extension that would work great for Richard here (and I  
suspect a _lot_ of others, too).

Oh, and before anyone asks, yes, I’ve thought about tackling it myself  
but I’m several projects (read months) from getting to that stage.

-Chris

---

<div class="post-metadata">

**Author:** ![richardchaven](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/e480ec/32.png) [@richardchaven](https://www.ruby-forum.com/u/richardchaven)\
**Post date:** [October 30, 2007, 11:34pm UTC](https://www.ruby-forum.com/t/how-do-you-migrate-content-between-environments/118783/4 "2007-10-30T23:34:59Z")

</div>

We’ve had to tackle this exact issue.

Best we could do, which is by no means perfect, but does work is using  
the Linux version of SQLyog ( [http://webyog.com/en/downloads.php](http://webyog.com/en/downloads.php) ) and  
what we were able to do, is simply build it into all our deployment  
scripts so it’s actually very automated for us.

The part that sold me on SJA is the fact that it’s very easily  
configurable so rolling it out across multiple servers was a breeze.

Syncing between our dev and production databases works pretty damn  
well, only ran into a couple of issues that haven’t seemed to be life  
or death anyways.

Cheers,

–  
Travis B.  
Not sent from my iPhone.

---

<div class="post-metadata">

**Author:** ![richardchaven](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/e480ec/32.png) [@richardchaven](https://www.ruby-forum.com/u/richardchaven)\
**Post date:** [October 31, 2007, 4:24am UTC](https://www.ruby-forum.com/t/how-do-you-migrate-content-between-environments/118783/5 "2007-10-31T04:24:48Z")

</div>

> My first thought was to dump the DEV database and import it  
> into the PROD  
> database. The problem with that is I risk overwriting  
> production data (!!)  
> and that doesn’t help me move non-database resources (i.e.  
> /public/images/\*).

If you think that overwriting production data is a risk, then  
you’re going about this the wrong way.

The only sane way to go about this is to make your production  
site read-only while you’re developing. Any other road WILL  
lead to madness. Trust me on that.

Copy the entire site directory to another location - everything,  
including public folders - anything that you think you’re going  
to need. Then, duplicate the database, copying it from live to  
development.

To do the database copy I run:

mysql -e “drop database radiant\_development”  
mysql -e “create database radiant\_development”  
mysqldump radiant\_production | mysql radiant\_development

I keep all the non-db files on my site under subversion, so I just  
do a commit in production and a checkout in development. Otherwise,  
you can just do a simple copy.

Then, you do all your development in the development area and when  
you’re happy, reverse the process.

I can hear some of you already saying “but we can’t leave our  
production environment locked for that long”. If that’s the case,  
just do an initial copy-live-to-test, take note of everything that  
you do while you develop it and then when you think you’ve nailed it  
do a fresh copy-live-to-test and then replay your changes by hand  
before doing a copy-test-to-live. Alternatively, you could just  
replay your changes by hand directly into live - but only if you’ve  
got an unfaltering belief in yourself.

Again, The only way to sanely move changes between two versions that  
both have modifications in them is by hand.

Dan.

---

<div class="post-metadata">

**Author:** ![richardchaven](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/e480ec/32.png) [@richardchaven](https://www.ruby-forum.com/u/richardchaven)\
**Post date:** [October 31, 2007, 2:42am UTC](https://www.ruby-forum.com/t/how-do-you-migrate-content-between-environments/118783/6 "2007-10-31T02:42:00Z")

</div>

!!! WARNING WARNING WARNING !!!  
I know next to nothing about rake migrations so the following text might  
just as well be labled SPAM. 🙂  
!!! WARNING WARNING WARNING !!!

Is there anyway to hook into a db:migration task to export/import the  
known  
radiant tables? That way we could at least grab the radiant stuff with  
a  
standard rake migration. If rake is anything like ant (for Java) it  
should  
be able to be extended to do this sort of thing. I envision something  
like  
the following:

## Export the development envrionment

rake development db:migrate:export \> mysite.db.tar

## [Optional] Store site in version control

svn commit -m “Keep this in case we need it later” mysite.db.tar

## Import into the production environment

rake production db:migrate:import \< mysite.db.tar

This is what we used to do with all of our environments when I was a  
corporate slave. Is this type of thing possible with rake? Am I  
barking up  
the wrong tree?

Thanx!  
Richard

---

<div class="post-metadata">

**Author:** ![richardchaven](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/e480ec/32.png) [@richardchaven](https://www.ruby-forum.com/u/richardchaven)\
**Post date:** [October 31, 2007, 3:36pm UTC](https://www.ruby-forum.com/t/how-do-you-migrate-content-between-environments/118783/7 "2007-10-31T15:36:52Z")

</div>

Matt,

You’re essentially talking about an audit trail which you can replay.  
Sounds like something that would fit deep within the guts of  
ActiveRecord but would not necessarily have to be related to Radiant.  
However, you would only have to capture changes (i.e. INSERT, SELECT,  
DELETE). You would still have to have some way of locking your  
production database.

Richard, you should look into Joyent Slingshot – Radiant has been  
ported to this mini-platform that allows offline editing and syncing  
with a live database. It may be the solution you’re looking for.

Sean

---

<div class="post-metadata">

**Author:** ![richardchaven](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/e480ec/32.png) [@richardchaven](https://www.ruby-forum.com/u/richardchaven)\
**Post date:** [October 31, 2007, 3:07pm UTC](https://www.ruby-forum.com/t/how-do-you-migrate-content-between-environments/118783/8 "2007-10-31T15:07:01Z")

</div>

Couldn’t we extend Radiant so that it trapped all SQL that caused  
update/delete/adds to a script?

- In the beginning, the development and production server have the  
exact same DB with the production server. Freeze the production  
server so that no changes can happen through /admin.

- Make some changes on the development server.

- Export these changes to a script: “rake db:export:changes” -\>  
changes.sql

- On production server: “mysql radiant\_production \< changes.sql”

I would need to read up more on ActiveRecord and if you are capturing  
just SQL you would have to be using the same DB engine for dev and  
production but does this idea seem feasible?

---

<div class="post-metadata">

**Author:** ![richardchaven](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/e480ec/32.png) [@richardchaven](https://www.ruby-forum.com/u/richardchaven)\
**Post date:** [November 1, 2007, 3:48pm UTC](https://www.ruby-forum.com/t/how-do-you-migrate-content-between-environments/118783/9 "2007-11-01T15:48:13Z")

</div>

I guess I shouldn’t be surprised that people are using import\_export,  
but it’s really not a stable or complete extension. I would appreciate  
any patches that can fix the problem because I don’t have time to work  
on it right now.

Sean

---

<div class="post-metadata">

**Author:** ![richardchaven](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/e480ec/32.png) [@richardchaven](https://www.ruby-forum.com/u/richardchaven)\
**Post date:** [November 9, 2007, 7:47pm UTC](https://www.ruby-forum.com/t/how-do-you-migrate-content-between-environments/118783/10 "2007-11-09T19:47:31Z")

</div>

Hi,

We use the attached rake file to version control Radiant changes via  
yml files. I think this is close to what you are describing.

–David W. Body / Iowa Ruby Brigade

---

<div class="post-metadata">

**Author:** ![richardchaven](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/e480ec/32.png) [@richardchaven](https://www.ruby-forum.com/u/richardchaven)\
**Post date:** [November 1, 2007, 3:35pm UTC](https://www.ruby-forum.com/t/how-do-you-migrate-content-between-environments/118783/11 "2007-11-01T15:35:57Z")

</div>

Sean,

I was just looking at the  
import\_export[http://dev.radiantcms.org/svn/radiant/tags/rel\_0-6-3/extensions/import\_export/](http://dev.radiantcms.org/svn/radiant/tags/rel_0-6-3/extensions/import_export/)extension  
and I think it would do almost everything I needed, if it worked  
properly. I tried it on my development machine and it looks like it  
hangs  
when trying to build the “users” table. I was getting a NIL error after  
it  
had cleared the table.

My migration plan would then be to export the DB using import\_export and  
store it in SVN right next to everything else (public/, images/, etc.)  
That  
way I could always get back to a certain version of my web-site.

First and formost is getting the extension to work properly. I have  
very  
limited Ruby (and Rails) experience and I don’t think I can debug it  
myself. ☹ I would love to but I’m just now learning how to use  
Radiant  
tags and debugging Rails is hard (or at least I don’t know how yet :).

Thanx!  
Richard

---

<div class="post-metadata">

**Author:** ![richardchaven](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/e480ec/32.png) [@richardchaven](https://www.ruby-forum.com/u/richardchaven)\
**Post date:** [November 9, 2007, 7:49pm UTC](https://www.ruby-forum.com/t/how-do-you-migrate-content-between-environments/118783/12 "2007-11-09T19:49:17Z")

</div>

I guess the attachment didn’t make it through, so here it is  
(fixtures.rake):

# make hash.to\_yaml sort by keys

# [http://code.whytheluckystiff.net/syck/ticket/3](http://code.whytheluckystiff.net/syck/ticket/3)

class Hash  
def to\_yaml( opts = {} )  
YAML::quick\_emit( object\_id, opts ) do |out|  
out.map( taguri, to\_yaml\_style ) do |map|  
sorted\_keys = keys  
sorted\_keys = begin  
sorted\_keys.sort  
rescue  
sorted\_keys.sort\_by {|k| k.to\_s} rescue sorted\_keys  
end

```
     sorted_keys.each do |k|
       map.add( k, fetch(k) )
     end
   end
 end

```

end  
end

class String  
def to\_unix  
self.gsub(/\r\n/, “\n”)  
end  
end

namespace “iowaruby” do  
desc ‘Create YAML test fixtures from data in an existing  
database. Defaults to development database. Set RAILS\_ENV to  
override.’  
task :extract\_fixtures =\> :environment do  
sql = “select \* from %s order by id”  
skip\_tables = [“schema\_info”, “sessions”, “users”]  
ActiveRecord::Base.establish\_connection  
(ActiveRecord::Base.connection.tables - skip\_tables).each do |  
table\_name|  
puts “Extracting #{table\_name}”  
i = “000”  
File.open(“#{RAILS\_ROOT}/fixtures/#{table\_name}.yml”, ‘w’) do |  
file|  
data = ActiveRecord::Base.connection.select\_all(sql %  
table\_name)  
file.write data.inject({}) { |hash, record|  
hash[“#{table\_name}\_#{i.succ!}”] = record  
hash  
}.to\_yaml.to\_unix  
end  
end  
end

desc ‘Load fixtures into a database. Defaults to development  
database. Set RAILS\_ENV to override.’  
task :load\_fixtures =\> :environment do  
require ‘active\_record/fixtures’  
ActiveRecord::Base.establish\_connection  
Dir.glob(File.join(RAILS\_ROOT, ‘fixtures’, ‘_.{yml,csv}')).each  
do |fixture\_file|  
puts "Loading #{File.basename(fixture\_file, '._’)}"  
Fixtures.create\_fixtures([‘fixtures’], File.basename  
(fixture\_file, ‘.\*’))  
end  
end

end