On 8 Sep 2008, at 17:21, Jonathan L. wrote:
Here’s one example: lets say my app is a specialized CMS, where
account owners can setup their own projects, pages and forms. I’d
like to run scenarios against setups that users have created.
Thinking further, wouldn’t it be neat to make this a user feature,
eg via “Validate This Project” button
The more I think about it, the more I come to the conclusion you
should just take (very) regular backups, and when a problem occurs,
isolate the situation.
Dan’s export suggestion would be really useful here. In case of data
loss, you could fire up a backup version of the app, export missing/
corrupt pages, and re-import into the production environment.
This would encourage you to define a robust data format too. It
wouldn’t directly lead you to catching edge cases, but it will give
you more confidence about recovering from them. And anyway, if your
tests-on-real-data throw up edge cases, they will still miss the ones
that real data doesn’t exist for. So maybe being able to say “well,
I’m only 98% sure your data is safe, but I’m 99.9% sure I can get it
back if there’s a problem”, is more reassuring than hoping you covered
I’ve found that testing can sometimes drive out new features from an
app. The one I’m starting to work on now, for example, is a daemon
process that sits and polls some RSS and XML services. But to test it
I’ve been driven to start writing a (crude, right now) socket-based
remote control and corresponding client interface. I can’t say if it
will ever be used, but it’s reassuring to know I can always go in and
prod it if I’m not sure it’s working right live.
Sometimes it feels like waste, but then insurance products have an
expected net loss, and people still consider them valuable.