# How to make ZenTest autotest run whenever my tests change **URL:** \ **Category:** Rails\ **Created:** [July 6, 2009, 8:42pm UTC](https://www.ruby-forum.com/t/how-to-make-zentest-autotest-run-whenever-my-tests-change/171630 "2009-07-06T20:42:18Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![Phlip](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/ed8c4c/32.png) [@Phlip](https://www.ruby-forum.com/u/Phlip)\ **Post date:** [July 6, 2009, 8:42pm UTC](https://www.ruby-forum.com/t/how-to-make-zentest-autotest-run-whenever-my-tests-change/171630/1 "2009-07-06T20:42:18Z")
Railsters: Despite Rails being the only Web platform designed for TDD, a lot of its test infrastructure is still “cargo cult” - imitating other test rigs instead of understanding their principles. Most importantly, tests should run instantly. There’s no excuse for breaking this rule, and if you invent a platform that can’t obey it then you are doing something wrong. And I have yet to meet a combination of editor and test script which does this. Even Visual Studio and C++ are faster and easier to work with. The Rails problem is apparently rake db:test:prepare, which clones your database instead of detecting whether it needs a clone. I have heard the fix is ZenTest’s autotest. This allegedly loads the Ruby “VM” once, awaits file changes, and responds correctly to each one. So I have a vanilla Rails 2.3.2 application, and I install autotest (and autotest -v does not work, so I can’t see which version number I actually get), and the gem system claims I have version 4.1.3. And I run autotest in the project folder, and it says: $ autotest (Not running features. To run features in autotest, set AUTOFEATURE=true.) /usr/bin/ruby1.8 -I.:lib:test -rubygems -e “%w[test/unit test/test\_helper.rb].each { |f| require f }” | unit\_diff -u Loaded suite -e Started Finished in 0.000437 seconds. 0 tests, 0 assertions, 0 failures, 0 errors Note the system did not even run the whole test batch, first. And note it sees and warns that it rejects my features folder. I need it to see my test folder (NOT a spec folder!). Then I make a non-trivial change to a correctly-named test file, save the file, and nothing triggers. (Pure Ubuntu Linux substrate, BTW.) So how do I trigger? If I could get that working, how can I TDD both story and unit tests AT THE SAME TIME, without using two different test runners? – Phlip ---
**Author:** ![Phlip](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/ed8c4c/32.png) [@Phlip](https://www.ruby-forum.com/u/Phlip)\ **Post date:** [July 6, 2009, 9:09pm UTC](https://www.ruby-forum.com/t/how-to-make-zentest-autotest-run-whenever-my-tests-change/171630/2 "2009-07-06T21:09:47Z")
On Mon, Jul 6, 2009 at 10:54 AM, Phlip[removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > Started > > Finished in 0.000437 seconds. > > 0 tests, 0 assertions, 0 failures, 0 errors sudo gem install autotest-rails. Autotest does not automatically intuit project directory structures, beyond the simplest, oldest Ruby idiom (lib/_.rb =\> test/test\__.rb). The autotest-\* set of support gems adds support for more complex directory structures, like Rails’. ~ j. ---
**Author:** ![Phlip](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/ed8c4c/32.png) [@Phlip](https://www.ruby-forum.com/u/Phlip)\ **Post date:** [July 6, 2009, 9:10pm UTC](https://www.ruby-forum.com/t/how-to-make-zentest-autotest-run-whenever-my-tests-change/171630/3 "2009-07-06T21:10:38Z")
Phlip wrote: > Railsters: > > Despite Rails being the only Web platform designed for TDD, Really? I have the impression that there are others…if nothing else, some of the Railsy PHP frameworks have copied this feature. […] > Most importantly, tests should run instantly. In most cases, mine do. The exceptions are those which use regexps or DOM wizardry for parsing HTML. I suspect Oniguruma might help here, but I haven’t tried it. > And I have yet to meet a combination of editor and test > script > which does this. RSpec + autotest + out-of-the-box test scripts + any editor you like. Done. > The Rails problem is apparently rake db:test:prepare, which clones your > database > instead of detecting whether it needs a clone. When would it _not_ need a clone? Tests are supposed to start from a clean environment. I think the real time sink may be loading the Rails environment, which seems to typically take several seconds on an otherwise fast test run. But I can wait that long for my tests to run. Jay Fields has posted some stuff on his blog about how to make Rails unit tests not touch the DB, but I don’t really see much need to do that. > I have heard the fix is ZenTest’s autotest. This allegedly loads the > Ruby “VM” > once, awaits file changes, and responds correctly to each one. Yup. That’s what it does as far as I understand. > So I have a vanilla Rails 2.3.2 application, and I install autotest (and > autotest -v does not work, -v is verbose, not version. The autotest script is part of the ZenTest package, and doesn’t appear to have a version number of its own. > so I can’t see which version number I > actually get), > and the gem system claims I have version 4.1.3. Of what? ZenTest? (FWIW, I’m using ZenTest 3.11.1; I’ll have to try the newer version.) > And I run autotest in > the > project folder, and it says: > > $ autotest > (Not running features. To run features in autotest, set > AUTOFEATURE=true.) > /usr/bin/ruby1.8 -I.:lib:test -rubygems -e “%w[test/unit > test/test\_helper.rb].each { |f| require f }” | unit\_diff -u > Loaded suite -e > Started > > Finished in 0.000437 seconds. > > 0 tests, 0 assertions, 0 failures, 0 errors > > Note the system did not even run the whole test batch, first. Bizarre. It does for me with RSpec. Are your tests in the directories that Autotest is searching (which would appear to be test/unit)? > And note > it sees > and warns that it rejects my features folder. Not quite. It is warning you that it did not even look for your features folder, and will not look for it unless you set AUTOFEATURE=true. So try that. (Personally, I think features are too time-consuming to run as often as tests, because I tend to have them do a lot of browser simulation.) > I need it to see my test > folder > (NOT a spec folder!). It should do that. Check your configuration. > Then I make a non-trivial change to a correctly-named test file, save > the file, > and nothing triggers. (Pure Ubuntu Linux substrate, BTW.) > > So how do I trigger? Again, this is likely a ZenTest/autotest config issue. What you’re describing as desired is exactly what ZT/AT should normally do. > If I could get that working, how can I TDD both story and unit tests AT > THE SAME > TIME, without using two different test runners? Set AUTOFEATURE=true, like it’s telling you in that message. ZT/AT will call rake test and cucumber as appropriate. > – > Phlip ## Best, Marnen Laibow-Koser [http://www.marnen.org](http://www.marnen.org) removed\_email\_address@domain.invalid ---
**Author:** ![Phlip](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/ed8c4c/32.png) [@Phlip](https://www.ruby-forum.com/u/Phlip)\ **Post date:** [July 6, 2009, 9:12pm UTC](https://www.ruby-forum.com/t/how-to-make-zentest-autotest-run-whenever-my-tests-change/171630/4 "2009-07-06T21:12:08Z")
John B. wrote: > > 0 tests, 0 assertions, 0 failures, 0 errors > > sudo gem install autotest-rails Yay thank you the good news is that runs now. (I would have gone with test/\*\*/\*suite.rb myself…) The bad news is it’s still slow as f—. Yes Gherkin is partly to blame. But does anyone have a replacement for rake db:test:prepare that does less?? – Phlip ---
**Author:** ![Phlip](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/ed8c4c/32.png) [@Phlip](https://www.ruby-forum.com/u/Phlip)\ **Post date:** [July 6, 2009, 9:22pm UTC](https://www.ruby-forum.com/t/how-to-make-zentest-autotest-run-whenever-my-tests-change/171630/5 "2009-07-06T21:22:39Z")
Marnen Laibow-Koser wrote: > > Despite Rails being the only Web platform designed for TDD, > > Really? I have the impression that there are others…if nothing else, > some of the Railsy PHP frameworks have copied this feature. designed for - not copied on. > > Most importantly, tests should run instantly. > > In most cases, mine do. The exceptions are those which use regexps or > DOM wizardry for parsing HTML. I suspect Oniguruma might help here, but > I haven’t tried it. Rails?? > RSpec + autotest + out-of-the-box test scripts + any editor you like. > Done. When the tests fail, what button do you type to navigate instantly to the test failure? > > The Rails problem is apparently rake db:test:prepare, which clones your > > database > > instead of detecting whether it needs a clone. > > When would it _not_ need a clone? Tests are supposed to start from a > clean environment. We do not build a new CPU with new libraries and Ruby installation between each test run. All test isolation is a trade-off of some kind. > Jay Fields has posted some stuff on his blog about how to make Rails > unit tests not touch the DB, but I don’t really see much need to do > that. Thanks - that’s the next thing I was going to … rail at. Some consultants who teach TDD use a mind-trick to motivate their newbs to decouple their code. In a big-iron environment where database _migrations_ are expensive, until you pay that down, you can get a slight benefit to unit tests that reach into code that decouples from the database. We should do that too. And when we need a fixture, to start such a test case, it should come from the database. – Phlip ---
**Author:** ![Phlip](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/ed8c4c/32.png) [@Phlip](https://www.ruby-forum.com/u/Phlip)\ **Post date:** [July 7, 2009, 12:42am UTC](https://www.ruby-forum.com/t/how-to-make-zentest-autotest-run-whenever-my-tests-change/171630/6 "2009-07-07T00:42:30Z")
> RSpec + autotest + out-of-the-box test scripts + any editor you like. > Done. FWIW, this is still not competing with using Alt+Tab to switch to a console, then running script/cucumber features/whatever.feature. Autotest seems to just get mired in unit tests that I did not recently change, or something! – Phlip ---
**Author:** ![Phlip](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/ed8c4c/32.png) [@Phlip](https://www.ruby-forum.com/u/Phlip)\ **Post date:** [July 7, 2009, 1:32am UTC](https://www.ruby-forum.com/t/how-to-make-zentest-autotest-run-whenever-my-tests-change/171630/7 "2009-07-07T01:32:10Z")
On Mon, Jul 6, 2009 at 12:10 PM, Phlip[removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > John B. wrote: > > > sudo gem install autotest-rails > > Yay thank you the good news is that runs now. (I would have gone with > test/\*\*/\*suite.rb myself…) Excellent, glad it worked for you. Suites were originally bad for autotest because it attempts to do two-way mapping between test and implementation files, and suites generally aggregate a bunch of test files. > The bad news is it’s still slow as f—. Yes Gherkin is partly to blame. But > does anyone have a replacement for rake db:test:prepare that does less?? Are you absolutely certain that it’s the DB load? Have you benchmarked ‘rake db:test:prepare’ against ‘rake environment’? The latter is basically the minimum price you pay for a Rake task that loads Rails. ~ j. ---
**Author:** ![Phlip](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/ed8c4c/32.png) [@Phlip](https://www.ruby-forum.com/u/Phlip)\ **Post date:** [July 7, 2009, 2:44am UTC](https://www.ruby-forum.com/t/how-to-make-zentest-autotest-run-whenever-my-tests-change/171630/8 "2009-07-07T02:44:03Z")
John B. wrote: > Are you absolutely certain that it’s the DB load? Nope! > Have you benchmarked > ‘rake db:test:prepare’ against ‘rake environment’? Of course not! > The latter is > basically the minimum price you pay for a Rake task that loads Rails. $ time rake db:test:prepare real 0m2.153s user 0m1.324s sys 0m0.324s $ time rake environment real 0m1.623s user 0m1.316s sys 0m0.280s Rhetorical question: So if that’s the fast part, what is so f—ing slow??? – Phlip ---
**Author:** ![Phlip](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/ed8c4c/32.png) [@Phlip](https://www.ruby-forum.com/u/Phlip)\ **Post date:** [July 7, 2009, 2:13am UTC](https://www.ruby-forum.com/t/how-to-make-zentest-autotest-run-whenever-my-tests-change/171630/9 "2009-07-07T02:13:47Z")
Phlip wrote: > Marnen Laibow-Koser wrote: > > > > Despite Rails being the only Web platform designed for TDD, > > > > Really? I have the impression that there are others…if nothing else, > > some of the Railsy PHP frameworks have copied this feature. > > designed for - not copied on. Copying a feature in the design phase is still designing it in. Or were you going to say that Ruby wasn’t really designed for most of its cool features because they were copied from Smalltalk and Lisp? 🙂 > > > Most importantly, tests should run instantly. > > > > In most cases, mine do. The exceptions are those which use regexps or > > DOM wizardry for parsing HTML. I suspect Oniguruma might help here, but > > I haven’t tried it. > > Rails?? Huh? Oniguruma is what I haven’t tried. > > RSpec + autotest + out-of-the-box test scripts + any editor you like. > > Done. > > When the tests fail, what button do you type to navigate instantly to > the test > failure? Cmd-Tab. The test failure is usually where I was last editing. 🙂 Seriously, you’re asking the wrong person. I don’t really care that much about that level of editor integration. If you do, I believe there is a TextMate bundle that will do what you want, but you will get better answers from others on this list. […] > We do not build a new CPU with new libraries and Ruby installation > between each > test run. No, because we can guarantee that those bits won’t be changed in a typical app. We cannot guarantee that about the DB, can we? I _would_ want a new Ruby installation for each test run if I thought my code were likely to modify the system Ruby installation as it ran. > All test isolation is a trade-off of some kind. Sure. > > Jay Fields has posted some stuff on his blog about how to make Rails > > unit tests not touch the DB, but I don’t really see much need to do > > that. > > Thanks - that’s the next thing I was going to … rail at. Some > consultants who > teach TDD use a mind-trick to motivate their newbs to decouple their > code. In a > big-iron environment where database _migrations_ are expensive, until > you pay > that down, you can get a slight benefit to unit tests that reach into > code that > decouples from the database. > > We should do that too. > […] Do what? You mentioned several things, and I’m not sure which one you’re referring to. ## Best, Marnen Laibow-Koser [http://www.marnen.org](http://www.marnen.org) removed\_email\_address@domain.invalid