# How can I have the tests for my Rails app to be executed in a random order? **URL:** \ **Category:** Rails\ **Created:** [September 8, 2009, 9:47am UTC](https://www.ruby-forum.com/t/how-can-i-have-the-tests-for-my-rails-app-to-be-executed-in-a-random-order/175115 "2009-09-08T09:47:39Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Piero\_Di\_Bello](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/8c91f0/32.png) [@Piero\_Di\_Bello](https://www.ruby-forum.com/u/Piero_Di_Bello)\ **Post date:** [September 8, 2009, 9:47am UTC](https://www.ruby-forum.com/t/how-can-i-have-the-tests-for-my-rails-app-to-be-executed-in-a-random-order/175115/1 "2009-09-08T09:47:39Z")
Hi How can I have the tests for my Rails app to be executed in a random order? Is there a simple solution using rake? I already posted my question to stackoverflow (http:// > order-with-rake) but the few responses I got were not able to solve the problem. Thanks a lot! Pietro Di Bello ---
**Author:** ![Piero\_Di\_Bello](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/8c91f0/32.png) [@Piero\_Di\_Bello](https://www.ruby-forum.com/u/Piero_Di_Bello)\ **Post date:** [September 8, 2009, 4:33pm UTC](https://www.ruby-forum.com/t/how-can-i-have-the-tests-for-my-rails-app-to-be-executed-in-a-random-order/175115/2 "2009-09-08T16:33:51Z")
On 8 Sep 2009, at 14:01, Marnen Laibow-Koser wrote: > the spec file. > > But why do you need this anyway? Each test should start from a clean > slate, so the order should be completely immaterial. Should, but it is possible for this not to be true. A random order would be hard to work with though - Run your suite and a test fails, run it again and it doesn’t any more. Fred ---
**Author:** ![Piero\_Di\_Bello](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/8c91f0/32.png) [@Piero\_Di\_Bello](https://www.ruby-forum.com/u/Piero_Di_Bello)\ **Post date:** [September 8, 2009, 3:01pm UTC](https://www.ruby-forum.com/t/how-can-i-have-the-tests-for-my-rails-app-to-be-executed-in-a-random-order/175115/3 "2009-09-08T15:01:13Z")
Piero Di Bello wrote: > Hi > How can I have the tests for my Rails app to be executed in a random > order? At least with RSpec, specs are executed in unpredictable order. I think it’s the same each time, but it’s not the order in which they appear in the spec file. But why do you need this anyway? Each test should start from a clean slate, so the order should be completely immaterial. ## Best, Marnen Laibow-Koser [http://www.marnen.org](http://www.marnen.org) removed\_email\_address@domain.invalid ---
**Author:** ![Piero\_Di\_Bello](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/8c91f0/32.png) [@Piero\_Di\_Bello](https://www.ruby-forum.com/u/Piero_Di_Bello)\ **Post date:** [September 8, 2009, 10:13pm UTC](https://www.ruby-forum.com/t/how-can-i-have-the-tests-for-my-rails-app-to-be-executed-in-a-random-order/175115/4 "2009-09-08T22:13:23Z")
On Sep 8, 3:01 pm, Marnen Laibow-Koser \ wrote: > Piero Di Bello wrote: > > > Hi > > How can I have the tests for my Rails app to be executed in a random > > order? > > At least with RSpec, specs are executed in unpredictable order. I think > it’s the same each time, but it’s not the order in which they appear in > the spec file. Unfortunately we don’t use Rspec on our project, although I think we may give it a try > But why do you need this anyway? Each test should start from a clean > slate, so the order should be completely immaterial. Well, actually, the answer is in your words 🙂 Each test _should_ start from a clean state, but what if it doesn’t? For us it’s a great value to have a suite of test executed every time in a random order (better, every single test method in every test class should be “shuffled”). Dependency between tests is a really bad anti-pattern, and we want to know if this happens as soon as possible. anyway, thanks for your response, Piero ---
**Author:** ![Piero\_Di\_Bello](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/8c91f0/32.png) [@Piero\_Di\_Bello](https://www.ruby-forum.com/u/Piero_Di_Bello)\ **Post date:** [September 8, 2009, 9:27pm UTC](https://www.ruby-forum.com/t/how-can-i-have-the-tests-for-my-rails-app-to-be-executed-in-a-random-order/175115/5 "2009-09-08T21:27:33Z")
Marnen Laibow-Koser wrote: > Piero Di Bello wrote: > > > Hi > > How can I have the tests for my Rails app to be executed in a random > > order? > > At least with RSpec, specs are executed in unpredictable order. I think > it’s the same each time, but it’s not the order in which they appear in > the spec file. > > But why do you need this anyway? Each test should start from a clean > slate, so the order should be completely immaterial. First I’ll say I completely agree with this. If tests/specs fail or pass depending on the order of execution then there must be serious problems with your test suite. Now to address the OP. I know of nothing “built-in” any of the testing frameworks for this. What I suppose you would need to do is write your own script that searches though your project for all test classes then executes them in a random order. This, however, would only randomize the testing classes, each test within each class would still execute in whatever order the test framework decided to run them. Which, would mean that you would probably also have to replace the test runner with your own that could randomize each method of each test class. In conclusion if this is a solid requirement of your project, I’m really glad it’s your project and not mine. 🙂 Good luck. ---
**Author:** ![Piero\_Di\_Bello](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/8c91f0/32.png) [@Piero\_Di\_Bello](https://www.ruby-forum.com/u/Piero_Di_Bello)\ **Post date:** [September 9, 2009, 9:37am UTC](https://www.ruby-forum.com/t/how-can-i-have-the-tests-for-my-rails-app-to-be-executed-in-a-random-order/175115/6 "2009-09-09T09:37:45Z")
On Sep 8, 3:44 pm, Frederick C. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > > think > > it’s the same each time, but it’s not the order in which they appear > > in > > the spec file. > > > But why do you need this anyway? Each test should start from a clean > > slate, so the order should be completely immaterial. > > Should, but it is possible for this not to be true. A random order > would be hard to work with though - Run your suite and a test fails, > run it again and it doesn’t any more. The whole point of randomizing test execution is to have a better chance to discover that the tests are not independent. Once you find a test that fails because of the order of execution, you fix it so that it does not happen anymore. Matteo ---
**Author:** ![Piero\_Di\_Bello](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/8c91f0/32.png) [@Piero\_Di\_Bello](https://www.ruby-forum.com/u/Piero_Di_Bello)\ **Post date:** [September 9, 2009, 9:40am UTC](https://www.ruby-forum.com/t/how-can-i-have-the-tests-for-my-rails-app-to-be-executed-in-a-random-order/175115/7 "2009-09-09T09:40:05Z")
On Sep 9, 7:18 am, Matteo V. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > On Sep 8, 3:44 pm, Frederick C. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) > wrote: > > Should, but it is possible for this not to be true. A random order > > would be hard to work with though - Run your suite and a test fails, > > run it again and it doesn’t any more. > > The whole point of randomizing test execution is to have a better > chance to discover that the tests are not independent. Once you find > a test that fails because of the order of execution, you fix it so > that it does not happen anymore. I get that, but unless you could say ‘run the tests again but in the same random order as before’ how are you going to be sure that you’ve actually fixed your code/tests ? Fred ---
**Author:** ![Piero\_Di\_Bello](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/8c91f0/32.png) [@Piero\_Di\_Bello](https://www.ruby-forum.com/u/Piero_Di_Bello)\ **Post date:** [September 8, 2009, 10:20pm UTC](https://www.ruby-forum.com/t/how-can-i-have-the-tests-for-my-rails-app-to-be-executed-in-a-random-order/175115/8 "2009-09-08T22:20:59Z")
On Sep 8, 9:27 pm, Robert W. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > > But why do you need this anyway? Each test should start from a clean > > This, however, would only randomize the testing classes, each test > within each class would still execute in whatever order the test > framework decided to run them. Which, would mean that you would probably > also have to replace the test runner with your own that could randomize > each method of each test class. > > In conclusion if this is a solid requirement of your project, I’m really > glad it’s your project and not mine. 🙂 Thanks for your ideas. I’ll definitely explore some of them. No, it’s not a solid requirement, but as I already said in a previous answer, we find it _very_ useful to discover dependencies between tests as soon as possible (and probably the Rspec authors’ find this important too, given that Rspec execute specs in a random order…). Thanks Piero