Why test Rails, itself?

Rails
1

I’m trying to learn the habit of writing tests, but now that I’m
getting into it, I’m starting to realize that many potential tests –
including some described in Agile Web D. w/Rails – don’t
test the application. Instead, they test Rails itself.

For example, what is the point of writing a unit test that simply
tests a validate statement in the model? Yes, it’s interesting (the
first time, at least) that the test works and that, golly-gee, Rails
works as advertised, as well, but is there any real use in doing this?

I can imagine a scenario where a I update Rails and suddenly a unit
test that tests a Rails validator fails, but I expect the Rails
development team would find this before me. Or is that too naive?

The reason I ask is that I’m starting to realize how time consuming it
is to write tests… it could easily double the amount of work. I
can’t afford that, so am trying to focus tests on my application code.
I just want to make sure I’m not missing some obvious benefit of
testing the whole Framework.

– John

2

Hi –

On 3/17/07, identry [email protected] wrote:

I can imagine a scenario where a I update Rails and suddenly a unit
test that tests a Rails validator fails, but I expect the Rails
development team would find this before me. Or is that too naive?

The reason I ask is that I’m starting to realize how time consuming it
is to write tests… it could easily double the amount of work. I
can’t afford that, so am trying to focus tests on my application code.
I just want to make sure I’m not missing some obvious benefit of
testing the whole Framework.

It’s a very interesting question. There really is no point testing
validates_format_of itself, since there’s already a test suite for it.
I’d say, though, that it’s good to have tests that walk your models
through everything they’re supposed to do. Among other things that
will come in handy if you accidentally comment out or alter the
validation. So you can think of your test suite as a kind of
documentation or mega-assertion pertaining to your whole application.

David

–
Q. What is THE Ruby book for Rails developers?
A. RUBY FOR RAILS by David A. Black (http://www.manning.com/black)
(See what readers are saying! http://www.rubypal.com/r4rrevs.pdf)
Q. Where can I get Ruby/Rails on-site training, consulting, coaching?
A. Ruby Power and Light, LLC (http://www.rubypal.com)

3

I would add to that:

  1. If you do not test it, you do not know if it works. Yes there are
    tests of Rails itself, but not of your use of Rails.
  2. The comment I have seen and come to believe is that if it is not
    tested then it is not guaranteed to work. The tests define what
    “work” is.

On a small project the programmer may know all the requirements and
remember them at all times, and always get it right. :slight_smile:

But, even on a 2 man project it helps to build tests for what you
expect the software to do.

The application tests will only test the cases that matter to the
applicaiton, while the Rails tests test all the cases the framwork
supports. But, you never know if the Rails tests cover the one case
you care about.

Michael

4

On Sat, Mar 17, 2007 at 04:31:55PM -0000, identry wrote :

For example, what is the point of writing a unit test that simply
tests a validate statement in the model? Yes, it’s interesting (the
first time, at least) that the test works and that, golly-gee, Rails
works as advertised, as well, but is there any real use in doing this?

+1

The reason I ask is that I’m starting to realize how time consuming it
is to write tests… it could easily double the amount of work. I
can’t afford that, so am trying to focus tests on my application code.
I just want to make sure I’m not missing some obvious benefit of
testing the whole Framework.

+1. Actually, it doubles the amount of work. Imagine you want to test
you method Father.add_kid. You’ll need to ‘manually’ add a kid to your
@father and then call your method add_kid and assert equality. You
simply rewrite your code! That really annoys me.

However, testing is really useful, but not for testing if rake
test:units passes: I have discovered ‘bugs’ (logic problems) when I
was writing tests.
You know, you’re beginning to write your
test_father_add_kid method with weird @father and suddently a
flash[:warning] rings in your head: ‘Hey! What about if @father has
already 12 kids? It can’t work because… blablabla’. So you
correct your ‘real method’ and cut/paste in your test just for ‘fun’
and be proud to see that it passes.

My $0.02.

–
,========================.
| Pierre-Alexandre M. |
| email : [email protected] |
`========================’

5

Hi –

On 3/17/07, MichaelLatta [email protected] wrote:

I would add to that:

  1. If you do not test it, you do not know if it works. Yes there are
    tests of Rails itself, but not of your use of Rails.

Yes – that’s what I mean about not testing validates_format_of
itself, but testing what you want your application to do, and if what
you want it to do involves calling validates_format_of, then the test
will fail if v_f_of is broken.

  1. The comment I have seen and come to believe is that if it is not
    tested then it is not guaranteed to work. The tests define what
    “work” is.

That still leaves the question: at what point can we consider x to
have been tested?

On a small project the programmer may know all the requirements and
remember them at all times, and always get it right. :slight_smile:

But, even on a 2 man project it helps to build tests for what you
expect the software to do.

I think testing is good even if you’re working alone on a project. But
that’s for another discussion :slight_smile:

The application tests will only test the cases that matter to the
applicaiton, while the Rails tests test all the cases the framwork
supports. But, you never know if the Rails tests cover the one case
you care about.

I think you can usually assume it doesn’t. But it’s still a question
of what point you hand off the responsibility to the framework – or
the language. I don’t have any code that specifically tests
String#capitalize, even though I use it and Rails uses it, because I
assume that there are tests in place that test it, in both the Ruby
and the Rails test suites. But if I have a method called title_case,
or something, I’ll test that, and if it fails then I can investigate
where it went wrong.

All of this relates, also, to the use of tests as an anchor for
refactoring. The fact that my title_case method uses capitalize is
incidental, just as it’s incidental that I prevent people from having
too-short passwords by using validates_size_of. There are other ways
to do these things, so it’s really a matter of testing for
functionality (I’ll let someone else be the one to introduce the word
‘behavior’ into the thread :slight_smile: rather than implementation.

David

–
Q. What is THE Ruby book for Rails developers?
A. RUBY FOR RAILS by David A. Black (http://www.manning.com/black)
(See what readers are saying! http://www.rubypal.com/r4rrevs.pdf)
Q. Where can I get Ruby/Rails on-site training, consulting, coaching?
A. Ruby Power and Light, LLC (http://www.rubypal.com)

6

On Sat, Mar 17, 2007 at 08:09:18PM +0100, Xavier N. wrote :

+1. Actually, it doubles the amount of work.

I find that kind of estimations hard to believe, because they sort of
assume the alternative amount of work equals development minus tests.

But that is not the case, you need to sum all the bug fixing that
comes afterwards, normally in production, and normally accompained by
stress, and the uncertainty to deploy yet again without the safety
net of a test suite, heh, perhaps you broke some other part of the
system, and sometime you do.

You’re right. But if you want to test your code on boundaries conditions
(which are the ones which most of time break your code), you’ll need to
imagine them first and then write your methods. It’s just like the
writing of
your app’s core. For your project, you have (let’s assume you’re not the
project leader and you don’t have to mess^W^understand what your
customers really want) a pretty precise idea of what your code should
do, and you spend time thinking how to implement it. For testing
purposes, you already know how to implement tests since it’s just like
the code in your app’s core, but you need to imagine all sort of
conditions: normal cases, boundaries, weirdy, etc.

Actually, I didn’t meant to say that’s development process and testing
require EXACTLY the same time, but testing process requires a lot of
time.

Now, in frameworks or libraries you just can’t afford publishing
something without tests. People are relying on your work, they are
using your software in their production systems. So extra care is
needed.

I completly agree. I am really surprised whenever I read on blogs: ‘hey!
checkout my acts_as_hibernate plugin. INSTALL: svn co http://svn. &
script/generate hibernate_plugin lots_of_messy_xml etc.’. What about
rake test:plugins?

I got accostumed to the idea using (and humbly writing) Perl modules
from CPAN. As you probably know, CPAN modules are expected to have a
test suite. That test suite is run when the module is installed in
your machine, locally, which is even better because all I know as
an author is they pass in my computer (and the ones volunteered by
CPAN testers). If the test suite does not pass module managers won’t
install them unless forced.

‘perl is dead’ (© my algorithmic teacher)

–
pam
I am not sure he’s reliable since he codes (maybe he is coded?)
in
python…

7

On Mar 17, 2007, at 6:48 PM, Pierre-Alexandre M. wrote:

On Sat, Mar 17, 2007 at 04:31:55PM -0000, identry wrote :

The reason I ask is that I’m starting to realize how time
consuming it
is to write tests… it could easily double the amount of work. I
can’t afford that, so am trying to focus tests on my application
code.
I just want to make sure I’m not missing some obvious benefit of
testing the whole Framework.

+1. Actually, it doubles the amount of work.

I find that kind of estimations hard to believe, because they sort of
assume the alternative amount of work equals development minus tests.

But that is not the case, you need to sum all the bug fixing that
comes afterwards, normally in production, and normally accompained by
stress, and the uncertainty to deploy yet again without the safety
net of a test suite, heh, perhaps you broke some other part of the
system, and sometime you do.

Now, in frameworks or libraries you just can’t afford publishing
something without tests. People are relying on your work, they are
using your software in their production systems. So extra care is
needed.

I got accostumed to the idea using (and humbly writing) Perl modules
from CPAN. As you probably know, CPAN modules are expected to have a
test suite. That test suite is run when the module is installed in
your machine, locally, which is even better because all I know as
an author is they pass in my computer (and the ones volunteered by
CPAN testers). If the test suite does not pass module managers won’t
install them unless forced.

– fxn

8

2007/3/17, identry [email protected]:

For example, what is the point of writing a unit test that simply
tests a validate statement in the model? Yes, it’s interesting (the
first time, at least) that the test works and that, golly-gee, Rails
works as advertised, as well, but is there any real use in doing this?

Maybe just some viewpoint shift is needed?
Take look at BDD and RSpec, write executable specification before
code, so first you state:

context "Data Model " do
specify “should not allow blank names” do
@model.name = ‘’
@model.should_not be_valid
end
end

and only then implement the spec:

class Model < ActiveRecord::Base
validates_presence_of :name
end

9

+1. Actually, it doubles the amount of work.

Based on the test writing that I’ve been doing the last few days, I’d
say that writing tests turns a ‘N’ job into a ‘2N’ job… possibly a
‘3N’ job…

Perhaps I’m just new at it and will get faster with practice, but
taking the time to think up tests takes at least as much time as it
does to write the code. And then you have to debug the tests, because
they are no more likely to work the first time than any other code.
I’ve just spent a couple hours writing Unit tests for a handful of
functions.

However…

But that is not the case, you need to sum all the bug fixing that
comes afterwards, normally in production, and normally accompained by
stress, and the uncertainty to deploy yet again without the safety
net of a test suite, heh, perhaps you broke some other part of the
system, and sometime you do.

I guess this is the point. An investment now will pay off in the
future.

Additionally, in writing a bunch of ‘pointless’ model tests, I did
find a bug in a migration… that is the model, and my tests, expected
the database to be set up one way, but the test failed and thereby
uncovered a bug that had been lurking in the database.

So testing the connection between the model code and the database is
one use of validating things that you already “know” are correct.

– John

10

Don’t look it as testing Rails. You are testing your model on testing
validates. Let’s say you write this class:

class Person < ActiveRecord::Base
validates_length_of :phone_number, :within => 7…10
…
end

So you write a test:

def test_phone_number_length_correct
person = Person.new(:phone_number => ‘1234567’)
person.valid?
assert !person.errors.invalid?(:phone_number), ‘Valid Phone Number
Coming Back as Invalid’
end

I think we would all agree that this test makes sure that a valid
phone number passes.

Now not thinking you do a find and replace and replace all 7s with 8s
or maybe number with numbers is more likely. So you have:

class Person < ActiveRecord::Base
validates_length_of :phone_numbers, :within => 7…10
…
end

Now you can place :phone_number => ‘12’ into your database because you
are now validating :phone_numbers.

If you have a test you run tests after you completed the find and
replace, and this error would be caught right away. That is why you
‘test Rails.’

Amos K.
Ramped Media Inc./US Postal Service
Programmer Analyst
dirtyInformation.com

11

Amos.L.King wrote:

person = Person.new(:phone_number => ‘1234567’)
person.valid?

Now not thinking you do a find and replace and replace all 7s with 8s
or maybe number with numbers is more likely.

Or you internationalize. :slight_smile:

(Good sample BTW!)

–
Phlip
http://c2.com/cgi/wiki?ZeekLand ← NOT a blog!!

12

Hi, it’s simply about testing your application. I tend to have
business login in the models. Thus, I don’t really expect this to
have been done by the core team because they do not know anything
about login with my modules. I also I tend to perform both functional
and integration testing as well.

Rails makes it very easy to perform testing at very levels of your
application. If you think that it’s pointless, then you should try
testing J2EE and/or PHP application. When I wrote code in these
technologies, I wished that there was a testing infrastructure like
Rails.

Next, I tend to write my tests first. Then I write the code for my
Rails application so that the tests pass. I’m still getting use to
doing things this way as well as writing tests in general.

Finally, if your client knows enough about Rails, you may be require
to provide these tests as part of the projtect.

-Conrad

13

To me it seems like testing is duplicating (some times 3-4 times) you
work; but in a good way.

So if I have the following:

class Person < ActiveRecord::Base
validates_length_of :phone_number, :within => 7…10
…
end

And the test:

def test_phone_number_length_correct
person = Person.new(:phone_number => ‘1234567’)
person.valid?
assert !person.errors.invalid?(:phone_number), ‘Valid Phone Number
Coming Back as Invalid’
end

I am basically saying in two separate places that my application
requires phone numbers to be in a certain format. By putting this in
two or more separate places (and using different syntax) I find myself
significantly less likely to make a mistake (because I have to think
about it in more than one way) and I find myself significantly more
likely to find the errors.

If the odds of coding something wrong (i.e. differently than I think I
am coding) are 20%. Then the odds of coding the same thing wrong twice
are about 4% (0.2 * 0.2). Writing tests has made me go from writing
code that does what I want 80% of the time to doing what I want 96% of
the time.

I’m sure real life difference isn’t as simple as that math, but I have
found the concept to be true.

PS. I’m not saying that repetition within you application code is
good. DRY is still the best way to go!!!