How much test data to use in specs

I’m not sure how much test data I should be using in my specs. I’m
writing specs for the Property model in my Rails app. Its “address”
attribute is going to be validated with this regex:
/\A\d+[a-z]? [-’, a-z]{2,128}\Z/i

At the moment, my plan is to spec out the following possibilities. A
property is invalid if its address:

  1. doesn’t begin with a digit;
  2. is shorter than 2 characters;
  3. is longer than 128 characters;

I should also test that certain characters, such as ! @ # $ etc,
invalidate a property.

However, all of that seems like blacklisting, and achieves poor
coverage of the regular expression.

Should I create a list of valid and invalid addresses to test against
the regex? That seems like a decent idea, but seems synonymous with
fixtures. What would you guys recommend?

Cheers,
Nick

On Mon, Aug 25, 2008 at 12:53 PM, Nick H. [email protected]
wrote:

I should also test that certain characters, such as ! @ # $ etc, invalidate
a property.

However, all of that seems like blacklisting, and achieves poor coverage of
the regular expression.

Should I create a list of valid and invalid addresses to test against the
regex? That seems like a decent idea, but seems synonymous with fixtures.
What would you guys recommend?

I might do something like the following…

describe Property, “email validations” do

[“[email protected]”, “can’t start with a digit”,
[email protected]”, “can’t end with a digit”
].in_groups_of(2) do |email, description|
it description do
prop = Property.new :email => email
prop.should_not be_valid
prop.should have(1).error_on(:email)
end
end
end


Zach D.
http://www.continuousthinking.com
http://www.mutuallyhuman.com

On 2008-08-25, at 13:29, Zach D. wrote:

 prop.should have(1).error_on(:email)

end
end
end


Zach D.
http://www.continuousthinking.com
http://www.mutuallyhuman.com

Hi Zach. That’s a great way of iterating over test data. Do you have
any suggestions for how much test data to use?

Cheers,
Nick

On Mon, Aug 25, 2008 at 2:50 PM, Nick H. [email protected]
wrote:

prop =  Property.new :email => email

Hi Zach. That’s a great way of iterating over test data. Do you have any
suggestions for how much test data to use?

I’d probably start with the first invalid email/description that I can
think of. Perhaps it can’t start with a digit. Then I’d update the
regexp to reflect that. Next I’d add a second invalid
email/description and update the regexp to reflect that. Rinse and
repeat until your regexp is where you need it. This should help you
not add redundant emails which test the same thing. When you get to
the case where you allow from 2 up to 128 characters, you could just
test those two edge cases (rather than emails which 2 characters, 3
characters, 4 characters, … 128 characters.


Zach D.
http://www.continuousthinking.com
http://www.mutuallyhuman.com

Hi all,

On 25 Aug 2008, at 17:53, Nick H. wrote:

At the moment, my plan is to spec out the following possibilities.
A property is invalid if its address:

  1. doesn’t begin with a digit;
  2. is shorter than 2 characters;
  3. is longer than 128 characters;

Personally I think that if those three points are your specification
then your RSpec should match that.

So:

  • it “should not begin with a digit”,
  • it “should not be shorter than 2 characters”,
  • it “should not be longer than 128 characters” and
  • it “should allow valid addresses” (this is for clarity, so you have
    an example of a valid address in your documentation).

If your regex allows or disallows more or less than those four
clauses then your specification does not match your implementation.

Baz

Rahoul B.
Web design and development: http://www.3hv.co.uk/
Nottingham Forest: http://www.eighteensixtyfive.co.uk/
Serious Rails Hosting: http://www.brightbox.co.uk/
Lifecast: http://www.madeofstone.net/

If your regex allows or disallows more or less than those four
clauses then your specification does not match your implementation.

So this is interesting, because my approach seems to be more like this:

describe “when the address does not begin with a digit” do
it “should be invalid” do … end
end

describe “when the address is less than two characters long” do
it “should be invalid” do … end
end

describe “when the address is more than two characters long” do

describe “and the address is less than 128 characters long” do
it “should be valid” do … end
end

describe “and the address is more than 128 characters long” do
it “should be invalid” do … end
end

end

Is anyone else using ExampleGroups like this? Or is it just me?!

cheers,
Matt

On Mon, Aug 25, 2008 at 3:02 PM, Zach D. [email protected]
wrote:

it description do
http://www.mutuallyhuman.com

Hi Zach. That’s a great way of iterating over test data. Do you have any
suggestions for how much test data to use?

I’d probably start with the first invalid email/description that I can
think of. Perhaps it can’t start with a digit. Then I’d update the
regexp to reflect that. Next I’d add a second invalid
email/description and update the regexp to reflect that. Rinse and
repeat until your regexp is where you need it.

This is nit-picky, I realize, but saying “until your regexp is where
you need it” suggests that you’ll know when you get there by looking
at the regexp. I’d suggest that you know when you get there by looking
at the code examples and feeling confident that you’ve covered all of
the edge cases.

This should help you
not add redundant emails which test the same thing. When you get to
the case where you allow from 2 up to 128 characters, you could just
test those two edge cases (rather than emails which 2 characters, 3
characters, 4 characters, … 128 characters.

If the boundaries are 2 and 128, I would test 1,2,3,127,128,129 - the
boundary and either side of it. Might seem like overkill, but I’ve
seen a bug or two make its way to production because the boundary
wasn’t clearly understood.

FWIW,
David

On 2008-08-27, at 12:57, Rahoul B. wrote:

Rahoul B.

This is what I ended up with:

Property addresses that are valid

  • can’t be 129 characters
  • can’t have a 1-letter street name
  • can’t have an &
  • can’t have an !
  • can’t have a "
  • can’t have 2 letters after the street number
  • can’t be missing a street number
  • can’t have only 3 characters
  • can’t be empty
  • can have 128 characters
  • can have 127 characters
  • can have only 5 characters
  • can have only 4 characters
  • can have a ,
  • can have a -
  • can have a ’
  • can have 1 letter after the street number
  • can have multiple spaces

Each of the “can …” examples are proper addresses in the sense that
they’re variations of:

  • “123 A Street With A Long Name”
  • “123B Maple Ave”
  • “123 O’Connor Street”
  • etc

One thing that I didn’t do is write examples for each invalid
character. That’d be too hairy and verbose. Instead, I picked a few,
and wrote examples for them, as you can see above (Eg: “can’t have an
&”).

Cheers,
Nick

On Wed, Aug 27, 2008 at 1:03 PM, Nick H. [email protected]
wrote:

B

  • can’t have an &
  • can have a ,
  • etc

One thing that I didn’t do is write examples for each invalid character.
That’d be too hairy and verbose. Instead, I picked a few, and wrote examples
for them, as you can see above (Eg: “can’t have an &”).

If you want to (not necessarily advising this, but I’ve seen it done)
you
can do this:

[’!’,’(’,’)’,’&’].each do |char|
it “can’t have #{char}” do

end
end

That makes the output very verbose, but the spec file is easy to grok.

what if my office were at

Route 102 & Yahoo! Way, Suite #123

:slight_smile:

On 27 Aug 2008, at 16:35, Matt W. wrote:

describe “and the address is more than 128 characters long” do
it “should be invalid” do … end
end

end

For this, trivial, example, I find that far too verbose.

The “specification” says “if it is less than 2 characters then
invalid” and “if it is more than 128 characters then invalid” .

But you are actually running three checks - “if less than 2”, “if
greater than 2 but less than 128” and “if greater than 2 and greater
than 128”.

B

Rahoul B.
Web design and development: http://www.3hv.co.uk/
Nottingham Forest: http://www.eighteensixtyfive.co.uk/
Serious Rails Hosting: http://www.brightbox.co.uk/
Lifecast: http://www.madeofstone.net/

On Wed, Aug 27, 2008 at 12:05 PM, Nick H. [email protected]
wrote:

need to prevent addresses from having symbols such as !, &, ", etc. If
people really want to put them in, why not?

Validation isn’t just about preventing users from doing things they want
to
do - it’s also to prevent errors.

At any rate, what consitutes a valid address is a business rule, and
isn’t a
programming issue.

///ark

We’d recommend that you start sending our your resumé. :slight_smile:

cr

On 2008-08-27, at 14:55, Jonathan L. wrote:

what if my office were at

Route 102 & Yahoo! Way, Suite #123

:slight_smile:

Good point. Thinking about it again, there’s probably not all that
much need to prevent addresses from having symbols such as !, &, ",
etc. If people really want to put them in, why not?

Does your office actually not have a proper street number?
-Nick

On Wed, Aug 27, 2008 at 2:24 PM, Nick H. [email protected]
wrote:

That makes the output very verbose, but the spec file is easy to grok.

Hi David. I ended up creating valid and invalid test data, as well as a
method to iterate over them and write the descriptions and examples for me.
You can have a look at it at http://pastie.org/261175 . If you have any
suggestions for improvements, I’d love to hear them.

Hey Nick,

I can definitely see some value in this, though there are a few things
that I’d do differently. Have a look at http://pastie.org/261642 and
let me know if you have any questions/comments. The only thing I’m not
sure about is the actual method name - it’s not speaking to me, but
I’m at a loss for a better one. Maybe after some coffee …

Cheers,
David

On 2008-08-27, at 15:24, Nick H. wrote:

Hi David. I ended up creating valid and invalid test data, as well
as a method to iterate over them and write the descriptions and
examples for me. You can have a look at it at http://pastie.org/
261175 . If you have any suggestions for improvements, I’d love to
hear them.

Cheers,
Nick

I forgot to mention that I’m using the “factories-and-workers” plugin:
http://github.com/dfl/factories-and-workers/
which provides the the “build_property” method.

On 2008-08-28, at 08:31, David C. wrote:


You can have a look at it at http://pastie.org/261175 . If you have

Cheers,
David

Thanks for taking a look and making some suggestions, David. Much
appreciated. I’ve incorporated some of them into the method.

The method’s name was “describe_property_attributes” because it was
being used to describe the Property (as in “real estate”) model in my
application. I’ve just finished abstracting it out so that it can now
be used with any model class. If you’re interested, I’ve pasted it at
http://pastie.org/261829

On 2008-08-27, at 14:46, David C. wrote:

If you want to (not necessarily advising this, but I’ve seen it
done) you can do this:

[’!’,’(’,’)’,’&’].each do |char|
it “can’t have #{char}” do

end
end

That makes the output very verbose, but the spec file is easy to grok.

Hi David. I ended up creating valid and invalid test data, as well as
a method to iterate over them and write the descriptions and examples
for me. You can have a look at it at http://pastie.org/261175 . If you
have any suggestions for improvements, I’d love to hear them.

Cheers,
Nick

I started using a set of methods to deal with this problem from code in
the RadiantCMS(http://radiantcms.org/) project.

http://gist.github.com/7936

I stopped using it after a while finding the tests did not read well.
Your method improves on Radiant’s which has tempted me to start testing
models like this again.

Thanks!

Joseph W.
http://www.joesniff.co.uk

Nick H. wrote:

On 2008-08-28, at 08:31, David C. wrote:


You can have a look at it at http://pastie.org/261175 . If you have

Cheers,
David

Thanks for taking a look and making some suggestions, David. Much
appreciated. I’ve incorporated some of them into the method.

The method’s name was “describe_property_attributes” because it was
being used to describe the Property (as in “real estate”) model in my
application. I’ve just finished abstracting it out so that it can now
be used with any model class. If you’re interested, I’ve pasted it at
http://pastie.org/261829

On Fri, Aug 29, 2008 at 9:03 AM, Nick H. [email protected]
wrote:

Thanks!

Joseph W.
http://www.joesniff.co.uk

Thanks for the vote of confidence, Joseph! I have two other methods that go
alongside #describe_model_attribute which you might find useful. I’m in the
process of wrapping them up into a plugin. Until that’s finished, I’ve
pasted them at http://pastie.org/262508 and http://pastie.org/262509 .

Hey Nick - I’m not clear why you need these, given that you already
have the describe_model_attributes macro, which you can use to handle
nils and will give you a meaningful error if there is a problem with
your factory. What’s your goal here?

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs