Second assert_tag failling in rails integration test

Hi all,

I’m fairly new to rails and ruby, but I’ve come across an interesting
glitch and I’m not sure whether I’ve just got something wrong with my
assumptions or if it really is an error in the underlying framework…

I’ve created a bare-bones set of tests to show you where the problem is.
What follows are the steps I went through to reproduce the issue.

  1. run “rails test”
  2. create a controller called “test_controller”:

class TestController < ApplicationController
def one
render_text(‘one’)
end

def two
render_text(‘two’)
end
end

  1. create an integration test:
    require “#{File.dirname(FILE)}/…/test_helper”

class MyTest < ActionController::IntegrationTest
def test_one
post(’/test/one’)
assert_tag(:content => /one/)
post(’/test/two’)
assert_match(/two/, @response.body)
assert_tag(:content => /two/)
end
end

  1. ran the integration test.

So, I expected the assertions to all pass, but instead I got the
following error:

Loaded suite test/integration/test_one
Started
F
Finished in 0.131389 seconds.

  1. Failure:
    test_one(MyTest)
    [test/integration/test_one.rb:9:in test_one' /usr/local/lib/ruby/gems/1.8/gems/actionpack-1.12.1/lib/action_controller/integration.rb:427:inrun’]:
    expected tag, but no tag found matching {:content=>/two/} in:
    “two”.
    is not true.

1 tests, 3 assertions, 1 failures, 0 errors

As you can see, it successfully passed the match against @response.body,
but failed the second assert_tag.

What’s interesting is that if you remove the initial assert_tag it works
just fine.

Any ideas?

Taryn

On May 3, 2006, at 10:53 PM, Taryn East wrote:

end

Any ideas?

assert_match /two/, @response.body


Eric H. - [email protected] - http://blog.segment7.net
This implementation is HODEL-HASH-9600 compliant

http://trackmap.robotcoop.com

On Thu, 2006-05-04 at 11:01 -0700, Eric H. wrote:

assert_tag(:content => /two/)

end
end

Any ideas?

assert_match /two/, @response.body

Yes, well that is the path we are currently forced to take (as you see
in the example I gave above). But I was still kind of wondering why
assert_tag doesn’t work the way that I expect it to.

To give a couple of other examples:

These ones work:

def test_one
post(’/test/one’)
assert_tag(:content => /one/)
assert_tag(:content => /n/)
post(’/test/two’)
assert_match(/two/, @response.body)
end

def test_two
post(’/test/one’)
post(’/test/two’)
assert_match(/two/, @response.body)
assert_tag(:content => /two/)
assert_tag(:content => /w/)
end

These ones don’t

def test_three
post(’/test/one’)
assert_tag(:content => /one/)
post(’/test/two’)
assert_match(/two/, @response.body)
assert_tag(:content => /two/) # breaks here
end
def test_four
post(’/test/one’)
assert_tag(:content => /one/)
assert_tag(:content => /n/)
post(’/test/two’)
assert_match(/two/, @response.body)
assert_tag(:content => /two/) # breaks here
assert_tag(:content => /w/)
end

ie - you need:

  • multiple assert_tag calls and at least two post/get calls.
  • At least one assert_tag has to occur both before and after the second
    get/post.
  • If there are multiple calls before the get/post - they all work.
  • It fails on the first assert_tag that occurs after the second
    get/post.

What I can only assume (not knowing the underlying architecture) is that
the first assert_tag does something to internal variables (the book says
it “parses the response into a DOM”)… then when the next get/post
occurs, the response changes… but the next assert_tag doesn’t check
that the response has changed - it is still using the old parsed DOM.

Cheers,
Taryn

On May 21, 2006, at 6:46 PM, Taryn East wrote:

You know I never did get a real response[1] to this question…

Does anyone else here actually use integration tests?

I don’t, but I know people do.

Does they work for you?

They seem to for the people that use them.

Do you not do what we are doing?

They probably don’t, integration testing isn’t about testing HTML.

Are we using the wrong kind of test here?

You should be writing functional tests when using assert_tag.


Eric H. - [email protected] - http://blog.segment7.net
This implementation is HODEL-HASH-9600 compliant

http://trackmap.robotcoop.com

You know I never did get a real response[1] to this question…

Does anyone else here actually use integration tests? Does they work for
you? Do you not do what we are doing? Are we using the wrong kind of
test here?
Has anybody got any ideas?

To recap on the issue I had:
the steps I went through to reproduce the issue.

  1. run “rails test”
  2. create a controller called “test_controller”:

class TestController < ApplicationController
def one
render_text(‘one’)
end
def two
render_text(‘two’)
end
end

  1. create an integration test:
    require “#{File.dirname(FILE)}/…/test_helper”

class MyTest < ActionController::IntegrationTest
def test_one
post(’/test/one’)
assert_tag(:content => /one/)
post(’/test/two’)
assert_match(/two/, @response.body)
assert_tag(:content => /two/)
end
end

  1. run the integration test.

So, I expected the assertions to all pass, but instead I got the
following error:

Loaded suite test/integration/test_one
Started
F
Finished in 0.131389 seconds.

  1. Failure:
    test_one(MyTest)
    [test/integration/test_one.rb:9:in test_one' /usr/local/lib/ruby/gems/1.8/gems/actionpack-1.12.1/lib/action_controller/integration.rb:427:inrun’]:
    expected tag, but no tag found matching {:content=>/two/} in:
    “two”.
    is not true.

1 tests, 3 assertions, 1 failures, 0 errors

To give a couple of other examples:

These ones work:

def test_one
post(’/test/one’)
assert_tag(:content => /one/)
assert_tag(:content => /n/)
post(’/test/two’)
assert_match(/two/, @response.body)
end

def test_two
post(’/test/one’)
post(’/test/two’)
assert_match(/two/, @response.body)
assert_tag(:content => /two/)
assert_tag(:content => /w/)
end

These ones don’t

def test_three
post(’/test/one’)
assert_tag(:content => /one/)
post(’/test/two’)
assert_match(/two/, @response.body)
assert_tag(:content => /two/) # breaks here
end
def test_four
post(’/test/one’)
assert_tag(:content => /one/)
assert_tag(:content => /n/)
post(’/test/two’)
assert_match(/two/, @response.body)
assert_tag(:content => /two/) # breaks here
assert_tag(:content => /w/)
end

ie - you need:

  • multiple assert_tag calls and at least two post/get calls.
  • At least one assert_tag has to occur both before and after the second
    get/post.
  • If there are multiple calls before the get/post - they all work.
  • It fails on the first assert_tag that occurs after the second
    get/post.

What I can only assume (not knowing the underlying architecture) is that
the first assert_tag does something to internal variables (the book says
it “parses the response into a DOM”)… then when the next get/post
occurs, the response changes… but the next assert_tag doesn’t check
that the response has changed - it is still using the old parsed DOM.

Cheers,
Taryn

[1] Yes, I got Eric’s response of:
assert_match /two/, @response.body
which we are already using… but while it’s possible to write a regexp
to match on something like:
assert_tag(:tag => ‘td’,
:attributes => {:id => ‘status-cell’},
:child => {:tag => ‘img’,
:attributes => {:alt => %r’Complete’}}
)
it’s really annoying… :stuck_out_tongue:

On Mon, 2006-05-22 at 11:47 -0700, Eric H. wrote:

On May 21, 2006, at 6:46 PM, Taryn East wrote:

Does anyone else here actually use integration tests?

I don’t, but I know people do.

I haven’t heard anything from anybody else that does… and I guess I’m
just beginning to wonder…

Does they work for you?

They seem to for the people that use them.

Do you not do what we are doing?

They probably don’t, integration testing isn’t about testing HTML.

We’re not actually testing the html itself, we’re testing the flow of
screens… which I assumed is what integration tests are for. It’s just
that to test that we’re arriving at the right screen with the right data
on it we use assert_tag.

We also use assert_response and assert_template (which seem to be the
main thing used in integration tests I’ve found) but you have to test
not only that an edit screen popped up, but that the right one popped
up - and the way to test that is to check that it was populated by the
data that you actually asked for. We don’t test that every bit of data
is there (that’s what I believe belongs in the realm of functional
testing - though maybe I have that wrong too) but we need to check that
something is right so for example we check for the existence of one key
field, or that one input has been pre-populated with previously-chosen
data. We check that, for example, this occurs whether or not the user is
still using the same session, or if they login with a different session
and come to the same page.

If we are doing this in the wrong way then I would be very appreciative
of any assistance in directing us towards the correct way to test this
sort of thing.

Are we using the wrong kind of test here?

You should be writing functional tests when using assert_tag.

Ok, so then what are integration tests for? It’s quite possible that we
have misunderstood what level of testing we need… but the article I
read on it here:
http://jamis.jamisbuck.org/articles/2006/03/09/integration-testing-in-rails-1-1

seemed to indicate that integration tests were used for screen flow…
which is what (afaics) we are doing.

In any case even if we are using integration tests completely
incorrectly… there is still a bug in the framework if assert_tag is
failing.

Cheers and thanks,
Taryn

On May 22, 2006, at 4:09 PM, Taryn East wrote:

On Mon, 2006-05-22 at 11:47 -0700, Eric H. wrote:

On May 21, 2006, at 6:46 PM, Taryn East wrote:

Does they work for you? They seem to for the people that use them.

Do you not do what we are doing?

They probably don’t, integration testing isn’t about testing HTML.

I’m going to take the liberty of rearranging this paragraph.

we’re testing the flow of screens… which I assumed is what
integration tests are for.

Correct.

We’re not actually testing the html itself, […] It’s just that to
test that we’re arriving at the right screen with the right data on
it we use assert_tag.

If you’re using assert_tag you’re testing the HTML.

assert_tag tests at the wrong level. Test assigns instead.

Your functional tests should ensure that data on the controller side
correctly shows up in the right place on the HTML. If you’re using
assert_tag you’re testing in two places.

We also use assert_response and assert_template (which seem to be
the main thing used in integration tests I’ve found) but you have
to test not only that an edit screen popped up, but that the
right one popped up and the way to test that is to check that it
was populated by the data that you actually asked for.

This is the responsibility of functional tests. When your functional
tests ensure that data is in the right spot then you can test based
purely on assigns in your integration tests and cut out a ton of
(duplicated) code.

We don’t test that every bit of data is there (that’s what I
believe belongs in the realm of functional testing - though maybe I
have that wrong too) but we need to check that something is right
so for example we check for the existence of one key field, or that
one input has been pre-populated with previously-chosen data. We
check that, for example, this occurs whether or not the user is
still using the same session, or if they login with a different
session and come to the same page.

And that data gets there via assigns, so exploit that fact and make
your tests simpler. The bigger the test the more likely it is wrong.

If we are doing this in the wrong way then I would be very
appreciative of any assistance in directing us towards the correct
way to test this sort of thing.

I actually think that even functional tests are too high level, so I
wrote Test::Rails to break functional tests down into controller
tests and view tests. I found my functional tests repeating
themselves too much so I needed to get closer to what the real
responsibilities of controllers and views are.

Controller tests ensure that URL params map to the correct assigns
and view tests ensure that assigns map to the correct HTML. Once I
know these tests all pass then integration tests can rely strictly on
assigns and I avoid tons of code.

seemed to indicate that integration tests were used for screen
flow… which is what (afaics) we are doing.

Testing is broken into multiple layers. What you test on one layer
shouldn’t be tested on another. What is tested on one layer you can
rely on in a higher layer.

Here’s an untested example:

class EntryControllerTest < Test::Rails::ControllerTestCase

fixtures :entries

def test_view
get :view, :id => entries(:frogs).id
assert_response :success
assert_template ‘entry/view’
assert_assigned :entry, entries(:frogs)
end

def test_edit
get :edit, :id => entries(:frogs).id
assert_response :success
assert_template ‘entry/edit’
assert_assigned :entry, entries(:frogs)
end

def test_update
post :update, :id => entries(:frogs).id, :body => ‘Frogs rock!’
assert_redirected_to “/entry/view/#{entries(:frogs).id}”
entries(:frogs).reload
assert_equal entries(:frogs).body, ‘Frogs rock!’
end

end

class EntryViewTest < Test::Rails::ViewTestCase

fixtures :entries

def test_view
assigns[:entry] = entries(:frogs)
render
assert_tag :tag => ‘p’, :content => entries(:frogs).body
end

def test_edit
assigns[:entry] = entries(:frogs)
render
form_url = “/entries/update/#{entries(:frogs).id}”
assert_post_form form_url
assert_tag_in_form form_url, :tag => ‘textarea’,
:content => entries(:frogs).body
end

end

Now I know that all the URLs and data will show up correctly I can do
an integration test.

class EntryIntegrationTest < ActionController::IntegrationTest

fixtures :entries

def test_editing
entry = entries(:frogs)
get “/entry/edit/#{entry.id}”
assert_response :success
assert_template ‘entry/edit’

 post "/entry/update/#{entry.id}" :body => 'Frogs rock!'
 assert_response :redirect

 follow_redirect!
 assert_success
 assert_template 'entry/view'

end

end

I already know all the URLs in the HTML will be correct for that
template so I don’t need to re-test that.

In any case even if we are using integration tests completely
incorrectly… there is still a bug in the framework if assert_tag
is failing.

I know exactly what the problem is, and it isn’t a problem if you
write your integration tests correctly.


Eric H. - [email protected] - http://blog.segment7.net
This implementation is HODEL-HASH-9600 compliant

http://trackmap.robotcoop.com