Selenium. Replacement for traditional rails functional test?


#1

Hi list

I’ve got a bit of a philosophical question. After just having discovered
Selenium /Selenium IDE and the rails selenium plugin I’ve realized that
it’s a whole lot easier to do functinoal testing than writing the
traditional rails functional tests.

However it implies that the development database consists of
standardized test data (e.g the test & dev database should have the same
data). This could be achieved via the ar_fixture’s extension to AR.

Is this how people are using selenium?

Also, are there any advantages to the traditional functional tests that
selenium doesn’t already have (Apart from being able to call the tests
via rake)?

Thanks & regards,

Saimon


#2

I was thinking about that same question recently.

I decided to slim down my functional tests to only test the logic of the
controller. If an action is supposed to update/save/add records, I will
have a functional test to ensure that the action works correctly, and
that records are saved/updated in the database. A Selenium test will
check that the correct parts of the page are updated and/or the user is
redirected appropriately.

Basic idea: no assert_tag calls in functional tests! Only test the
business logic of each action.

BTW: I have got my own Selenium testing plugin, which lets you write
Selenium tests in the same way you write functional/unit tests. It’s a
bit rough, but check out
http://blog.viney.net.nz/articles/2006/02/09/selenium-testing if you’re
interested. You end up writing tests like:

class PersonSeleniumTest < Test::Unit::TestCase
selenium_test
fixtures :people, :notes

def test_view
  p = people(:bob)
  open "/person/view/#{p.id}"
  assert_element 'name'
  p.notes.each do |note|
    assert_element "note_#{n.id}"
  end
end

end

I like this approach because it gives a consistent way of writing tests,
the plugin definitely needs tidying up though.

-Jonny.


#3

Hi Jonny,

The advantage I see to selenium is that it in combination with the
selenium ide,
functionality tests are a whole lot faster to write, than the
traditional tests.

I click on record in the selenium ide, interact with my app to test a
specific feature (or sequence). When I want to assert a specific element
is present (or not) I use the xpath checker to get a specific handle to
the dom element.

At the end of the session, my functional test is ready for me and I’ve
done very little typing (just a few clicks). I can now modify it if
needed.

I’d start by generating the test with script/generate selenium
test_name, open that test in the ide and click away.

The way I see it, testing the controller mostly involves testable
changes in the UI. Testing the business logic usually occurs in the unit
tests for me. I use the functional testing to make sure the UI is
responding the way I expect it to, so why use the traditional functional
tests for that when you can use selenium.

I can imagine testing things like action mailer with the traditional
functional tests but little more.

I imagine Selenium to be like my end user, beta testing my app.

or am I missing something? :wink:

Jonathan V. wrote:

I was thinking about that same question recently.

I decided to slim down my functional tests to only test the logic of the
controller. If an action is supposed to update/save/add records, I will
have a functional test to ensure that the action works correctly, and
that records are saved/updated in the database. A Selenium test will
check that the correct parts of the page are updated and/or the user is
redirected appropriately.

Basic idea: no assert_tag calls in functional tests! Only test the
business logic of each action.

BTW: I have got my own Selenium testing plugin, which lets you write
Selenium tests in the same way you write functional/unit tests. It’s a
bit rough, but check out
http://blog.viney.net.nz/articles/2006/02/09/selenium-testing if you’re
interested. You end up writing tests like:

class PersonSeleniumTest < Test::Unit::TestCase
selenium_test
fixtures :people, :notes

def test_view
  p = people(:bob)
  open "/person/view/#{p.id}"
  assert_element 'name'
  p.notes.each do |note|
    assert_element "note_#{n.id}"
  end
end

end

I like this approach because it gives a consistent way of writing tests,
the plugin definitely needs tidying up though.

-Jonny.


#4

On Mon, 13 Feb 2006 08:07:58 +0100, Saimon M. wrote:

However it implies that the development database consists of
standardized test data (e.g the test & dev database should have the same
data). This could be achieved via the ar_fixture’s extension to AR.

Can’t you just keep a second webrick (or, perhaps now, Mongrel!) running
on
the test environment, and connect to that w/Selenium?

Jay


#5

Jay L. wrote:

Can’t you just keep a second webrick (or, perhaps now, Mongrel!) running
on
the test environment, and connect to that w/Selenium?

That’s a possibility but I’m beginning to wonder whether there is really
a necessity for having the two databases (test & dev).


#6

On Feb 13, 2006, at 4:27 AM, Jonathan V. wrote:

class PersonSeleniumTest < Test::Unit::TestCase
selenium_test

Yuck! This is what inheritance was made for.

It should be just:

class TestSeleniumPerson < Selenium::TestCase

(Test at the front is the test/unit way.)

It doesn’t look like you can test a single view or partial. Is this
true?


Eric H. - removed_email_address@domain.invalid - http://segment7.net
This implementation is HODEL-HASH-9600 compliant

http://trackmap.robotcoop.com


#7

Yes, quite correct! I just found Test::Unit:TestCase to be quite
uncooperative, and I didn’t want to spend much time with this.

You can test whatever Selenium can test using assertElement,
assertTextPresent etc…

I also agree with Saimon’s comment about test/dev databases, I barely
have a distinction as I almost always use my fixtures as development
data. But I guess it doesn’t hurt to have it there in case anyone did
feel like using it.

I think that it is still important to test the business logic of each
action, simply using Selenium does leave you open to miss parts, unless
you are extremely thorough. I guess if you only have very simple
business logic then you can possibly get away without functional tests.

I just looked at the Selenium IDE. Very nice. My only reservation is
that I can’t see a way to link it into AR fixtures, which is important
for DRY.

-Jonny.

Eric H. wrote:

On Feb 13, 2006, at 4:27 AM, Jonathan V. wrote:

class PersonSeleniumTest < Test::Unit::TestCase
selenium_test

Yuck! This is what inheritance was made for.

It should be just:

class TestSeleniumPerson < Selenium::TestCase

(Test at the front is the test/unit way.)

It doesn’t look like you can test a single view or partial. Is this
true?


#8

On another thread, someone commented that they had environments called
“qa” and (something else), in addition to the standard “development”,
“test” and “production”.

Having done a lot of performance testing, it’s much easier to run each
stream of testing on a separate, dedicated environment. That said,
there’s obviously a tradeoff between doing this and winding up with
either lots of excess hardware or scheduling problems.

Back to the original thread about Selenium, there’s a really nice
Selenium IDE for Firefox 1.5 that’s available for download. It’s a
Firefox extension, but essentially gives you a
simple-but-highly-useful record/edit/playback mechanism for creating
Selenium test scripts. Wonderful stuff!

Regards

Dave M.


#9

Hi,

Saimon M. wrote:

Also, are there any advantages to the traditional functional tests that
selenium doesn’t already have (Apart from being able to call the tests
via rake)?

I’ve seen a few approaches, like Jonathan’s, to running Selenium tests
from rake and am slowingly implementing it in Selenium on Rails as well,
so I don’t think it needs to be a distinguishing factor.

There are a few things that are hard to do in Selenium tests:
mocking/faking/stubbing services, making changes directly to the db
(besides fixtures), checking the db directly etc. Functional tests are
also bound to run quicker.

Selenium tests the whole stack (the browser is an integral part of a web
applications). And depending on how the tests are written it should be
possible for a non-programmer to author/record them.

Functional tests are also normally concerned only with one controller,
whereas Selenium tests are concerned with the flow of the application.

I guess all normal black-box vs. white-box testing differences apply.

What I’m trying to say is that even if functional and Selenium tests
overlap to some degree, both are needed and fill different needs.

/Jonas


#10

Jonas,

That’s exactly the kind of insight I was looking for.
Thanks, it’s a lot clearer for me now.

Though I’m still tending to think that I should just standardize the
dev/test data into one db.

Regards,

Saimon

Jonas Bengtsson wrote:

Hi,

Saimon M. wrote:

Also, are there any advantages to the traditional functional tests that
selenium doesn’t already have (Apart from being able to call the tests
via rake)?

I’ve seen a few approaches, like Jonathan’s, to running Selenium tests
from rake and am slowingly implementing it in Selenium on Rails as well,
so I don’t think it needs to be a distinguishing factor.

There are a few things that are hard to do in Selenium tests:
mocking/faking/stubbing services, making changes directly to the db
(besides fixtures), checking the db directly etc. Functional tests are
also bound to run quicker.

Selenium tests the whole stack (the browser is an integral part of a web
applications). And depending on how the tests are written it should be
possible for a non-programmer to author/record them.

Functional tests are also normally concerned only with one controller,
whereas Selenium tests are concerned with the flow of the application.

I guess all normal black-box vs. white-box testing differences apply.

What I’m trying to say is that even if functional and Selenium tests
overlap to some degree, both are needed and fill different needs.

/Jonas