Hi,
We are upgrading to cucumber 0.1.99 in the process we also upgraded to
the latest versions of webrat and rspec. However now we are having
difficulties checking for ActionMailer.deliveries in the selenium
features. I seem to recall being able to check for these without
problems using webrat’s wrapper for selenium, but maybe my memory is
tricking me…
Lately we were solving this by using Polomium, here is our config
However, with the new configuration for webrat selenium features:
Webrat.configure do |config|
config.mode = :selenium
end
We can’t seem to get Polonium working nicely with webrat.
Is there something wrong with our configuration or how are you guys
checking for ActionMailer deliveries in your selenium stories?
Cheers,
Rai
Raimond G. wrote:
gist:39038 · GitHub
checking for ActionMailer deliveries in your selenium stories?
Cheers,
Rai
One option is to use ARMailer[1] to queue your mail. As long as your
selenium process and test process are using the same DB without
transactions getting in the way then ARMailer should work fine for what
you want to do.
Thanks to the guys at weplay EmailSpec[2] works happily with ARMailer so
you can use the same helpers that EmailSpec provides for the regular
in-memory ActionMailer test deliveries.
-Ben
-
http://seattlerb.rubyforge.org/ar_mailer/classes/ActionMailer/ARMailer.html
2. GitHub - email-spec/email-spec: Collection of RSpec/MiniTest matchers and Cucumber steps for testing email in a ruby app using ActionMailer or Pony
On Wed, Feb 18, 2009 at 3:02 PM, Ben M. [email protected] wrote:
One option is to use ARMailer[1] to queue your mail. As long as your
selenium process and test process are using the same DB without transactions
getting in the way then ARMailer should work fine for what you want to do.
Is changing production infrastructure solely for the sake of making
tests work a recommended practice? To me it feels a bit backwards.
Raimond: I can’t address the problems you’re having with Polonium, as
I’d never heard of it until this thread (and Googling it was
surprisingly hard; I finally found
http://github.com/pivotal/polonium). But it does seem to me like
Webrat and its various dependencies all manage to break each other’s
APIs with every minor release, so this doesn’t surprise me. I’ve had
to roll back versions of Nokogiri several times for things like this.
If you’re at the level of integration testing, checking
ActionMailer.deliveries may not be the way to go anyway. I’d look
instead to see if the e-mails were actually sent. As I see it, even
if ActionMailer thinks it’s done its job, mail could still fail for
other reasons and that’s a failed system interaction. Here’s an idea
I’ve seen that’s independent of delivery method; it’s basically just
overriding the ActionMailer recipients in test mode so that all
e-mails go to one address:
From that point, your tests could use Ruby’s standard mail libraries
to retrieve the mail and scan it, or scrape ‘mail’ from the shell, or
read the entrails of your enemies, or whatever else you like to
confirm that the right stuff is in the message.
–
Have Fun,
Steve E. ([email protected])
ESCAPE POD - The Science Fiction Podcast Magazine
http://www.escapepod.org
On Wed, Feb 18, 2009 at 3:15 PM, Stephen E. [email protected] wrote:
On Wed, Feb 18, 2009 at 3:02 PM, Ben M. [email protected] wrote:
One option is to use ARMailer[1] to queue your mail. As long as your
selenium process and test process are using the same DB without transactions
getting in the way then ARMailer should work fine for what you want to do.
Is changing production infrastructure solely for the sake of making
tests work a recommended practice? To me it feels a bit backwards.
I’m not commenting on this particular situation, but if I want to use
some library, but I can’t figure out how to write tests for my code
that uses it, then I don’t use it.
Basically, if I have to change something to make it testable, I do.
Pat
Stephen E. wrote:
On Wed, Feb 18, 2009 at 3:02 PM, Ben M. [email protected] wrote:
One option is to use ARMailer[1] to queue your mail. As long as your
selenium process and test process are using the same DB without transactions
getting in the way then ARMailer should work fine for what you want to do.
Is changing production infrastructure solely for the sake of making
tests work a recommended practice? To me it feels a bit backwards.
Well, in this case the ARMailer is an arguably better alternative than
waiting on an SMTP connection for ActionMailer in production settings.
ARMailer was not created to solve testing woes… it was created it solve
production woes. 
The fact that it helps in testing is just a side
effect.
-Ben
On Wed, Feb 18, 2009 at 5:43 PM, Pat M. [email protected]
wrote:
Basically, if I have to change something to make it testable, I do.
Yeah. Testability is a positive attribute. If you’d change code to
make it more readable or more speedy, why not more testable?
///ark
On Wed, Feb 18, 2009 at 6:27 PM, Ben M. [email protected] wrote:
Well, in this case the ARMailer is an arguably better alternative than
waiting on an SMTP connection for ActionMailer in production settings.
ARMailer was not created to solve testing woes… it was created it solve
production woes. The fact that it helps in testing is just a side
effect.
Oh, sure. I wasn’t trying to be critical against ARMailer itself; I
do think that e-mail is the sort of side process that ought to be
brought out of the request critical path one way or another. But
that’s unrelated here. It just seems like the sort of process change
that might be overkill for ‘I upgraded my gems and then my testing
toolchain broke.’ Testability may be a factor in initial tool
selection, but in this case the tool was testable until the tests
were broken by an unrelated change. Once a production process is in
place, I think changes to it should be driven by business
requirements.
I just had a sympathetic twitch for the original poster. I could be
on some other forum, asking about agility training for my dog or
something, and somebody will say “See, if you had a cat, you wouldn’t
have that problem.”
Yeah, maybe, but the fact that my dog is a dog wasn’t really the
problem.
–
Have Fun,
Steve E. ([email protected])
ESCAPE POD - The Science Fiction Podcast Magazine
http://www.escapepod.org
On Wed, Feb 18, 2009 at 8:43 PM, Pat M. [email protected]
wrote:
I’m not commenting on this particular situation, but if I want to use
some library, but I can’t figure out how to write tests for my code
that uses it, then I don’t use it.
I take the opposite view. My goal in coding isn’t to write good
tests, it’s to solve somebody’s problem. My own or someone else’s.
The right tool to help solve the problem is the right tool. Testing
plays a supporting role, not a decision-making role.
Anyway, there are very few cases where I can’t figure out some
straightforward way to test a behavior. The last time it happened,
ironically enough, was when I found a bug in Merb’s Autotest hooks for
Rspec. There was a loop termination issue such that Rspec worked fine
until you tried to run Cucumber right after it with the merb-cucumber
gem. Writing specs to prove that your specs terminate was… Well,
after a couple hours I gave up and just submitted a patch with “I know
this is the wrong way to do things, but here. My head hurts.”
–
Have Fun,
Steve E. ([email protected])
ESCAPE POD - The Science Fiction Podcast Magazine
http://www.escapepod.org
Stephen E. wrote:
do think that e-mail is the sort of side process that ought to be
brought out of the request critical path one way or another. But
that’s unrelated here. It just seems like the sort of process change
that might be overkill for ‘I upgraded my gems and then my testing
toolchain broke.’ Testability may be a factor in initial tool
selection, but in this case the tool was testable until the tests
were broken by an unrelated change. Once a production process is in
place, I think changes to it should be driven by business
requirements.
Yeah, and so I probably wouldn’t change to ARMailer in production if
that was an issue or wasn’t really needed. So I agree with you there.
However, I wouldn’t have a problem in using ARMailer in my tests but
still use straight ActionMailer in production. ActionMailer works and I
don’t need to test that. What I’m interested in testing is if I am
sending the correct emails at the correct time. If I really need to
test this part of the app in conjunction with selenium or another
secondary process then setting up ARMailer just for that purpose seems
like a reasonable investment. Once you have that tested the only point
of failure in production would be your ActionMailer settings. I’ll point
out that you will face the same risk when using ActionMailer in :test
mode anyways. So coming up with a method to test your settings would be
needed in either case. My method for doing this is clicking through the
site once to fire off an email and never changing the working production
settings again.
An alternative to using ARMailer would be to use something like
Mailtrap[1] that acts as a dummy SMTP server to capture the emails from
the app. You would then need to provide a way for the tests to access
these emails to verify them.
Both ways involve some extra work. If it isn’t worth the investment
then the only other option is to test the emails apart from the selenium
tests. It just depends on how important it is that you test everything
at the same time.
-Ben
- GitHub - mmower/mailtrap: A mock SMTP server for use in Rails development.
On Thu, Feb 19, 2009 at 12:13 AM, Ben M. [email protected] wrote:
Both ways involve some extra work. If it isn’t worth the investment then
the only other option is to test the emails apart from the selenium tests.
It just depends on how important it is that you test everything at the same
time.
Good point. I have to confess, when I first read the message about
using the Selenium framework to test e-mail delivery, my first thought
was: “Er… Selenium? Are you automating the browser to check your
Gmail or something…?”
–
Have Fun,
Steve E. ([email protected])
ESCAPE POD - The Science Fiction Podcast Magazine
http://www.escapepod.org
-
http://seattlerb.rubyforge.org/ar_mailer/classes/ActionMailer/ARMailer.html
P.S. I’ll follow up with a confirmation post just to let the people know
how it worked out for us with ARMailer.
Yeap everything worked like a charm, once again checking email contents
like there is no tomorrow 
Rai
Thanks guys, very interesting points.
As a good collegue of mine says, its all about trade offs 
I agree with changing your production code to make it more testable,
definitely, especially early on in the development process, once your
site is up and running it would have to be done with extra care,
we don’t wanna be sending hundreds of emails accidentally, and I must
confess I can unfortunately relate, even though it was a team effort to
achieve it Thus I’m happy you mentioned Duncan Beevers blog post,
Steve, cheers. I’ll definitely recommend it as a precaution tool to the
team.
For my particular situation ARMailer sounds just like what we need to
get the build passing again, and will certainly shot out a couple of
emails from the staging server to make sure emails are actually being
send.
“Er… Selenium? Are you automating the browser to check your
Gmail or something…?”
Well this might sound like a crazy idea… but I admit that I’ve
discussed this before with my team, and concluded that, without
including speed limit constraints, it was the most thorough way of
testing our signup and confirmation system
Thanks again for all your help!
Cheers,
Rai
P.S. I’ll follow up with a confirmation post just to let the people know
how it worked out for us with ARMailer.
Hi Rai,
Good that you have found a solution…
but I still don’t understand why you want to test sending mails from the
browser. In my project, we use Cucumber + Webrat for writing functional
features.
and Cucumber + selenium for acceptance features.
Functional tests can enable ruby code and test for the number of mails
sent
and to whom, etc
Acdeptance only tests what the user would see…
Let me know if it makes sense… and it would be great if you could
elaborate
on your purpose of testing mails from the browser…
Thanks,
Manasi
–
View this message in context:
http://www.nabble.com/Cucumber-%2B-Webrat-%2B-Selenium-%2B-ActionMailer-tp22086863p22350981.html
Sent from the rspec-users mailing list archive at Nabble.com.
Manasi V. wrote:
Hi Rai,
Good that you have found a solution…
but I still don’t understand why you want to test sending mails from the
browser. In my project, we use Cucumber + Webrat for writing functional
features.
and Cucumber + selenium for acceptance features.
Functional tests can enable ruby code and test for the number of mails
sent
and to whom, etc
Acdeptance only tests what the user would see…
Let me know if it makes sense… and it would be great if you could
elaborate
on your purpose of testing mails from the browser…
Thanks,
Manasi
Hi Manasi,
We have a bit of both, sometimes we can test email sending with Cucumber
- Webrat, but in this particular case, the feature that sent out emails
also used Javascript.
We could have split the feature in two, one to test the Javascript with
a Selenium feature and one to test the emails with a Webrat feature, but
IMHO adding a couple of steps to the selenium feature to check for
emails, is a good choice to help to clearly see the whole intent of the
feature.
Rai