# Mechanize select list help...?

**URL:** <https://www.ruby-forum.com/t/mechanize-select-list-help/150288>\
**Category:** Ruby\
**Created:** [October 6, 2008, 11:04pm UTC](https://www.ruby-forum.com/t/mechanize-select-list-help/150288 "2008-10-06T23:04:05Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![prot](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/a88e57/32.png) [@prot](https://www.ruby-forum.com/u/prot)\
**Post date:** [October 6, 2008, 11:04pm UTC](https://www.ruby-forum.com/t/mechanize-select-list-help/150288/1 "2008-10-06T23:04:05Z")

</div>

Hi.

I’m using the excellent WWW::Mechanize to screen scrape a site for UK  
frost dates (don’t ask 😉

there’s a lot of issues with the HTML not being grand, so I thought  
that’s where I am going wrong in my code, but I’d be really grateful if  
somebody could give me a steer on this as I’ve been trying for hours,  
and the documentation only gets me half-way 🙂

Here’s the code. All I want to do is select each of the 100 or so towns  
in the select list, follow the link via the submit button and scrape the  
first and last frost dates from the resulting page.

Here’s the code:

require ‘rubygems’  
require ‘mechanize’

agent = WWW::Mechanize.new  
page = agent.get(‘[First and last frost dates for towns and cities in the UK and USA](http://www.gardenaction.co.uk/main/weather1.asp)’)

town\_results = page.form\_with(:action =\> ‘create\_cookie.asp’) do |e|  
e.fields.name(‘Town’).options.each do |s|  
s.select  
end  
end.submit

p town\_results.search(“/\<p align="left"\>HOME TOWN:(.\*)\<Form  
Method=Post Action="create\_cookie.asp"\>/”)

I think I’m actually getting as a result the page itself back not the  
results page (which should be  
[http://www.gardenaction.co.uk/main/weather1-results.asp](http://www.gardenaction.co.uk/main/weather1-results.asp))

Can anyone give me some advice here? It should be obvious I’m new to  
Ruby and OO so am fully expecting to have gone wrong here with instance  
variables or the like 🙂

thanks in advance.

andy

---

<div class="post-metadata">

**Author:** ![prot](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/a88e57/32.png) [@prot](https://www.ruby-forum.com/u/prot)\
**Post date:** [October 7, 2008, 12:36am UTC](https://www.ruby-forum.com/t/mechanize-select-list-help/150288/2 "2008-10-07T00:36:32Z")

</div>

On Oct 6, 5:02 pm, Andy P. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> Here’s the code. All I want to do is select each of the 100 or so towns
> 
> results page (which should behttp://www.gardenaction.co.uk/main/weather1-results.asp)
> 
> Can anyone give me some advice here? It should be obvious I’m new to  
> Ruby and OO so am fully expecting to have gone wrong here with instance  
> variables or the like 🙂

I don’t think it’s the ruby; you need to think it through a bit more.  
How many times will you need to submit the form? Once per town,  
correct? Therefore, the submit and parse should be inside the loop.

Try this for starters:

agent = WWW::Mechanize.new  
page = agent.get(‘[First and last frost dates for towns and cities in the UK and USA](http://www.gardenaction.co.uk/main/weather1.asp)’)

form = page.form\_with(:action =\> ‘create\_cookie.asp’)  
form.fields.name(‘Town’).options.each do |town|  
form[‘Town’] = town  
page2 = form.submit  
puts page2.body  
exit #remove when you’re ready to process them all  
end

---

<div class="post-metadata">

**Author:** ![prot](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/a88e57/32.png) [@prot](https://www.ruby-forum.com/u/prot)\
**Post date:** [October 7, 2008, 8:18pm UTC](https://www.ruby-forum.com/t/mechanize-select-list-help/150288/3 "2008-10-07T20:18:11Z")

</div>

Thanks for the help Mark. You’re right I needed to think it through a  
bit more. Plus, I was unnecessarily using the select method.

Now I’ve got to find a way to grab the stuff from the proceeding pages  
that I need…on to the docs again.

cheers, andy