# Mechanize question **URL:** \ **Category:** Ruby\ **Created:** [November 21, 2006, 2:29pm UTC](https://www.ruby-forum.com/t/mechanize-question/76822 "2006-11-21T14:29:04Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![splattael](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/858c86/32.png) [@splattael](https://www.ruby-forum.com/u/splattael)\ **Post date:** [November 21, 2006, 2:29pm UTC](https://www.ruby-forum.com/t/mechanize-question/76822/1 "2006-11-21T14:29:04Z")
Hello, I am using mechanize 0.6.3. On Aaron’s blog I have found this example: form.selectlist.options[2].select however, for me, ‘puts form.methods.sort’ revealed that form does not have a method ‘selectlist’. What’s up? I am doing something wrong? Here is the code I am using: require ‘rubygems’ require ‘mechanize’ agent = WWW::Mechanize.new page = agent.get ‘[www.some-page.com](http://www.some-page.com)’ form = page.forms.with.name(‘formname’).first and this form does not have a method selectlist. (just in case: page.forms.with.name(‘formname’) == ‘WWW::Mechanize::Form’ and not nil or other kind of nonsense 🙂 Thanks, Peter \_\_ [http://www.rubyrailways.com](http://www.rubyrailways.com) ---
**Author:** ![splattael](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/858c86/32.png) [@splattael](https://www.ruby-forum.com/u/splattael)\ **Post date:** [November 21, 2006, 2:47pm UTC](https://www.ruby-forum.com/t/mechanize-question/76822/2 "2006-11-21T14:47:09Z")
Peter S. wrote: > Hello, > > I am using mechanize 0.6.3. On Aaron’s blog I have found this example: > > form.selectlist.options[2].select > > however, for me, ‘puts form.methods.sort’ revealed that form does not > have a method ‘selectlist’. What’s up? I am doing something wrong? Meanwhile, after browsing the RDoc of WWW::Mechanize::GlobalForm I have found that indeed, Form does not have a selectlist method. OK but then How I am supposed to get the selectlist of a form? the $1.000.000 question: In the same RDoc I read this: ‘Class Form does not work in the case there is some invalid (unbalanced) html involved…’ Well, on my page, the tag is not even closed. Can this be fixed somehow? Thanks, Peter \_\_ [http://www.rubyrailways.com](http://www.rubyrailways.com) ---
**Author:** ![splattael](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/858c86/32.png) [@splattael](https://www.ruby-forum.com/u/splattael)\ **Post date:** [November 21, 2006, 7:54pm UTC](https://www.ruby-forum.com/t/mechanize-question/76822/3 "2006-11-21T19:54:34Z")
Hey, I don’t know the answer to this question specifically, but I did some work with Mechanize recently and I found that it was pretty much doing everything we needed it to do, just sometimes returning things in forms we didn’t expect. Pretty much everything we stumbled on, we solved by getting the return value and doing .class on it to find out what it was coming back to us as. Going on the million-dollar question, which I actually only just noticed, I think the HTML we were working against with Mechanize – this was a consulting thing, so I don’t have the code in front of me, it’s on somebody else’s laptop – but I think the HTML was pretty bad. totally noncompliant, non-validating. we used Hpricot a lot, which is pretty great, we might have actually given up on Mechanize for the HTML-parsing and just used it for setting and getting cookies, things like that. I don’t quite recall, but definitely have a look at Hpricot, it’s pretty great and I think it was written by why the lucky stiff. ---
**Author:** ![splattael](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/858c86/32.png) [@splattael](https://www.ruby-forum.com/u/splattael)\ **Post date:** [November 21, 2006, 8:00pm UTC](https://www.ruby-forum.com/t/mechanize-question/76822/4 "2006-11-21T20:00:37Z")
On 11/21/06, Giles B. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > Going on the million-dollar question, which I actually only just > noticed, I think the HTML we were working against with Mechanize – > this was a consulting thing, so I don’t have the code in front of me, > it’s on somebody else’s laptop – but I think the HTML was pretty bad. > totally noncompliant, non-validating. we used Hpricot a lot, which is > pretty great, we might have actually given up on Mechanize for the > HTML-parsing and just used it for setting and getting cookies, things > like that. I don’t quite recall, but definitely have a look at > Hpricot, it’s pretty great and I think it was written by why the lucky > stiff. Mechanize now has direct support (and is implemented on top of) hpricot. [IIRC] Aaron is likely to have more info on this, of course. ---
**Author:** ![splattael](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/858c86/32.png) [@splattael](https://www.ruby-forum.com/u/splattael)\ **Post date:** [November 24, 2006, 12:00am UTC](https://www.ruby-forum.com/t/mechanize-question/76822/5 "2006-11-24T00:00:14Z")
On Fri, Nov 24, 2006 at 04:16:48AM +0900, Aaron P. wrote: > Mechanize uses HPricot for its HTML parsing. If Hpricot can handle the > form tag that isn’t closed, then you should be fine. If HPricot cannot > handle the unbalanced form tag, you can write a pluggable parser to fix > up your HTML before it is run through HPricot. If Hpricot cannot handle the tag, please open a ticket[1] or mail me, so I can fix it and add to my tests. This way we all get to benefit from these wild tags you’ve captured. \_why [1] [https://code.whytheluckystiff.net/hpricot/](https://code.whytheluckystiff.net/hpricot/) ---
**Author:** ![splattael](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/858c86/32.png) [@splattael](https://www.ruby-forum.com/u/splattael)\ **Post date:** [November 23, 2006, 8:17pm UTC](https://www.ruby-forum.com/t/mechanize-question/76822/6 "2006-11-23T20:17:48Z")
On Tue, Nov 21, 2006 at 10:46:26PM +0900, Peter S. wrote: > Meanwhile, after browsing the RDoc of WWW::Mechanize::GlobalForm I have > found that indeed, Form does not have a selectlist method. > > OK but then How I am supposed to get the selectlist of a form? The select list is treated like a regular field. Say you have a select list with name ‘foo’, you could find it like this: form.fields.name(‘foo’) -or, with method missing magic- form.foo > the $1.000.000 question: In the same RDoc I read this: > > ‘Class Form does not work in the case there is some invalid (unbalanced) > html involved…’ > > Well, on my page, the tag is not even closed. Can this be fixed > somehow? Mechanize uses HPricot for its HTML parsing. If Hpricot can handle the form tag that isn’t closed, then you should be fine. If HPricot cannot handle the unbalanced form tag, you can write a pluggable parser to fix up your HTML before it is run through HPricot. > Thanks, > Peter > > \_\_ > [http://www.rubyrailways.com](http://www.rubyrailways.com) Hope that helps. –Aaron ---
**Author:** ![splattael](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/858c86/32.png) [@splattael](https://www.ruby-forum.com/u/splattael)\ **Post date:** [November 24, 2006, 9:31am UTC](https://www.ruby-forum.com/t/mechanize-question/76822/7 "2006-11-24T09:31:05Z")
\_why wrote: > On Fri, Nov 24, 2006 at 04:16:48AM +0900, Aaron P. wrote: > > > Mechanize uses HPricot for its HTML parsing. If Hpricot can handle the > > form tag that isn’t closed, then you should be fine. If HPricot cannot > > handle the unbalanced form tag, you can write a pluggable parser to fix > > up your HTML before it is run through HPricot. > > If Hpricot cannot handle the tag, please open a ticket[1] or mail me, so I can > fix it and add to my tests. This way we all get to benefit from these wild tags > you’ve captured. Sure 🙂 I am just beginning with mechanize so I don’t have a real-life testcase yet, but since I am going to scrape tens or maybe hundreds of pages with HPricot + mechanize in the near future, I guess something will pop up sooner or later… Peter \_\_ [http://www.rubyrailways.com](http://www.rubyrailways.com) ---
**Author:** ![splattael](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/858c86/32.png) [@splattael](https://www.ruby-forum.com/u/splattael)\ **Post date:** [November 24, 2006, 8:50pm UTC](https://www.ruby-forum.com/t/mechanize-question/76822/8 "2006-11-24T20:50:40Z")
Peter S. wrote: > Meanwhile, after browsing the RDoc of WWW::Mechanize::GlobalForm I have > somehow? > You mean, like, closing the … tag pair? Well, yes, > closing > it is always an option. What is the issue with editing the page and > making > sure it is valid HTML?