# How do I debug WWW::Mechanize problems?

**URL:** <https://www.ruby-forum.com/t/how-do-i-debug-www-mechanize-problems/51685>\
**Category:** Ruby\
**Created:** [December 2, 2005, 7:55pm UTC](https://www.ruby-forum.com/t/how-do-i-debug-www-mechanize-problems/51685 "2005-12-02T19:55:41Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![dan](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/eada6e/32.png) [@dan](https://www.ruby-forum.com/u/dan)\
**Post date:** [December 2, 2005, 7:55pm UTC](https://www.ruby-forum.com/t/how-do-i-debug-www-mechanize-problems/51685/1 "2005-12-02T19:55:41Z")

</div>

Hi, I’m trying to scrape my mileage summary from United. Mechanize  
starts out correctly, but then gets redirected back to the United  
homepage instead of the Mileage Summary page.

Is there an rdoc for mechanize, as gems doesn’t install one? The only  
examples I could find (besides the one it ships with) were  
[http://www.ntecs.de/blog/Blog/WWW-Mechanize.rdoc](http://www.ntecs.de/blog/Blog/WWW-Mechanize.rdoc) and  
[http://www.zenspider.com/pipermail/ruby/2005-July/002068.html](http://www.zenspider.com/pipermail/ruby/2005-July/002068.html). So,  
I’m having trouble just figuring out the process to debug these errors.  
Is there anything equivalent to the mech-dump that Perl’s Mechanize  
ships with  
[http://search.cpan.org/dist/WWW-Mechanize/lib/WWW/Mechanize/FAQ.pod](http://search.cpan.org/dist/WWW-Mechanize/lib/WWW/Mechanize/FAQ.pod)?

Anyway, here’s the code and the debugging output (run with my username  
and password). (If you have a United frequent flyer account, you can  
replace username and password with your own to recreate.)

require ‘mechanize’  
agent = WWW::Mechanize.new{|a| a.log = Logger.new(STDERR) }

page = agent.get(‘[http://www.united.com](http://www.united.com)’)  
agent.user\_agent\_alias = ‘Windows IE 6’

form = page.forms.find {|f| f.name == ‘MileagePlus’}  
form.fields.find {|f| f.name == ‘userId’}.value = username  
form.fields.find {|f| f.name == ‘password’}.value = password  
puts page.forms  
page = agent.submit(form, form.buttons.first)

output = File.open(“output.html”, “w”)  
output.puts page.body  
puts page.body

> ruby united.rb  
> I, [2005-12-02T10:02:10.791000 #4816] INFO – : GET:  
> [http://www.united.com](http://www.united.com)  
> D, [2005-12-02T10:02:10.961000 #4816] DEBUG – : request-header: accept  
> =\> _/_  
> D, [2005-12-02T10:02:11.021000 #4816] DEBUG – : header: connection :  
> close  
> D, [2005-12-02T10:02:11.021000 #4816] DEBUG – : header: content-type :  
> text/html  
> D, [2005-12-02T10:02:11.021000 #4816] DEBUG – : header: date : Fri, 02  
> Dec 2005 18:02:12 GMT  
> D, [2005-12-02T10:02:11.021000 #4816] DEBUG – : header: server :  
> Netscape-Enterprise/6.0  
> I, [2005-12-02T10:02:11.512000 #4816] INFO – : status: 200  
> I, [2005-12-02T10:02:18.091000 #4816] INFO – : POST:  
> [https://www.ua2go.com/ci/DoLogin.jsp?stamp=NEWCOOKY\*itn/ord=NEWREC,itn/air/united](https://www.ua2go.com/ci/DoLogin.jsp?stamp=NEWCOOKY*itn/ord=NEWREC,itn/air/united)  
> warning: peer certificate won’t be verified in this SSL session  
> D, [2005-12-02T10:02:19.313000 #4816] DEBUG – : query:  
> {“submit2”=\>“Login”, “sel\_return\_to”=\>“&return\_to=ff\_acct\_hist”,  
> “userId”=\>“username”, “password”=\>“password”}  
> D, [2005-12-02T10:02:19.313000 #4816] DEBUG – : request-header: accept  
> =\> _/_  
> D, [2005-12-02T10:02:19.313000 #4816] DEBUG – : request-header:  
> content-type =\> application/x-www-form-urlencoded  
> D, [2005-12-02T10:02:19.313000 #4816] DEBUG – : request-header:  
> user-agent =\> Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)  
> D, [2005-12-02T10:02:19.313000 #4816] DEBUG – : request-header:  
> referer =\> [http://www.united.com](http://www.united.com)  
> D, [2005-12-02T10:02:19.313000 #4816] DEBUG – : request-header:  
> content-length =\> 92  
> D, [2005-12-02T10:02:19.553000 #4816] DEBUG – : cookie received:  
> JSESSIONID=DQMtsIaIkMsQbuTwzMK9Nciy064tQh1KBNY1EB1Q3M3ibLa5g7J9!151650267;  
> path=/  
> D, [2005-12-02T10:02:19.553000 #4816] DEBUG – : cookie received:  
> testcookie=1  
> D, [2005-12-02T10:02:19.553000 #4816] DEBUG – : header: connection :  
> close  
> D, [2005-12-02T10:02:19.553000 #4816] DEBUG – : header: content-type :  
> text/html  
> D, [2005-12-02T10:02:19.553000 #4816] DEBUG – : header: date : Fri, 02  
> Dec 2005 18:02:21 GMT  
> D, [2005-12-02T10:02:19.553000 #4816] DEBUG – : header: server :  
> Netscape-Enterprise/6.0  
> D, [2005-12-02T10:02:19.553000 #4816] DEBUG – : header: set-cookie :  
> JSESSIONID=DQMtsIaIkMsQbuTwzMK9Nciy064tQh1KBNY1EB1Q3M3ibLa5g7J9!151650267;  
> path=/, testcookie=1  
> D, [2005-12-02T10:02:19.553000 #4816] DEBUG – : header: location :  
> [https://www.ua2go.com/ci/DoLogin.jsp?setCookie=1&stamp=NEWCOOKY\*itn/ord=NEWREC,itn/air/united](https://www.ua2go.com/ci/DoLogin.jsp?setCookie=1&stamp=NEWCOOKY*itn/ord=NEWREC,itn/air/united)  
> I, [2005-12-02T10:02:19.553000 #4816] INFO – : status: 302  
> I, [2005-12-02T10:02:19.553000 #4816] INFO – : follow redirect to:  
> [https://www.ua2go.com/ci/DoLogin.jsp?setCookie=1&stamp=NEWCOOKY\*itn/ord=NEWREC,itn/air/united](https://www.ua2go.com/ci/DoLogin.jsp?setCookie=1&stamp=NEWCOOKY*itn/ord=NEWREC,itn/air/united)  
> I, [2005-12-02T10:02:19.553000 #4816] INFO – : GET:  
> [https://www.ua2go.com/ci/DoLogin.jsp?setCookie=1&stamp=NEWCOOKY\*itn/ord=NEWREC,itn/air/united](https://www.ua2go.com/ci/DoLogin.jsp?setCookie=1&stamp=NEWCOOKY*itn/ord=NEWREC,itn/air/united)  
> warning: peer certificate won’t be verified in this SSL session  
> D, [2005-12-02T10:02:19.824000 #4816] DEBUG – : use cookie:  
> JSESSIONID=DQMtsIaIkMsQbuTwzMK9Nciy064tQh1KBNY1EB1Q3M3ibLa5g7J9!151650267;  
> testcookie=1  
> D, [2005-12-02T10:02:19.824000 #4816] DEBUG – : request-header: accept  
> =\> _/_  
> D, [2005-12-02T10:02:19.824000 #4816] DEBUG – : request-header:  
> user-agent =\> Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)  
> D, [2005-12-02T10:02:19.824000 #4816] DEBUG – : request-header: cookie  
> =\>  
> JSESSIONID=DQMtsIaIkMsQbuTwzMK9Nciy064tQh1KBNY1EB1Q3M3ibLa5g7J9!151650267;  
> testcookie=1  
> D, [2005-12-02T10:02:19.824000 #4816] DEBUG – : request-header:  
> referer =\>  
> [https://www.ua2go.com/ci/DoLogin.jsp?stamp=NEWCOOKY\*itn/ord=NEWREC,itn/air/united](https://www.ua2go.com/ci/DoLogin.jsp?stamp=NEWCOOKY*itn/ord=NEWREC,itn/air/united)  
> D, [2005-12-02T10:02:20.174000 #4816] DEBUG – : cookie received:  
> rememberMP=-1; [domain=.ua2go.com](http://domain=.ua2go.com); expires=Friday, 02-Dec-2005 18:02:23  
> GMT; path=/  
> D, [2005-12-02T10:02:20.174000 #4816] DEBUG – : cookie received:  
> ciToken=792ad6bc2fb42b7d178c34e2f98fab5c6931842a1f415c7d123f896c82db46a0;  
> [domain=.ua2go.com](http://domain=.ua2go.com); path=/  
> D, [2005-12-02T10:02:20.174000 #4816] DEBUG – : cookie received:  
> ciToken=792ad6bc2fb42b7d178c34e2f98fab5cf8cf3f5e1ce627d58e2ed9576b828ef1;  
> [domain=.ua2go.com](http://domain=.ua2go.com); path=/  
> D, [2005-12-02T10:02:20.174000 #4816] DEBUG – : header: connection :  
> close  
> D, [2005-12-02T10:02:20.174000 #4816] DEBUG – : header: content-type :  
> text/html  
> D, [2005-12-02T10:02:20.174000 #4816] DEBUG – : header: date : Fri, 02  
> Dec 2005 18:02:21 GMT  
> D, [2005-12-02T10:02:20.224000 #4816] DEBUG – : header: server :  
> Netscape-Enterprise/6.0  
> D, [2005-12-02T10:02:20.224000 #4816] DEBUG – : header: set-cookie :  
> rememberMP=-1; [domain=.ua2go.com](http://domain=.ua2go.com); expires=Friday, 02-Dec-2005 18:02:23  
> GMT; path=/,  
> ciToken=792ad6bc2fb42b7d178c34e2f98fab5c6931842a1f415c7d123f896c82db46a0;  
> [domain=.ua2go.com](http://domain=.ua2go.com); path=/,  
> ciToken=792ad6bc2fb42b7d178c34e2f98fab5cf8cf3f5e1ce627d58e2ed9576b828ef1;  
> [domain=.ua2go.com](http://domain=.ua2go.com); path=/  
> D, [2005-12-02T10:02:20.224000 #4816] DEBUG – : header: location :  
> [http://www.united.com/index/Home?stamp=NEWCOOKY\*itn/ord=NEWREC,itn/air/united](http://www.united.com/index/Home?stamp=NEWCOOKY*itn/ord=NEWREC,itn/air/united)  
> I, [2005-12-02T10:02:20.234000 #4816] INFO – : status: 302  
> I, [2005-12-02T10:02:20.234000 #4816] INFO – : follow redirect to:  
> [http://www.united.com/index/Home?stamp=NEWCOOKY\*itn/ord=NEWREC,itn/air/united](http://www.united.com/index/Home?stamp=NEWCOOKY*itn/ord=NEWREC,itn/air/united)  
> I, [2005-12-02T10:02:20.234000 #4816] INFO – : GET:  
> [http://www.united.com/index/Home?stamp=NEWCOOKY\*itn/ord=NEWREC,itn/air/united](http://www.united.com/index/Home?stamp=NEWCOOKY*itn/ord=NEWREC,itn/air/united)  
> D, [2005-12-02T10:02:20.325000 #4816] DEBUG – : use cookie:  
> JSESSIONID=DQMtsIaIkMsQbuTwzMK9Nciy064tQh1KBNY1EB1Q3M3ibLa5g7J9!151650267;  
> testcookie=1; rememberMP=-1;  
> ciToken=792ad6bc2fb42b7d178c34e2f98fab5c6931842a1f415c7d123f896c82db46a0;  
> ciToken=792ad6bc2fb42b7d178c34e2f98fab5cf8cf3f5e1ce627d58e2ed9576b828ef1  
> D, [2005-12-02T10:02:20.325000 #4816] DEBUG – : request-header: accept  
> =\> _/_  
> D, [2005-12-02T10:02:20.325000 #4816] DEBUG – : request-header:  
> user-agent =\> Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)  
> D, [2005-12-02T10:02:20.325000 #4816] DEBUG – : request-header: cookie  
> =\>  
> JSESSIONID=DQMtsIaIkMsQbuTwzMK9Nciy064tQh1KBNY1EB1Q3M3ibLa5g7J9!151650267;  
> testcookie=1; rememberMP=-1;  
> ciToken=792ad6bc2fb42b7d178c34e2f98fab5c6931842a1f415c7d123f896c82db46a0;  
> ciToken=792ad6bc2fb42b7d178c34e2f98fab5cf8cf3f5e1ce627d58e2ed9576b828ef1  
> D, [2005-12-02T10:02:20.325000 #4816] DEBUG – : request-header:  
> referer =\>  
> [https://www.ua2go.com/ci/DoLogin.jsp?setCookie=1&stamp=NEWCOOKY\*itn/ord=NEWREC,itn/air/united](https://www.ua2go.com/ci/DoLogin.jsp?setCookie=1&stamp=NEWCOOKY*itn/ord=NEWREC,itn/air/united)  
> D, [2005-12-02T10:02:20.435000 #4816] DEBUG – : cookie received:  
> united=DQMujVlTtYtjKEjdBoIwk2nLn1vdm82kMn1pSndGtFNa7nIBOEFy!-232476364;  
> path=/  
> D, [2005-12-02T10:02:20.435000 #4816] DEBUG – : cookie received:  
> sessionid=NEWCOOKY_itn/ord=NEWREC; [domain=.united.com](http://domain=.united.com); path=/  
> D, [2005-12-02T10:02:20.435000 #4816] DEBUG – : header: connection :  
> close  
> D, [2005-12-02T10:02:20.435000 #4816] DEBUG – : header: content-type :  
> text/html  
> D, [2005-12-02T10:02:20.435000 #4816] DEBUG – : header: date : Fri, 02  
> Dec 2005 18:02:21 GMT  
> D, [2005-12-02T10:02:20.435000 #4816] DEBUG – : header: server :  
> Netscape-Enterprise/6.0  
> D, [2005-12-02T10:02:20.435000 #4816] DEBUG – : header: set-cookie :  
> united=DQMujVlTtYtjKEjdBoIwk2nLn1vdm82kMn1pSndGtFNa7nIBOEFy!-232476364;  
> path=/, sessionid=NEWCOOKY_itn/ord=NEWREC; [domain=.united.com](http://domain=.united.com); path=/  
> D, [2005-12-02T10:02:20.435000 #4816] DEBUG – : header: content-length  
> : 0  
> D, [2005-12-02T10:02:20.435000 #4816] DEBUG – : header: location :  
> [http://www.united.com/](http://www.united.com/)  
> I, [2005-12-02T10:02:20.435000 #4816] INFO – : status: 302  
> I, [2005-12-02T10:02:20.435000 #4816] INFO – : follow redirect to:  
> [http://www.united.com/](http://www.united.com/)  
> I, [2005-12-02T10:02:20.435000 #4816] INFO – : GET:  
> [http://www.united.com/](http://www.united.com/)  
> D, [2005-12-02T10:02:20.665000 #4816] DEBUG – : use cookie:  
> JSESSIONID=DQMtsIaIkMsQbuTwzMK9Nciy064tQh1KBNY1EB1Q3M3ibLa5g7J9!151650267;  
> testcookie=1; rememberMP=-1;  
> ciToken=792ad6bc2fb42b7d178c34e2f98fab5c6931842a1f415c7d123f896c82db46a0;  
> ciToken=792ad6bc2fb42b7d178c34e2f98fab5cf8cf3f5e1ce627d58e2ed9576b828ef1;  
> united=DQMujVlTtYtjKEjdBoIwk2nLn1vdm82kMn1pSndGtFNa7nIBOEFy!-232476364;  
> sessionid=NEWCOOKY\*itn/ord=NEWREC  
> D, [2005-12-02T10:02:20.665000 #4816] DEBUG – : request-header: accept  
> =\> _/_  
> D, [2005-12-02T10:02:20.665000 #4816] DEBUG – : request-header:  
> user-agent =\> Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)  
> D, [2005-12-02T10:02:20.665000 #4816] DEBUG – : request-header: cookie  
> =\>  
> JSESSIONID=DQMtsIaIkMsQbuTwzMK9Nciy064tQh1KBNY1EB1Q3M3ibLa5g7J9!151650267;  
> testcookie=1; rememberMP=-1;  
> ciToken=792ad6bc2fb42b7d178c34e#WWW::Form:0x34644c0  
> #WWW::Form:0x35fcd78  
> #WWW::Form:0x35d1530  
> #WWW::Form:0x35a71e8  
> #WWW::Form:0x35a3c78  
> #WWW::Form:0x35e8300

```
      - dan

```