\
**Category:** Ruby\
**Created:** [May 17, 2010, 7:16pm UTC](https://www.ruby-forum.com/t/scraping-html-behind-a-log-in/188748 "2010-05-17T19:16:02Z")\
**Posts on this page:** 2\
**Page:** 1
**Author:**  [@csitol](https://www.ruby-forum.com/u/csitol)\
**Post date:** [May 17, 2010, 7:16pm UTC](https://www.ruby-forum.com/t/scraping-html-behind-a-log-in/188748/1 "2010-05-17T19:16:02Z")
I’ve used libraries, such as hpricot, to scrape html from public sites.
Does anyone know how to get at data behind a user log-in? I’m trying to
automate a service that periodically grabs data for a user. You could
think of this as analogous to what [mint.com](http://mint.com) does with aggregating user
data in the financial sector.
---
**Author:**  [@csitol](https://www.ruby-forum.com/u/csitol)\
**Post date:** [May 17, 2010, 7:27pm UTC](https://www.ruby-forum.com/t/scraping-html-behind-a-log-in/188748/2 "2010-05-17T19:27:39Z")
On Mon, May 17, 2010 at 7:16 PM, Phil Mcdonnell
[removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:
> I’ve used libraries, such as hpricot, to scrape html from public sites.
> Does anyone know how to get at data behind a user log-in? I’m trying to
> automate a service that periodically grabs data for a user. You could
> think of this as analogous to what [mint.com](http://mint.com) does with aggregating user
> data in the financial sector.
You can try Mechanize. I’ve used it to login into a Wiki and edit pages.
I think it was also integrated with Hpricot, or maybe they changed to
Nokogiri. In any case you have an html parser at your disposal.
[http://mechanize.rubyforge.org/mechanize/](http://mechanize.rubyforge.org/mechanize/)
Checking this, I see it’s nokogiri.
Jesus.