How to imitate a browser?

Hey guys,

so I’m trying to write a simple script for extraction of my profile data
from one site. In order to get the data I have to log in and then
download a page.

At first I tried to use Ruby’s Net::HTTP library, but the problem is
that the site runs PHP and logs users in via storing cookies (therefore
the low-level network approach doesn’t work).

Any ideas?

On Sun, Oct 3, 2010 at 4:28 PM, Marek O. [email protected]
wrote:

Any ideas?
Use Mechanize:

http://mechanize.rubyforge.org/mechanize/

It manages cookies and allows you to fill forms, click links and so on.

Jesus.

Hi Marek,

Have you tried the combo Nokogiri (http://nokogiri.org/) / Mechanize
(http://mechanize.rubyforge.org/mechanize/) to do the job?

I recommend you to take a look at:


From: Marek O. [email protected]
To: ruby-talk ML [email protected]
Sent: Sun, October 3, 2010 11:28:38 AM
Subject: How to imitate a browser?

Hey guys,

so I’m trying to write a simple script for extraction of my profile data
from one site. In order to get the data I have to log in and then
download a page.

At first I tried to use Ruby’s Net::HTTP library, but the problem is
that the site runs PHP and logs users in via storing cookies (therefore
the low-level network approach doesn’t work).

Any ideas?

so I’m trying to write a simple script for extraction of my profile data
from one site.

http://ruby-toolbox.com/categories/http_clients.html
http://ruby-toolbox.com/categories/browser_testing.html (watir,
especially)

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs