Mechanize problem

Hi!
I rarely code but when I try I use Ruby. So this is a beginners problem
:slight_smile:

Im trying to fetch a number from a website and im using mechanize but it
seems to be impossible. Im doing this:

require ‘Mechanize’
agent = Mechanize.new
agent.get(“http://www.linasmatkasse.se”)
agent.page.search(’#counter’)

This div should contain a value but it doesnt. I get this in irb:

[#<Nokogiri::XML::Element:0x83fe5ff0 name=“div”
attributes=[#<Nokogiri::XML::Attr:0x83fde2b4 name=“id”
value=“counter”>]>]

This div contains a number like 26 100 002 when looking at the website.

What is it that im doing wrong?

Any suggestion is appreciated! :slight_smile:

Br
cristian

On Tue, Dec 11, 2012 at 5:00 PM, cristian cristian
[email protected] wrote:

agent.page.search(’#counter’)

Any suggestion is appreciated! :slight_smile:

Br
cristian


Posted via http://www.ruby-forum.com/.

When you just load the html, that div is empty:

One of the scripts on that page is filling it in.

Thank you both!

Now I understand why my code didnt work and I solved my problem!

Br
cristian

The problem is that when you view the page in your browser, your browser
executes some javascript code that sets the value of the counter

.
However, when you use mechanize to access the page, mechanize retrieves
a plain text file containing the lines of javascript code as well as the
html–but mechanize is not a browser, so it cannot execute the
javascript code (nor does mechanize display the colors specified by the
html). All mechanize does is examine the text file for information.
You
can see the same text file that mechanize sees by loading the web page
in your browser and clicking on View/Source. Then use the search
function
to search for ‘counter’. When you find the counter
, you’ll see
that
it contains no text.

Luckily for you, there are now advanced programs that can retrieve the
text file containing the javascript and html, load the text file into a
browser(which executes the javascript), then the program can examine
what is actually displayed in the browser:

require ‘watir-webdriver’

b = Watir::Browser.new
b.goto(“http://www.linasmatkasse.se”)

counter_div = b.div id: ‘counter’
puts counter_div.text

–output:–
26 308 814

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs