I’m trying to parse a page with Hpricot in order to retrieve a value.
I use Xpather (a firefox extension) in order to get the path of this
value. But when I use this path with Hpricot, it doesn’t work. I have
to change it so that it works.
Well, it depends. It’d be helpful to see the page you’re working with.
You might want to try asking the Hpricot mailing list as well (To
join: Send a message to [email protected] Cc: [email protected]).
HPricot doesn’t include the whole XPath syntax. You can write a little
function which translates XPath expressions with brackets to HPricot
expressions. I wrote a function for that, but my SVN is right now down
and I can’t get it. Drop an answer if you still need it
Firefox includes some missing HTML tags. I ran in it, when I had to
write a little script. is added in
, probably some more
things, but I didn’t find them. You can see the difference when you
download the page with open-uri. Not many pages add .
Well, it depends. It’d be helpful to see the page you’re working with.
You might want to try asking the Hpricot mailing list as well (To
join: Send a message to [email protected] Cc: [email protected]).
Chris
Hi Chris, thanks for your answer
Here is the page I’m working with : http://finance.yahoo.com
I want to retrieve value of Nasdaq (up left of the page).
I sent the same message on Hpricot ML this afternoon, actually no
answer.
Here’s my path, given by Xpather :
things, but I didn’t find them. You can see the difference when you
download the page with open-uri. Not many pages add .
Hi Thomas
I’m very interested in your function. Do you know where I can find
differences between XPath syntax and Hpricot syntax ? What reference
did you use to write your function ?
On Dec 18, 11:21 pm, Chris S. [email protected] wrote:
Looking at the page source, the span element that contains the value
actually has an id (yfs_l10_^ixic), but it doesn’t look stable, does
it?
I take that back. That id is almost definitely stable.
I’ll post it as soon as the SVN server is up again.
Do you know where I can find
differences between XPath syntax and Hpricot syntax ? What reference
did you use to write your function ?
I used http://code.whytheluckystiff.net/hpricot/wiki/SupportedXpathExpressions
and related pages to get started with it. I found the table/tbody
thing because I didn’t get any further with it and thought, that I did
something wrong until I downloaded the page without Firefox using
open-uri.
Hi Chris, thanks for your answer
Here is the page I’m working with : http://finance.yahoo.com
I want to retrieve value of Nasdaq (up left of the page).
I see. It’s pretty easy to get using element attributes, which
resilient to page changes. If Yahoo decides to add a div in the
hierarchy, or add a new exchange, this shouldn’t suddenly fail:
On Dec 18, 11:21 pm, Chris S. [email protected] wrote:
Looking at the page source, the span element that contains the value
actually has an id (yfs_l10_^ixic), but it doesn’t look stable, does
it?
I take that back. That id is almost definitely stable.
doc.at(‘span[@id=“yfs_l10_^ixic”]’)
Chris
Yes, thanks, it runs.
There’s something I can’t understand : in the Xpath expression I
posted later, when a node has several child DIVs, I access them with
an index (div[2]…), but in Hpricot syntax, DIVs aren’t accessed
using an index. So, what trick Hpricot uses to locate “the good” div ?
Hi Chris, thanks for your answer
Here is the page I’m working with : http://finance.yahoo.com
I want to retrieve value of Nasdaq (up left of the page).
I sent the same message on Hpricot ML this afternoon, actually no
answer.
Hi, Celine.
I know it’s not nearly as fun as screen-scraping, but you can get the
value
for Nasdaq (and many other quotes) on Yahoo! Finance by querying the
right
URL for the CSV. The current value can be obtained by fetching:
Just replace [name] with the index or stock you wish to query and each
bracketed date info with integers.
I’ve replied to a topic before that involved Yahoo! Finance, but it was
specifically about searching for a symbol. Since it’s not your case,
here’s
hoping that directly fetching it will suffice.
Yes, thanks, it runs.
There’s something I can’t understand : in the Xpath expression I
posted later, when a node has several child DIVs, I access them with
an index (div[2]…), but in Hpricot syntax, DIVs aren’t accessed
using an index. So, what trick Hpricot uses to locate “the good” div ?
I’m not sure I understand. Hpricot certainly can access elements that
way:
At some point the path you’re using fails. That’s why. You could check
node by node, going one level lower each time to see where you start
getting nil from your search. And then you could see what you need to
do to fix the path. That’s what I just did:
Tools like Xpather and Firebug can give you paths, but they’re not
going to work all the time. But, as I said before, there’s a span with
an id attribute that lets you pluck the data without worrying about a
full path, so this is sort of moot.
HTH,
Chris
This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.
