Using Hpricot with HTML's

Ruby
1

I have been working on this program for class all day and am not getting
anywhere, so I figured I would try to ask you guys on the forum. Here is
my code so far:

#!/usr/bin/ruby

Lab 19

require ‘rubygems’
require ‘fileutils’
require ‘hpricot’
require ‘net/http’

tamper_args = {
‘my_countries’ => ‘,“Germany”, “Japan”, “United Kingdom”, “United
States”’,
‘my_vars’ => ‘,+pop’,
‘my_yrs’ => ‘,1950,1960,1970,1980,1990,2000’,
‘selected_vars’ => ‘POP’,
‘format_selected’ => ‘csv’
}

host_name =
“http://pwt.econ.upenn.edu/php_site/pwt62/pwt62_retrieve.php”

puts tamper_screen = Net::HTTP.post_form(URI.parse(host_name),
tamper_args)
puts page_parse = Hpricot(tamper_screen.to_s)

puts page_parse.search(“pre”)

It is designed to go through the host_name link and give the populations
for the 4 countries listed, with years 1950, 1960… through 2000, and
I have to get the data in csv format. I am not sure why I am not able to
do my search the pre tag, which is where the data is stored in the html
doc.

I think it has to do with the .to_s for the page_parse variable, but my
program gives me an error when tamper_screen is not a string.

Any help would be greatly appreciated.

2

Hi Rick,

On Sun, Nov 23, 2008 at 04:43:20AM +0900, Rick Rick wrote:

require ‘hpricot’

It is designed to go through the host_name link and give the populations
for the 4 countries listed, with years 1950, 1960… through 2000, and
I have to get the data in csv format. I am not sure why I am not able to
do my search the pre tag, which is where the data is stored in the html
doc.

I think it has to do with the .to_s for the page_parse variable, but my
program gives me an error when tamper_screen is not a string.

Try using the body method. post_form returns a response object, and you
need to grab the body of the response:

require ‘rubygems’
require ‘fileutils’
require ‘hpricot’
require ‘net/http’

tamper_args = {
‘my_countries’ => ‘,“Germany”, “Japan”, “United Kingdom”, “United
States”’,
‘my_vars’ => ‘,+pop’,
‘my_yrs’ => ‘,1950,1960,1970,1980,1990,2000’,
‘selected_vars’ => ‘POP’,
‘format_selected’ => ‘csv’
}

host_name =
“http://pwt.econ.upenn.edu/php_site/pwt62/pwt62_retrieve.php”

tamper_screen = Net::HTTP.post_form(URI.parse(host_name), tamper_args)

page_parse = Hpricot(tamper_screen.body)
puts page_parse.search(“pre”).inner_text

3

Thank you so much!!! I can finally get past that part.

4

Just to continue along with this same program, I am now trying to change
my parameters for the tamper_args to allow the user to key in a country
name, and the program automatically changes the country in
‘my_countries’ to whatever counties the user wants to examine… Is
there a way to allow for this?