Scraping amazon pages with nokogiri

ruby latest
nokogiri 1.6.3.1

I am trying to get title and price from Amazon product pages like,

require ‘open-uri’
require ‘nokogiri’

url =
http://www.amazon.co.jp/Advanced-EXCEL-David-Bolocan/dp/0830604685/ref=sr_1_1?ie=UTF8&qid=undefined&sr=8-1&keywords=advanced+excel

charset = nil
html = open(url) do |f|
charset = f.charset
f.read
end

doc = Nokogiri::HTML.parse(html, nil, charset)

p doc.title

the result is

…1.0/open-uri.rb:353:in `open_http’: 503 Service Unavailable
(OpenURI::HTTPError)

Does that mean amazon is refusing the scraping act?

Is there any way to do it?

Soichi I. wrote in post #1156501:

Does that mean amazon is refusing the scraping act?

When I run your code, I get:

“Amazon.co.jp: Advanced EXCEL: David Bolocan: 洋書”

0.5.4 503 Service Unavailable

The server is currently unable to handle the request due to a temporary
overloading or maintenance of the server. The implication is that this
is a temporary condition which will be alleviated after some delay. If
known, the length of the delay MAY be indicated in a Retry-After header.
If no Retry-After is given, the client SHOULD handle the response as it
would for a 500 response