# Data extraction using Scrubyt

**URL:** <https://www.ruby-forum.com/t/data-extraction-using-scrubyt/154928>\
**Category:** Ruby\
**Created:** [December 5, 2008, 8:08am UTC](https://www.ruby-forum.com/t/data-extraction-using-scrubyt/154928 "2008-12-05T08:08:37Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![vmvipin](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/v/c0e974/32.png) [@vmvipin](https://www.ruby-forum.com/u/vmvipin)\
**Post date:** [December 5, 2008, 8:08am UTC](https://www.ruby-forum.com/t/data-extraction-using-scrubyt/154928/1 "2008-12-05T08:08:37Z")

</div>

Hi All,

I need to fetch some information from [http://www.ebay.in](http://www.ebay.in).  
My required fields are : Name of the product, Image, Price and the link  
to that product.

am able to get the data using this method.  
require ‘rubygems’  
require ‘scrubyt’

google\_data = Scrubyt::Extractor.define do  
fetch ‘[http://www.ebay.in](http://www.ebay.in)’  
fill\_textfield ‘satitle’, ‘ipod shuffle’  
submit  
record  
“/html/body/div[2]/div[4]/div[2]/div/div/div[2]/div[2]/div/div/div[3]/div/div/table/tr”  
do  
name “/td[2]/div/a”  
price “/td[5]”  
image “/td/a/img” do  
url “src”, :type =\> :attribute  
end  
link “/td[2]/div/a” do  
url “href”, :type =\> :attribute  
end  
end  
end

google\_data.to\_xml.write($stdout, 1)

but my problem is for some products its not working properly. (div may  
be changed). is there any better solution for this?

Thanks in advance,  
Vipin

---

<div class="post-metadata">

**Author:** ![vmvipin](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/v/c0e974/32.png) [@vmvipin](https://www.ruby-forum.com/u/vmvipin)\
**Post date:** [December 5, 2008, 10:13am UTC](https://www.ruby-forum.com/t/data-extraction-using-scrubyt/154928/2 "2008-12-05T10:13:15Z")

</div>

You need to create smarter XPaths, relying on CSS id/class attributes  
or other properties rather than a full XPath from the root - for  
example:

require ‘rubygems’  
require ‘scrubyt’

ebay\_data = Scrubyt::Extractor.define do

```
  fetch 'http://www.ebay.in/'
  fill_textfield 'satitle', 'ipod'
  submit

  record "//table[@class='nol']" do
    name "//td[@class='details']/div/a"
  end

```

end

puts ebay\_data.to\_xml

etc.

This way your scraper will be more robust and prone to page changes.

HTH,  
Peter

* * *

[http://www.rubyrailways.com](http://www.rubyrailways.com)  
[http://scrubyt.org](http://scrubyt.org)

---

<div class="post-metadata">

**Author:** ![vmvipin](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/v/c0e974/32.png) [@vmvipin](https://www.ruby-forum.com/u/vmvipin)\
**Post date:** [December 6, 2008, 8:31am UTC](https://www.ruby-forum.com/t/data-extraction-using-scrubyt/154928/3 "2008-12-06T08:31:20Z")

</div>

On 2008.12.06., at 4:46, Vipin Vm wrote:

> Hi Peter,
> 
> Thanks for the Help… its working fine 🙂

Glad that I could help. I am just working on a new release btw, so  
stay tuned!

Cheers,  
Peter

* * *

[http://www.rubyrailways.com](http://www.rubyrailways.com)  
[http://scrubyt.org](http://scrubyt.org)

---

<div class="post-metadata">

**Author:** ![vmvipin](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/v/c0e974/32.png) [@vmvipin](https://www.ruby-forum.com/u/vmvipin)\
**Post date:** [December 6, 2008, 4:52am UTC](https://www.ruby-forum.com/t/data-extraction-using-scrubyt/154928/4 "2008-12-06T04:52:53Z")

</div>

Hi Peter,

Thanks for the Help… its working fine 🙂

Vipin

Peter S. wrote:

> You need to create smarter XPaths, relying on CSS id/class attributes  
> or other properties rather than a full XPath from the root - for  
> example:
> 
> require ‘rubygems’  
> require ‘scrubyt’
> 
> ebay\_data = Scrubyt::Extractor.define do
> 
> ```
> fetch 'http://www.ebay.in/'
> fill_textfield 'satitle', 'ipod'
> submit
> 
> record "//table[@class='nol']" do
> name "//td[@class='details']/div/a"
> end
> 
> ```
> 
> end
> 
> puts ebay\_data.to\_xml
> 
> etc.
> 
> This way your scraper will be more robust and prone to page changes.
> 
> HTH,  
> Peter
> 
> * * *
> 
> [http://www.rubyrailways.com](http://www.rubyrailways.com)  
> [http://scrubyt.org](http://scrubyt.org)