# Performance comparison between screen scrapers

**URL:** <https://www.ruby-forum.com/t/performance-comparison-between-screen-scrapers/81684>\
**Category:** Ruby\
**Created:** [January 11, 2007, 9:36am UTC](https://www.ruby-forum.com/t/performance-comparison-between-screen-scrapers/81684 "2007-01-11T09:36:42Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![chuboy](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/439d5e/32.png) [@chuboy](https://www.ruby-forum.com/u/chuboy)\
**Post date:** [January 11, 2007, 9:36am UTC](https://www.ruby-forum.com/t/performance-comparison-between-screen-scrapers/81684/1 "2007-01-11T09:36:42Z")

</div>

Does anyone know how the following screen scrapers perform against one  
another?

- ScrAPI
- RubyfulSoup
- HTree
- Hpricot

I’m trying to write up a tool where a person enters in a URL, and I use  
an AJAX call to scrape the contents of that URL for title, description,  
etc. So speed is really important (I suppose, regular expressions would  
be the fastest, but I need something that is tree-based and supports  
HTML tidying)

Thanks  
Conrad

---

<div class="post-metadata">

**Author:** ![chuboy](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/439d5e/32.png) [@chuboy](https://www.ruby-forum.com/u/chuboy)\
**Post date:** [January 11, 2007, 10:52am UTC](https://www.ruby-forum.com/t/performance-comparison-between-screen-scrapers/81684/2 "2007-01-11T10:52:11Z")

</div>

On 1/11/07, Conrad C. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> etc. So speed is really important (I suppose, regular expressions would  
> be the fastest, but I need something that is tree-based and supports  
> HTML tidying)
> 
> Thanks  
> Conrad

There was a comparision done on this list some time ago. Search for lib  
names.

---

<div class="post-metadata">

**Author:** ![chuboy](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/439d5e/32.png) [@chuboy](https://www.ruby-forum.com/u/chuboy)\
**Post date:** [January 11, 2007, 11:26am UTC](https://www.ruby-forum.com/t/performance-comparison-between-screen-scrapers/81684/3 "2007-01-11T11:26:23Z")

</div>

On Thu, 11 Jan 2007 08:36:44 -0000, Conrad C. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
wrote:

> etc. So speed is really important (I suppose, regular expressions would  
> be the fastest, but I need something that is tree-based and supports  
> HTML tidying)
> 
> Thanks  
> Conrad

I don’t know about ScrAPI or HTree, but I recently blogged an informal  
benchmark run between Rubyful Soup, Hpricot, and the (still  
developmental)  
libxml2 HTML parser binding in Libxml-ruby. It’s at:

[http://cloverhead.blogspot.com/2006/12/bit-of-benchmarking.html](http://cloverhead.blogspot.com/2006/12/bit-of-benchmarking.html)

---

<div class="post-metadata">

**Author:** ![chuboy](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/439d5e/32.png) [@chuboy](https://www.ruby-forum.com/u/chuboy)\
**Post date:** [January 19, 2007, 4:31pm UTC](https://www.ruby-forum.com/t/performance-comparison-between-screen-scrapers/81684/4 "2007-01-19T16:31:48Z")

</div>

Conrad C. wrote:

> etc. So speed is really important (I suppose, regular expressions would  
> be the fastest, but I need something that is tree-based and supports  
> HTML tidying)
> 
> Thanks  
> Conrad
> 
> –  
> Posted via [http://www.ruby-forum.com/](http://www.ruby-forum.com/).

I haven’t used them all but Hpricot is fast (the parser is written in C  
with Ragel), error tolerant and perfect for this task. Take a look at  
its website for a guide on how to use it.