# Using Scrubyt on bad markup pages

**URL:** <https://www.ruby-forum.com/t/using-scrubyt-on-bad-markup-pages/166650>\
**Category:** Ruby\
**Created:** [April 28, 2009, 8:39am UTC](https://www.ruby-forum.com/t/using-scrubyt-on-bad-markup-pages/166650 "2009-04-28T08:39:05Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![rolinster](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/49beb7/32.png) [@rolinster](https://www.ruby-forum.com/u/rolinster)\
**Post date:** [April 28, 2009, 8:39am UTC](https://www.ruby-forum.com/t/using-scrubyt-on-bad-markup-pages/166650/1 "2009-04-28T08:39:05Z")

</div>

I am having trouble scrubbing a page that has bad markup. After  
fetching the page, the Scrubyt::Extractor exits while parsing the  
document. The Apple Safari web inspector shows numerous errors from the  
page:

 is not allowed inside . Moving into the . Unmatched encountered. Ignoring tag. Unmatched encountered. Ignoring tag. Unmatched encountered. Ignoring tag.

Is there anyway to scrub a page with scrubyt that is poorly formated? I  
am using the latest version (0.4.1) of scrubyt.

Thanks,  
Rolin

---

<div class="post-metadata">

**Author:** ![rolinster](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/49beb7/32.png) [@rolinster](https://www.ruby-forum.com/u/rolinster)\
**Post date:** [April 28, 2009, 10:16am UTC](https://www.ruby-forum.com/t/using-scrubyt-on-bad-markup-pages/166650/2 "2009-04-28T10:16:57Z")

</div>

On Apr 27, 2009, at 23:39 , Rolin Nelson wrote:

> Is there anyway to scrub a page with scrubyt that is poorly  
> formated? I  
> am using the latest version (0.4.1) of scrubyt.

switch to mechanize and update your gems. scrubyt depends on hpricot  
and a very old version of mechanize. Mechanize now uses nokogiri  
instead of hpricot and is much more resilient with errors.

---

<div class="post-metadata">

**Author:** ![rolinster](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/49beb7/32.png) [@rolinster](https://www.ruby-forum.com/u/rolinster)\
**Post date:** [April 28, 2009, 3:48pm UTC](https://www.ruby-forum.com/t/using-scrubyt-on-bad-markup-pages/166650/3 "2009-04-28T15:48:33Z")

</div>

Ryan D. wrote:

> On Apr 27, 2009, at 23:39 , Rolin Nelson wrote:
> 
> > Is there anyway to scrub a page with scrubyt that is poorly  
> > formated? I  
> > am using the latest version (0.4.1) of scrubyt.
> 
> switch to mechanize and update your gems. scrubyt depends on hpricot  
> and a very old version of mechanize. Mechanize now uses nokogiri  
> instead of hpricot and is much more resilient with errors.

Thank you, I will try to use Mechanize directly. However, when I  
installed scrubyt 0.4.1 it did appear to have a dependency on nokogiri.  
I’ve cut and pasted the standard output.

$ sudo gem install scrubyt-0.4.11.gem  
Password:  
Building native extensions. This could take a while…  
Successfully installed scrubyt-0.4.1  
Successfully installed nokogiri-1.2.3  
2 gems installed  
Installing ri documentation for scrubyt-0.4.1…  
Installing ri documentation for nokogiri-1.2.3…  
Installing RDoc documentation for scrubyt-0.4.1…  
Installing RDoc documentation for nokogiri-1.2.3…
