\
**Category:** Ruby\
**Created:** [November 28, 2005, 10:33am UTC](https://www.ruby-forum.com/t/htmltokenizer-bug/48989 "2005-11-28T10:33:23Z")\
**Posts on this page:** 6\
**Page:** 1
**Author:**  [@hsanson](https://www.ruby-forum.com/u/hsanson)\
**Post date:** [November 28, 2005, 10:33am UTC](https://www.ruby-forum.com/t/htmltokenizer-bug/48989/1 "2005-11-28T10:33:23Z")
I am using htmltokenizer to extract the links of some web pages, my
script
worked perfectly until I started to parse pages with “\<” and “\>” chars
in the
text.
a html string like this
this is a
causes the htmlparser to raise and exception; Error, tag is nil…
Is there a patch or any way to make htmlparser to parse this text??
regards,
Horacio
---
**Author:**  [@hsanson](https://www.ruby-forum.com/u/hsanson)\
**Post date:** [November 28, 2005, 11:30am UTC](https://www.ruby-forum.com/t/htmltokenizer-bug/48989/2 "2005-11-28T11:30:12Z")
On 28/11/05, Horacio S. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:
> Is there a patch or any way to make htmlparser to parse this text??
I think most _browsers_ would choke on that 🙂
Have you tried using entities instead ?
( \< instead of \< and \> instead of \>)
---
**Author:**  [@hsanson](https://www.ruby-forum.com/u/hsanson)\
**Post date:** [November 28, 2005, 1:56pm UTC](https://www.ruby-forum.com/t/htmltokenizer-bug/48989/3 "2005-11-28T13:56:03Z")
Horacio S. wrote:
> Is there a patch or any way to make htmlparser to parse this text??
>
> regards,
> Horacio
Your HTML isn’t valid. Either use the proper entities (\< = \< and \> =
\>) or make a CDATA section, though the latter isn’t really that
well-supported in most browsers.
\
Cheers,
Daniel
---
**Author:**  [@hsanson](https://www.ruby-forum.com/u/hsanson)\
**Post date:** [December 2, 2005, 2:18am UTC](https://www.ruby-forum.com/t/htmltokenizer-bug/48989/4 "2005-12-02T02:18:12Z")
Sorry for the late reply.
I’m surprised no one mentioned RubyfulSoup:
> **[Rubyful Soup: "The brush has got entangled in it!"](https://www.crummy.com/software/RubyfulSoup/)**
>
> Rubyful Soup: an HTML parser for Ruby, optimized for screen-scraping.
If I understand your problem correctly, it’s exactly what you need: a
forgiving html parser.
Dan
---
**Author:**  [@hsanson](https://www.ruby-forum.com/u/hsanson)\
**Post date:** [December 2, 2005, 3:30am UTC](https://www.ruby-forum.com/t/htmltokenizer-bug/48989/5 "2005-12-02T03:30:56Z")
Daniel A. wrote:
> Sorry for the late reply.
>
> I’m surprised no one mentioned RubyfulSoup:
>
> [Rubyful Soup: "The brush has got entangled in it!"](http://www.crummy.com/software/RubyfulSoup/)
>
> If I understand your problem correctly, it’s exactly what you need: a
> forgiving html parser.
I recently tried using RubyfulSoup to parse a Web page, and it had some
peculiar behavior, such as stripping all attributes. Either I was not
using it correctly, or it was a bit too casual in making sense of the
input.
I ended up using some crude string parsing to extract just the subset of
the page I wanted, which gave me well-formed XML suitable for REXML
manipulation. I got a phenomenal speed increase from that as well;
RubyfulSoup seems quite slow.
## James
[http://www.ruby-doc.org](http://www.ruby-doc.org) - Ruby Help & Documentation
[Ruby Code & Style](http://www.artima.com/rubycs/) - Ruby Code & Style: Writers wanted
[http://www.rubystuff.com](http://www.rubystuff.com) - The Ruby Store for Ruby Stuff
[http://www.jamesbritt.com](http://www.jamesbritt.com) - Playing with Better Toys
[http://www.30secondrule.com](http://www.30secondrule.com) - Building Better Tools
---
**Author:**  [@hsanson](https://www.ruby-forum.com/u/hsanson)\
**Post date:** [November 28, 2005, 2:44pm UTC](https://www.ruby-forum.com/t/htmltokenizer-bug/48989/6 "2005-11-28T14:44:39Z")
Well the problem is that this HTML is not mine, retrieving the pages
from the
Internet.
Guess I will skip this page from my script.
thanks,
Horacio
Monday 28 November 2005 21:52ã?Daniel S. ã?ã??ã¯æ?¸ãã¾ã?ã?: