\
**Category:** Ruby\
**Created:** [October 6, 2006, 4:13am UTC](https://www.ruby-forum.com/t/stripping-unwanted-html/71217 "2006-10-06T04:13:32Z")\
**Posts on this page:** 7\
**Page:** 1
**Author:**  [@wildal](https://www.ruby-forum.com/u/wildal)\
**Post date:** [October 6, 2006, 4:13am UTC](https://www.ruby-forum.com/t/stripping-unwanted-html/71217/1 "2006-10-06T04:13:32Z")
Hi everyone:
I’m trying to strip html with the exception of a few html tags.
I have found the following code:
def strip\_tags(html)
if html.index("\<")
text = “”
tokenizer = HTML::Tokenizer.new(html)
```
while token = tokenizer.next
node = HTML::Node.parse(nil, 0, 0, token, false)
# result is only the content of any Text nodes
text << node.to_s if node.class == HTML::Text
end
# strip any comments, and if they have a newline at the end (ie.
```
line with
# only a comment) strip that too
text.gsub(/[\n]?/m, “”)
else
html # already plain text
end
end
I’m trying to understand what is going on in this code but cannot find
documenation for HTML::Tokenizer or HTML::Node.parse. Does anyone know
the use of the parameters in the parse method?
In the while loop, how do you access the html tag. If I could access
the html tags, I could then decide if I wanted to keep the tag or not.
Thanks for reading,
Wild Al
---
**Author:**  [@wildal](https://www.ruby-forum.com/u/wildal)\
**Post date:** [October 6, 2006, 5:38am UTC](https://www.ruby-forum.com/t/stripping-unwanted-html/71217/2 "2006-10-06T05:38:42Z")
Wild Al wrote:
> Hi everyone:
>
> I’m trying to strip html with the exception of a few html tags.
>
> I have found the following code:
>
> def strip\_tags(html)
> if html.index(“\<”)
> text = “”
> tokenizer = HTML::Tokenizer.new(html)
>
> ```
> while token = tokenizer.next
> node = HTML::Node.parse(nil, 0, 0, token, false)
> # result is only the content of any Text nodes
> text << node.to_s if node.class == HTML::Text
> end
> # strip any comments, and if they have a newline at the end (ie.
>
> ```
>
> line with
> # only a comment) strip that too
> text.gsub(/[\n]?/m, “”)
> else
> html # already plain text
> end
> end
>
> I’m trying to understand what is going on in this code but cannot find
> documenation for HTML::Tokenizer or HTML::Node.parse. Does anyone know
> the use of the parameters in the parse method?
>
> In the while loop, how do you access the html tag. If I could access
> the html tags, I could then decide if I wanted to keep the tag or not.
>
> Thanks for reading,
> Wild Al
Al- I recently needed a parser similar to Perl’s HTML::Scrubber. [This](http://www.underpantsgnome.com/2006/09/09/using-hpricot-to-scrub-html/)
is what I came up with , you may find it useful.
Michael
---
**Author:**  [@wildal](https://www.ruby-forum.com/u/wildal)\
**Post date:** [October 6, 2006, 7:50am UTC](https://www.ruby-forum.com/t/stripping-unwanted-html/71217/3 "2006-10-06T07:50:41Z")
Wild Al wrote:
/ …
> In the while loop, how do you access the html tag. If I could access
> the html tags, I could then decide if I wanted to keep the tag or not.
Why not create your own method? This should give you some ideas:
#!/usr/bin/ruby -w
data = File.read("/path/page.html")
data.scan(/\<(\w+?)\>/) { |tag|
puts tag
}
This puts out the tag name extracted from each HTML tag in the page.
---
**Author:**  [@wildal](https://www.ruby-forum.com/u/wildal)\
**Post date:** [October 6, 2006, 11:20am UTC](https://www.ruby-forum.com/t/stripping-unwanted-html/71217/4 "2006-10-06T11:20:55Z")
Hey, that’s RoR’s strip\_tags method 🙂
I ran into the same issue so went down your path and instead of
fretting about no docs, just hacked my way through using IRB.
Here’s a modified version of what I came up with, maybe you’ll find it
useful?
def strip\_tags\_except(html, exceptions = [])
if html.index("\<")
text = “”
tokenizer = HTML::Tokenizer.new(html)
while token = tokenizer.next
case node = HTML::Node.parse(nil, 0, 0, token, false)
when HTML::Tag
text \<\< node.to\_s if exceptions.include?(node.name)
when HTML::Text
text \<\< node.to\_s
end
end
text
else
html
end
end
The one I had also stripped attributes and closed up dangling tags if
it found any. Have a look at RoR’s strip\_links for more examples of
HTML::Node/HTML::Tokenizer usage.
---
**Author:**  [@wildal](https://www.ruby-forum.com/u/wildal)\
**Post date:** [October 6, 2006, 11:24am UTC](https://www.ruby-forum.com/t/stripping-unwanted-html/71217/5 "2006-10-06T11:24:56Z")
eden wrote:
> Have a look at RoR’s strip\_links for more examples of
> HTML::Node/HTML::Tokenizer usage.
Sorry, I meant sanitize:
[http://api.rubyonrails.com/classes/ActionView/Helpers/TextHelper.html#M000516](http://api.rubyonrails.com/classes/ActionView/Helpers/TextHelper.html#M000516)
---
**Author:**  [@wildal](https://www.ruby-forum.com/u/wildal)\
**Post date:** [October 8, 2006, 9:54pm UTC](https://www.ruby-forum.com/t/stripping-unwanted-html/71217/6 "2006-10-08T21:54:13Z")
eden wrote:
> Here’s a modified version of what I came up with, maybe you’ll find it
> useful?
>
> def strip\_tags\_except(html, exceptions = [])
> if html.index("\<")
> text = “”
> tokenizer = HTML::Tokenizer.new(html)
> while token = tokenizer.next
> case node = HTML::Node.parse(nil, 0, 0, token, false)
> when HTML::Tag
> text \<\< node.to\_s if exceptions.include?(node.name)
> when HTML::Text
> text \<\< node.to\_s
> end
> end
> text
> else
> html
> end
> end
I found this method very useful; it is exactly what I needed. Thanks.
To all others: your suggestions helped too, especially in understanding
ruby. Thanks again…
---
**Author:**  [@wildal](https://www.ruby-forum.com/u/wildal)\
**Post date:** [October 9, 2006, 7:37am UTC](https://www.ruby-forum.com/t/stripping-unwanted-html/71217/7 "2006-10-09T07:37:03Z")
Glad I could help. One security-related caveat.
The method I posted doesn’t strip attributes, so it may be possible for
someone to “hack” your site by putting javascript onto one of the
allowed tags.
You can fix that by changing the last line of the first branch of the
if statement from “text” to “sanitize(text)”, eg:
if html.index("\<")
…
sanitize(text)
else
…