# Roll my own TokenFilter subclass

**URL:** <https://www.ruby-forum.com/t/roll-my-own-tokenfilter-subclass/104521>\
**Category:** Ferret\
**Created:** [May 18, 2007, 11:13pm UTC](https://www.ruby-forum.com/t/roll-my-own-tokenfilter-subclass/104521 "2007-05-18T23:13:25Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![richyjones](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/3da27b/32.png) [@richyjones](https://www.ruby-forum.com/u/richyjones)\
**Post date:** [May 18, 2007, 11:13pm UTC](https://www.ruby-forum.com/t/roll-my-own-tokenfilter-subclass/104521/1 "2007-05-18T23:13:25Z")

</div>

Hi all,

I’d like to write my own TokenStream Filter (in lucene this would be a  
subclass of a TokenFilter, which ferret seems to lack) but I’m not  
sure how to go about it. Specifically, it’s not clear how I’d create  
a non-trivial TokenStream to pass out to any filters that wrapped  
mine.

Can anyone point me towards a code example? Thanks.

---

<div class="post-metadata">

**Author:** ![richyjones](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/3da27b/32.png) [@richyjones](https://www.ruby-forum.com/u/richyjones)\
**Post date:** [May 19, 2007, 12:41am UTC](https://www.ruby-forum.com/t/roll-my-own-tokenfilter-subclass/104521/2 "2007-05-19T00:41:53Z")

</div>

> Can anyone point me towards a code example? Thanks.

It’s not exactly what you want, but you might glean some help from our  
Multilingual Ferret Tools at  
[http://svn.lingr.com/plugins/multilingual\_ferret\_tools](http://svn.lingr.com/plugins/multilingual_ferret_tools)

Best Regards,

Danny