# Parsers for input to index?

**URL:** <https://www.ruby-forum.com/t/parsers-for-input-to-index/92924>\
**Category:** Ferret\
**Created:** [April 25, 2007, 7:14pm UTC](https://www.ruby-forum.com/t/parsers-for-input-to-index/92924 "2007-04-25T19:14:53Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![dmonahan](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/c2a13f/32.png) [@dmonahan](https://www.ruby-forum.com/u/dmonahan)\
**Post date:** [April 25, 2007, 7:14pm UTC](https://www.ruby-forum.com/t/parsers-for-input-to-index/92924/1 "2007-04-25T19:14:53Z")

</div>

The documents we want to index come in many formats; e.g., HTML, PDF,  
RTF, Word, Excel, etc., etc., etc. I’ve been searching to find parsers  
that will translate each of these formats to indexable text, but have  
had little success. Any help will be appreciated.

---

<div class="post-metadata">

**Author:** ![dmonahan](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/c2a13f/32.png) [@dmonahan](https://www.ruby-forum.com/u/dmonahan)\
**Post date:** [April 25, 2007, 7:40pm UTC](https://www.ruby-forum.com/t/parsers-for-input-to-index/92924/2 "2007-04-25T19:40:45Z")

</div>

Hi Dick,

you may need to turn to using some external tools.

something similar to this was discussed before and some tools suggested.

See: [Indexing mostly-binary documents (.ppt) - Ferret - Ruby-Forum](http://www.ruby-forum.com/topic/103374)

assuming the text is stored ASCII single byte, you could fall back on  
the “strings” command as a last resort. It should be installed already  
on modern GNU/Linux distros. Try cygwin for windows. It reads in any  
data and outputs all “printable character sequences”.

John.

On Wed, 2007-04-25 at 19:14 +0200, Dick Monahan wrote:

> The documents we want to index come in many formats; e.g., HTML, PDF,  
> RTF, Word, Excel, etc., etc., etc. I’ve been searching to find parsers  
> that will translate each of these formats to indexable text, but have  
> had little success. Any help will be appreciated.

–  
[http://johnleach.co.uk](http://johnleach.co.uk)

---

<div class="post-metadata">

**Author:** ![dmonahan](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/c2a13f/32.png) [@dmonahan](https://www.ruby-forum.com/u/dmonahan)\
**Post date:** [April 25, 2007, 8:24pm UTC](https://www.ruby-forum.com/t/parsers-for-input-to-index/92924/3 "2007-04-25T20:24:08Z")

</div>

Hello Dick, and all (first post),

Here are some more that I use:

HTML to text: Vilistextum

> **[Blog - Mikula Beutl](https://mikulabeutl.com/blog/?utm_source=redirect&utm_medium=bhaak.dyndns.org&utm_campaign=%2Fvilistextum%2F)**
>
> Find Guides: Click on a Topic below this line or use the Search: Search Latest Articles:

also lynx:

> **[Lynx Information](http://lynx.browser.org/)**
>
> Index of Information resources about the Lynx browser for the World Wide Web

PDF to text: pdftotext, from Xpdf  
[http://www.foolabs.com/xpdf/](http://www.foolabs.com/xpdf/)

WordPerfect to text: wpd2text, from libwpd  
[http://libwpd.sourceforge.net/](http://libwpd.sourceforge.net/)

Converting other text encodings: iconv  
[http://www.gnu.org/software/libiconv/](http://www.gnu.org/software/libiconv/)

-Stuart Sierra