# Natural language detection library

**URL:** <https://www.ruby-forum.com/t/natural-language-detection-library/103511>\
**Category:** Ruby\
**Created:** [May 7, 2007, 1:18pm UTC](https://www.ruby-forum.com/t/natural-language-detection-library/103511 "2007-05-07T13:18:26Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![tnitsche](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/t/958977/32.png) [@tnitsche](https://www.ruby-forum.com/u/tnitsche)\
**Post date:** [May 7, 2007, 1:18pm UTC](https://www.ruby-forum.com/t/natural-language-detection-library/103511/1 "2007-05-07T13:18:26Z")

</div>

Hi,

does anyone know a Ruby library/module to detect which natural language  
is a given input text?

Cheers

Thomas

---

<div class="post-metadata">

**Author:** ![tnitsche](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/t/958977/32.png) [@tnitsche](https://www.ruby-forum.com/u/tnitsche)\
**Post date:** [May 7, 2007, 1:56pm UTC](https://www.ruby-forum.com/t/natural-language-detection-library/103511/2 "2007-05-07T13:56:00Z")

</div>

You can use the zlib library for this as does deplate:

from: [Deplate 0.8.5 – convert wiki-like markup to latex, docbook, html, or “html-slides” -- Modules](http://deplate.sourceforge.net/Modules.html#hd008001001)

> The algorithm of this plugin is based on D Benedetto & E Caglioti  
> & V Loreto ?Language Trees and Zipping?[1]. It?s a direct port of  
> Dirk Holtwick?s ?Guess language of text using ZIP?[2].

[1] [Format selector for cond-mat/0108530](http://xxx.uni-augsburg.de/format/cond-mat/0108530)  
[2] [http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/355807](http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/355807)

Ruby code:

> **[CVS Info for project deplate](http://deplate.cvs.sourceforge.net/deplate/deplate/lib/deplate/guesslanguage.rb?view=markup)**
>
> The world's largest development and download repository of Open Source code and applications

Works quite well for me.

Thomas.

---

<div class="post-metadata">

**Author:** ![tnitsche](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/t/958977/32.png) [@tnitsche](https://www.ruby-forum.com/u/tnitsche)\
**Post date:** [May 8, 2007, 4:50pm UTC](https://www.ruby-forum.com/t/natural-language-detection-library/103511/3 "2007-05-08T16:50:47Z")

</div>

On May 7, 1:54 pm, micathom [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> Ruby code:[CVS Info for project deplate](http://deplate.cvs.sourceforge.net/deplate/deplate/lib/deplate/guessl)…
> 
> Works quite well for me.
> 
> Thomas.

thx, I don’t have a clue how it works, but it’s great 😉

---

<div class="post-metadata">

**Author:** ![tnitsche](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/t/958977/32.png) [@tnitsche](https://www.ruby-forum.com/u/tnitsche)\
**Post date:** [May 9, 2007, 10:10am UTC](https://www.ruby-forum.com/t/natural-language-detection-library/103511/4 "2007-05-09T10:10:52Z")

</div>

> thx, I don’t have a clue how it works, but it’s great 😉

You need some base corpus/sample (I use the GPL License) for each  
language and calculate an index number for this on initiatlization.  
Then you compare a sample text’s index with the base index.

An example for how to use this can be found at:

> **[CVS Info for project deplate](http://deplate.cvs.sourceforge.net/deplate/deplate/lib/deplate/mod/guesslanguage.rb?view=markup)**
>
> The world's largest development and download repository of Open Source code and applications

First call #register(language\_name, text) for each corpus, then call  
#guess\_with\_diff(text) to guess a text’s language.

---

<div class="post-metadata">

**Author:** ![tnitsche](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/t/958977/32.png) [@tnitsche](https://www.ruby-forum.com/u/tnitsche)\
**Post date:** [May 9, 2007, 11:36am UTC](https://www.ruby-forum.com/t/natural-language-detection-library/103511/5 "2007-05-09T11:36:06Z")

</div>

Sorry, I meant I have no idea how the algorithm is working. The code  
itself works like a charm.  
Actually I’m using the “Declaration of Human Rights” 😉

Cheers,

Thomas