Whistlepig: real-time search for ruby

Hi all,

I’ve released Whistlepig 0.5.

Whistlepig is a minimalist realtime full-text search index. Its goal is
to be as small and feature-free as possible, while still remaining
useful, performant and scalable to large corpora. If you want realtime
full-text search without the frills, Whistlepig may be for you.

Whistlepig is written in ANSI C99. It currently provides a C API and
Ruby bindings.




require ‘rubygems’
require ‘whistlepig’

include Whistlepig

index = Index.new “index”

adding a document is easy

entry1 = Entry.new
entry1.add_string “body”, “hello there bob”
docid1 = index.add_entry entry1 # => 1

let’s add another one

entry2 = Entry.new
entry2.add_string “body”, “goodbye bob”
docid2 = index.add_entry entry2 # => 2

documents are immediately available for search

q1 = Query.new “body”, “bob”
results1 = index.search q1 # => [2, 1]

queries can be constructed programmatically

q2 = q1.and Query.new(“body”, “hello”)
results2 = index.search q2 # => [1]

you can add labels to documents

index.add_label docid2, “funny”

labels can be mixed into queries with the ~ operator

q3 = Query.new “body”, “bob ~funny”
results3 = index.search q3 # => [2]

you can have arbitrary fields!

entry3 = Entry.new
entry3.add_string “body”, “hello joe”
entry3.add_string “subject”, “what do you know?”
docid3 = index.add_entry entry3 # => 3

fielded search uses a :

q4 = Query.new “body”, “subject:know hello”
results4 = index.search q4 # => [3]

and there’s much much more…

I’m using it to implement a gmail-like email server.
(GitHub - wmorgan/heliotrope: A personal, threaded, search-centric email server.). It might be useful to you too.