Forum: Ruby on Rails scaling full text indexing(ferret vs solr vs hyperstraier)

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
3518ff338e5eee9b32eb1815daabc602?d=identicon&s=25 Marcelo Barbudas (Guest)
on 2009-03-08 13:15
(Received via mailing list)
Hi,

Does any have experience scaling full text search in RoR?

Right now our project is running a simple setup with ferret and
acts_as_ferret. We are thinking about deploying a feature that would
send 50x more search requests.

So we probably have to rethink our solution. How do services like
search.twitter.com (the former Summize) use?

Or in what direction should I look?

--
Thanks,
M.
4375b4aca04b8d72482fd409fb6d0b47?d=identicon&s=25 Roderick van Domburg (roderickvd)
on 2009-03-08 14:40
Marcelo Barbudas wrote:
> Right now our project is running a simple setup with ferret and
> acts_as_ferret. We are thinking about deploying a feature that would
> send 50x more search requests.

With Ferret you can scale reads horizontally: you can have multiple read
servers on a single index. You can only have one write server on a
single index or you'll risk data corruption.

Another strategy is partitioning: having separate indices for buckets of
data. Each index could run on it's own server or cluster of servers.

--
Roderick van Domburg
http://www.nedforce.com
3518ff338e5eee9b32eb1815daabc602?d=identicon&s=25 Marcelo Barbudas (Guest)
on 2009-03-08 22:53
(Received via mailing list)
> With Ferret you can scale reads horizontally: you can have multiple read
> servers on a single index. You can only have one write server on a
> single index or you'll risk data corruption.
>
> Another strategy is partitioning: having separate indices for buckets of
> data. Each index could run on it's own server or cluster of servers.
>

Would it be easier to scale with hyperestraier or something else?

--
M.
988320371d9a18f0d50375188e01d54a?d=identicon&s=25 Peter Vandenabeele (Guest)
on 2009-03-12 22:04
(Received via mailing list)
On Sun, Mar 8, 2009 at 1:14 PM, Marcelo Barbudas <nostef@gmail.com>
wrote:
> Does any have experience scaling full text search in RoR?

One option that worked very well for me is ultrasphinx.

  http://blog.evanweaver.com/files/doc/fauna/ultrasp...

IIRC, 2 limitations of ultrasphinx are:

* new entries can only be found after reindexing
  (full reindexing or delta indexing)
* you need a separate sphinx process somewhere on a server
  (if you run a shared hosting system, this may be an issue)

If you can live with those 2 limitations, ultrasphinx is a very good
candidate.

HTH,

Peter
This topic is locked and can not be replied to.