Forum: Ferret Warming up a new Searcher/Reader (Ferret 0.10.9 win32)

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
332b27a7e8304eaaf473752432a3f244?d=identicon&s=25 Neville Burnell (Guest)
on 2007-03-05 05:14
(Received via mailing list)
Hi,

I have a largish index [700MB]  which is updated from time to time,
requiring me to close and recreate the Ferret::Search::Searcher to use
the latest index.

My problem is that the first few searches on the new index are slow [by
comparison to before the close/recreate], I'm guessing because the new
index is being loaded into RAM by my OS and into Ferret as needed.

I'm thinking of "warming up" the OS and Ferret by processing a query or
two, and I'd appreciate any details about the way Ferret loads the
index, so that I can construct a good "warm up" query.

Kind Regards

Neville
B5e329ffa0cc78efbfc7ae2d084c149f?d=identicon&s=25 David Balmain (Guest)
on 2007-03-06 04:42
(Received via mailing list)
On 3/5/07, Neville Burnell <Neville.Burnell@bmsoft.com.au> wrote:
> comparison to before the close/recreate], I'm guessing because the new index
> is being loaded into RAM by my OS and into Ferret as needed.
>
> I'm thinking of "warming up" the OS and Ferret by processing a query or two,
> and I'd appreciate any details about the way Ferret loads the index, so that
> I can construct a good "warm up" query.

Hi Neville,

Ferret loads the index for each field when it is searched. So if you
only search one field, only that field's index will be loaded.

Once each fields index is loaded, ferret should be fully warmed up as
far as simple queries like phrase and boolean go. If you are sorting
search results, then sort indexes will also need to be built. A new
sort index is built for each sorted field depending on sort type (int,
float, string, byte) and sort direction (normal and reverse). So if
you are sorting your search results you will also need to try each
type of sort that you might use to warm up that part of the index.

That's all I can think of at the moment. One thing I have planned for
the future is adding the ability to autoload all the indexes and to
save sort indexes rather than building them each time you open an
index reader.

Cheers,
Dave
332b27a7e8304eaaf473752432a3f244?d=identicon&s=25 Neville Burnell (Guest)
on 2007-03-06 05:21
(Received via mailing list)
Thanks for the detail Dave!

> One thing I have planned for the future is adding the ability
> to autoload all the indexes

Awesome - this sounds like just what I'm looking for.

Cheers,

Nev
This topic is locked and can not be replied to.