Warming up a new Searcher/Reader (Ferret 0.10.9 win32)


#1

Hi,

I have a largish index [700MB] which is updated from time to time,
requiring me to close and recreate the Ferret::Search::Searcher to use
the latest index.

My problem is that the first few searches on the new index are slow [by
comparison to before the close/recreate], I’m guessing because the new
index is being loaded into RAM by my OS and into Ferret as needed.

I’m thinking of “warming up” the OS and Ferret by processing a query or
two, and I’d appreciate any details about the way Ferret loads the
index, so that I can construct a good “warm up” query.

Kind Regards

Neville


#2

On 3/5/07, Neville B. removed_email_address@domain.invalid wrote:

comparison to before the close/recreate], I’m guessing because the new index
is being loaded into RAM by my OS and into Ferret as needed.

I’m thinking of “warming up” the OS and Ferret by processing a query or two,
and I’d appreciate any details about the way Ferret loads the index, so that
I can construct a good “warm up” query.

Hi Neville,

Ferret loads the index for each field when it is searched. So if you
only search one field, only that field’s index will be loaded.

Once each fields index is loaded, ferret should be fully warmed up as
far as simple queries like phrase and boolean go. If you are sorting
search results, then sort indexes will also need to be built. A new
sort index is built for each sorted field depending on sort type (int,
float, string, byte) and sort direction (normal and reverse). So if
you are sorting your search results you will also need to try each
type of sort that you might use to warm up that part of the index.

That’s all I can think of at the moment. One thing I have planned for
the future is adding the ability to autoload all the indexes and to
save sort indexes rather than building them each time you open an
index reader.

Cheers,
Dave


#3

Thanks for the detail Dave!

One thing I have planned for the future is adding the ability
to autoload all the indexes

Awesome - this sounds like just what I’m looking for.

Cheers,

Nev