Index.optimize

In the documentation, it says that optimize “should only be called
when the index will no longer be updated very often, but will be read
a lot”. Does this mean it actually has a detrimental impact on updates
and inserts? In my project there will be many more reads than updates,
but there will still be a lot of updates. So should I be calling
Optimize once a day or something like that, during a low traffic time,
or is that going to make updates slower? What are your recommendations
for optimizing the index for a high-traffic site? Does optimizing the
index lock the index for a fair period of time if there are 100s of
thousands of documents? Will read requests be denied during the
optimization process?


Cheers,
Jordan F.
[email protected]

On 8/3/06, Jordan F. [email protected] wrote:

In the documentation, it says that optimize “should only be called
when the index will no longer be updated very often, but will be read
a lot”. Does this mean it actually has a detrimental impact on updates
and inserts?

Hi Jordan,

Optimizing the index won’t effect inserts and will actually make
updates a little faster. The problem is the time it takes to optimize
the index. Optimization is an expensive process so it should be
avoided when doing a lot of indexing.

In my project there will be many more reads than updates,
but there will still be a lot of updates. So should I be calling
Optimize once a day or something like that, during a low traffic time,
or is that going to make updates slower?

The only detrimental performance impact optimizing has is the time it
takes to actually perform the optimization.

What are your recommendations
for optimizing the index for a high-traffic site? Does optimizing the
index lock the index for a fair period of time if there are 100s of
thousands of documents? Will read requests be denied during the
optimization process?

Yes, optimization can hold the lock for a long period of time, but an
IndexWriter has a write-lock open on the index the whole time it is
open. The IndexWriter won’t effect the performance of IndexReaders.
You can have as many processes as you like reading the index at the
same time and it won’t matter how you are writing to the index. You
should keep in mind that the IndexReaders need to be refreshed (closed
and opened again) to make use of latest data added to the index. The
Index class can be setup to handle this for you but if you are
concerned about performance you should stick to using the IndexWriter
and IndexReader classes or at least understand how the Index class
makes use of those two classes (check out the Ruby source for the
Index class).

So the basic answer to your question is; optimize whenever you have
the cpu cycles available to perform the opimization process.
Optimizing the index won’t negatively effect the performance of the
index in any way.

Please let me know if I haven’t been clear on anything.

Cheers,
Dave

On 8/3/06, David B. [email protected] wrote:

the index. Optimization is an expensive process so it should be

What are your recommendations
should keep in mind that the IndexReaders need to be refreshed (closed
index in any way.

Please let me know if I haven’t been clear on anything.

Cheers,
Dave


Ferret-talk mailing list
[email protected]
http://rubyforge.org/mailman/listinfo/ferret-talk

Crystal clear, thank you very much. I definitely intend on getting
more into the internals of Ferret to understand how it really works,
but for now I need to get something going quickly, so your answer is
just what I needed. Right now I’m really just using acts_as_ferret but
modifying it slightly so that the updates get queued and then all
happen periodically, instead of when a record is updated…Eventually
I would like to bypass acts_as_ferret altogether (not that it isn’t a
great tool, I just want to have more control) and use Ferret directly.
Thanks for your help.


Cheers,
Jordan F.
[email protected]

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs