Forum: Ferret Rép : ignored or forgotten valu es in reindexing

Hi,

Le 27 août 08 à 02:51, Chris Strom a écrit :
>>
>
> I have seen this problem as well.  Same versions of ferret/aaf
> (0.11.6/0.4.3).  Differences are Rails 2.0 and using SQL Server.

I use posgresql, didn't mention it.


> The limited amount of investigation that I have been able to do
> suggests a fence post error in the bulk index pagination.  Of 2000+
> objects, 2 are missing.  I'm only basing the fence post conjecture on
> the number of missing objects & the default pagination window (1000).
> I have not even had a chance to investigate changing the window size,
> let alone investigate deeper.


I observed this behaviour with reindexing, even from script/console,
without any pagination interference.

Jean-Christophe Michel
jc.michel@symetrie.com

On Wed, Aug 27, 2008 at 10:30 AM, lists.jc.michel@symetrie.com
<lists.jc.michel@symetrie.com> wrote:
>> The limited amount of investigation that I have been able to do
>> suggests a fence post error in the bulk index pagination.  Of 2000+
>> objects, 2 are missing.  I'm only basing the fence post conjecture on
>> the number of missing objects & the default pagination window (1000).
>> I have not even had a chance to investigate changing the window size,
>> let alone investigate deeper.
>
>
> I observed this behaviour with reindexing, even from script/console, without
> any pagination interference.

Reindexing uses the BulkIndexer as well does it not?  Even from
script/console, unless I am mistaken.

-Chris

Hi!

I must admit that I don't do regular testing with postgresql, maybe
its some db related issue. I'll look into it. Could you please open a
ticket at projects.jkraemer.net/acts_as_ferret ?

cheers,
Jens

On 27.08.2008, at 16:30, lists.jc.michel@symetrie.com wrote:

>>> some values;
>> The limited amount of investigation that I have been able to do
> Jean-Christophe Michel
> jc.michel@symetrie.com
>
> _______________________________________________
> Ferret-talk mailing list
> Ferret-talk@rubyforge.org
> http://rubyforge.org/mailman/listinfo/ferret-talk
>

--
Jens Krämer
webit! Gesellschaft für neue Medien mbH
Schnorrstraße 76 | 01069 Dresden
Telefon +49351467660 | Telefax +493514676666
kraemer@webit.de | www.webit.de

Amtsgericht Dresden | HRB 15422
GF Sven Haubold

Le 28 août 08 à 09:25, Jens Krämer a écrit :
> I must admit that I don't do regular testing with postgresql, maybe  
> its some db related issue. I'll look into it. Could you please open  
> a ticket at projects.jkraemer.net/acts_as_ferret ?

Sure ! but I'd like to diagnose a bit more. How can I get some error
output instead of silently ignored rows ?

Hi,

Le 28 août 08 à 09:25, Jens Krämer a écrit :
> I must admit that I don't do regular testing with postgresql, maybe  
> its some db related issue. I'll look into it. Could you please open  
> a ticket at projects.jkraemer.net/acts_as_ferret ?

Thanks for your quick reply.
I think I found a mistake of mine with this other thread :

Le 28 août 08 à 17:19, Sheldon Maloff a écrit :
> Now, corrupt indexes in development is another issue. In  
> development, you are not running a DRb server. Each mongrel is  
> hitting the index directly. You typically have only one mongrel  
> running in development. But if you open an interactive script/ 
> console session, and play with your models side-by-side a running  
> mongrel, you WILL corrupt your Ferret index. That's because both  
> the mongrel and the script/console will be writers to the same  
> index, something that Ferret doesn't support. Heck, running a rake  
> db:migrate along side a running mongrel will cause index  
> corruption, for the same reason: multiple writers.

In fact I probably had mongrel and console running together when I
reindexed in development.
I shut down and dropped my index dir, then did bulk_index on my class
in console.
Seems all my words were indexed.

Maybe we could think to some error messages issued by each index
writer :
could be like
   logger.debug 'Man, chances are you are messing up your index by
writing concurrently to it' if (development) && writer_flag
The flag being a development only feature, enabled by default,
disabled in production only.
What do you think of this ?