[0.10.0] Index#search is not thread safe?

Ruby Apps Ferret
1

This script (Parked at Loopia) give this result :

“1”
“0”
“0”
“0”

Why the other thread does not have the same result ?
Maybe, it’s not the correct way to use the index in a multi threaded
environement but I don’t know how to do.

Any ideas ?

2

On Sat, Aug 26, 2006 at 04:12:57PM +0200, Florent S. wrote:

This script (Parked at Loopia) give this result :

“1”
“0”
“0”
“0”

Why the other thread does not have the same result ?
Maybe, it’s not the correct way to use the index in a multi threaded
environement but I don’t know how to do.

This should work, imho. I even tried to synchronize access to the index
like that:

guard = Mutex.new
[…]
and in the thread’s block:
guard.synchronize do
result = index.search(‘id:42’)
p result.total_hits
end

but the output stayed the same.

Any ideas ?
You might try to use a Searcher instance for your searching in the
threads. Maybe the problem lies only in the Index class.

Your best bet however is to create separate Index instances in each
thread.

Jens

–
webit! Gesellschaft für neue Medien mbH www.webit.de
Dipl.-Wirtschaftsingenieur Jens Krämer [email protected]
Schnorrstraße 76 Tel +49 351 46766 0
D-01069 Dresden Fax +49 351 46766 66

3

Your best bet however is to create separate Index instances in each
thread.

Thanks for your answer, but this example was inspirate by what I do with
Rails and Ferret. And the main problem is that I have many ajax calls,
each using the index… If I open/close for each ajax call an index, it
will be too expensive, no ? What are your opinion ?

PS: I will try tomorrow with a Searcher, stay tuned :slight_smile:

Thanks again.

4

On Sun, Aug 27, 2006 at 07:43:49PM +0200, Florent S. wrote:

Your best bet however is to create separate Index instances in each
thread.

Thanks for your answer, but this example was inspirate by what I do with
Rails and Ferret. And the main problem is that I have many ajax calls,
each using the index… If I open/close for each ajax call an index, it
will be too expensive, no ? What are your opinion ?

if you’re using Rails, you usually won’t encounter multithreaded access
to a single index instance, since Rails is not threaded (or at least is
not intended to run in a multithreaded way).

So in any Rails app you’ll have at most multiple OS processes accessing
one physical index, but not multiple threads accessing one instance of
class Index. Locking problems on physical index level may occur, though.

PS: I will try tomorrow with a Searcher, stay tuned :slight_smile:

A Searcher should not try to write-lock, so you’ll be on the safe side
as long as only the batch-indexing process will write to the index.

Jens

–
webit! Gesellschaft für neue Medien mbH www.webit.de
Dipl.-Wirtschaftsingenieur Jens Krämer [email protected]
Schnorrstraße 76 Tel +49 351 46766 0
D-01069 Dresden Fax +49 351 46766 66

5

if you’re using Rails, you usually won’t encounter multithreaded access
to a single index instance, since Rails is not threaded (or at least is
not intended to run in a multithreaded way).

But, tell me if I’m wrong, in dev env Rails use webrick, and webrick
create
a thread for each client, so each ajax call. Rails does not create
threads,
it’s true, but webrick does. And the instance of DispatchServlet is
share
between all client thread, yes or not ?

6

I don’t know how Webrick handles Rails, but with lighty/fastcgi you have
seperate fastcgi processes that don’t share anything and each only
handle one request at a time.

Thanks a lot for you answer.

I thought both dev/prod environement had the same way to deal with
threads and in production (with lighttpd/fastcgi) each rails process
could handle several clients at the same time, but I was wrong. I’ve
just re-read the FCGI dispatcher and you are right Jens. Thanks again.

7

On Mon, Aug 28, 2006 at 11:46:55AM +0200, Florent S. wrote:

it’s true, but webrick does. And the instance of DispatchServlet is
share
between all client thread, yes or not ?

I don’t know how Webrick handles Rails, but with lighty/fastcgi you have
seperate fastcgi processes that don’t share anything and each only
handle one request at a time.

Jens

–
webit! Gesellschaft für neue Medien mbH www.webit.de
Dipl.-Wirtschaftsingenieur Jens Krämer [email protected]
Schnorrstraße 76 Tel +49 351 46766 0
D-01069 Dresden Fax +49 351 46766 66

8

Thanks Dave, I’ll check it.

9

On 8/28/06, Jens K. [email protected] wrote:

environement but I don’t know how to do.
end

but the output stayed the same.

This was a bug that has been fixed in the current working copy. The
fix will be released in version 0.10.2.

Cheers,
Dave