Duplicate search results

Ruby Apps Ferret
1

i’m using Ferret 0.9.0 with acts_as_ferret (the one from
svn.jkraemer.net), and
i’m getting duplicate results, as described in this thread:
http://rubyforge.org/pipermail/ferret-talk/2005-December/000048.html

is there a way to configure the indexes created by acts_as_ferret to use
:key =>
:id, as described in that thread? i’ve poked around in the code, and had
trouble
even finding where the index was being created.

or could something else be causing this? every time i re-save (or
update) a
model, a new document is being created in the index.

ferret is rad, many thanks for all the work on it.

best,
john

John McGrath

This message was sent using IMP, the Internet Messaging Program.

2

Hi John,

On Wed, Apr 05, 2006 at 12:09:37AM -0400, John McGrath wrote:

i’m using Ferret 0.9.0 with acts_as_ferret (the one from svn.jkraemer.net), and
i’m getting duplicate results, as described in this thread:
http://rubyforge.org/pipermail/ferret-talk/2005-December/000048.html

is there a way to configure the indexes created by acts_as_ferret to use :key =>
:id, as described in that thread? i’ve poked around in the code, and had trouble
even finding where the index was being created.

acts_as_ferret uses the id as a key by default. Could you please check
if this problem still occurs with the current trunk (located at
svn://projects.jkraemer.net/acts_as_ferret/trunk/plugin/acts_as_ferret )

Cheers,
Jens

–
webit! Gesellschaft für neue Medien mbH www.webit.de
Dipl.-Wirtschaftsingenieur Jens Krämer [email protected]
Schnorrstraße 76 Tel +49 351 46766 0
D-01069 Dresden Fax +49 351 46766 66

3

hi jens, updated to trunk, but still having the same issue

in my Book.rb model i have the acts_as_ferret line with no attributes:
acts_as_ferret. to keep things simple it’s the only model using
acts_as_ferret
right now.

my controller code is based on the example in the comments above
find_id_by_contents:

@res << Book.find_id_by_contents(params[:search][:search_term])

if i zero both the index and the db, then add a single book, one (and
only one)
book appears in the ‘books’ table, but the line above from the
controller
returns 3 results from the newly-created index at
index/development/Book.

it’s coming from somewhere in the plugin, i think, because when i add
this line
to find_id_by_contents, right after the call to ferret_index.search(q,
option):
logger.debug(“hits is: #{hits.size}”)

this shows up in the log:
hits is: 3

best,
john

Quoting Jens K. [email protected]:

:id, as described in that thread? i’ve poked around in the code, and had

This message was sent using IMP, the Internet Messaging Program.

4

still banging my head against this. at a suggestion from jens i checked
the
index size with irb, and it is indeed larger than it should be – extra
entries
are being put in. i also got the demo from svn and ran the unit tests,
some of
which failed. the failing tests look like they’re related to multiple id
issues, too – i’ll paste them in below.

i’m slowly working my way through the code, but ferret is new to me
(ruby too,
really), so it might take a while. if anyone has any pointers in the
meanwhile,
i’d really appreciate it. since my problems have been so consistent, and
since i
get similar issues with the demo, i’m guessing it might have something
to do
with changes to ActiveRecord in Rails 1.1.

fwiw, my env details again: mac os 10.4.6, mysql 5.0.18, ruby 1.8.4,
rails 1.1,
ferret 0.9, acts_as_ferret from trunk. i removed all other plugins from
my app
in case they were conflicting, and still had the same problems.

here’s the rake output from the unit tests:

dewey:~/user/squirl/aaf_demo/aaf_demo john$ rake
(in /Users/john/user/squirl/aaf_demo/aaf_demo)
/opt/local/bin/ruby -Ilib:test
“/opt/local/lib/ruby/gems/1.8/gems/rake-0.7.0/lib/rake/rake_test_loader.rb”
“test/unit/comment_test.rb” “test/unit/content_test.rb”
Loaded suite
/opt/local/lib/ruby/gems/1.8/gems/rake-0.7.0/lib/rake/rake_test_loader
Started
…FFF.F.
Finished in 22.529063 seconds.

  1. Failure:
    test_id_multi_search(ContentTest) [./test/unit/content_test.rb:96]:
    <5> expected but was
    <6>.

  2. Failure:
    test_indexed_method(ContentTest) [./test/unit/content_test.rb:43]:
    <1> expected but was
    <0>.

  3. Failure:
    test_multi_index(ContentTest) [./test/unit/content_test.rb:67]:
    <5> expected but was
    <6>.

  4. Failure:
    test_multi_search(ContentTest) [./test/unit/content_test.rb:85]:
    <5> expected but was
    <6>.

14 tests, 70 assertions, 4 failures, 0 errors
/opt/local/bin/ruby -Ilib:test
“/opt/local/lib/ruby/gems/1.8/gems/rake-0.7.0/lib/rake/rake_test_loader.rb”
“test/functional/content_controller_test.rb”
Loaded suite
/opt/local/lib/ruby/gems/1.8/gems/rake-0.7.0/lib/rake/rake_test_loader
Started
…
Finished in 3.15368 seconds.

9 tests, 36 assertions, 0 failures, 0 errors
rake aborted!
Test failures

Quoting John McGrath [email protected]:

@res << Book.find_id_by_contents(params[:search][:search_term])
logger.debug(“hits is: #{hits.size}”)

Hi John,

:id, as described in that thread? i’ve poked around in the code, and had

–
webit! Gesellschaft für neue Medien mbH www.webit.de
Dipl.-Wirtschaftsingenieur Jens Krämer [email protected]
Schnorrstraße 76 Tel +49 351 46766 0
D-01069 Dresden Fax +49 351 46766 66

Ferret-talk mailing list
[email protected]
http://rubyforge.org/mailman/listinfo/ferret-talk

This message was sent using IMP, the Internet Messaging Program.

5

Hmmm, strange happenings are afoot …

Tell me, do you get the same results with ferret 0.3.2?

Kasper

John McGrath wrote:

get similar issues with the demo, i’m guessing it might have something
to do
with changes to ActiveRecord in Rails 1.1.

fwiw, my env details again: mac os 10.4.6, mysql 5.0.18, ruby 1.8.4,
rails 1.1,
ferret 0.9, acts_as_ferret from trunk. i removed all other plugins from
my app
in case they were conflicting, and still had the same problems.

6

Hi again,

I could reproduce these Results with Ferret 0.9 (Ruby-Version).

The C extension bails out with errors instead failures on the same
tests,
because the MultiReader class, which is used to search across indexes
is not accessible from outside of ferret.

Using Ferret 0.3.2, all tests except the test_indexed_method
(which is expected to fail for now and has nothing to do with the
problem here) pass both with Rails 1.0 and Rails 1.1.

All the failures happen on an OR query accross multiple Indexes
that returns more results than expected, simple one-term queries
run fine.

A test I added locally, that updates a single Object, saves it again and
checks if it’s still only one in the index runs fine in all scenarios
(Ferret 0.3.2, 0.9-ruby, 0.9-C, Rails 1.1, Linux).

To sum up:

  • it doesn’t seem to be a Rails issue.
  • Ferret 0.9/Ruby behaves somewhat different when using OR queries
    across multiple Indexes, but that should not be related to your
    problem.
  • tests creating and searching on a single index seem to pass in your
    environment (e.g. the content_controller_test)

very strange…

here’s a test method I just added to content_test.rb (can’t commit it
right now) that passes with all variations of Ferret, too.

def test_update
contents_from_ferret = Content.find_by_contents(‘useless’)
assert_equal 1, contents_from_ferret.size
assert_equal @content.id, contents_from_ferret.first.id
@content.description = ‘Updated description, still useless’
@content.save
contents_from_ferret = Content.find_by_contents(‘useless’)
assert_equal 1, contents_from_ferret.size
assert_equal @content.id, contents_from_ferret.first.id
contents_from_ferret = Content.find_by_contents(‘updated AND
description’)
assert_equal 1, contents_from_ferret.size
assert_equal @content.id, contents_from_ferret.first.id
contents_from_ferret = Content.find_by_contents(‘updated OR
description’)
assert_equal 1, contents_from_ferret.size
assert_equal @content.id, contents_from_ferret.first.id
end

hope we can solve this somehow…

Jens

On Thu, Apr 06, 2006 at 09:56:52AM +0200, Kasper W. wrote:

with changes to ActiveRecord in Rails 1.1.
Ferret-talk mailing list
[email protected]
http://rubyforge.org/mailman/listinfo/ferret-talk

–
webit! Gesellschaft für neue Medien mbH www.webit.de
Dipl.-Wirtschaftsingenieur Jens Krämer [email protected]
Schnorrstraße 76 Tel +49 351 46766 0
D-01069 Dresden Fax +49 351 46766 66