Strange search result with conditions in find_by_contents

Ruby Apps Ferret
1

Hi, guys:
Strange search result with conditions in find_by_contents!
first of all, i’ve installed the acts_as_ferret to my project vender
folder by ‘ruby script/plugin install
svn://projects.jkraemer.net/acts_as_ferret/tags/stable/acts_as_ferret’

in my SearchController
def searchforum
if !params[:doSearch].nil?
if params[:searchTerms].nil? || params[:searchTerms] == “”
flash[:notice] = ‘Please enter some words to search on.’
else
@conditions = " 1 = 1";
if !params[:dateRange].nil? && params[:dateRange] != “”
@conditions += " and creationDate >= " + params[:dateRange]
end
if !params[:forumID].nil? && params[:forumID] != “”
@conditions += " and forum_id = " + params[:forumID]
end

@total, @topics = Topic.full_text_search(params[:searchTerms], :page

=> (params[:page]||1))
@total, @topics = Topic.full_text_search(params[:searchTerms], {:page =>
(params[:page]||1)}, {:conditions => @conditions})
@pages = pages_for(@total)
end
end
end
in my model Topic:
def self.full_text_search(q, options = {}, find_options = {})
return nil if q.nil? or q==""
default_options = {:limit => 10, :page => 1}
options = default_options.merge options

get the offset based on what page we’re on

options[:offset] = options[:limit] * (options.delete(:page).to_i-1)

now do the query with our options

results = Topic.find_by_contents(q, options, find_options)
puts options
puts find_options
return [results.total_hits, results]
end

i display 10 results in one page. In SearchController, if i run this
code:
@total, @topics = Topic.full_text_search(params[:searchTerms], :page =>
(params[:page]||1))
and input a search term, it will get 16 result messages. when i click to
the next page, it display the next 6 results, and the total_hits is
still 16.

but if i run this code in SearchController:
@total, @topics = Topic.full_text_search(params[:searchTerms], {:page =>
(params[:page]||1)}, {:conditions => @conditions})
and input the same search term, it will also get 16 result messages. but
when i click to the next page, it display the next 6 results, and the
total_hits changes to 6 messages, and only display one page.

speak in shortly: searching with conditions in find_by_contents, it got
16 total_hits, it display the first 10 result messages in first page,
when i click to the next page, it display the next 6 results, but the
total_hits changes to 6, and only got one page to display.

i don’t know why.

2

and in the ApplicationController
def pages_for(size, options = {})
default_options = {:per_page => 10}
options = default_options.merge options
pages = Paginator.new self, size, options[:per_page],
(params[:page]||1)
return pages
end

thanks in advance!

3

Hi!
On Wed, Jul 18, 2007 at 05:41:46PM +0200, Chengcai He wrote:
[…]

speak in shortly: searching with conditions in find_by_contents, it got
16 total_hits, it display the first 10 result messages in first page,
when i click to the next page, it display the next 6 results, but the
total_hits changes to 6, and only got one page to display.

The reason for the odd behaviour you experience is as follows:
Aaf first queries ferret for your search results, it gets a
number of ids matching your query (and your paging). Now it uses this
set of ids to run a query against the DB, combining it with your own
conditions.

So in the end there is no simple way for aaf to determine the total
number of hits (without running the query more than once).

Additionally it might happen that you get less results than expected,
since Ferret is delivering :limit hits, but these might well be cut down
to less than :limit by your active record conditions.

There are several ways to work around this:

  • Get the total_hits separately by calling Model.find_by_contents
    without the paging options. Doesn’t solve the second problem I
    mentioned above.

  • Use :limit and :offset as part of your find_options, so the paging
    takes place in the database. Might work okay, didn’t ever try it.

  • Put the additional data in your index, too, and do not use
    active_record conditions at all. Just append your additional query
    criteria to the query entered by the user.

  • If you find a way how aaf could deliver consistent results in this
    scenario with any other method I can’t think of right now, submit a
    patch :slight_smile:

Btw, do never ever directly append request parameters to conditions
like this:

@conditions = " 1 = 1";
if !params[:dateRange].nil? && params[:dateRange] != “”
@conditions += " and creationDate >= " + params[:dateRange]
end
if !params[:forumID].nil? && params[:forumID] != “”
@conditions += " and forum_id = " + params[:forumID]
end

but use something like

conditions = [ ‘creationDate >= ? AND forum_id = ?’,
params[:dateRange], forum_id ]

to avoid exploitation of your app via sql injection. In newer Rails
there’s a Hash based notation for the conditions, too (that might be
easier to assemble in your case).

cheers,
Jens

–
Jens Krämer
http://www.jkraemer.net/ - Blog
http://www.omdb.org/ - The new free film database

4

one more question:
in the Model
acts_as_ferret({ :fields => {:username => {:store => :yes, :boost =>
30}, :subject => {:store => :yes, :boost => 20}, :body => {:store =>
:yes, :boost => 10}}, :remote => true }, { :analyzer =>
Ferret::Analysis::RegExpAnalyzer.new(/./, false) })

def username
return self.user.login
end

search as:
@total, @topics = Topic.full_text_search(params[:searchTerms], {:page =>
(params[:page]||1)}, {:conditions => @conditions})

if i input a query term to search, i will got the result: @total = 14,
@topics.length = 12
but when i input the user login to search the topics posted by the user,
it got:
@total = 14, @topics.length = 0. It just display the paginate things,
but none of the result topic displayed!

I’m totally lost!

5

search terms @total @topics.length
激流中国 12 12
中国 14 12
hecc 16 0

‘hecc’ is the user login name.

6

On Thu, Jul 19, 2007 at 05:40:45AM +0200, Chengcai He wrote:

I’m totally lost!

Look at your log files, see what queries ferret runs against the index,
and see what sql statements are issued to your DB. I guess without the
conditions everything is ok?

cheers,
Jens

–
Jens Krämer
webit! Gesellschaft für neue Medien mbH
Schnorrstraße 76 | 01069 Dresden
Telefon +49 351 46766-0 | Telefax +49 351 46766-66
[email protected] | www.webit.de

Amtsgericht Dresden | HRB 15422
GF Sven Haubold, Hagen Malessa