Is there a way to do incremental search?

Ruby Apps Ferret
#1

Say the user first enters “ruby” for search and gets 1000 results. Then
he can search “rails” just in the 1000 results just returned.

The common scenario is some kind of advanced search. User can
incrementally add criteria and the program will narrow the results step
by step.

I know that at least I can use all the criteria as a whole to do the
searching, but this is a waste and I’m hoping there is a better way.

#2

On Thu, May 10, 2007 at 09:12:19AM +0200, Allen Young wrote:

Say the user first enters “ruby” for search and gets 1000 results. Then
he can search “rails” just in the 1000 results just returned.

The common scenario is some kind of advanced search. User can
incrementally add criteria and the program will narrow the results step
by step.

I know that at least I can use all the criteria as a whole to do the
searching, but this is a waste and I’m hoping there is a better way.

Given Ferret’s speed this is no waste, just try it.

Jens

–
Jens Krämer
webit! Gesellschaft für neue Medien mbH
Schnorrstraße 76 | 01069 Dresden
Telefon +49 351 46766-0 | Telefax +49 351 46766-66
[email protected] | www.webit.de

Amtsgericht Dresden | HRB 15422
GF Sven Haubold, Hagen Malessa

#3

Jens K. wrote:

Given Ferret’s speed this is no waste, just try it.

But what if I put a bunch of joins and conditions in find_options? Will
it be still fast? I’m implementing search feature for material industry.
You know that how many properties a material may have, or just think
about a search according to the chemistry composition of the material,
so many chemistry elements.

Allen

#4

On Thu, May 10, 2007 at 10:05:13AM +0200, Allen Young wrote:

Jens K. wrote:

Given Ferret’s speed this is no waste, just try it.

But what if I put a bunch of joins and conditions in find_options? Will
it be still fast? I’m implementing search feature for material industry.
You know that how many properties a material may have, or just think
about a search according to the chemistry composition of the material,
so many chemistry elements.

You should try to use ferret instead of your DB as much as possible.

Joins and conditions are applied after the ferret search to further
narrow down the result set and, if speed is an issue, should only be
used for things that really can’t go into the index, i.e. checking for
user permissions.

It may be possible to gain some additional speed by using the results of
a previous query as some kind of scope for the next one. I.e. you could
keep the ids of your result set and use them as a base for your next
query. However Ferret’s API does not directly support incremental
queries, you’d have to implement this yourself. Not impossible but imho
you should only do this once you have ‘real’ data so you can measure
what you gain by optimizing things.

Jens

–
Jens Krämer
webit! Gesellschaft für neue Medien mbH
Schnorrstraße 76 | 01069 Dresden
Telefon +49 351 46766-0 | Telefax +49 351 46766-66
[email protected] | www.webit.de

Amtsgericht Dresden | HRB 15422
GF Sven Haubold, Hagen Malessa

#5

Jens K. wrote:

You should try to use ferret instead of your DB as much as possible.

How can I use search a decimal type attribute using ferret? For example,
I want all materials whose description contains “ruby” and resistance is
between 0.1 and 0.5? I have a lot of decimal and integer type attributes
which I don’t know how to index with ferret. Any advice about this?

allen

#6

On Thu, May 10, 2007 at 11:43:22AM +0200, Allen Young wrote:

Jens K. wrote:

You should try to use ferret instead of your DB as much as possible.

How can I use search a decimal type attribute using ferret? For example,
I want all materials whose description contains “ruby” and resistance is
between 0.1 and 0.5? I have a lot of decimal and integer type attributes
which I don’t know how to index with ferret. Any advice about this?

Use untokenized fields for such values, and normalize them to a fixed
length before indexing. I’d also normalize the decimals to integers.

Regarding your incremental queries, the QueryFilter class might be
useful:
http://ferret.davebalmain.com/api/classes/Ferret/Search/QueryFilter.html

Jens

–
Jens Krämer
webit! Gesellschaft für neue Medien mbH
Schnorrstraße 76 | 01069 Dresden
Telefon +49 351 46766-0 | Telefax +49 351 46766-66
[email protected] | www.webit.de

Amtsgericht Dresden | HRB 15422
GF Sven Haubold, Hagen Malessa

#7

Jens K. wrote:

Use untokenized fields for such values, and normalize them to a fixed
length before indexing. I’d also normalize the decimals to integers.

And how should I construct the query string? “ruby 0.1 <= resistance <=
0.5”?

Regarding your incremental queries, the QueryFilter class might be
useful:
http://ferret.davebalmain.com/api/classes/Ferret/Search/QueryFilter.html
I’ll check this out. Thanks a lot.

allen

#8

On Thu, May 10, 2007 at 12:21:51PM +0200, Allen Young wrote:

Allen Young wrote:

And how should I construct the query string? “ruby 0.1 <= resistance <=
0.5”?
Sorry, I’ve just realized this is a stupid question, I should use
“:description(ruby) :resistance(>=0.1)”.

you can also construct your query objects manually, in this case check
out RangeQuery:
http://ferret.davebalmain.com/api/classes/Ferret/Search/RangeQuery.html

Another question. What if resistance attribute is not in the materials
table but in some other table which has a one-to-one relationship with
materials table?

define a method that retrieves the value and add the method’s name as a
field’s name to your call to acts_as_ferret. You might want to search
the list for ‘indexed method’ for an example of this.

Jens

–
Jens Krämer
webit! Gesellschaft für neue Medien mbH
Schnorrstraße 76 | 01069 Dresden
Telefon +49 351 46766-0 | Telefax +49 351 46766-66
[email protected] | www.webit.de

Amtsgericht Dresden | HRB 15422
GF Sven Haubold, Hagen Malessa

#9

Allen Young wrote:

And how should I construct the query string? “ruby 0.1 <= resistance <=
0.5”?
Sorry, I’ve just realized this is a stupid question, I should use
“:description(ruby) :resistance(>=0.1)”.

Another question. What if resistance attribute is not in the materials
table but in some other table which has a one-to-one relationship with
materials table?

allen

#10

Jens K. wrote:

On Thu, May 10, 2007 at 12:21:51PM +0200, Allen Young wrote:

Allen Young wrote:
Another question. What if resistance attribute is not in the materials
table but in some other table which has a one-to-one relationship with
materials table?

define a method that retrieves the value and add the method’s name as a
field’s name to your call to acts_as_ferret. You might want to search
the list for ‘indexed method’ for an example of this.

There are about 100 attributes reside in several different tables. That
means I need to define all this methods manually?

#11

Jens K. wrote:

On Thu, May 10, 2007 at 12:48:15PM +0200, Allen Young wrote:

There are about 100 attributes reside in several different tables. That
means I need to define all this methods manually?

that depends - some metaprogramming might help make it a less daunting
task.

i.e.

class OtherClass

define which fields you want to have indexed and how:

def ferret_fields
{ :field1 => { :store => :yes }, … }
end
end

class MyModel

collect field list for aaf

ferret_fields = { :name => {}, … }
ferret_fields.update! OtherClass.ferret_fields

acts_as_ferret :fields => ferret_fields

define getters

OtherClass.ferret_fields.keys.each do |field|
define_method :“ferret_#{field}” do
I think this should be :"#{field}" or I must define ferret_feilds = {
:ferret_field1 } in OtherClass instead.
other_object.send(field)
end
end
end

It seems that define_method doesn’t work at all. I got many errors
saying that there is no method defined for things like ferret_field1. I
even tried to define a simple method dynamically with the following
code at the class level of my model class.

define_method :say_hello { puts ‘hello’ }

But unexpectedly, method_missing is called, so it means that
define_method doesn’t work. Is there anything wrong about my using with
define_method?

allen

#12

Allen Young wrote:

It seems that define_method doesn’t work at all. I got many errors
saying that there is no method defined for things like ferret_field1. I
even tried to define a simple method dynamically with the following
code at the class level of my model class.

define_method :say_hello { puts ‘hello’ }

But unexpectedly, method_missing is called, so it means that
define_method doesn’t work. Is there anything wrong about my using with
define_method?

Well, I got something. If I write define_method is this way:

define_method(:say_hello) { puts ‘hello’ }

then the “say_hello” method can be defined dynamically. Quite strange!
But I still cannot figure out why

OtherClass.ferret_fields.keys.each do |field|
define_method :“ferret_#{field}” do
other_object.send(field)
end
end

doesn’t work. Is this becouse define_method is in the context of
OtherClass and thus dynamically adds all the method to OtherClass?

allen

#13

On Mon, May 14, 2007 at 01:42:26PM +0200, Allen Young wrote:

define_method?

Well, I got something. If I write define_method is this way:

define_method(:say_hello) { puts ‘hello’ }

syntax weirdness …

OtherClass and thus dynamically adds all the method to OtherClass?
no, but you could try to add the parens there as well:
define_method(:“ferret_#{field}”) do
…

Jens

–
Jens Krämer
webit! Gesellschaft für neue Medien mbH
Schnorrstraße 76 | 01069 Dresden
Telefon +49 351 46766-0 | Telefax +49 351 46766-66
[email protected] | www.webit.de

Amtsgericht Dresden | HRB 15422
GF Sven Haubold, Hagen Malessa

#14

On Thu, May 10, 2007 at 12:48:15PM +0200, Allen Young wrote:

There are about 100 attributes reside in several different tables. That
means I need to define all this methods manually?

that depends - some metaprogramming might help make it a less daunting
task.

i.e.

class OtherClass

define which fields you want to have indexed and how:

def ferret_fields
{ :field1 => { :store => :yes }, … }
end
end

class MyModel

collect field list for aaf

ferret_fields = { :name => {}, … }
ferret_fields.update! OtherClass.ferret_fields

acts_as_ferret :fields => ferret_fields

define getters

OtherClass.ferret_fields.keys.each do |field|
define_method :“ferret_#{field}” do
other_object.send(field)
end
end
end

you can also join various (textual) attributes together and let them
form a single field in the index where this is appropriate.

Indexing data from a lot of relationships is not trivial when it comes
to updating the index - whenever some record changes it’s parent
object(s) (that acts as the root and goes into the Ferret index first)
has to be re-indexed.

Jens

–
Jens Krämer
webit! Gesellschaft für neue Medien mbH
Schnorrstraße 76 | 01069 Dresden
Telefon +49 351 46766-0 | Telefax +49 351 46766-66
[email protected] | www.webit.de

Amtsgericht Dresden | HRB 15422
GF Sven Haubold, Hagen Malessa

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs