Forum: Ruby on Rails filter in memory vs. hit the database again?

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
8e7e5b934fcbcdfb475072685755457f?d=identicon&s=25 Taylor Strait (taylorstrait)
on 2009-01-10 20:44
Is it generally preferred to filter a large collection of objects in
memory or in the database. In the case of a single query, I know that
the database is preferred.  But what about multiple filtering based on
atributes?  For example,

    @users = User.find(:all)
      @males = @users.find_all {|user| user if user.gender == 'male'}
      @females = @users.find_all {|user| user if user.gender ==
'female'}

Then these are filtered AGAIN in the view:

<tr>
  <td>males</td>
  <td><%= @males.find_all{|user| user.ethnicity == 'white'}.size %></td>
  <td><%= @males.find_all{|user| user.ethnicity == 'black'}.size %></td>
  <td><%= @males.find_all{|user| user.ethnicity == 'asian'}.size %></td>
  <td><%= @males.find_all{|user| user.ethnicity == 'hispanic'}.size
%></td>
  <td><%= @males.find_all{|user| user.ethnicity == 'other'}.size %></td>
</tr>

My question is - should i hit the database separately for each of these
groups and subgroups, or filter in memory as I am doing now?  Eventually
there could be 100,000+ records.
81b61875e41eaa58887543635d556fca?d=identicon&s=25 Frederick Cheung (Guest)
on 2009-01-10 22:41
(Received via mailing list)
Taylor Strait wrote:
> Then these are filtered AGAIN in the view:
>
> My question is - should i hit the database separately for each of these
> groups and subgroups, or filter in memory as I am doing now?  Eventually
> there could be 100,000+ records.

First a general and unhelpful answer: try both (maybe fake up some
data to see what happens as the database groups) and see what happens.

In this case you're only (at least what you've shown) using the number
of people of the various types, so you could do something like

User.male.count :group => 'ethnicity'
(assuming you had a male named scope doing the obvious) which will get
you the counts by ethnicity

Fred
8e7e5b934fcbcdfb475072685755457f?d=identicon&s=25 Taylor Strait (taylorstrait)
on 2009-01-12 23:54
Frederick Cheung wrote:

> In this case you're only (at least what you've shown) using the number
> of people of the various types, so you could do something like
>
> User.male.count :group => 'ethnicity'
> (assuming you had a male named scope doing the obvious) which will get
> you the counts by ethnicity
>
> Fred

Thanks.  I will give that a try.
This topic is locked and can not be replied to.