Forum: Ruby on Rails How to get cursor / lazy semantics from find(:all)

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
Kevin S. (Guest)
on 2009-02-23 22:37
(Received via mailing list)
I'm trying to figure out it there's some alternative to find(:all)
that can avoid instantiating every object at once, when I know that
I'm just going to iterate over the collection and throw it away.  E.g.

def batch_process
  User.find(:all).each do |user|
    batch_process_one_user(user)
  end
end

Unfortunately, this may instantiate millions of User objects and use
up GB of memory, which is not acceptable.  I've been digging through
docs and code trying to figure out how I can do this within Rails,
rather than having to go down to the raw DB connection and build from
there, or pushing everything into a stored procedure.


Thanks,

-kevin
Frederick C. (Guest)
on 2009-02-23 22:38
(Received via mailing list)
On Feb 23, 8:28 pm, Kevin S. <removed_email_address@domain.invalid>
wrote:
> I'm trying to figure out it there's some alternative to find(:all)  
> that can avoid instantiating every object at once, when I know that  
> I'm just going to iterate over the collection and throw it away.  E.g.
>

http://github.com/rails/rails/commit/d13623ca46c82...
?

Fred
Kevin S. (Guest)
on 2009-02-24 00:20
(Received via mailing list)
Thanks!  That's close enough and easy to backport.  In principle, a
solution that did O(1) queries, rather than O(N) would be nice, but
the reduction in memory use at least gets me back to something
tolerable.

-kevin
This topic is locked and can not be replied to.