Forum: Ruby on Rails pagination with 500000 items

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
Luma (Guest)
on 2007-07-29 13:39
(Received via mailing list)
hi

I've got a pagination with about 500000 items, and I don't know if it
will be slow. I need only the first 1000, so what I'm trying to do is
@item_pages, @items = paginate(:products, :order => "title", :per_page
=> 15, :limit => 1000)

But paginate seems to have no limit option. Will the pagination be
fast without this option?

Luma
Adam G. (Guest)
on 2007-07-29 14:29
Luma wrote:
> But paginate seems to have no limit option. Will the pagination be
> fast without this option?

Pagination works by running two queries, one that returns a count of the
total number of results, and one that returns the requested page of data
only. This second query returning the page of data already uses LIMIT
and OFFSET in the SQL query, so there is no need for an additional limit
option.

Cheers,

Adam
Lon B. (Guest)
on 2007-07-29 14:31
(Received via mailing list)
In reality no user of your application will page through 500,000 items.

If you only need the first 1,000 items, why not limit the results to
1,000
and paginate those in viewable chunks?

You can use a combination of tagging and searching to help users get the
"right" 1,000 records to look through.
Roderick v. (Guest)
on 2007-07-29 14:55
Luma wrote:
> But paginate seems to have no limit option. Will the pagination be
> fast without this option?

The built-in paginator is a known performance and scalability bottleneck
and is likely to be removed from the Rails core in upcoming releases.
Luckily, there's a couple of very good plugins out there that've tackled
the issue: paginating_find [1] and will_paginate [2].

[1] http://cardboardrocket.com/pages/paginating_find
[2] http://errtheblog.com/post/929

--
Roderick van Domburg
http://www.nedforce.nl
toby privett (Guest)
on 2007-07-30 13:35
(Received via mailing list)
> I've got a pagination with about 500000 items, and I don't know if it
> will be slow. I need only the first 1000, so what I'm trying to do is
> @item_pages, @items = paginate(:products, :order => "title", :per_page
> => 15, :limit => 1000)

The built in rails pagination will explode with that many items. I
strongly recomment find_by_sql with a :limit parameter.

I recently used some the info here:
http://www.igvita.com/blog/2006/09/22/eager-find-b...
along with the paginating_find plugin from here:
http://cardboardrocket.com/pages/paginating_find

Workign well with 100s of 1000s or records
Pratik N. (Guest)
on 2007-07-30 13:37
(Received via mailing list)
Just use will_paginate and not look back.

On 7/29/07, Luma <removed_email_address@domain.invalid> wrote:
> fast without this option?
>
> Luma
>
>
> >
>


--
Cheers!
- Pratik
http://m.onkey.org
Jet (Guest)
on 2007-07-31 03:20
(Received via mailing list)
I second that..
The Paginator in rails has an issue with getting all the records for
you..
We use paginating_find plugin here...works good so far.

cheers...

On Jul 29, 8:55 pm, Roderick van Domburg <rails-mailing-l...@andreas-
Rick O. (Guest)
on 2007-07-31 05:18
(Received via mailing list)
> The built-in paginator is a known performance and scalability bottleneck
> and is likely to be removed from the Rails core in upcoming releases.
> Luckily, there's a couple of very good plugins out there that've tackled
> the issue: paginating_find [1] and will_paginate [2].

It's gone in edge rails, though there's a plugin for backwards
compatibility purposes.

--
Rick O.
http://lighthouseapp.com
http://weblog.techno-weenie.net
http://mephistoblog.com
Camo (Guest)
on 2007-07-31 05:21
(Received via mailing list)
Watch this http://railscasts.com/episodes/51

Will explains things for you
Luma (Guest)
on 2007-07-31 13:59
(Received via mailing list)
thanks for that many answers. I'll try the plugins.
tonypm (Guest)
on 2007-08-04 17:04
(Received via mailing list)
Hi,

thanks all for the pagination tips, and thanks too for pointing out
the RailsCasts.   For some reason I had not picked up on these yet,
but have just been and had a look and they are really cool.


Tonypm
This topic is locked and can not be replied to.