I’m wondering if this is a typical thing.
What I’m doing is getting about 5000+ records from a database and
displaying all of it on a page.
I’ve used the benchmark helper to benchmark the views for that page
and that takes about 25 seconds to render that view. Is this pretty
typical?
What I’m doing is using find_by_sql to get all the values that I need
from the tables and saving that to an instance variable in the
controller. Then I iterate it through that instance variable using
.each_with_index (I need the index value as well) to render the View.
The view is just a table. So I’m just rendering
| @foo.bar | ...
Nothing complicated.
And that’s taking about 25 sec according to the benchmark helper.
Is there anything I’m doing in the algorithm that’s wrong? Is there
any way to speed things up? I’m getting pressure from the higher ups
saying that 25 seconds isn’t acceptable. They need something like 6
seconds.
The reason I use find_by_sql is so that I don’t hit the database
through each iteration. And I figure that’ll help in the speed. Also,
I tested the find_by_sql SQL query I wrote and checked the speed on
that. That’s not the issue (it took about 0.2 seconds for mysql to
dump all the data).
What I’m doing is getting about 5000+ records from a database and
displaying all of it on a page.
I would start by questioning why you would need to present 5K+ objects
on a single page. It would there are better ways to present massive
amounts of records than just to dump them all on one screen.
But if you must, I’d encourage you to go straight to the metal. Grab
the connection (ActiveRecord::Base.connection), then use select_all to
get hashes back instead of AR objects. If that’s not fast enough,
investigate what methods your particular database adapter has for
returning records as arrays instead of as hashes.
David Heinemeier H.
http://www.loudthinking.com – Broadcasting Brain
http://www.basecamphq.com – Online project management
http://www.backpackit.com – Personal information manager
http://www.rubyonrails.com – Web-application framework
On Jun 2, 2006, at 16:59, Marston A. wrote:
I’m wondering the same thing, so it will be great to have peoples
input
on this. Personally I have something similar setup but with much less
data than you, < 100 rows and iterating via a table. My page actually
takes 6 seconds to load which I think is way to long.
Less than 100 rows, that’s sounds strange. Can you reproduce that in
a minimal example?
– fxn
David Heinemeier H. wrote:
What I’m doing is getting about 5000+ records from a database and
displaying all of it on a page.
I would start by questioning why you would need to present 5K+ objects
on a single page. It would there are better ways to present massive
amounts of records than just to dump them all on one screen.
Well who know what and how PHBs think. I personally think they’re only
going to show 100 at a time, but they want it optimised enough so that
5000+ records can render in about 6-7 seconds. Or maybe they do want to
show 5000+ at the same time for some crazy reason.
We are a Java shop and we have application servers running that can
pretty much do that (display 5000+ records in a very short amount of
time). So I’m guess they want something comparable.
But if you must, I’d encourage you to go straight to the metal. Grab
the connection (ActiveRecord::Base.connection), then use select_all to
get hashes back instead of AR objects. If that’s not fast enough,
investigate what methods your particular database adapter has for
returning records as arrays instead of as hashes.
Wow, that worked really well. The benchmarks for rendering the view came
down from 25 sec to about 7 seconds. That’s crazy. Thanks David.
I’m wondering the same thing, so it will be great to have peoples input
on this. Personally I have something similar setup but with much less
data than you, < 100 rows and iterating via a table. My page actually
takes 6 seconds to load which I think is way to long.