Forum: Ruby on Rails Performance suggestions or best practices ideas?

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
26ca59adf1da8aabe0387889f7f8c994?d=identicon&s=25 Mike Lopke (mlopke)
on 2006-05-03 00:49
Any suggestions on applications that involve alot of calcualtions on a
fairly large data set?

My app uses a set of raw data ~5k, applies some default/override rules
on the raw data and does some calculations on the data in combination
with a list of assumptions.  A ranked list along with detailed metrics
is generated.  The end user can manipulate some of the rules and
assumptions to generate different metrics for comparison.

My current approach maintains the original data set, default/overrides
and assumptions as seperate models and does all the ranking calculations
on the fly.      The performance is a bit sluggish if I'm looking at
more that 400 objects and this is with only one user.

Any suggestions on caching options to explore?  Should I consider using
a seperate model for storing the results?  This would make the ranking
and comparisons quicker, but I'm concerned about the overhead of having
to write large amount of temporary data to the database.

Thanks,
Mike
3bb23e7770680ea44a2d79e6d10daaed?d=identicon&s=25 M. Edward (Ed) Borasky (Guest)
on 2006-05-03 19:06
(Received via mailing list)
Could you do similar calculations in, say, Microsoft Excel? How does the
performance of your application compare with that of Excel? If Excel is
a *lot* faster, I'd say go hunting for tuning opportunities. But if
they're comparable, and scale roughly the same way with increasing
problem size, you're probably not going to be able to squeeze much
performance out without a better algorithm.

What sort of calculations are they? What is the 5K ... numbers, rows of
a database table ...??

Mike Lopke wrote:
> and assumptions as seperate models and does all the ranking calculations
>
>
>
>
>
>
>
>

--
M. Edward (Ed) Borasky

http://linuxcapacityplanning.com
26ca59adf1da8aabe0387889f7f8c994?d=identicon&s=25 Mike Lopke (mlopke)
on 2006-05-03 20:36
The app looks at potential investments.  Raw data is 5K rows of mixed
info.  ( numbers, categories, dates, etc. )

The calculations are on par with Excel.  I don't think there is alot of
room for improvement on the calculation algorithm.  I was looking for a
way to avoid the expense of recalculating the same set of metrics.  The
end user goes through these steps:

1.  Set initial query paramters ( $to invest, investment type, etc. )
2.  App gets raw data, combines with overrides/defaults/assumptions &
does metrics calculation to generate ranked results
3.  User looks at ranked results
4.  User looks at details for particular investment
5.  User returns to ranked results to investigate other opportunities
..

I wanted to cache the ranked results rather than have to regenerate it
each time I return to the list.  Is there a way to have a persistant
object in memory or is my only option to write the data to a table.  A
new ranked results would be generated for each new initial query mod or
a change in the overrides/defaults/assumptions.

Thanks,
Mike
3b330e8ad92a811462ce777b10651c4f?d=identicon&s=25 Isaac Reuben (Guest)
on 2006-05-03 21:48
(Received via mailing list)
On 5/3/06, Mike Lopke <reglopke@cableone.net> wrote:
> Is there a way to have a persistant
> object in memory or is my only option to write the data to a table.

You could store an array of the ranking scores and id's of the
investment options in the session[] object.  That way they would
persist between pages.

- Isaac
26ca59adf1da8aabe0387889f7f8c994?d=identicon&s=25 Mike Lopke (mlopke)
on 2006-05-03 22:22
Isaac Reuben wrote:
> On 5/3/06, Mike Lopke <reglopke@cableone.net> wrote:
>> Is there a way to have a persistant
>> object in memory or is my only option to write the data to a table.
>
> You could store an array of the ranking scores and id's of the
> investment options in the session[] object.  That way they would
> persist between pages.
>
> - Isaac

Thanks.

I've considered using the session[] object.  This, however, brings up an
interesting question.  What is the practical limit on what can be stored
in it?  Lets say I have 500 investments I'm considering and each has 20
metrics I've calculated.  My hunch is that it would cause problems if I
tried storing this in the session but I don't really know.

Mike
6076c22b65b36f5d75c30bdcfb2fda85?d=identicon&s=25 Ezra Zygmuntowicz (Guest)
on 2006-05-03 23:37
(Received via mailing list)
On May 3, 2006, at 1:22 PM, Mike Lopke wrote:

>
> tried storing this in the session but I don't really know.
>
> Mike


	For something like this you might want to look at memcached and Eric
Hodels CachedModel extension.

http://dev.robotcoop.com/Libraries/cached_model/fi...

	Or you can setup a small drb server to hold this info and stored and
retrieve the results.

Cheers-
-Ezra
26ca59adf1da8aabe0387889f7f8c994?d=identicon&s=25 Mike Lopke (mlopke)
on 2006-05-05 23:54
> 	For something like this you might want to look at memcached and Eric
> Hodels CachedModel extension.
>
> http://dev.robotcoop.com/Libraries/cached_model/fi...
>
> 	Or you can setup a small drb server to hold this info and stored and
> retrieve the results.
>
> Cheers-
> -Ezra

Thanks.  I'll give it a look.
This topic is locked and can not be replied to.