Forum: Ruby on Rails Large External Data Integration

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
26ca59adf1da8aabe0387889f7f8c994?d=identicon&s=25 Mike Lopke (mlopke)
on 2006-03-03 23:54
I'm looking for best practices / ideas on how to refresh large external
data sources.

What if your rails app relies on a large data set that you get via some
service and it needs to be updated periodically.  For instance, lets say
your rails app uses weather report and airline flight information.  You
can get both data sets via some feed mechanism and you would like to use
data that is no more than 1 hour old.

One approach on this would be to simply clear and reload tables
containing this data on a periodic basis, but I would not like to
interrupt service.  The next Idea I had was to have two sets of tables
and a switching mechanism to point to one side while the other is being
updated.

Any thoughts or ideas?  I'm also looking at using the Cron Plugin
managing this.

Thanks,
Mike
2073c7e0e7d2311727b680a887a375be?d=identicon&s=25 Leon Leslie (Guest)
on 2006-03-04 20:56
(Received via mailing list)
Use a version marker in the affected tables.
store the current version in another table maybe an application settings
table.
update with current version + 1
Once updated update current version.
On site only show current version.


On 3/3/06, Mike Lopke <reglopke@cableone.net> wrote:
> containing this data on a periodic basis, but I would not like to
> --
> Posted via http://www.ruby-forum.com/.
> _______________________________________________
> Rails mailing list
> Rails@lists.rubyonrails.org
> http://lists.rubyonrails.org/mailman/listinfo/rails
>


--
Never be afraid to try something new. Remember, amateurs built the
ark; professionals built the Titanic!
This topic is locked and can not be replied to.