Caching application data

Rails
1

Hi,

say my application has a small 20 row information table in the database
that stores names of airlines:
“continental”, “delta”, united" , etc…

These names will be used in various pages of the application for all the
website users for different actions. They do not expect to change often.

what I do now is - when a user logs in - these airline names are queried
from the db and stored in session. Then every action that needs them
will get them from session instead of querying the database.

What I wanted to do is not query this table again and again for every
action. It just occured to me that i merely replaced a db query for a
session file access. Maybe a little faster in some cases for also not
the best way to go.

What’s the best solution for this? can I store this array of strings in
some global constant after fetching them from the database for the first
time when a user logs in? how? what will happen when I change the
contents of the table in the database every few weeks? (i guess i still
have this problem with sessions too, if a user is logged in while I make
the change)

Thanks!

2

Alon wrote:

Hi,

say my application has a small 20 row information table in the database
that stores names of airlines:
“continental”, “delta”, united" , etc…

These names will be used in various pages of the application for all the
website users for different actions. They do not expect to change often.

what I do now is - when a user logs in - these airline names are queried
from the db and stored in session. Then every action that needs them
will get them from session instead of querying the database.

What I wanted to do is not query this table again and again for every
action. It just occured to me that i merely replaced a db query for a
session file access. Maybe a little faster in some cases for also not
the best way to go.

What’s the best solution for this? can I store this array of strings in
some global constant after fetching them from the database for the first
time when a user logs in? how? what will happen when I change the
contents of the table in the database every few weeks? (i guess i still
have this problem with sessions too, if a user is logged in while I make
the change)

Thanks!

You could use class methods accessors to a class variable.

Simply fetch your data from the DB if the class variable is nil, and use
the var if it has a value. The trick is to reset the cached value to
nil after any change that might affect it.

class Airlines
def self.all_airlines
@@all_airlines ||= Airlines.find(:all)
end
def self.expire_airlines
@@all_airlines = nil
end
end

Airlines.all_airlines # calls DB
Airlines.all_airlines # uses already fetched @@all_airlined
Airlines.expire_airlines
Airlines.all_airlines # calls DB

3

Alex,

thanks a bunch! makes a lot of sense. Will use your suggestion.

Alon.

4

Alon wrote:

Hmm… seems to be querying the database every single time for some
reason…

this is how it looks like (categories instead of airlines in this
example)

class Category < ActiveRecord::Base

def self.get_categories
@@all_categories ||= find(:all, :order => “name”)
end

def self.expire_categories
@@all_categories = nil
end
end

in some controller

@cat = Category.get_categories # this statement always ends up in a db
query

expire_categories is not used yet in my code so nobody is setting it to
nil

I think the best thing to do is at the start of each day, the first
person to access the database dumps that information into an xml file.
From then on you only need to read the file.

This approch will limit db access quite a bit.

You might want to name the file after todays date so if the file does
not exist, then we can assume its time for a new query.

This may not seem like an elegant approach but it is when you consider
the load your placing elsewhere.

5

CatLady [] wrote:

I think the best thing to do is at the start of each day, the first
person to access the database dumps that information into an xml file.
From then on you only need to read the file.

This approch will limit db access quite a bit.

You might want to name the file after todays date so if the file does
not exist, then we can assume its time for a new query.

This may not seem like an elegant approach but it is when you consider
the load your placing elsewhere.

I don’t know if I would recommend reading from the file system as a
better approach to reading from the DB. If you are going to dump into a
file I would suggest YAML, it’s extremely easy to marshall/unmarshall
ruby objects into/out of it. It’s all built in, nice and easy. Plus it’s
more lightweight.

With that said have you thought of using something like Memcached? It
was built for exactly this purpose. It’s extremely efficient, and has
been proven successful time and time again. It’s extremely to hook up to
ruby/rails as well. I use it everyday and swear by it.

If you can’t hook up Memcached, then you could create a simple Singleton
class and cache the data in there. Pretty simple and straightforward,
and it works. Very lightweight.

6

Thanks catlady and Mark - very good suggestions and very appreciated.

If you can’t hook up Memcached, then you could create a simple Singleton
class and cache the data in there. Pretty simple and straightforward,
and it works. Very lightweight.

Yes, singleton class is my preference actually, as suggested by Alex.
However, as I indicated in my last posting above, it doesn’t seem to
work for me - the application works but it seems to be accessing the
database every single time for the data… I am probably doing something
wrong - I’ll appreciate if somebody can see if it’s something obvious
(short code snipet above)

Alon.

7

Hmm… seems to be querying the database every single time for some
reason…

this is how it looks like (categories instead of airlines in this
example)

class Category < ActiveRecord::Base

def self.get_categories
@@all_categories ||= find(:all, :order => “name”)
end

def self.expire_categories
@@all_categories = nil
end
end

in some controller

@cat = Category.get_categories # this statement always ends up in a db
query

expire_categories is not used yet in my code so nobody is setting it to
nil

8

Alon wrote:

Thanks catlady and Mark - very good suggestions and very appreciated.
Yes, singleton class is my preference actually, as suggested by Alex.
However, as I indicated in my last posting above, it doesn’t seem to
work for me - the application works but it seems to be accessing the
database every single time for the data… I am probably doing something
wrong - I’ll appreciate if somebody can see if it’s something obvious
(short code snipet above)

Alon.

The reason it wasn’t working for you was because you weren’t using a
singleton. Here’s a very very simple singleton cache

require ‘singleton’
class Cache
include Singleton
attr_reader :cache

def initialize
@cache = {}
end

def get(key)
self.cache[key.to_s.downcase]
end

def put(key, value)
self.cache[key.to_s.downcase] = value
value
end

def delete (key)
self.cache.delete(key.to_s.downcase)
end

def reset
self.cache.clear
end

end

c = Cache.instance
puts c.get(:foo)
c.put(:foo, “hello”)
3.times { puts c.get(:foo) }
c.delete(:foo)
puts c.get(:foo)

If you run that you’ll see when it does the loop it keeps returning
“hello” during the loop block. You can start getting clever with this
sort of thing and have the get method yield to a block if it can’t find
the key, etc… Anyway, you get the idea.

Also, I don’t know if you were running in development mode, but if you
were remember that AR models and controllers get reloaded ever single
time you make a request. So if you were expecting them to hold some sort
of state, apart from sessions in the controller, then that might be part
of your problem as well.

What I’ve outlined briefly above will hold state for the entire length
of your application. You can’t kill a singleton until you kill the ruby
process.

Anyway, hope this helps.

9

Thanks again Mark.

Also, I don’t know if you were running in development mode, but if you
were remember that AR models and controllers get reloaded ever single
time you make a request. So if you were expecting them to hold some sort
of state, apart from sessions in the controller, then that might be part
of your problem as well.

That’s exactly what happened. I actually suspected it to be the case,
however for some reason in the past my models were not getting loaded
every time in development mode, but only the views and controllers. I
discovered this because most changes I was making to my model classes
were not taking effect unless I restarted webricks. I actually just saw
some people’s posts about it and somebody included a link to an erratta
for David’s Agile book with an indication of this bug.

Having said that - it appear to have been loaded every single time in
this case, which confuses me, but most importantly my problem is solved
so thanks a lot to everybody!

Alon.

10

Alon wrote:

end
expire_categories is not used yet in my code so nobody is setting it to
nil

–
Posted via http://www.ruby-forum.com/.

In development mode, the models get reloaded on each request (IIRC), so
the value of @@all_categories gets reset every time you call the page.

_Kevin

11

Serialization in either a YAML file or maybe as ruby source code that
can be eval’d seem to be the most obvious solutions to that problem, to
me. Another idea that you might try is putting all that data in a
value-object class (not unlike TimeZone).

12

Use memcached, dump it onto that. Really fast.

Vish

13

Hi~

On Nov 6, 2006, at 5:37 AM, Mark B. wrote:

def initialize
end

find
What I’ve outlined briefly above will hold state for the entire length
of your application. You can’t kill a singleton until you kill the
ruby
process.

Anyway, hope this helps.

While this may seem like the solution it is not. This will not work

when you have multiple mongrels or fcgi’s running for your app.
Because each process will have its own copy of the singleton cache
and your users are not garaunteed to always hit the same backend on
each request> So the data in the cache will be inconsistent depending
on which backend serves a certain request.

The only way to do this correctly is to make it a drb server or just

use memcached. The singleton cache stuff is great if you can
garauntee that your app will never use more then one backend. But
that will not scale.

Cheers-

– Ezra Z.
– Lead Rails Evangelist
– [email protected]
– Engine Y., Serious Rails Hosting
– (866) 518-YARD (9273)