Does Rails persist in RAM between requests?

Rails
1

I’m trying to implement a data caching system and getting nowhere
because I cannot get a ruby global variable to persist across page
requests.*

This leads me to really wonder how the entire process of mongrel-

ruby->rails is working.

If I run a plain Ruby shell script, it starts, it ends, done.
Processing the file is the extent of the life cycle of the code. I
get that.

Is Rails essentially just a giant nested set of scripts with a single
start and finish life cycle as well?

Does Rails persist as an in-memory application even when not
responding to requests? If so, I am totally baffled by global
variables being re-initialized with each page request.

Perhaps understanding this process will help me see if what I am
trying to is even supposed to work or not.

– gw

2

On Oct 25, 2007, at 11:02 PM, Philip H. wrote:

Does Rails persist as an in-memory application even when not
responding to requests? If so, I am totally baffled by global
variables being re-initialized with each page request.

Yes.

OK, at least that point is settled. I figured it had to be, but…

Are you running in development mode? Development mode reloads an
awful
lot b/n each request and might be undoing whatever you’re trying to
persist.

Yep. Hmm. Y’think it would even wipe out global vars? I suppose it
could – I had a mechanism in my own framework ('nother language) to
do that very thing, but I could control turn it on/off at will rater
than tying it exclusively to a mode. Hmm… OK, I’ll experiment with
running is a production mode I gueess.

Thanks! At least it is something new to try.

– gw

3

Is Rails essentially just a giant nested set of scripts with a single
start and finish life cycle as well?

No.

Does Rails persist as an in-memory application even when not
responding to requests? If so, I am totally baffled by global
variables being re-initialized with each page request.

Yes.

Are you running in development mode? Development mode reloads an awful
lot b/n each request and might be undoing whatever you’re trying to
persist.

-philip

4

On Oct 25, 2007, at 8:53 PM, Greg W. wrote:

Yep. Hmm. Y’think it would even wipe out global vars? I suppose it
could – I had a mechanism in my own framework ('nother language) to
do that very thing, but I could control turn it on/off at will rater
than tying it exclusively to a mode. Hmm… OK, I’ll experiment with
running is a production mode I gueess.

Not likely dev mode is dorking around with initialization stuff. It
mostly tries to reload models to reflect code changes. Try this.
Start a new mongrel on another port so you can be sure you have a
real fresh copy with real fresh code. Put logging statements in your
caching code to determine the state of the variable you’re trying to
access. I always use RAILS_DEFAULT_LOGGER.debug but do as you see fit.

Tail your development.log to see what that reveals. If you are
totally stuck, try ruby-debug (DATANOISE.COM
2006/07/12/tutorial-on-ruby-debug) and set a breakpoint.

G’luck.

5

On 10/25/07, Greg W. [email protected] wrote:

I’m trying to implement a data caching system and getting nowhere
because I cannot get a ruby global variable to persist across page
requests.*

Can you look into how the session code works?

Is Rails essentially just a giant nested set of scripts with a single
start and finish life cycle as well?

Sort of.

Does Rails persist as an in-memory application even when not
responding to requests? If so, I am totally baffled by global
variables being re-initialized with each page request.

It depends.

For instance, web servers can be configured to run at most N requests
through a single process, then let it die. This is a common strategy
to “fixing” memory leaks.

So, your application might restart fully, or it might just reset a lot
of state. You can’t really rely on either behavior.

In development mode, it doesn’t always RELOAD everything, it just
checks the timestamps on a lot more files. For instance, in
production I’ve found view templates will automatically reload,
probably because they are actually re-read from disk each time.
Controllers and plug-ins will not, nor will any configuration files.
In development mode, controllers will reload if changed, but most
configuration will not.

–Michael

6

On Oct 25, 2007, at 9:02 PM, s.ross wrote:

Start a new mongrel on another port so you can be sure you have a
real fresh copy with real fresh code.

This yields some differences. Still some screwy behavior at times,
but getting further. Whatever is going on it seems to be some
condition induced by my attempts to stop/start everything maybe?

OK, I’ll hack at it for a while longer with these ideas.

Thanks for the fresh angle.

– gw

7

On Oct 25, 2007, at 8:40 PM, Greg W. wrote:

I’m trying to implement a data caching system and getting nowhere
because I cannot get a ruby global variable to persist across page
requests.*

Finally got somewhere with this one. Starting a new port generate
some different results. Still not perfect, but at least different.
So, I took that thoiughtto the extreme and just rebooted the whole
machine. Now it works.

Very freaky that two things didn’t work (global var across pages, and
added a var to the environment page), but everything else did work.
Wasted a lot of time on this one. I hope its not an omen!

Thanks everyone for poking at it with me. Hopefully tomorrow I can
make some real progress (the alleged Rails productivity meter is a
bit in the yellow zone after today :wink:

– gw

8

On 10/25/07, Greg W. [email protected] wrote:

Not likely dev mode is dorking around with initialization stuff. It
OK, I’ll hack at it for a while longer with these ideas.

Thanks for the fresh angle.

Have you tried to run it in production mode to see how the behavior
changes?

Pat

9

On 10/26/07, Greg W. [email protected] wrote:

I’m trying to implement a data caching system and getting nowhere
because I cannot get a ruby global variable to persist across page
requests.*

Do consider “memcached”.

Regards,
Isak

10

On 10/26/07, Isak H. [email protected] wrote:

On 10/26/07, Greg W. [email protected] wrote:

I’m trying to implement a data caching system and getting nowhere
because I cannot get a ruby global variable to persist across page
requests.*

Do consider “memcached”.

Never mind, I see from one of your other topics on this issue that
you’re already aware of memcached.

Data in a global/singleton class should persist fine across requests
to the same rails instance, but if you need to share the cache between
multiple rails processes, memcached is the way to go.

You could also use the file system, relying on the OS to keep your
data cached in memory, but that sounds awkward at best…

11

On 10/26/07, Greg W. [email protected] wrote:

[…] One thing about memecached on
Rails is that I don’t see how to use it for arbitrary data chunks.
Rails is obsessed with caching HTML, but I don’t see any provisions
for arbitrary data.

Regards,
Isak

12

On Oct 26, 2007, at 1:23 AM, Isak H. wrote:

On 10/26/07, Greg W. [email protected] wrote:

I’m trying to implement a data caching system and getting nowhere
because I cannot get a ruby global variable to persist across page
requests.*

Do consider “memcached”.

I will, but I just didn’t need the added complexity today* :slight_smile:

I did get my global var cache working, but unlike the prev
environment I worked with, I can see where this idea will be kinda
useless in Rails as each instance in production will be maintaining a
redundant copy of what I store.

However, I’m trying to learn a bunch of new stuff, so just did not
need to branch into memcached today. One thing about memecached on
Rails is that I don’t see how to use it for arbitrary data chunks.
Rails is obsessed with caching HTML, but I don’t see any provisions
for arbitrary data. I have stuff I want to load (from txt, db, even
web service) and pre-parse/munge it to a particular state, store it,
retrieve it in that state and finish the job based on other
application logic influences.

For that, I am accustomed to building my own RAM caching via globals,
but that’s in a laguage where the environment is truly global unlike
it would be in a mulit-process mongrel/Rails setup.

So I see memcached being in my future soon enough, but need more time
to read up on it.

Meanwhile I can move dev forward with my cheesy home-grown one for a
while.

  • I’ve been using Lasso for years now. It’s totally turn-key. All
    this stuff I see the Rails world cobbling together out of pieces of
    open-source pet projects, and spending who knows how many days and
    weeks it take to learn how to put it together, Lasso just has out of
    the box, ready to go. It’s not perfect, but edags the setup and
    hosting and all that is so much easier, and optimized for web apps.
    Oh, well.

– gw

13

What are you trying to store in a global variable? Why not put it into
the session? Maybe you can avoid it alltogether?

I’m saying this because I came to RoR from a background of many years of
client/server development where I was used to build up a lot of state in
the app server or even the client. And one of the first things I tried
was to store state in global variables. But that does not work because
of the stateless design of RoR. You simply cannot rely on how long your
app state lives in the server. Don’t confuse caching with app state.

Try to design your app stateless, that’s what I learned from Rails. And
it works beautifully.

14

On Oct 26, 2007, at 7:41 AM, Gernot K. wrote:

your
app state lives in the server. Don’t confuse caching with app state.

Try to design your app stateless, that’s what I learned from Rails.
And
it works beautifully.

This isn’t data that belongs in a session.

I use globals as a mini memcache (at least I did with Lasso, and now
I’m assuming I can recreate that with Ruby). It’s a hash of data
being stored in partially processed formats.

In my case I’ll have a particular global dedicated to caching a
particular type of data. The global will be created by the system
that needs it, when it needs it. This way I can build sub-systems
that create their own environment needs. If I need a db mapper, I
install that. It creates it’s own global for caching the various
cross reference structures created from mapping definitions.

Example: I load value list definitions from a text file (or db, or
even web service, source doesn’t matter). Many value lists are stored
in one file, or retrieved with one query. These definitions do not
change, they’re part of fixed options in the application design. I
lazy load the lists as they’re needed, parse each list into a data
structure suitable for passing as options to a renderer, then cache
each parsed list.

I have my own value list renderer class that provides some options
and application-integration capabilities different than the Rails
one. Some lists require on the fly additions or subtractions at
various points in the application (modular app hierarchy, or
permissions based adjustments). I can pull the base list from the
cache and modify it in the controller before passing a var with the
final list to the renderer in the view.

Also, I might go ahead and store a baseline HTML rendering of a value
list in the cache, and then use regex to alter its final state. This
is usually the fastest way to generate valuelists. On pages with a
handful of them it doesn’t matter so much. On pages with many of them
or, several long ones, it can make a difference.

I know Rails has some tools for this, but IMO they’re more cumbersome
than my system, and they simply don’t allow for some of the
flexibility I need.

My using globals tends to be for application config/reference, so
while using globals is good for creating machine-specific caches, I
wouldn’t use it for data where it would make more sense to use
something like memcached.

– gw

15

On Oct 26, 2007, at 4:11 AM, Isak H. wrote:

On 10/26/07, Greg W. [email protected] wrote:

[…] One thing about memecached on
Rails is that I don’t see how to use it for arbitrary data chunks.
Rails is obsessed with caching HTML, but I don’t see any provisions
for arbitrary data.

Serialization - Wikipedia

Sure, but in the context of Rails, Rails’ own cache APIs all seem to
be geared towards caching HTML or database queries. So if I generate
this chunk of data internal to my app (whether it is just text or an
object that could be serialized), I don’t see any Rails APIs for,
say, memecached that are for storing that particular chunk. From what
I can tell so far, Rails integration of memcached is as a sub-service
to its own higher level APIs for storing HTML and db queries.

So, there has to be some way to talk to memcached directly so I can
build my own higher level API for storing arbitrary data. I’m sure
it’s there somewhere, but I don’t see it documented clearly… yet.

– gw

16

Sure, but in the context of Rails, Rails’ own cache APIs all seem to
be geared towards caching HTML or database queries. So if I generate
this chunk of data internal to my app (whether it is just text or an
object that could be serialized), I don’t see any Rails APIs for,
say, memecached that are for storing that particular chunk. From what
I can tell so far, Rails integration of memcached is as a sub-service
to its own higher level APIs for storing HTML and db queries.

So, there has to be some way to talk to memcached directly so I can
build my own higher level API for storing arbitrary data. I’m sure
it’s there somewhere, but I don’t see it documented clearly… yet.

http://dev.robotcoop.com/Libraries/memcache-client/index.html

Download the memcache_util.rb file and look at it.

Once loaded you can do things like:

CACHE.set(“key”, “val”)
CACHE.get(“key”)

etc…

And you’re talking directly to memcache… no rails stuff…

-philip

17

On Oct 26, 2007, at 5:22 PM, Philip H. wrote:

Sure, but in the context of Rails, Rails’ own cache APIs all seem to

dev.robotcoop.com
Download the memcache_util.rb file and look at it.
Once loaded you can do things like:
CACHE.set(“key”, “val”)
CACHE.get(“key”)
And you’re talking directly to memcache… no rails stuff…

OK, cool. Does this replace the interface already in Rails? Or work
in parallel?

I will get to it, but probably not for a while. Trying to fry other
fish first.

– gw

18

OK, cool. Does this replace the interface already in Rails? Or work
in parallel?

Doesn’t interfer with Rails at all. memcache-client could be happily
used
in pure ruby land. Unless you configure the other memcache related
things, you won’t get any memcache auto-magic in rails.

19

On Oct 26, 2007, at 5:44 PM, Philip H. wrote:

OK, cool. Does this replace the interface already in Rails? Or work
in parallel?

Doesn’t interfer with Rails at all. memcache-client could be
happily used
in pure ruby land. Unless you configure the other memcache related
things, you won’t get any memcache auto-magic in rails.

Good. sounds like a better place to start for me – work w/ it
directly, then figure out the Rails implementation if I need it.

Thx.

– gw

20

On Oct 26, 2007, at 6:48 PM, CCH wrote:

On Oct 26, 11:40 am, Greg W. [email protected] wrote:

I’m trying to implement a data caching system and getting nowhere
because I cannot get a ruby global variable to persist across page
requests.*

cch: Stay away from global variables. Even if you managed to get it to
work in development mode, the moment you run it in production with a
pack of mongrels, the global variable will not persit from one mongrel
process to another.

Understood, but in this case, that’s OK because the global isn’t
really being used for the purposes of sharing, it is being used as a
high speed replacement for text files or db. The original data is
more or less static. It gets loaded mostly from text files, sometimes
a database. Loaded once, parsed, then used for repetitive reads. It
doesn’t change.

We’re talking about a relatively small amount of data–50 to 100 KB
max. So, each process instance can create it’s own global, that’s not
a problem.

– gw