Page cache more than 100K files inside a single directory?

Rails
1

I get a model containing more than 100K rows of data, and page caching
is used for this model. As a cached page will generate a html file in
related directory,
(http://example.com/data/:rowid)
day after day, more than 100k files will be generated inside a single
directory.
Is it possible? If so, is it the right thing to do?

2

On 5/18/07, Nanyang Z. [email protected] wrote:

I get a model containing more than 100K rows of data, and page caching
is used for this model. As a cached page will generate a html file in
related directory,
(http://example.com/data/:rowid)
day after day, more than 100k files will be generated inside a single
directory.
Is it possible? If so, is it the right thing to do?

That depends on your file system and has little to do with ruby or
rails. You’re probably better off partitioning that somehow. Here’s
what I usually do:

http://www.37signals.com/svn/archives2/id_partitioning.php

–
Rick O.
http://lighthouseapp.com
http://weblog.techno-weenie.net
http://mephistoblog.com

3

Rick O. wrote:

You’re probably better off partitioning that somehow.
Here’s what I usually do:
ID Partitioning - Signal vs. Noise (by 37signals)

This is what I need!

That depends on your file system and has little to do with ruby or
rails.

I don’t think it has little to do with RoR. I think caches_page method
should do this automatically, because most of file system Do have file
number limits.

If RoR doesn’t do this, I will need to do it myself. So the ways of
writing page cached files and fetching them will need to be changed. I
think I need to rewrite
Module ActionController::Caching::Pages::ClassMethods .

right?

4

I’ll explain my idea.
Here is what I want to do when cache page is on:

  1. If the last part of a url is a digital number, it will be identified
    as rowid automatically.
  2. Depending on these numbers, cached page files will be stored in a
    subdiretory of page caching directory. (like 1.html stored as /0/1.html,
    19043.html as /1/9043.html, 677899.html as /67/7899.hrml…).
  3. Fetching, deleting cached page files will follow this idea.

By doing this, I still can visit http://example.com/data/677899 to reach
cached page file, and needn’t worry about too many files to store in one
directory.

Rick, can you tell me which methods I need to rewrite? As you’re far
more family with rails than me, you help can save a lot of my time.