Forum: Ruby on Rails page cache more than 100K files inside a single directory?

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
Nanyang Z. (Guest)
on 2007-05-18 09:16
I get a model containing more than 100K rows of data, and page caching
is used for this model. As a cached page will generate a html file in
related directory,
(http://example.com/data/:rowid)
day after day, more than 100k files will be generated inside a single
directory.
Is it possible? If so, is it the right thing to do?
Rick O. (Guest)
on 2007-05-18 09:32
(Received via mailing list)
On 5/18/07, Nanyang Z. <removed_email_address@domain.invalid> wrote:
>
> I get a model containing more than 100K rows of data, and page caching
> is used for this model. As a cached page will generate a html file in
> related directory,
> (http://example.com/data/:rowid)
> day after day, more than 100k files will be generated inside a single
> directory.
> Is it possible? If so, is it the right thing to do?

That depends on your file system and has little to do with ruby or
rails.  You're probably better off partitioning that somehow.  Here's
what I usually do:

http://www.37signals.com/svn/archives2/id_partitioning.php

--
Rick O.
http://lighthouseapp.com
http://weblog.techno-weenie.net
http://mephistoblog.com
Nanyang Z. (Guest)
on 2007-05-18 10:35
Rick O. wrote:

> You're probably better off partitioning that somehow.
> Here's what I usually do:
> http://www.37signals.com/svn/archives2/id_partitioning.php

This is what I need!

> That depends on your file system and has little to do with ruby or
> rails.

I don't think it has little to do with RoR. I think caches_page method
should do this automatically, because most of file system Do have file
number limits.


If RoR doesn't do this, I will need to do it myself. So the ways of
writing page cached files and fetching them will need to be changed. I
think I need to rewrite
Module ActionController::Caching::Pages::ClassMethods .

right?
Nanyang Z. (Guest)
on 2007-05-18 11:05
I'll explain my idea.
Here is what I want to do when cache page is on:
1. If the last part of a url is a digital number, it will be identified
as rowid automatically.
2. Depending on these numbers, cached page files will be stored in a
subdiretory of page caching directory. (like 1.html stored as /0/1.html,
19043.html as /1/9043.html, 677899.html as /67/7899.hrml...).
3. Fetching, deleting cached page files will follow this idea.

By doing this, I still can visit http://example.com/data/677899 to reach
cached page file, and needn't worry about too many files to store in one
directory.

Rick, can you tell me which methods I need to rewrite? As you're far
more family with rails than me, you help can save a lot of my time.
This topic is locked and can not be replied to.