# Feedback wanted on current setup, and a question about cachi **URL:** \ **Category:** Deployment\ **Created:** [January 24, 2007, 11:28pm UTC](https://www.ruby-forum.com/t/feedback-wanted-on-current-setup-and-a-question-about-cachi/83300 "2007-01-24T23:28:56Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![joe-dumoulin](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/j/9dc877/32.png) [@joe-dumoulin](https://www.ruby-forum.com/u/joe-dumoulin)\ **Post date:** [January 24, 2007, 11:28pm UTC](https://www.ruby-forum.com/t/feedback-wanted-on-current-setup-and-a-question-about-cachi/83300/1 "2007-01-24T23:28:56Z")
Here’s where I explain my current setup, and ask for feedback. I’m looking for a solution that’s easy to set up and Just Works. This is for [http://www.tanga.com](http://www.tanga.com). Due to the nature of the site, traffic is awfully spikey, meaning that we can get 100x normal traffic in a matter of seconds. I’m being hosted at Textdrive. An accelerator is sort of like a VPS, I suppose. Running on OpenSolaris. In a few days, I should have: ``` * Four 1/16 accelerators. Each of these will be running 10 ``` Mongrels. A Big IP will be doing load balancing feeding the requests directly to the Mongrels. SSL processing is done by the Big IP (I hope). ``` * One 1/8 accelerator doing database (postgresql). It also sends ``` out a ton of email and sms notifications. ``` * All the accelerators share the same /home mount point that's on a ``` x4500. All the Mongrels on the different Accelerators run off the same Rails code (i.e. the code is in /home/deploy/apps/tanga). Each Mongrel writes to the same production.log file that’s in /home. QUESTION: How should page/fragment caching work? I guess we need a webserver in there somewhere that determines whether the static cache file should be sent or if the request needs to be sent to the Mongrels. I’m not opposed to using something like BingoDisk, but that doesn’t solve the issue of how the cache issue should work. Right now, Apache first checks for the existence of the requested file, and if it doesn’t find it, it sends the request to Mongrel. ---
**Author:** ![joe-dumoulin](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/j/9dc877/32.png) [@joe-dumoulin](https://www.ruby-forum.com/u/joe-dumoulin)\ **Post date:** [January 25, 2007, 12:42am UTC](https://www.ruby-forum.com/t/feedback-wanted-on-current-setup-and-a-question-about-cachi/83300/2 "2007-01-25T00:42:06Z")
Memcached fragments? [http://blog.zvents.com/2006/11/3/rails-plugin-extended-fragment-cache](http://blog.zvents.com/2006/11/3/rails-plugin-extended-fragment-cache) Eliot ---
**Author:** ![joe-dumoulin](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/j/9dc877/32.png) [@joe-dumoulin](https://www.ruby-forum.com/u/joe-dumoulin)\ **Post date:** [January 25, 2007, 12:51am UTC](https://www.ruby-forum.com/t/feedback-wanted-on-current-setup-and-a-question-about-cachi/83300/3 "2007-01-25T00:51:07Z")
The issue is how the cached files get to the user. I don’t think memcache helps with that. ---
**Author:** ![joe-dumoulin](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/j/9dc877/32.png) [@joe-dumoulin](https://www.ruby-forum.com/u/joe-dumoulin)\ **Post date:** [January 25, 2007, 1:05am UTC](https://www.ruby-forum.com/t/feedback-wanted-on-current-setup-and-a-question-about-cachi/83300/4 "2007-01-25T01:05:10Z")
On Jan 24, 2007, at 2:28 PM, Joe Van D. wrote: > ``` > * All the accelerators share the same /home mount point that's > > ``` > > solve the issue of how the cache issue should work. Right now, Apache > first checks for the existence of the requested file, and if it > doesn’t > find it, it sends the request to Mongrel. You will have to step up to something like memcached for your fragment store, page caching is harder to do when you go straight to the mongrels from the hardware load balancers, I never came up with a solution to page caching that I liked with a setup like this. What I do when I have nodes like that behind a hardware l/b is run one nginx server on each node. Nginx serves static content and page caches and proxies to mongrel for the rest. This neatly sidesteps the hassle of an asset\_host. Especially since you have the shared filesystem to put cached pages and fragments on you may as well take advantage of it. Also one flaw I see in your setup is that you have all the nodes logging their production log to the same file on a shared filesystem. This is going to degrade performance a lot when you have 4 nodes trying to get a lock so they can write to the logs. And rails does this all day long. It’s one thing to have a cluster of mongrels on the same host write to the same logs, thats fine,. It’s another thing to have 4 server nodes that share the same filesystem that are each running clusters of mongrels. We use GFS as our shared filesystem in a totally similar way to what you are doign but GFS has something called context dependent symlinks. These allow rails to blissfully write to the production log at the same path on the shared filesystem, but really the log directory is just a symlink to the node’s hostname. Like this: ey00-s00076 shared # ls -lsa total 36 4 drwxrwxr-x 6 ez ez 3864 Dec 14 12:08 . 4 drwxr-xr-x 4 ez ez 3864 Jan 23 17:08 … 4 lrwxrwxrwx 1 ez ez 9 Dec 2 11:30 log -\> @hostname 4 drwxr-xr-x 2 ez ez 3864 Jan 24 01:26 ey00-s00048 4 drwxr-xr-x 2 ez ez 3864 Jan 24 01:27 ey00-s00069 see the log -\> @hostname ? that makes it so each node in the cluster can still write logs to app/shared/log, but the file will be actually in a dir named after the current hostname. Separating the logs out increased the performance of rails apps in a cluster like this at least 20% Cheers- – Ezra Z. – Lead Rails Evangelist – removed\_email\_address@domain.invalid – Engine Y., Serious Rails Hosting – (866) 518-YARD (9273) ---
**Author:** ![joe-dumoulin](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/j/9dc877/32.png) [@joe-dumoulin](https://www.ruby-forum.com/u/joe-dumoulin)\ **Post date:** [January 25, 2007, 2:17am UTC](https://www.ruby-forum.com/t/feedback-wanted-on-current-setup-and-a-question-about-cachi/83300/5 "2007-01-25T02:17:19Z")
On Jan 24, 4:03 pm, Ezra Z. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > > ``` > > * All the accelerators share the same /home mount point that's > > > > ``` > > > > solve the issue of how the cache issue should work. Right now, Apache > > first checks for the existence of the requested file, and if it > > doesn’t > > find it, it sends the request to Mongrel.You will have to step up to something like memcached for your > > fragment store, page caching is harder to do when you go straight to > > the mongrels from the hardware load balancers, I never came up with a > > solution to page caching that I liked with a setup like this. Oh, sorry. The cached files will be stored on a NFS drive (for now). I’ll look at using memcache later. My question was more about how to get the static/cached files to the user, which it looks like nginx might do. > trying to get a lock so they can write to the logs. And rails does > this all day long. It’s one thing to have a cluster of mongrels on > the same host write to the same logs, thats fine,. It’s another thing > to have 4 server nodes that share the same filesystem that are each > running clusters of mongrels. Hm, good point. > 4 drwxrwxr-x 6 ez ez 3864 Dec 14 12:08 . > cluster like this at least 20% > So, if each machine’s log is written to a different file, doesn’t it > make it difficult to look through each production.log file when you’re > trying to track down problems? I wonder if there’s a good way to > combine all the log files when you’re debugging. ---
**Author:** ![joe-dumoulin](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/j/9dc877/32.png) [@joe-dumoulin](https://www.ruby-forum.com/u/joe-dumoulin)\ **Post date:** [January 25, 2007, 2:39am UTC](https://www.ruby-forum.com/t/feedback-wanted-on-current-setup-and-a-question-about-cachi/83300/6 "2007-01-25T02:39:21Z")
Hi~ On Jan 24, 2007, at 5:16 PM, Joe Van D. wrote: > > > An accelerator is sort of like a VPS, I suppose. Running on > > > out a ton of email and sms notifications. > > > webserver in there somewhere that determines whether the static > > > fragment store, page caching is harder to do when you go straight to > > > one nginx server on each node. Nginx serves static content and page > > > this all day long. It’s one thing to have a cluster of mongrels on > > > at the same path on the shared filesystem, but really the log > > > 4 drwxr-xr-x 2 ez ez 3864 Jan 24 01:27 ey00-s00069 > > > So, if each machine’s log is written to a different file, doesn’t it > > > make it difficult to look through each production.log file when you’re > > > trying to track down problems? I wonder if there’s a good way to > > > combine all the log files when you’re debugging. Since all the real log directories live on the shared filesystem you can tail all the production logs at once like this: $ tail -f /data/appname/shared/ey00-s000\*/production.log That will tail all the log files at once with labels for which file entries come from. Cheers- – Ezra Z. – Lead Rails Evangelist – removed\_email\_address@domain.invalid – Engine Y., Serious Rails Hosting – (866) 518-YARD (9273)