I just came across varnish. We serve a lot of MP3 files on a regular
basis. Does it make sense to have Varnish cache those and serve them
out of
cache? Does anyone have any experience with Varnish and Nginx installs?
Are they complimentary or exclusionary?
Thanks
On Sun, 2009-03-29 at 19:00 -0400, Ilan B. wrote:
I just came across varnish. We serve a lot of MP3 files on a regular
basis. Does it make sense to have Varnish cache those and serve them
out of cache? Does anyone have any experience with Varnish and Nginx
installs? Are they complimentary or exclusionary?
They are complimentary. That being said, I don’t think it makes much
sense to cache static files with an application. Your OS kernel
already caches disk accesses and running another application will cut
into that cache memory. Varnish makes more sense for dynamic content.
Regards,
Cliff
My company is a heavy user of varnish cache since 2006. We have 6
varnish servers with 550 GB of RAM caching static pictures for our
community. At peak each server is handling 40k active connections per
second.
If you only have one server you should use a dedicated static server and
let the file system cache(linux caches last read data in ram), but it
won’t scale beyond that server. But I assume you have many GBs of mp3s
and need more than one cache. Our setup have pools of varnish servers
delivering photos from RAM, and we distribute traffic between the
varnishes by hash method so one server only serve his part of the photos
and caches them in memory.
Varnish is also a very hardware friendly software. It makes smokeping
look nice. 
Posted at Nginx Forum:
How did you get 61 GB of RAM in one server?
HP Blades with AMD CPU support up to 128 GB of RAM. We have 3 x AMD with
128 GB and 3 x Intel 64 GB. The new intel HP blades support 144 GB RAM.
At peak each varnish server is handling 40k established connections, not
active connections as I wrote by mistake. About 7000 to 8000 hits per
second.
Posted at Nginx Forum:
On Apr 13, 2009, at 10:05 AM, russlndr wrote:
HP Blades with AMD CPU support up to 128 GB of RAM. We have 3 x AMD
with 128 GB and 3 x Intel 64 GB. The new intel HP blades support 144
GB RAM.At peak each varnish server is handling 40k established connections,
not active connections as I wrote by mistake. About 7000 to 8000
hits per second.
Are you also using nginx in this setup? We are using nginx + varnish
for http://gravatar.com (nginx → varnish → apache ) and are
running into issues with the nginx boxes getting CPU bound (softirq,
not user). We are only able to get about 8000 req/sec (1k est
concurrent TCP connections) per nginx server before this happens.
Anyone else seeing this at high concurrency? We have already done the
obvious stuff like disable connection tracking, and are using
irqbalance to push the interrupt handlers from eth1 and eth0 onto
separate cores. We are running Debian Linux Sarge (2.6.18) 64-bit.
Any suggestions?
@Barry A.
Why don’t you guys just use nginx to serve the files and let the file
system and OS hande caching of static data? Just curious?
On Mon, Apr 13, 2009 at 1:55 AM, CryptWizard [email protected]
wrote:
How did you get 61 GB of RAM in one server?
64 bit architecture supports a theoretical limit of 16 exabytes of RAM.
On Apr 14, 2009, at 6:55 PM, Victor I. wrote:
@Barry A.
Why don’t you guys just use nginx to serve the files and let the file
system and OS hande caching of static data? Just curious?
Not sure exactly what you mean, but nothing is “static files” –
everything is generated dynamically.
@Barry A.
Sorry, I should of been more specific. I also follow the varnish-misc
list, and I have read that gravatar uses varnish specifically for their
thumbnails. That is what I meant by static data. So the question still
stands, won’t it make more sense to avoid the double buffer problem and
just use filesystem cache?
On Apr 15, 2009, at 1:43 PM, Victor I. wrote:
@Barry A.
Sorry, I should of been more specific. I also follow the varnish-misc
list, and I have read that gravatar uses varnish specifically for
their
thumbnails. That is what I meant by static data. So the question
still
stands, won’t it make more sense to avoid the double buffer problem
and
just use filesystem cache?
Yes, except the vast majority images are generated dynamically, so
they don’t exist on a filesystem anywhere 
Hi Kiril,
-----Original Message-----
From: Kiril A. [mailto:kupokomapa-
[email protected]]
Posted At: Sunday, March 29, 2009 7:57 PM
Posted To: gmane.comp.web.nginx.english
Conversation: varnish?
Subject: Re: varnish?You can check also http://www.ncache.org
Isn’t better to use the built-in cache, available into 0.7.5x branch?
Thanks for the link, though. I haven’t got the time to look at Igor’s
new
cache system.
Floren
Barry A. wrote:
Yes, except the vast majority images are generated dynamically, so
they don’t exist on a filesystem anywhere
Wouldn’t it be faster to have the image generation script push the file
to a memcached store on creation, and have nginx check this store before
falling back to the generation script?
–
Phillip B Oldham
The Activity People
[email protected] mailto:[email protected]
Policies
This e-mail and its attachments are intended for the above named
recipient(s) only and may be confidential. If they have come to you in
error, please reply to this e-mail and highlight the error. No action
should be taken regarding content, nor must you copy or show them to
anyone.
This e-mail has been created in the knowledge that Internet e-mail is
not a 100% secure communications medium, and we have taken steps to
ensure that this e-mail and attachments are free from any virus. We must
advise that in keeping with good computing practice the recipient should
ensure they are completely virus free, and that you understand and
observe the lack of security when e-mailing us.
On Apr 15, 2009, at 8:58 PM, zepolen wrote:
On Wed, Apr 15, 2009 at 10:14 PM, Barry A. <[email protected]
wrote:
Yes, except the vast majority images are generated dynamically, so
they
don’t exist on a filesystem anywhereCan’t proxy_store help you with that?
Maybe, but at 30,000 req/sec, reading/writing content from disk is too
slow.
On Wed, Apr 15, 2009 at 10:14 PM, Barry A.
[email protected] wrote:
Yes, except the vast majority images are generated dynamically, so they
don’t exist on a filesystem anywhere
Can’t proxy_store help you with that?
I’ve been using it with nginx to read thumbnails from an origin server
and save them to local disk, works pretty well.
On Apr 16, 2009, at 3:20 AM, Phillip B Oldham wrote:
Barry A. wrote:
Yes, except the vast majority images are generated dynamically, so
they don’t exist on a filesystem anywhere
Wouldn’t it be faster to have the image generation script push the
file to a memcached store on creation, and have nginx check this
store before falling back to the generation script?
Maybe, but speed isn’t the main issue here, the requests are served
generally “fast enough” Its scaling a single box to serve a huge
number of requests/sec.
BTW, I enabled http keep alives in nginx, and it seems to have reduced
softirq by about 5% which is pretty good. I guess setting up and
tearing down all of those TCP connections has a little bit of
overhead. We have sacrificed a little bit of RAM to deal with the
huge number of established connections (50k ish), but thats ok. I’m
wondering if there are even more optimizations like this which could
reduce sofirq even more.
On Thu, Apr 16, 2009 at 5:48 PM, Barry A. [email protected]
wrote:
Maybe, but at 30,000 req/sec, reading/writing content from disk is too slow.
Have you tried it? Typically only the first access will be slow, after
that it should be cached by the file system.
This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.
Sponsor our Newsletter | Privacy Policy | Terms of Service | Remote Ruby Jobs