Hey,
Do you think it's a good idea to add Varnish to Nginx to serve static
files?
I'm running a dedicated server with 2GB RAM and a pentium dual core
E2180 for static files.
I'm only using Nginx 0.6.32 on Centos 5/linux 2.6.24: no database, no
php, no gzip mod...
Nginx sends 60 images/seconde (max 100 images/sec) among 3.200.000 jpeg
images (one image = 5-10kB).
Thanks in advance!
My Nginx file :
--------------------------------------------------
user nginx nginx;
worker_processes 4;
events
{
worker_connections 1024;
}
http
{
include mime.types;
default_type application/octet-stream;
access_log off;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
client_body_timeout 5;
client_header_timeout 5;
keepalive_timeout 5 5;
send_timeout 5;
gzip off;
server
{
listen 80;
root /home/www;
expires 90d;
# CSS & Javascript files configuration
location ~* ^.+.(css|js)$
{
gzip_static on;
gzip_http_version 1.1;
gzip_proxied off;
gzip_min_length 0;
gzip_disable "MSIE [1-6]\.";
gzip_vary off;
}
}
}
on 09.10.2008 22:50
on 10.10.2008 02:04
From my experience, 60 small static files / second used 2% cpu on a OS X Xeon Server. Are you suspecting that your server is being limited in some way ? Cheers Dave
on 10.10.2008 09:58
I think you should increase this, as you have more CPU and also you are using 4 workers. Try to set this at lets say "worker_connections 2048;". It might work.
on 10.10.2008 14:03
Dave Cheney wrote: > From my experience, 60 small static files / second used 2% cpu on a > OS X Xeon Server. Are you suspecting that your server is being limited > in some way ? Right now, I don't have problems. I just want to optimize my web server for the best result (it takes 400-500ms to serve a 5kB image). http://img29.picoodle.com/img/img29/3/10/10/f_monitoring0m_f2441cf.png http://img33.picoodle.com/img/img33/3/10/10/f_monitoring0m_bb25457.png Those images show my web site performance (from pingdom.com and site-perf.com). I'm wondering if there is something to reduce those timings... Thanks.
on 10.10.2008 14:38
Compression, gzip. You should try it, maybe for jpeg is not actually worth it... Hmm, try what can i say.
on 10.10.2008 15:01
Have you actually tried putting varnish/ncache in place and benchmarking the before/after results? As I've found with all server set-ups: YMMV. Gen Gennix wrote: > 400-500ms to serve a 5kB image). > > Thanks. > > Attachments: > http://www.ruby-forum.com/attachment/2797/monitoring00.png > > -- *Phillip B Oldham* The Activity People phill@theactivitypeople.co.uk <mailto:phill@theactivitypeople.co.uk> ------------------------------------------------------------------------ *Policies* This e-mail and its attachments are intended for the above named recipient(s) only and may be confidential. If they have come to you in error, please reply to this e-mail and highlight the error. No action should be taken regarding content, nor must you copy or show them to anyone. This e-mail has been created in the knowledge that Internet e-mail is not a 100% secure communications medium, and we have taken steps to ensure that this e-mail and attachments are free from any virus. We must advise that in keeping with good computing practice the recipient should ensure they are completely virus free, and that you understand and observe the lack of security when e-mailing us.
on 10.10.2008 15:44
> Compression, gzip. You should try it, maybe for jpeg is not actually > worth it... Hmm, try what can i say. I already use static gzip on css and javascript file. And for jpeg, it is not worth it, indeed. Moreover, if you look at the monitoring image (in attachment), the loss of time is while sending request and receiving http-headers from server (25% of the total time). I was wondering if we can reduce it? PS: watch the last line of the image in attachment: it's the javascript from Google Analytics, and the timing is really, really short!
on 10.10.2008 15:50
> Have you actually tried putting varnish/ncache in place and benchmarking > the before/after results? As I've found with all server set-ups: YMMV. I'm trying to get tips and advices from experienced users like you before diving in the benchmarking work. And don't worry, I will share my varnish+nginx benchmarks and experiences at the end... ^_-
on 10.10.2008 16:15
On Fri, Oct 10, 2008 at 02:03:50PM +0200, Gen Gennix wrote: > > http://img29.picoodle.com/img/img29/3/10/10/f_monitoring0m_f2441cf.png > > http://img33.picoodle.com/img/img33/3/10/10/f_monitoring0m_bb25457.png > > Those images show my web site performance (from pingdom.com and > site-perf.com). > > I'm wondering if there is something to reduce those timings... What I see on these images is that TCP connection times are about 100ms-500ms. These values depend on client speed connection and kernel internel processing only. User-level applications can not affect on it. Thus, if you have 100-500ms TCP round trip, then small data chunks will be processed in the same time, even if nginx will not block on disk reads. As to disk blocking, you have 30G images and 2G memory. nginx is probably blocking on disk - see "wa" percents in top. You may try to increase worker_processes to 10-20. As to using varnish+nginx, I think varnish will not help in this case.
on 11.10.2008 01:26
I have tried both approaches. nginx on ZFS (arc caching) is superior in performance to varnish in front of nginx. I can easily choke varnish with many requests. http://letsgetdugg.com/view/Nginx_hidden_performance_option works very well for us in our production environment. -Victor
on 11.10.2008 01:30
oh yes I forgot, I have confronted one of the developers on #varnish with my results. The said developer stated that nginx currently does have the edge performance wise, even though they have the "superior" architecture. -Victor
on 12.10.2008 11:33
On Sat, Oct 11, 2008 at 7:26 AM, Victor Iggy <lists@ruby-forum.com> wrote: > -- > Posted via http://www.ruby-forum.com/. On Sat, Oct 11, 2008 at 7:30 AM, Victor Iggy <lists@ruby-forum.com> wrote: > oh yes I forgot, I have confronted one of the developers on #varnish > with my results. The said developer stated that nginx currently does > have the edge performance wise, even though they have the "superior" > architecture. > and would this be on solaris only? or does it apply to other platforms as well? -jf -- In the meantime, here is your PSA: "It's so hard to write a graphics driver that open-sourcing it would not help." -- Andrew Fear, Software Product Manager, NVIDIA Corporation http://kerneltrap.org/node/7228
on 12.10.2008 20:22
don't know, benchmark and find out. I have seen nginx edge out varnish in performance on solaris and freebsd. -Victor
on 14.10.2008 20:24
Serving thumbnails efficiently is not easy. See http://highscalability.com/youtube-architecture (scroll down to 'Serving thumbnails'). You're spending a good chunk of time doing disk seeks. Even if read time is 0, average seek time is about 10ms. So you're looking at a limit of 100 images/sec. This is the ideal case. Add read time + TCP delays + other delays, you're looking 50-60 images/sec. Serve off memory, try using memcache. -praveen On Fri, Oct 10, 2008 at 5:03 AM, Gen Gennix <lists@ruby-forum.com> wrote: > > > Attachments: > http://www.ruby-forum.com/attachment/2797/monitoring00.png > > -- > Posted via http://www.ruby-forum.com/. > > -- Praveen Can you yell "Movie!" in a crowded firestation? http://tigertrail.net
on 14.10.2008 20:36
umm, linux file cache? solaris ZFS ARC cache? who says you will be doing disk seeks for frequently used files? seems like varnish is clashing with what is already in place. -Victor
on 14.10.2008 21:40
Yes. There is caching at the OS level too, which needs to be tuned (like cache sizes, inode vs content etc). I did not look at your benchmarks, but was commenting more on Gen's post. In the graphs he's given, the latency between connect and sending first byte (the green part) is high, which might explain disk seek/read. -p On Tue, Oct 14, 2008 at 11:36 AM, Victor Iggy <lists@ruby-forum.com> wrote: > umm, linux file cache? solaris ZFS ARC cache? who says you will be doing > disk seeks for frequently used files? > > seems like varnish is clashing with what is already in place. > > -Victor > -- > Posted via http://www.ruby-forum.com/. > > -- Praveen Can you yell "Movie!" in a crowded firestation? http://tigertrail.net