Forum: NGINX Nginx VS Nginx + Varnish

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
Gen G. (Guest)
on 2008-10-10 00:50
Hey,

Do you think it's a good idea to add Varnish to Nginx to serve static
files?


I'm running a dedicated server with 2GB RAM and a pentium dual core
E2180 for static files.

I'm only using Nginx 0.6.32 on Centos 5/linux 2.6.24: no database, no
php, no gzip mod...

Nginx sends 60 images/seconde (max 100 images/sec) among 3.200.000 jpeg
images (one image = 5-10kB).


Thanks in advance!



My Nginx file :
--------------------------------------------------
user              nginx nginx;
worker_processes  4;


events
{
    worker_connections  1024;
}


http
{
   include                mime.types;
   default_type           application/octet-stream;

   access_log             off;

   sendfile               on;
   tcp_nopush             on;
   tcp_nodelay            on;

   client_body_timeout    5;
   client_header_timeout  5;
   keepalive_timeout      5   5;
   send_timeout           5;


   gzip                   off;

   server
      {
         listen       80;

         root         /home/www;

         expires      90d;

         # CSS & Javascript files configuration
         location ~* ^.+.(css|js)$
            {
               gzip_static        on;
               gzip_http_version  1.1;
               gzip_proxied       off;
               gzip_min_length    0;
               gzip_disable       "MSIE [1-6]\.";
               gzip_vary          off;
            }
      }
}
Dave C. (Guest)
on 2008-10-10 04:04
(Received via mailing list)
From my experience, 60 small static files / second used 2% cpu on a
OS X Xeon Server. Are you suspecting that your server is being limited
in some way ?

Cheers

Dave
Robert G. (Guest)
on 2008-10-10 11:58
I think you should increase this, as you have more CPU and also you are
using 4 workers. Try to set this at lets say "worker_connections
2048;". It might work.
Gen G. (Guest)
on 2008-10-10 16:03
Attachment: monitoring00.png (0 Bytes)
Dave C. wrote:
> From my experience, 60 small static files / second used 2% cpu on a
> OS X Xeon Server. Are you suspecting that your server is being limited
> in some way ?

Right now, I don't have problems.

I just want to optimize my web server for the best result (it takes
400-500ms to serve a 5kB image).


http://img29.picoodle.com/img/img29/3/10/10/f_moni...

http://img33.picoodle.com/img/img33/3/10/10/f_moni...

Those images show my web site performance (from pingdom.com and
site-perf.com).

I'm wondering if there is something to reduce those timings...

Thanks.
Robert G. (Guest)
on 2008-10-10 16:38
Compression, gzip. You should try it, maybe for jpeg is not actually
worth it... Hmm, try what can i say.
Phillip B Oldham (Guest)
on 2008-10-10 17:01
(Received via mailing list)
Attachment: phill.vcf (0 Bytes)
Have you actually tried putting varnish/ncache in place and benchmarking
the before/after results? As I've found with all server set-ups: YMMV.

Gen G. wrote:
> 400-500ms to serve a 5kB image).
>
> Thanks.
>
> Attachments:
> http://www.ruby-forum.com/attachment/2797/monitoring00.png
>
>

--

*Phillip B Oldham*
The Activity People
removed_email_address@domain.invalid 
<mailto:removed_email_address@domain.invalid>

------------------------------------------------------------------------

*Policies*

This e-mail and its attachments are intended for the above named
recipient(s) only and may be confidential. If they have come to you in
error, please reply to this e-mail and highlight the error. No action
should be taken regarding content, nor must you copy or show them to
anyone.

This e-mail has been created in the knowledge that Internet e-mail is
not a 100% secure communications medium, and we have taken steps to
ensure that this e-mail and attachments are free from any virus. We must
advise that in keeping with good computing practice the recipient should
ensure they are completely virus free, and that you understand and
observe the lack of security when e-mailing us.
Gen G. (Guest)
on 2008-10-10 17:44
> Compression, gzip. You should try it, maybe for jpeg is not actually
> worth it... Hmm, try what can i say.

I already use static gzip on css and javascript file.
And for jpeg, it is not worth it, indeed.

Moreover, if you look at the monitoring image (in attachment), the loss
of time is while sending request and receiving http-headers from server
(25% of the total time).

I was wondering if we can reduce it?

PS: watch the last line of the image in attachment: it's the javascript
from Google Analytics, and the timing is really, really short!
Gen G. (Guest)
on 2008-10-10 17:50
> Have you actually tried putting varnish/ncache in place and benchmarking
> the before/after results? As I've found with all server set-ups: YMMV.

I'm trying to get tips and advices from experienced users like you
before diving in the benchmarking work.

And don't worry, I will share my varnish+nginx benchmarks and
experiences at the end... ^_-
Igor S. (Guest)
on 2008-10-10 18:15
(Received via mailing list)
On Fri, Oct 10, 2008 at 02:03:50PM +0200, Gen G. wrote:

>
> http://img29.picoodle.com/img/img29/3/10/10/f_moni...
>
> http://img33.picoodle.com/img/img33/3/10/10/f_moni...
>
> Those images show my web site performance (from pingdom.com and
> site-perf.com).
>
> I'm wondering if there is something to reduce those timings...

What I see on these images is that TCP connection times are about
100ms-500ms. These values depend on client speed connection and kernel
internel processing only. User-level applications can not affect on it.
Thus, if you have 100-500ms TCP round trip, then small data chunks
will be processed in the same time, even if nginx will not block on disk
reads.

As to disk blocking, you have 30G images and 2G memory. nginx is
probably
blocking on disk - see "wa" percents in top. You may try to increase
worker_processes to 10-20.

As to using varnish+nginx, I think varnish will not help in this case.
Victor I. (Guest)
on 2008-10-11 03:26
I have tried both approaches. nginx on ZFS (arc caching) is superior in
performance to varnish in front of nginx. I can easily choke varnish
with many requests.

http://letsgetdugg.com/view/Nginx_hidden_performance_option

works very well for us in our production environment.

-Victor
Victor I. (Guest)
on 2008-10-11 03:30
oh yes I forgot, I have confronted one of the developers on #varnish
with my results. The said developer stated that nginx currently does
have the edge performance wise, even though they have the "superior"
architecture.

-Victor
Jeffrey 'jf' Lim (Guest)
on 2008-10-12 13:33
(Received via mailing list)
On Sat, Oct 11, 2008 at 7:26 AM, Victor I. 
<removed_email_address@domain.invalid>
wrote:
> --
> Posted via http://www.ruby-forum.com/.


On Sat, Oct 11, 2008 at 7:30 AM, Victor I. 
<removed_email_address@domain.invalid>
wrote:
> oh yes I forgot, I have confronted one of the developers on #varnish
> with my results. The said developer stated that nginx currently does
> have the edge performance wise, even though they have the "superior"
> architecture.
>

and would this be on solaris only? or does it apply to other platforms
as well?

-jf

--
In the meantime, here is your PSA:
"It's so hard to write a graphics driver that open-sourcing it would not
help."
    -- Andrew Fear, Software Product Manager, NVIDIA Corporation
http://kerneltrap.org/node/7228
Victor I. (Guest)
on 2008-10-12 22:22
don't know, benchmark and find out. I have seen nginx edge out varnish
in performance on solaris and freebsd.

-Victor
Praveen (Guest)
on 2008-10-14 22:24
(Received via mailing list)
Serving thumbnails efficiently is not easy. See
http://highscalability.com/youtube-architecture (scroll down to
'Serving thumbnails').

You're spending a good chunk of time doing disk seeks. Even if read
time is 0, average seek time is about 10ms. So you're looking at a
limit of 100 images/sec. This is the ideal case. Add read time + TCP
delays + other delays, you're looking 50-60 images/sec.

Serve off memory, try using memcache.

-praveen

On Fri, Oct 10, 2008 at 5:03 AM, Gen G. <removed_email_address@domain.invalid>
wrote:
>
>
> Attachments:
> http://www.ruby-forum.com/attachment/2797/monitoring00.png
>
> --
> Posted via http://www.ruby-forum.com/.
>
>



--
Praveen

Can you yell "Movie!" in a crowded firestation?

http://tigertrail.net
Victor I. (Guest)
on 2008-10-14 22:36
umm, linux file cache? solaris ZFS ARC cache? who says you will be doing
disk seeks for frequently used files?

seems like varnish is clashing with what is already in place.

-Victor
Praveen (Guest)
on 2008-10-14 23:40
(Received via mailing list)
Yes. There is caching at the OS level too, which needs to be tuned
(like cache sizes, inode vs content etc). I did not look at your
benchmarks, but was commenting more on Gen's post. In the graphs he's
given, the latency between connect and sending first byte (the green
part) is high, which might explain disk seek/read.

-p

On Tue, Oct 14, 2008 at 11:36 AM, Victor I. 
<removed_email_address@domain.invalid>
wrote:
> umm, linux file cache? solaris ZFS ARC cache? who says you will be doing
> disk seeks for frequently used files?
>
> seems like varnish is clashing with what is already in place.
>
> -Victor
> --
> Posted via http://www.ruby-forum.com/.
>
>



--
Praveen

Can you yell "Movie!" in a crowded firestation?

http://tigertrail.net
This topic is locked and can not be replied to.