Forum: NGINX SSL slow on nginx

2974d09ac2541e892966b762aad84943?d=identicon&s=25 khav (Guest)
on 2014-06-28 07:15
(Received via mailing list)
For my site , ssl seems to be slow even though i got A+ on sslabs
(implemented ocsp stapling, Forward Secrecy , spdy)

Here is the result from pingdom

http://tools.pingdom.com/fpt/#!/cc2MfH/https://www...


Notice the high connect time and high ssl negociation time


Here is my nginx conf

server {
    listen       443 spdy default_server; #Change to 443 when SSL is on
  ssl on;
  ssl_certificate    /etc/ssl/filterbypass.me.crt; #(or .pem)
    ssl_certificate_key    /etc/ssl/filterbypass.me.key.nopass;
  ssl_client_certificate /etc/ssl/intermediate.crt;
  ssl_protocols SSLv3 TLSv1 TLSv1.1 TLSv1.2;
   ssl_ciphers
ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS;
    ssl_prefer_server_ciphers on;
  ssl_buffer_size 8k;
    ssl_session_cache shared:SSL:10m;
  ssl_dhparam /etc/ssl/dhparam.pem;
    ssl_session_timeout 10m;
  ssl_stapling on;
    ssl_stapling_verify on;
  ssl_trusted_certificate  /etc/ssl/trustchain.crt;
  resolver 8.8.8.8 8.8.4.4;
  add_header Strict-Transport-Security "max-age=31536000;
includeSubDomains";
  #rest of config goes here
}

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,251277,251277#msg-251277
671d9faabfe3d3382be736b93fbfa1d5?d=identicon&s=25 Steve Holdoway (Guest)
on 2014-06-29 09:07
(Received via mailing list)
Hi!
On Sat, 2014-06-28 at 01:14 -0400, khav wrote:
>
>
ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS;
> includeSubDomains";
>   #rest of config goes here
> }
>

Setting up a local DNS resolver, and adding it in as the first one may
make a bit of a help, as may implementing SPDY.

However, ( using webpagetest as my preferred tool... from Dulles as the
server is reported to be in Utah:
http://www.webpagetest.org/result/140629_3J_518/ ) I'd say that the
problem may well be with the available power on the server itself: for
example, it takes .8s to download a 20KB woff file. Given the services
that you're wanting to offer, I think you're going to need a load more
grunt.

hth,

Steve
--
Steve Holdoway BSc(Hons) MIITP
http://www.greengecko.co.nz
Linkedin: http://www.linkedin.com/in/steveholdoway
Skype: sholdowa
A8108a0961c6087c43cda32c8616dcba?d=identicon&s=25 Maxim Dounin (Guest)
on 2014-07-01 00:40
(Received via mailing list)
Hello!

On Sat, Jun 28, 2014 at 01:14:16AM -0400, khav wrote:

> For my site , ssl seems to be slow even though i got A+ on sslabs
> (implemented ocsp stapling, Forward Secrecy , spdy)

Note that SSL Labs grades are about security, not about speed.

> Here is the result from pingdom
>
> http://tools.pingdom.com/fpt/#!/cc2MfH/https://www...
>
>
> Notice the high connect time and high ssl negociation time

The test is done from Netherlands, the site is in US, RTT seems to
be about 170ms.  So, a connect will take 170ms minimum (1*RTT), and an
SSL
handshake without a cached session will take 340ms minimum
(2*RTT).  And these are only network costs, not counting any
computational costs for SSL.

Pingdom numbers for the first pages as I see are as follows:

SSL 168ms
Connect 572ms
Send 0ms
Wait 304ms
Receive 6ms

They seems to be labeled incorrectly (1st number is
actually connect time, while 2nd one is SSL handshake, not vice
versa as they are labeled).  Connect time seems pretty much
normal, just 1 RTT, close to minimum possible.  SSL handshake
time is a bit more than it could be, about 3 RTT for some reason.
Simple test here suggests that the cause is likely CPU usage on your
server - response to ServerHello is noticeably delayed.

One of the possible reasons is that you prefer ciphers with
forward secrecy, and they are CPU hungry, especially DH ones:

>    ssl_ciphers
>
ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS;
>     ssl_prefer_server_ciphers on;

You may try "openssl speed rsa2048" to find out how many
handshakes per core your server can handle.  Note that ECDH with
256 bit curve will result in about 2x slowdown compared to plain
RSA, and DH with 2048 bit params - up to 10x slowdown.

Additionally, make sure that:

- you've properly tuned number of worker processes to match your
  server cores, see http://nginx.org/r/worker_processes;

- number of handshakes per seconds isn't reaching numbers your
  server can handle, use "openssl speed" to find out.

--
Maxim Dounin
http://nginx.org/
2974d09ac2541e892966b762aad84943?d=identicon&s=25 khav (Guest)
on 2014-07-01 09:10
(Received via mailing list)
Thanks Maxim and GreenGecko for the insights


The worker process does match my number of cpu cores (running on 8 cores
atm)


How can i know  the number of handshakes per seconds occurring on the
server

The openssl speed result have been posted on
http://pastebin.com/hNeVhJfa
for readability


You can find my full list of ssl ciphers here --->
http://pastebin.com/7xJRJgJC

If you can suggest "faster ciphers" with same level of compatibility , i
would be awesome

Will a faster cpu actually solve the issue ?
My cpu load never reached a value > 0.50 as far as i know and average is
like 0.30

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,251277,251353#msg-251353
A8108a0961c6087c43cda32c8616dcba?d=identicon&s=25 Maxim Dounin (Guest)
on 2014-07-01 15:51
(Received via mailing list)
Hello!

On Tue, Jul 01, 2014 at 03:10:07AM -0400, khav wrote:

> Thanks Maxim and GreenGecko for the insights
>
>
> The worker process does match my number of cpu cores (running on 8 cores
> atm)

Good.  It may be also good idea to make sure you don't have
multi_accept enabled, just in case.

> How can i know  the number of handshakes per seconds occurring on the
> server

First of all, count the number of connections per second (and
requests per second) - it should be trivial, and may be extracted
even with nginx stub_status module.  I would generally recommend
using logs though.  With logs, you should be also able to count
number of uncached handshakes - by using $ssl_session_reused
variable and the $connection_requests one.

See here:

http://nginx.org/r/$ssl_session_reused
http://nginx.org/r/$connection_requests
http://nginx.org/r/log_format

> The openssl speed result have been posted on http://pastebin.com/hNeVhJfa
> for readability

So, basically, your server is able to do about 800 plain RSA
handshakes per second per core, 6400 handshakes total.

But as previously noted, things can be very much worse with DH
ciphers, especially if you are using 2048 bit dhparams (or
larger).

> If you can suggest "faster ciphers" with same level of compatibility , i
> would be awesome

It may be good idea to disable DH regardless of the level of
compatibility.  It's just too slow.

> Will a faster cpu actually solve the issue ?
> My cpu load never reached a value > 0.50 as far as i know and average is
> like 0.30

You mean - 50% CPU usage across all CPUs?  That's looks high
enough, though not critical.  But it may be a good idea to look
into per-CPU stats, as well as per process CPU usage.

Note well, CPU is a bottleneck I assumed based on few external
tests.  It may not be a CPU, but, e.g., a packet loss somewhere.
And, as I already said, numbers shown by Pingdom are close to
theoretical minimum, and I don't think there is much room for
improvement.  The one extra RTT probably deserves investigation,
but I can't say it's an "issue" - it might be even legitimate.

--
Maxim Dounin
http://nginx.org/
2974d09ac2541e892966b762aad84943?d=identicon&s=25 khav (Guest)
on 2014-07-01 17:00
(Received via mailing list)
I am currently using 1024 bit dhparams for maximum compatibility


Here is my ssllabs report :
https://www.ssllabs.com/ssltest/analyze.html?d=fil...

If i remove the DH from my cipher suites , will handshake simulation be
still a success for all browsers listed in the ssllabs report above


What is the best cipher suite according to you that is both fast and has
maximum compatibility ?

Thanks again

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,251277,251396#msg-251396
Please log in before posting. Registration is free and takes only a minute.
Existing account

NEW: Do you have a Google/GoogleMail, Yahoo or Facebook account? No registration required!
Log in with Google account | Log in with Yahoo account | Log in with Facebook account
No account? Register here.