SSL slow on nginx

Server NGINX
1

For my site , ssl seems to be slow even though i got A+ on sslabs
(implemented ocsp stapling, Forward Secrecy , spdy)

Here is the result from pingdom

Notice the high connect time and high ssl negociation time

Here is my nginx conf

server {
listen 443 spdy default_server; #Change to 443 when SSL is on
ssl on;
ssl_certificate /etc/ssl/filterbypass.me.crt; #(or .pem)
ssl_certificate_key /etc/ssl/filterbypass.me.key.nopass;
ssl_client_certificate /etc/ssl/intermediate.crt;
ssl_protocols SSLv3 TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers
ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS;
ssl_prefer_server_ciphers on;
ssl_buffer_size 8k;
ssl_session_cache shared:SSL:10m;
ssl_dhparam /etc/ssl/dhparam.pem;
ssl_session_timeout 10m;
ssl_stapling on;
ssl_stapling_verify on;
ssl_trusted_certificate /etc/ssl/trustchain.crt;
resolver 8.8.8.8 8.8.4.4;
add_header Strict-Transport-Security “max-age=31536000;
includeSubDomains”;
#rest of config goes here
}

Posted at Nginx Forum:

2

Hi!
On Sat, 2014-06-28 at 01:14 -0400, khav wrote:

ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS;

includeSubDomains";
#rest of config goes here
}

Setting up a local DNS resolver, and adding it in as the first one may
make a bit of a help, as may implementing SPDY.

However, ( using webpagetest as my preferred tool… from Dulles as the
server is reported to be in Utah:
WebPageTest Test - Running web page performance and optimization tests... ) I’d say that the
problem may well be with the available power on the server itself: for
example, it takes .8s to download a 20KB woff file. Given the services
that you’re wanting to offer, I think you’re going to need a load more
grunt.

hth,

Steve

Steve H. BSc(Hons) MIITP

Linkedin: http://www.linkedin.com/in/steveholdoway
Skype: sholdowa

3

Thanks Maxim and GreenGecko for the insights

The worker process does match my number of cpu cores (running on 8 cores
atm)

How can i know the number of handshakes per seconds occurring on the
server

The openssl speed result have been posted on

for readability

You can find my full list of ssl ciphers here —>

If you can suggest “faster ciphers” with same level of compatibility , i
would be awesome

Will a faster cpu actually solve the issue ?
My cpu load never reached a value > 0.50 as far as i know and average is
like 0.30

Posted at Nginx Forum:

4

Hello!

On Tue, Jul 01, 2014 at 03:10:07AM -0400, khav wrote:

Thanks Maxim and GreenGecko for the insights

The worker process does match my number of cpu cores (running on 8 cores
atm)

Good. It may be also good idea to make sure you don’t have
multi_accept enabled, just in case.

How can i know the number of handshakes per seconds occurring on the
server

First of all, count the number of connections per second (and
requests per second) - it should be trivial, and may be extracted
even with nginx stub_status module. I would generally recommend
using logs though. With logs, you should be also able to count
number of uncached handshakes - by using $ssl_session_reused
variable and the $connection_requests one.

See here:

http://nginx.org/r/$ssl_session_reused
http://nginx.org/r/$connection_requests
http://nginx.org/r/log_format

The openssl speed result have been posted on openssl speed test for rsa2048 - Pastebin.com
for readability

So, basically, your server is able to do about 800 plain RSA
handshakes per second per core, 6400 handshakes total.

But as previously noted, things can be very much worse with DH
ciphers, especially if you are using 2048 bit dhparams (or
larger).

If you can suggest “faster ciphers” with same level of compatibility , i
would be awesome

It may be good idea to disable DH regardless of the level of
compatibility. It’s just too slow.

Will a faster cpu actually solve the issue ?
My cpu load never reached a value > 0.50 as far as i know and average is
like 0.30

You mean - 50% CPU usage across all CPUs? That’s looks high
enough, though not critical. But it may be a good idea to look
into per-CPU stats, as well as per process CPU usage.

Note well, CPU is a bottleneck I assumed based on few external
tests. It may not be a CPU, but, e.g., a packet loss somewhere.
And, as I already said, numbers shown by Pingdom are close to
theoretical minimum, and I don’t think there is much room for
improvement. The one extra RTT probably deserves investigation,
but I can’t say it’s an “issue” - it might be even legitimate.

–
Maxim D.
http://nginx.org/

5

Hello!

On Sat, Jun 28, 2014 at 01:14:16AM -0400, khav wrote:

For my site , ssl seems to be slow even though i got A+ on sslabs
(implemented ocsp stapling, Forward Secrecy , spdy)

Note that SSL Labs grades are about security, not about speed.

Here is the result from pingdom

Website Speed Test | Pingdom Tools

Notice the high connect time and high ssl negociation time

The test is done from Netherlands, the site is in US, RTT seems to
be about 170ms. So, a connect will take 170ms minimum (1RTT), and an
SSL
handshake without a cached session will take 340ms minimum
(2RTT). And these are only network costs, not counting any
computational costs for SSL.

Pingdom numbers for the first pages as I see are as follows:

SSL 168ms
Connect 572ms
Send 0ms
Wait 304ms
Receive 6ms

They seems to be labeled incorrectly (1st number is
actually connect time, while 2nd one is SSL handshake, not vice
versa as they are labeled). Connect time seems pretty much
normal, just 1 RTT, close to minimum possible. SSL handshake
time is a bit more than it could be, about 3 RTT for some reason.
Simple test here suggests that the cause is likely CPU usage on your
server - response to ServerHello is noticeably delayed.

One of the possible reasons is that you prefer ciphers with
forward secrecy, and they are CPU hungry, especially DH ones:

ssl_ciphers

ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS;

ssl_prefer_server_ciphers on;

You may try “openssl speed rsa2048” to find out how many
handshakes per core your server can handle. Note that ECDH with
256 bit curve will result in about 2x slowdown compared to plain
RSA, and DH with 2048 bit params - up to 10x slowdown.

Additionally, make sure that:

  • you’ve properly tuned number of worker processes to match your
    server cores, see Core functionality;

  • number of handshakes per seconds isn’t reaching numbers your
    server can handle, use “openssl speed” to find out.

–
Maxim D.
http://nginx.org/

6

I am currently using 1024 bit dhparams for maximum compatibility

Here is my ssllabs report :
https://www.ssllabs.com/ssltest/analyze.html?d=filterbypass.me

If i remove the DH from my cipher suites , will handshake simulation be
still a success for all browsers listed in the ssllabs report above

What is the best cipher suite according to you that is both fast and has
maximum compatibility ?

Thanks again

Posted at Nginx Forum: