Bad Decompression error after default ssl_session_timeout

Server NGINX
1

Hi,

I have an LB setup with nginx for an ssl enabled site which load balance
with 2 apache servers. All the servers are CentOS5.5* and OpenSSL
0.9.8e-fips-rhel5 01 Jul 2008. Also we are using the same SSL
certificate on all the 3 servers.

It does load balance perfectly untill 5m. After that it raises an
error:

[crit] 5179#0: *6 SSL_do_handshake() failed (SSL: error:1408F06B:SSL
routines:SSL3_GET_RECORD:bad decompression) while SSL handshaking to
upstream, client: clientip, server: lb.abcd.net, request: “GET /search/
HTTP/1.1”, upstream: “https://server1-ip:443/search/”, host:
“lb.abcd.net”

This error happens for both server1 and server2. After this, the load
balancer is not working.

The following are the nginx conf.

http {
include mime.types;
default_type application/octet-stream;
autoindex off;
ssi off;
server_tokens off;

log_format  main  '$remote_addr [$time_local] - "$request" - '
                  '$status - $body_bytes_sent - "$http_referer"';

log_format load_b '$remote_addr [$time_local] - "$request" - $status

  • ’
    'worker_addr $upstream_addr - ’
    'worker_status $upstream_status - ’
    'worker_response_time $upstream_response_time - ’
    'total_processing_time $request_time - ’
    ‘content_type $upstream_http_content_type’;

    access_log logs/access.log main;

    sendfile on;
    keepalive_timeout 65;

    gzip on;
    gzip_http_version 1.1;
    gzip_proxied expired no-cache no-store private auth;
    gzip_types text/plain application/xml text/css
    application/x-javascript text/xml;
    gzip_disable “MSIE [1-6].”;

    proxy_ssl_session_reuse on;

    upstream loadbalancer {
    server server1-ip:443 weight=1 max_fails=5 fail_timeout=3m;
    server server2-ip:443 weight=1 max_fails=5 fail_timeout=3m;
    }
    server {
    listen 443 ssl;
    server_name lb.abcd.net;
    location ~* ^.+.(jpg|jpeg|gif|png|ico|css|txt|js|htm|html)$ {
    expires 24h;
    add_header Cache-Control public;
    root /home/abc/media;
    }

      ssl_certificate      /root/Apache_New_SSL_Keys/abcd.co.uk.crt;
  ssl_certificate_key  /root/Apache_New_SSL_Keys/abcd.key.nopass;
  ssl_session_timeout  3m;
  ssl_protocols  SSLv3;

  proxy_redirect off;
  proxy_set_header Host $host;
  proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

location / {
proxy_pass https://loadbalancer;
access_log logs/access_lb.log load_b;
}

    error_page  403 templates/403.html;
    error_page  404 templates/404.html;
    error_page   500 502 503 504  /50x.html;

    location = /50x.html {
        alias templates/500.html;
    }
}

}

What can be the issue? Thanks in advance.

Posted at Nginx Forum:

2

Hello!

On Tue, May 22, 2012 at 03:15:31AM -0400, lima wrote:

[crit] 5179#0: *6 SSL_do_handshake() failed (SSL: error:1408F06B:SSL
routines:SSL3_GET_RECORD:bad decompression) while SSL handshaking to
upstream, client: clientip, server: lb.abcd.net, request: “GET /search/
HTTP/1.1”, upstream: “https://server1-ip:443/search/”, host:
“lb.abcd.net”

This error happens for both server1 and server2. After this, the load
balancer is not working.

[…]

What can be the issue? Thanks in advance.

This looks like problem with session resumption and compression in
OpenSSL version you are using. Obvious workaround is to use

proxy_ssl_session_reuse off;

in nginx config, see Module ngx_http_proxy_module.

Alternatively you may try upgrading openssl or recompiling one you
are using without zlib support.

Maxim D.

3

Hello!

On Wed, May 23, 2012 at 03:03:33AM -0400, lima wrote:

Thanks for the reply.

The first solution has solved the problem!! unfortunately we cannot make
proxy_ssl_session_reuse off, as it may affect the performance. And the
second option given by you is also cannot be done as the system team has
some concern over it. The Openssl version we are using is the latest one
supported by the CentOS version we are using.

Is it possible to configure/compile nginx without zlib support?

This isn’t about nginx and zlib support, it’s about OpenSSL and
zlib support.

The zlib support in the OpenSSL version you are using is known to
have problems, and it can’t be switched off dynamically as the
SSL_OP_NO_COMPRESSION option appeared only in OpenSSL 1.0.0.

Another workaround you may try is to force SSLv2 between nginx and
backend servers (by using appropriate settings on backends), it
should eliminate compression as it’s not supported in SSLv2
protocol.

You may also try compiling nginx statically with newer version of
OpenSSL (or the same one, but without zlib support) by using
./configure --with-openssl=…, it might help as well.

Again, is the gzip module creating problem here? We tried with gzip on
and off but still it was giving the same problem.

No, gzip module is completely unrelated.

Maxim D.

4

On Wed, May 23, 2012 at 03:03:33AM -0400, lima wrote:

Again, is the gzip module creating problem here? We tried with gzip on
and off but still it was giving the same problem.

Try the attached patch.

5

Thanks for the reply.

The first solution has solved the problem!! unfortunately we cannot make
proxy_ssl_session_reuse off, as it may affect the performance. And the
second option given by you is also cannot be done as the system team has
some concern over it. The Openssl version we are using is the latest one
supported by the CentOS version we are using.

Is it possible to configure/compile nginx without zlib support?

Again, is the gzip module creating problem here? We tried with gzip on
and off but still it was giving the same problem.

Thanks again.

Posted at Nginx Forum:

6

Hi Maxim,

That helped a lot!!. Thank you very much.
It is working fine with SSLv2. I will see if I can upgrade the OpenSSL
version and try with SSLv3.

Thanks again.

Posted at Nginx Forum: