502 Bad Gateway/while SSL handshaking to upstream

I have been fighting with this one for some time, would appreciate
some assistance.

in my test environment on ec2, I have:

[ nginx -> haproxy ] -> [ apache w/ ajp -> tomcat ] -> [ mysql cluster ]

nginx and haproxy on the same machine, apache and tomcat on the same
machine - and the mysql cluster has 2-4 sql nodes+data nodes.

I have been testing with a single GET request, which exercises all of
the above (ex. ab -n 200 -c 200). the net of the problem is that 95%
of the time, 1/3 to 1/2 of my connections result in a 502 returned by
nginx. I’ve yet to have the full 200 completed requests.

the nginx error.log contains both types of errors below, but there is
a direct relation to the failures - if 93/200 connections result in a
502, I will have exactly 93 SSL handshake errors - so I’ve narrowed
the problem down I believe.

a)
2010/04/23 07:49:43 [error] 18430#0: *364 SSL_do_handshake() failed
(SSL: error:1408F119:SSL routines:SSL3_GET_RECORD:decryption failed or
bad record mac) while SSL handshaking to upstream, client: 174.1.X.X,
server: api.example.com, request: “GET /1.0/getName HTTP/1.0”,
upstream: “https://127.0.0.1:8003/1.0/getName”, host:
api.example.com

b)
2010/04/23 07:49:41 [info] 18430#0: *260 peer closed connection in SSL
handshake while SSL handshaking to upstream, client: 174.10.X.X,
server: api.example.com, request: “GET /1.0/getName HTTP/1.0”,
upstream: “https://127.0.0.1:8003/1.0/getName”, host:
api.example.com

unfortunately, it’s unclear why this is happening. nginx config:

server {
listen 0.0.0.0:443;
server_name api.example.com;

  location ~ /* {
    proxy_pass         https://api_443;
  }

  access_log            logs/api.mobioid.com-access.log main;

  ssl                   on;
  keepalive_timeout     70;
  ssl_certificate       /u/nginx/ssl/api.example.com.crt;
  ssl_certificate_key   /u/nginx/ssl/api.example.com.key;

}

upstream api_443   { server 127.0.0.1:8003; }

and haproxy config on port 8003:

listen api_443 127.0.0.1:8003
mode tcp
option ssl-hello-chk
server s1.example.com 10.242.X.X:443 weight 1 minconn 3
maxconn 125 check inter 20000

api.example.com is a trusted cert, s1.example.com/apache is a
self-signed cert.

a previous email thread by David Taveras last month, asked about
essentially the same problem:

http://nginx.org/pipermail/nginx/2010-March/019120.html

and my config is also similar. unfortunately no one had responded to his
query.

I’m hopeful the community will be able to lend a hand here!

teege

and I’m running the following versions, meant to include this
originally:

$ nginx/sbin/nginx -v
nginx version: nginx/0.7.65
$ haproxy/bin/haproxy -v
HA-Proxy version 1.4.2 2010/03/17
$ apache/bin/httpd -v
Server version: Apache/2.2.14 (Unix)
$ tomcat/bin/version.sh
Server version: Apache Tomcat/6.0.20

teege

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs