Websocket proxying module questions

Server NGINX
1

Hello,

I’m wondering if it’s possible to make a module for proxying websockets
for nginx.

I read Emiller’s Guide to Nginx Module Development – Evan Miller …

From what I understand nginx normally works like this:

  1. keep connection with client
  2. connect to proxy and make request
  3. do something else
  4. get response from proxy and reply to client

Now, a websocket connection will drop when it issues the upgrade command
to nginx. A couple of questions

  1. Does nginx need a persistent connection per client with the proxy to
    do websocket proxying?
  2. For proper websocket proxying would the proxy rather have to be a
    gateway or tunneling proxy in nature?
  3. If 2 holds, does that even fit into the nginx architecture?

I know there is handshake issues with draft 76, but assuming I ignore
them, is it possible to implement a websocket proxy to nginx?
If so, can anyone give me some hints on how to do that? If not what
would be the alternative? If I were to build a websocket module
for nginx, that does all the handshaking inside nginx, how would the
backend notify nginx that there’s new data available?

Regards,
Reza

Posted at Nginx Forum:

2

ok another update.~m~16~m~5890598851256073…~m~16~m~~j~{“buffer”:[]}.
doesn’t belong in the handshake response. it’s actually websocket
framing.

the socket.io server will respond not only with the handshake response
but immediately throw a session id back at nginx. since nginx doesn’t
know anything about websocket framing, it will think that it belongs to
the challenge response and mess it up. any suggestions on how to switch
from http to websocket framing?

Posted at Nginx Forum:

3

ok little update. first problem is obviously the missing 8 byte of the
handshake.

what happens is the following:

clients sends handshake

GET /socket.io/websocket HTTP/1.1
Upgrade: WebSocket
Connection: Upgrade
Host: websocket.local
Origin: http://websocket.local
Sec-WebSocket-Key1: '6F24 >168D1m/2T7b
Sec-WebSocket-Key2: 3 84w6 5 u{U45 7 17
Cookie: socketio=websocket

12345678

the server would forward the following:
GET /socket.io/websocket HTTP/1.0
Host: websocket.local
X-Real-IP: 127.0.0.1
X-Forwarded-For: 127.0.0.1
Connection: close
Upgrade: WebSocket
Origin: http://websocket.local
Sec-WebSocket-Key1: '6F24 >168D1m/2T7b
Sec-WebSocket-Key2: 3 84w6 5 u{U45 7 17
Cookie: socketio=websocket

note the missing websocket key3.

This is easily worked around(fixed) by adding

        if (ngx_strcasestrn(h->key.data, "upgrade", 7 - 1) &&
            ngx_strcasestrn(h->value.data, "websocket", 9 - 1)) {
                r->headers_in.content_length_n = 8;
                ngx_log_debug2(NGX_LOG_DEBUG_HTTP,

r->connection->log, 0,
“Upgrade header found”);
}

to ngx_http_request.c:ngx_http_process_request_headers
the next thing i don’t get, node will response with a handshake:
HTTP/1.1 101 WebSocket Protocol Handshake
Upgrade: WebSocket
Connection: Upgrade
Sec-WebSocket-Origin: http://websocket.local
Sec-WebSocket-Location: ws://websocket.local/socket.io/websocket

n.E…Q.}…/.(s.~m~16~m~5890598851256073…~m~16~m~~j~{“buffer”:[]}.

nginx will return the following:
HTTP/1.1 101 WebSocket Protocol Handshake
Server: nginx/0.7.67
Date: Fri, 19 Nov 2010 13:41:48 GMT
Transfer-Encoding: chunked
Connection: keep-alive
Upgrade: WebSocket
Sec-WebSocket-Origin: http://websocket.local
Sec-WebSocket-Location: ws://websocket.local/socket.io/websocket

44
n.E…Q.}…/.(s.~m~16~m~5890598851256073…~m~16~m~~j~{“buffer”:[]}.

Question:

  1. where do the 44 come in? how do they fit in?
  2. can i store the state of the connection as being a websocket now. so
    that it doesn’t rewrite the connection header?

Regards,
Reza

Posted at Nginx Forum:

4

another quick update, adding a content-length: 16 header in my socket.io
instance(note that’s not part of the websocket draft). nginx will
forward the handshake response properly. and chrome and safari will
establish a websocket connection which works only server → client. i
explained it a bit more detailed in irc

14:02 [ timebomb ] no
14:02 [ timebomb ] server to client
14:02 [ timebomb ] there’s 2 problems
14:03 [ timebomb ] no.1 when you make a high timeout proxy connection,
nginx will assume the websocket frames are just the rest of the previous
html response. so it will keep forwarding the websocket requests from
the server back to the client
14:03 [ timebomb ] however the frontend side isn’t able to send data
back over that same connection
14:04 [ timebomb ] no.2 once the websocket connection is established
the framing is something like 00 data ff
14:04 [ timebomb ] it will look like this:
14:04 [ timebomb ] 127.0.0.1 - - [21/Nov/2010:13:28:27 +0100] “GET
/socket.io/websocket HTTP/1.1” 101 589 “-” “-”
14:04 [ timebomb ] 127.0.0.1 - - [21/Nov/2010:13:28:27 +0100]
“\x00~m~4~m~~h~1#” 400 173 “-” “-”
14:04 [ timebomb ] nginx will return a 400 bad request answer
14:05 [ timebomb ] the best solution i can think of right now is to
wrap that request in a html get at frontend side and then unwrap it
internally at the proxy side
14:07 [ timebomb ] also i needed to add a content-length: 16 header to
my node.js handshake response and a content-length: 8Â internally in
nginx for the handshake request otherwise nginx would drop the websocket
key 3
14:08 [ Evet ] things getting more complicated
14:09 *** netzsportler [[email protected]] has quit
[Ping timeout: 245 seconds]
14:09 [ timebomb ] about that wrap and unwrapping … it would be
easily avoided if there was a way to avoid the html filter chain once
the connection is established
14:09 [ timebomb ] i just don’t know how and if that’s possible

Posted at Nginx Forum: