Nginx convert UTF8 request to ISO-8859-1

Server NGINX
1

Hi,

IE8 (maybe also IE9/IE10) doesn’t auto encode url (firefox do), and can
make utf8 requests
If you put “http:///?test=ééé” in the address bar, the é
will not
be html encoded, and will be sent encoded in utf8 (c3a9 in hex, i’ve
checked with wireshark)

The problem is that the fastcgi backend (mono webapp, unix socket)
get the é in ISO-8859-1 (e9 in hex, i’ve checked with socat)

Is it normal that nginx (1.4.1) convert the request encoding from UTF8
to ISO-8859-1?

Is there a workaround (linux/nginx conf)? (haven’t found any yet)

What the RFCs are saying? (HTTP request encoding, Fastcgi param
encoding)

I’m not using rewrite in nginx, i’m just passing the request to
a fastcgi unix socket.
(I will provide a minimal test conf/do more tests tomorrow)

To reproduce:
Request: curl -O “http:///?test=ééé”
Request capture: sudo tcpdump -X ‘port 80’
Backend capture: sudo socat -t100 -x -v
UNIX-LISTEN:/path/to/sock,mode=777,reuseaddr,fork
UNIX-CONNECT:/path/to/sock.original

Thanks in advance
Etienne

2

On Tue, Sep 24, 2013 at 10:19:51PM +0200, [email protected]
wrote:

Hi there,

If you put “http:///?test=” in the address bar, the will not
be html encoded, and will be sent encoded in utf8 (c3a9 in hex, i’ve checked
with wireshark)

The problem is that the fastcgi backend (mono webapp, unix socket)
get the in ISO-8859-1 (e9 in hex, i’ve checked with socat)

When I use:

==
server {
listen 8080;
location = / {
fastcgi_param QUERY_STRING $query_string;
fastcgi_pass 127.0.0.1:9;
}
}

and

tcpdump -nn -i any -X -s 0 port 8080 or port 9

and

curl http://localhost:8080/?key=

followed by some bytes, I don’t see any difference in the bytes in
the to-8080 “GET /?key=” and the to-9 “QUERY_STRINGkey=” parts of the
tcpdump output.

What am I doing that is different to you?

f

Francis D. [email protected]

3

Hi,

----- Mail original -----

If you put “http:///?test=ééé” in the address bar,
server {
tcpdump -nn -i any -X -s 0 port 8080 or port 9

and

curl http://localhost:8080/?key=

followed by some bytes, I don’t see any difference in the bytes in
the to-8080 “GET /?key=” and the to-9 “QUERY_STRINGkey=” parts of the
tcpdump output.

What am I doing that is different to you?

Sorry today i’m not able to reproduce my ‘bug’
Also not able to send utf8 url with IE
We (me and my collegue) must have misread the wireshark dump…
(User error - Wikipedia)

With curl & IE i’ve tested nginx works perfectly (UTF8 in - UTF8 out /
Latin1 in - Latin1 out)

Thanks and sorry