The "chunkin" module: Experimental chunked input support for Nginx

Server NGINX
1

Hi, folks!

Pushed by those cutting-edge users on the list, I’ve quickly worked
out “chunkin” module which adds HTTP 1.1 chunked input support for
Nginx without the need to patching the core:

http://github.com/agentzh/chunkin-nginx-module

This module registers an access-phase handler that will eagerly read
and decode incoming request bodies when a “Transfer-Encoding: chunked”
header triggers a 411 error page in Nginx (hey, that’s what you have
to pay for avoiding patching the core ;)). For requests that are not
in the “chunked” transfer encoding, this module is a “no-op”.

To enable the magic, just turn on the “chunkin” config option like this:

chunkin on;
location /foo { … }
…

No other modification is required in your nginx.conf file. (The
“chunkin” directive is not allowed to use in the location block BTW.)

This module is still considered highly experimental and there must be
some serious bugs lurking somewhere. But you’re encouraged to play and
test it in your non-production environment and report any quirks to me
:slight_smile:

Efforts have been made to reduce data copying, dynamic memory
allocation, and other performance tuning, thus unfortunately raising
the risk of potential buffer handling bugs caused by premature
optimizations :stuck_out_tongue:

This module is not supposed to be merged into the Nginx core because
I’ve used Ragel to generate the chunked encoding parser for joy :slight_smile:

The following Nginx versions have been successfully tested by this
module’s (very limited) test suite:

0.8.0 ~ 0.8.24
0.7.21 ~ 0.7.63

The test suite definitely needs more test cases and the code is hacky
in various places. If you’re willing to contribute, feel free to ask
me for a commit bit in a private email :slight_smile:

Have fun!
-agentzh

2

Hi, agentzh

You are so awesome for the “thunkin” module implementation of ngix.
I am a beginner for ngix and linux.
I don’t know how to do a little for your “thunkin” module.

Kai. Wang

Posted at Nginx Forum:

3

This plugin is great. But I think that this feature should be built-in.
So it is very nice if someone can make a patch that adds HTTP 1.1
chunked input support for
Nginx

4

On Tue, Nov 17, 2009 at 10:28 AM, Dinh P. [email protected]
wrote:

This plugin is great.

Thanks :slight_smile:

But I think that this feature should be built-in.
So it is very nice if someone can make a patch that adds HTTP 1.1
chunked input support for
Nginx

Totally agreed even though I’m reluctant to do it myself :stuck_out_tongue:

Cheers,
-agentzh

5

On Mon, Nov 16, 2009 at 7:48 PM, wangkai [email protected] wrote:

Hi, agentzh

You are so awesome for the “thunkin” module implementation of ngix.

Well, I’m assuming you’re serious. It’s “chunkin” not “thunkin” :slight_smile:

I am a beginner for ngix and linux.
I don’t know how to do a little for your “thunkin” module.

To show how this module works in action, I’d use the “echo” module to
demonstrate the actual effect.

With both “echo” and “chunkin” modules compiled into your nginx, you
can try the following configuration:

chunkin on;
location /body {
    echo_read_request_body;           # try to read the request body
    echo_request_body;                    # emit the request body
    echo;                                            # print out a

trailing new line
}

Then from your command line:

$ curl -d ‘hello, world’ -H ‘Transfer-Encoding: chunked’
‘http://localhost/body’
hello, world

$ curl -d ‘hello, world’ ‘http://localhost/body’
hello, world

The first “curl” command enforces the “chunked” encoding to be used in
the HTTP request.

Then try turning off the “chunkin” directive:

chunkin off;                                      # turn off this

module
location /body {
echo_read_request_body; # try to read the request body
echo_request_body; # emit the request body
echo; # print out a
trailing new line
}

And try to send the previous requests again:

$ curl -d ‘hello, world’ -H ‘Transfer-Encoding: chunked’
‘http://localhost/body’

411 Length Required

411 Length Required

nginx/0.8.24

$ curl -d ‘hello, world’ ‘http://localhost/body’
hello, world

The first response just shows the fact that nginx does not support the
“chunked” encoding itself while the second just tests the normal
transfer encoding :slight_smile:

Note that the “chunkin” module does not rely on “echo” to function. I
just used “echo” for the sake of easy demonstration. FWIW, I also
makes heavy use of the “echo” module in the test suite of the
“chunkin” module, for example,

http://github.com/agentzh/chunkin-nginx-module/blob/master/test/t/sanity.t

I’d add this demo to the “chunkin” module’s wiki page later:

http://wiki.nginx.org/NginxHttpChunkinModule

Please consult the “echo” module’s wiki page for more detailed
explanation for the directives used above:

http://wiki.nginx.org/NginxHttpEchoModule

Cheers,
-agentzh