Simple module question

Server NGINX
1

Hi,

I just want to make a simple module which get uploaded image to a
directory with a custom name. All things seem to be good but
ngx_http_read_client_request_body create local image with the form
header (filename=“test.jpg” Content-Type :application-octet-stream
etc…). Do you know a simple way to remove it ? I want avoid a new
reading from disk.

Thanks

2

Chavelle V. ha scritto:

Hi,

I just want to make a simple module which get uploaded image to a
directory with a custom name. All things seem to be good but
ngx_http_read_client_request_body create local image with the form
header (filename=“test.jpg” Content-Type :application-octet-stream
etc…). Do you know a simple way to remove it ? I want avoid a new
reading from disk.

Nginx does not decode the request body.

I’m planning to write a function that does the decoding, for use in
mod_wsgi, but I don’t now when I will find the time.

It will not exacly be what you want, but the function can be reused.
If you manage to write such function before me, please let me know (it’s
not an easy task)!

Thanks

Manlio P.

3

Hi!

In my patch for processing uploads you can find an API to process
multipart/form-data bodies:

http://www.grid.net.ru/nginx/nginx-upload-patch.html

What you need are files upload.c and upload.h.

I’m actually working now on the next version of this module which will
be able to cut out and store uploaded files in temporal directory and
proxy the remaining part of the body to a backend with brief information
about files uploaded.

However I’m now faced with a curious problem regarding nginx internals
and I expect a solution in 1-2 weeks.

4

Hi!

I’ve forgotten to clarify, that HTML form uploads are encapsulated in
multipart/form-data format, thus “decoding of a request body” in your
case would probably mean “decoding mulipart/form-data”. That’s why my
previous message considers processing of multipart/form-data format.

5

Valery K. wrote:

In my patch for processing uploads you can find an API to process
multipart/form-data bodies:

Nginx upload module

What you need are files upload.c and upload.h.

It’s exactly what I want. Many thanks !
Your module should be on NginxModules page :slight_smile:

6

Valery K. ha scritto:

Hi!

In my patch for processing uploads you can find an API to process
multipart/form-data bodies:

Nginx upload module

What you need are files upload.c and upload.h.

Thanks!

I’m actually working now on the next version of this module which will
be able to cut out and store uploaded files in temporal directory and
proxy the remaining part of the body to a backend with brief information
about files uploaded.

My idea is to:

  1. Modify the content-type from multipart/form-data to
    application/x-www-form-urlencoded
  2. Any non file parts are urlencoded and written to the new body
  3. Files are written to temporary files, and in the new body is written:
    name.filename=filename&name.path=path&name.content-type=content-type
    where name is the field name and path is the temporary path used to
    store the file.

With this method an application needs only to care about urlencoded
request body.

However I’m now faced with a curious problem regarding nginx internals
and I expect a solution in 1-2 weeks.

Manlio P.

7

Manlio P. wrote:

application/x-www-form-urlencoded
2) Any non file parts are urlencoded and written to the new body
3) Files are written to temporary files, and in the new body is written:
name.filename=filename&name.path=path&name.content-type=content-type
where name is the field name and path is the temporary path used to
store the file.

With this method an application needs only to care about urlencoded
request body.

The problem with application/x-www-form-urlencoded is that it is
percent-encoded and multipart/form-data is not, thus requiring us to
deal with encoding, etc. Dealing with encodings is generally a bad idea
and I prefer to think about request body as a stream of bytes.

Another problem is that if we rename attributes to name.filename= and
name.path= this will require modifying existing applications and such
thing would be quite unuseful.

Therefore what I do now is transferring almost identical request body to
the backend with file fields converted to normal fields containing
following line:

The current code is available under

http://www.grid.net.ru/nginx/modules/nginx_upload_module-1.99.0.tar.gz

but it has a problem with generating header for error responses. Use it
on your own risk.

8

Valery K. ha scritto:

  1. Any non file parts are urlencoded and written to the new body
    deal with encoding, etc. Dealing with encodings is generally a bad idea
    and I prefer to think about request body as a stream of bytes.

You are right, your method is more correct.

Thanks Manlio P.

9

Chavelle V. wrote:

Valery K. wrote:

In my patch for processing uploads you can find an API to process
multipart/form-data bodies:

Nginx upload module

What you need are files upload.c and upload.h.

It’s exactly what I want. Many thanks !
Your module should be on NginxModules page :slight_smile:

Thanks, but the first version of this module is not mature enough from
my point of view. It solves a problem with extracting files from
multipart/form-data, but it doesn’t solve the problem of handling
uploads them self.

However I’ll appreciate for any constructive feedback about my work.