Asynchronous file processing

Hi all,

I’m working on a native nginx module in which I want to read an input
file
and perform some manipulations on its data. Since the files I’m reading
are
big and accessed over NFS, I want to use asynchronous I/O for reading
them,
and I want to implement it as a pipeline of chunks, i.e. read a chunk,
process the chunk, add the chunk to the response chain.

My questions are as follows:

  1. Is there any sample / documentation / basic guidelines you can give
    me on
    how to progressively output data back to the client ?
  2. Does nginx support generating responses with chunked-encoding (I may
    not
    be able to determine the response size without processing the whole
    file,
    and would prefer to avoid keeping the entire processed buffer in memory
    at
    once) ?
  3. Can you send me some guidelines on how to perform asynchronous file
    read
    (my understanding is that I need to use ngx_file_aio_read and if I get
    NGX_AGAIN, set the handler/data members of file->aio to my completion
    callback) ?

Thank you in advance,

Eran

Posted at Nginx Forum: