FreeBSD & Linux disk AIO support

Experimental patch to support disk AIO on FreeBSD and Linux.

./configure --with-file-aio …

nginx.conf:

location / {
aio on;
output_buffers 1 128k;
}


Linux: at least 2.6.22 kernel is required. Probably, you need to enable
directio as well (otherwise kernel may fall back to synchronous IO,
however, I’m not sure):

location / {
aio on;
directio 1;
output_buffers 1 128k;
}


FreeBSD: should work on 4+, however, 7.x is preferable: on 6.x the
network
subsystem is GiantLock’ed if AIO is used.

Kernel must support AIO:

options VFS_AIO
or
kldload aio

Probably you need to tune the sysctls:

vfs.aio.max_aio_queue (default) 1024
vfs.aio.max_aio_queue_per_proc 256
vfs.aio.max_aio_per_proc 32
vfs.aio.max_aio_procs 32

You may diagnose load using

vfs.aio.num_queue_count
vfs.aio.num_aio_procs

If kernel AIO queue overflows then nginx fall back to synchronous IO.

Hi, I try to build nginx using the patch, but it doesnt work for me. I
have customer build kernel and all the “aio” related headers are in
/lib/modules//source and build. Using configure, it is
settings this to /usr/include and that doesnt work:

make -f objs/Makefile
make[1]: Entering directory /users/root/downloads/source/nginx-0.8.10' gcc -c -O3 -mcpu=prescott -march=prescott -mtune=prescott -pipe -fomit-frame-pointer -I src/core -I src/event -I src/event/modules -I src/os/unix -I objs \ -o objs/src/core/nginx.o \ src/core/nginx.c In file included from src/core/ngx_config.h:25, from src/core/nginx.c:7: src/os/unix/ngx_linux_config.h:85:25: error: sys/eventfd.h: No such file or directory make[1]: *** [objs/src/core/nginx.o] Error 1 make[1]: Leaving directory/users/root/downloads/source/nginx-0.8.10’
make: *** [build] Error 2

On Wed, Aug 26, 2009 at 02:22:28PM +0200, Robert G. wrote:

-o objs/src/core/nginx.o \
src/core/nginx.c

In file included from src/core/ngx_config.h:25,
from src/core/nginx.c:7:
src/os/unix/ngx_linux_config.h:85:25: error: sys/eventfd.h: No such file
or directory
make[1]: *** [objs/src/core/nginx.o] Error 1
make[1]: Leaving directory `/users/root/downloads/source/nginx-0.8.10’
make: *** [build] Error 2

Try the attached updated patch.

Igor S. wrote:

On Wed, Aug 26, 2009 at 02:22:28PM +0200, Robert G. wrote:

-o objs/src/core/nginx.o \
src/core/nginx.c

In file included from src/core/ngx_config.h:25,
from src/core/nginx.c:7:
src/os/unix/ngx_linux_config.h:85:25: error: sys/eventfd.h: No such file
or directory
make[1]: *** [objs/src/core/nginx.o] Error 1
make[1]: Leaving directory `/users/root/downloads/source/nginx-0.8.10’
make: *** [build] Error 2

Try the attached updated patch.

The patch worked, but got another error:

objs/src/core/ngx_output_chain.o: In function ngx_output_chain': ngx_output_chain.c:(.text+0x6aa): undefined reference tongx_file_aio_read’
collect2: ld returned 1 exit status
make[1]: *** [objs/nginx] Error 1
make[1]: Leaving directory `/users/root/downloads/source/nginx-0.8.10’
make: *** [build] Error 2

On Wed, Aug 26, 2009 at 03:11:55PM +0200, Robert G. wrote:

make[1]: Leaving directory /users/root/downloads/source/nginx-0.8.10' make[1]: *** [objs/nginx] Error 1 make[1]: Leaving directory/users/root/downloads/source/nginx-0.8.10’
make: *** [build] Error 2

The new patch.

Igor S. wrote:

On Wed, Aug 26, 2009 at 03:11:55PM +0200, Robert G. wrote:

make[1]: Leaving directory /users/root/downloads/source/nginx-0.8.10' make[1]: *** [objs/nginx] Error 1 make[1]: Leaving directory/users/root/downloads/source/nginx-0.8.10’
make: *** [build] Error 2

The new patch.

Hi, it worked, it was build perfectly, but how can I know now if it’s
using aio or not and what would be the “improvements” ?

Is there a way to test this?

On Wed, Aug 26, 2009 at 03:49:12PM +0200, Robert G. wrote:

Hi, it worked, it was build perfectly, but how can I know now if it’s
using aio or not and what would be the “improvements” ?

Is there a way to test this?

You need set

location / {
aio on;
directio 128k;
output_buffers 1 128k;
}

I do not know how to learn if nginx uses AIO on Linux or not.
I can say it only for FreeBSD. On Linux I looked in debug log.

Igor S. wrote:

On Wed, Aug 26, 2009 at 03:49:12PM +0200, Robert G. wrote:

Hi, it worked, it was build perfectly, but how can I know now if it’s
using aio or not and what would be the “improvements” ?

Is there a way to test this?

You need set

location / {
aio on;
directio 128k;
output_buffers 1 128k;
}

I do not know how to learn if nginx uses AIO on Linux or not.
I can say it only for FreeBSD. On Linux I looked in debug log.

OK i guess, I have set it up for a few vhosts, Im not sure there is some
kind of improvement, as most of the vhosts are with php, but don’t see
any slowness neither, so … I’m not sure :slight_smile:

P.S. I’m running it on Ubuntu 8.04.3 with 2.6.30.5 custom kernel.

On Wed, Aug 26, 2009 at 04:20:56PM +0200, Robert G. wrote:

location / {
any slowness neither, so … I’m not sure :slight_smile:

P.S. I’m running it on Ubuntu 8.04.3 with 2.6.30.5 custom kernel.

You should see improvements on a host where whole hot static content
does not fit in the host physical memory.

-------- Original-Nachricht --------

Datum: Wed, 26 Aug 2009 17:59:30 +0400
Von: Igor S. [email protected]
An: [email protected]
Betreff: Re: FreeBSD & Linux disk AIO support

make: *** [build] Error 2
location / {
aio on;
directio 128k;
output_buffers 1 128k;
}

I seem to be not able to find any documentation about the aio module in
nginx. Do you have somewhere hidden a documentation about the parameters
nginx supports in relation to aio module?

On Wed, Aug 26, 2009 at 04:55:34PM +0200, Steve wrote:

On Wed, Aug 26, 2009 at 03:11:55PM +0200, Robert G. wrote:
Hi, it worked, it was build perfectly, but how can I know now if it’s
}

I seem to be not able to find any documentation about the aio module in nginx. Do you have somewhere hidden a documentation about the parameters nginx supports in relation to aio module?

This is new patch.

-------- Original-Nachricht --------

Datum: Wed, 26 Aug 2009 19:02:20 +0400
Von: Igor S. [email protected]
An: [email protected]
Betreff: Re: FreeBSD & Linux disk AIO support

The new patch.
aio on;
directio 128k;
output_buffers 1 128k;
}

I seem to be not able to find any documentation about the aio module in
nginx. Do you have somewhere hidden a documentation about the parameters
nginx supports in relation to aio module?

This is new patch.

Okay. Then it has nothing to do with the --with-aio_module configure
switch that has been in nginx for quite a while. Right?

On Wed, Aug 26, 2009 at 05:37:20PM +0200, Steve wrote:

-------- Original-Nachricht --------

make[1]: Leaving directory
using aio or not and what would be the “improvements” ?

I seem to be not able to find any documentation about the aio module in
nginx. Do you have somewhere hidden a documentation about the parameters
nginx supports in relation to aio module?

This is new patch.

Okay. Then it has nothing to do with the --with-aio_module configure switch that has been in nginx for quite a while. Right?

No. The --with-aio_module is for very experimental socket AIO module.

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs