Writev function not implemented

Server NGINX
#1

Hello,
I have compiled nginx on ubuntu 12.04 but I’m seeing a really strange
error when I try and POST a file through nginx. I get a line in my logs
that says:

2013/08/02 17:01:11 [crit] 26#0: *7 writev()
“/var/lib/nginx/client_body_temp/0000000001” failed (38: Function not
implemented), client: 172.16.42.1, server: , request: “POST
/tenants/cwstest/stunts/51fbe5d696bb27002d000001/uploads/51fbe5d796bb27002d000002/encrypt
HTTP/1.1”, host: “localhost:49172”

I then get a 500 error.

I have put together a gist with all the relevant information:
https://gist.github.com/ismell/6156617

If anyone could provide some assistance that would be great.

Thanks,
Raul

#2

2013/08/02 17:01:11 [crit] 26#0: *7 writev()
“/var/lib/nginx/client_body_temp/0000000001” failed (38: Function not
implemented)

On what filesystem does /var/lib/nginx/client_body_temp reside (like
‘cat
/proc/mounts’)?

rr

#3

The filesystem is AUFS. It’s mounted inside of a docker container.

[email protected]:/# cat /proc/mounts
rootfs / rootfs rw 0 0
none / aufs rw,relatime,si=2418709ef08a7cdd 0 0
proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0
sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0
rpool/ROOT/ubuntu-1 /sbin/init zfs ro,relatime,xattr 0 0
data/docker /etc/resolv.conf zfs ro,relatime,xattr 0 0
devpts /dev/tty1 devpts
rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0
devpts /dev/pts devpts rw,relatime,mode=600,ptmxmode=666 0 0
devpts /dev/ptmx devpts rw,relatime,mode=600,ptmxmode=666 0 0

So my assumption is that AUFS does not support writev? So I need to
somehow mount a different filesystem?

#4

On 05/08/13 21:13, Rangel, Raul wrote:

The filesystem is AUFS. It’s mounted inside of a docker container.

So my assumption is that AUFS does not support writev? So I need to somehow
mount a different filesystem?

Hi
I can’t comment about AUFS, but you can change where those temp files
are stored if you wanted to make a small partition dedicated as a temp
directory
http://nginx.org/en/docs/http/ngx_http_core_module.html#client_body_temp_path

#5

So I just tried you little script inside my container (AUFS on top of
ZFS):

[email protected]:/# ./a.out
writev: Function not implemented
[email protected]:/#

Then I tried my script outside of the container (ZFS):
[email protected]:~/Projects/service/services/upload$ ./a.out
6

Here is my uname:
Linux slagathor 3.8.0-27-generic #40~precise3-Ubuntu SMP Fri Jul 19
14:38:30 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux

The plot thickens…

#6

Rangel, Raul wrote in post #1117817:

So my assumption is that AUFS does not support writev? So I need to
somehow mount a different filesystem?

I wrote a quick and dirty C program to test writev() on AUFS, and it
worked like a charm here (3.8 Debian kernel).

https://gist.github.com/jpetazzo/6160048

What’s the underlying filesystem? ZFS?

#7

On 5 August 2013 23:22, Rangel, Raul [email protected] wrote:

Here is my uname:
Linux slagathor 3.8.0-27-generic #40~precise3-Ubuntu SMP Fri Jul 19 14:38:30 UTC
2013 x86_64 x86_64 x86_64 GNU/Linux

The plot thickens…

Not really. Check http://aufs.sourceforge.net/ for the string
“writev”. It’s not implemented.

J

#8

I believe it is implemented.

I re-did my tests on:

  • 3.10 (Debian)
  • 3.8 (Debian)
  • 3.8.0-27-generic (Ubuntu, the same as yours)
  • 3.2.0-40 (Ubuntu)
  • 2.6.38.2 (in-house)

They all worked.

I don’t understand exactly how AUFS passes writev to the underlying
filesystem, but there might be some weird interaction with ZFS. I tried
with tmpfs, ext4, and btrfs, they all worked.

Then I wondered if it could have been caused by something special in
Docker, so I tried within a Docker container (not just in a manual AUFS
mount) - and it worked.

Would you mind trying with a non-ZFS backend? (I’m asking just because
it will be much faster for you to test with a non-ZFS backend, than for
me to re-install ZFS on my Linux machine :-))

#9

Actually, I went ahead and rebuilt SPL and ZFS on my machine, and did an
AUFS mount over ZFS… And wvtest ran, no problem.

#10

Rangel, Raul wrote in post #1117896:

The first one was I used -v /var/lib/nginx to create a volume which bind
mounted a zfs directory inside my container. This worked correctly.

I was about to suggest that as a workaround.
I’m glad that it worked!

The second was I created an ext4 partition and used docker -g to set the
graph path. When I tried my test again it worked. So it does seem to be
a strange interaction between AUFS and zfs.

I’m wondering why your setup is working but mine isn’t.

Indeed. I’m using ZFS on Linux with DKMS, as packaged by Debian:
ii spl-dkms 0.6.1-2
ii zfs-dkms 0.6.1-1~wheezy

Which flavor and version of ZFS are you using?

#11

are there any official recommendations from nginx to safeguard against
the BREACH exploit ?

http://breachattack.com/

http://arstechnica.com/security/2013/08/gone-in-30-seconds-new-attack-plucks-secrets-from-https-protected-pages/

#12

So I tried two different things.

The first one was I used -v /var/lib/nginx to create a volume which bind
mounted a zfs directory inside my container. This worked correctly.

The second was I created an ext4 partition and used docker -g to set the
graph path. When I tried my test again it worked. So it does seem to be
a strange interaction between AUFS and zfs.

I’m wondering why your setup is working but mine isn’t.

#13

And what if we are using gzip_static?
As far as I understand, we have to block gzipping page code. But what
about
.js .css with no secure content?

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,241591,241794#msg-241794

#14

On Aug 12, 2013, at 21:32 , offmind wrote:

And what if we are using gzip_static?
As far as I understand, we have to block gzipping page code. But what about
.js .css with no secure content?

Statically gzipped files do not depend on user input so they are not
subject
to BREACH.

–
Igor S.
http://nginx.com/services.html

#15

On Aug 7, 2013, at 1:48 , Jonathan V. wrote:

are there any official recommendations from nginx to safeguard against the
BREACH exploit ?

http://breachattack.com/

http://arstechnica.com/security/2013/08/gone-in-30-seconds-new-attack-plucks-secrets-from-https-protected-pages/

“gzip off” от SSL-enabled sites.

–
Igor S.
http://nginx.com/services.html