# XML files truncated

**URL:** <https://www.ruby-forum.com/t/xml-files-truncated/189403>\
**Category:** NGINX\
**Created:** [May 31, 2010, 11:44am UTC](https://www.ruby-forum.com/t/xml-files-truncated/189403 "2010-05-31T11:44:22Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![axel](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/6de8d8/32.png) [@axel](https://www.ruby-forum.com/u/axel)\
**Post date:** [May 31, 2010, 11:44am UTC](https://www.ruby-forum.com/t/xml-files-truncated/189403/1 "2010-05-31T11:44:22Z")

</div>

Hi guys,

I recently migrated from Apache to Nginx. Really cool. My website is  
much faster. BUT I have a really weird issue that didn’t occur with  
Apache. It seems that some big files are sometimes truncated by the  
server. For instance, I generate a 200 entries videos sitemap (4600  
lines) for Google (Google allows up to 50 000 entries FYI). Quite often,  
Google cannot parse this XML file because it is truncated in the middle  
and the XML is therefore no longer valid. It seems I have the same issue  
when I browse this XML into Safari or Firefox. When I try a feed  
validator ([Feed Validator for Atom and RSS](http://validator.w3.org/feed/check.cgi)), the test fails 8  
times out of 10 and the source is clearly truncated.

I tried to disable gzip encoding or to adapt quite a lot of settings.  
Nothing did the trick. Have U any idea that could help?  
Thks a lot

Axel

---

<div class="post-metadata">

**Author:** ![axel](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/6de8d8/32.png) [@axel](https://www.ruby-forum.com/u/axel)\
**Post date:** [May 31, 2010, 12:22pm UTC](https://www.ruby-forum.com/t/xml-files-truncated/189403/2 "2010-05-31T12:22:25Z")

</div>

On 31.05.2010 13:43, Axel wrote:

> I recently migrated from Apache to Nginx. Really cool. My website is much faster. BUT I have a really weird issue that didn’t occur with Apache. It seems that some big files are sometimes truncated by the server. For instance, I generate a 200 entries videos sitemap (4600 lines) for Google (Google allows up to 50 000 entries FYI). Quite often, Google cannot parse this XML file because it is truncated in the middle and the XML is therefore no longer valid. It seems I have the same issue when I browse this XML into Safari or Firefox. When I try a feed validator ([http://validator.w3.org/feed/check.cgi](http://validator.w3.org/feed/check.cgi)), the test fails 8 times out of 10 and the source is clearly truncated.
> 
> I tried to disable gzip encoding or to adapt quite a lot of settings. Nothing did the trick. Have U any idea that could help?  
> Thks a lot

enable error\_log at notice level (or higher) and see in logs when error  
occurs.

–  
Anton Y.

---

<div class="post-metadata">

**Author:** ![axel](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/6de8d8/32.png) [@axel](https://www.ruby-forum.com/u/axel)\
**Post date:** [May 31, 2010, 12:39pm UTC](https://www.ruby-forum.com/t/xml-files-truncated/189403/3 "2010-05-31T12:39:23Z")

</div>

Hi Anton,

very good idea. Here what I get:

2010/05/31 12:36:52 [crit] 22005#0: \*5 open()  
“/usr/local/nginx/fastcgi\_temp/1/00/0000000001” failed (13: Permission  
denied) while reading upstream, client: 93.17.75.xxx, server:  
[www.xxx.com](http://www.xxx.com), request: “GET /rss/restaurants/sitemap HTTP/1.1”, upstream:  
“fastcgi://127.0.0.1:9000”, host: “[www.xxx.com](http://www.xxx.com)”

Weird …

Thks  
Axel

Le 31 mai 2010 à 12:21, Anton Y. a écrit :

---

<div class="post-metadata">

**Author:** ![axel](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/6de8d8/32.png) [@axel](https://www.ruby-forum.com/u/axel)\
**Post date:** [May 31, 2010, 12:42pm UTC](https://www.ruby-forum.com/t/xml-files-truncated/189403/4 "2010-05-31T12:42:19Z")

</div>

Hello!

On Mon, May 31, 2010 at 11:43:21AM +0200, Axel wrote:

> issue when I browse this XML into Safari or Firefox. When I try  
> a feed validator ([http://validator.w3.org/feed/check.cgi](http://validator.w3.org/feed/check.cgi)), the  
> test fails 8 times out of 10 and the source is clearly  
> truncated.
> 
> I tried to disable gzip encoding or to adapt quite a lot of  
> settings. Nothing did the trick. Have U any idea that could  
> help?  
> Thks a lot

1. As Anton already suggested, try looking into error log. From  
symptoms you describe it looks like nginx can’t write to  
fastcgi\_temp\_path/proxy\_temp\_path, it should complain in logs.

2. If the above doesn’t help, please follow this topic:

> **[GitHub - nginxinc/nginx-wiki: ARCHIVED -- Source for the now archived NGINX Wiki...](https://github.com/nginxinc/nginx-wiki)**
>
> ARCHIVED -- Source for the now archived NGINX Wiki section of https://www.nginx.com

You have to provide config, nginx -V and debug log of a request  
which exposes the problem.

Maxim D.

---

<div class="post-metadata">

**Author:** ![axel](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/6de8d8/32.png) [@axel](https://www.ruby-forum.com/u/axel)\
**Post date:** [May 31, 2010, 1:20pm UTC](https://www.ruby-forum.com/t/xml-files-truncated/189403/5 "2010-05-31T13:20:56Z")

</div>

Hi again,

nginx is running under www-data:www-data (it is maybe a bad idea isn’t  
it?). I doublechecked filesystem permission and issued a chown -R  
www-data: fastcgi\_temp && chmod -R 664 fastcgi\_temp.  
This didn’t solve the issue. Here is some debug information below.

* * *

root@o:/usr/local/nginx# nginx -V  
nginx version: nginx/0.7.65  
built by gcc 4.3.3 (Ubuntu 4.3.3-5ubuntu4)  
configure arguments: --with-cc-opt=-O3 --user=www-data --group=www-data  
–add-module=./modules/nginx\_mp4\_streaming\_public  
–with-http\_stub\_status\_module

## CONFIG

user www-data www-data;  
worker\_processes 5;

events {  
use epoll;  
worker\_connections 2048;  
}

http {  
include mime.types;  
include fastcgi\_params;  
default\_type application/octet-stream;

```
sendfile off;
tcp_nopush on;
tcp_nodelay on;
server_tokens off;
ignore_invalid_headers on;

```

# client\_body\_buffer\_size 8k;

# client\_header\_buffer\_size 1k;

# client\_max\_body\_size 1k;

# large\_client\_header\_buffers 1 1k;

```
client_body_timeout 5;
client_header_timeout 5;
keepalive_timeout 15 14;
send_timeout 5;

gzip on;
gzip_comp_level 4;
gzip_proxied any;
gzip_buffers 16 8k;
gzip_types text/html text/css application/x-javascript text/xml 

```

application/xml application/xml+rss text/javascript  
application/xhtml+xml;

```
expires 1d;

error_log /var/log/nginx/error.log notice;
access_log /var/log/nginx/access.log;

include /usr/local/nginx/sites-enabled/*;

```

}

server  
{  
listen 80;  
server\_name [www.xxx.com](http://www.xxx.com);  
root /var/www/xxx/prod/public;  
index index.php;

```
            log_format main '$remote_addr - $remote_user 

```

[$time\_local] “$request” ’  
'$status $body\_bytes\_sent “$http\_referer” ’  
‘“$http\_user\_agent” “$http\_x\_forwarded\_for”’;  
access\_log /var/log/nginx/xxx\_access.log main;

```
            location /
            {

                    try_files $uri $uri/ @kohana;

                    fastcgi_pass 127.0.0.1:9000;
                    fastcgi_index index.php;
                    fastcgi_param SCRIPT_FILENAME 

```

$document\_root/$fastcgi\_script\_name;

```
                    include fastcgi_params;
            }

            location ~ /\. {
                    deny all;
            }

            location @kohana
            {
                    fastcgi_pass 127.0.0.1:9000;
                    fastcgi_index index.php;
                    fastcgi_param SCRIPT_FILENAME 

```

$document\_root/index.php;  
include fastcgi\_params;  
}

```
            location ~* 

```

.css|.js|.jpg|.jpeg|.png|.gif|.swf|.svg|.tiff$ {  
expires 30d;  
}  
}

## DEBUG LOG:

2010/05/31 12:36:52 [crit] 22005#0: \*5 open()  
“/usr/local/nginx/fastcgi\_temp/1/00/0000000001” failed (13: Permission  
denied) while reading upstream, client: 93.17.75.xxx, server: xxx,  
request: “GET /rss/restaurants/sitemap HTTP/1.1”, upstream:  
“fastcgi://127.0.0.1:9000”, host: “[www.xxx.com](http://www.xxx.com)”  
2010/05/31 12:41:37 [warn] 22008#0: \*312 an upstream response is  
buffered to a temporary file  
/usr/local/nginx/fastcgi\_temp/7/00/0000000007 while reading upstream,  
client: 93.17.75.xxx, server: [www.xxx.com](http://www.xxx.com), request: “GET  
/rss/restaurants/sitemap HTTP/1.1”, upstream:  
“fastcgi://127.0.0.1:9000”, host: “[www.xxx.com](http://www.xxx.com)”  
2010/05/31 12:55:47 [crit] 22008#0: \*1234 open()  
“/usr/local/nginx/fastcgi\_temp/6/04/0000000046” failed (13: Permission  
denied) while reading upstream, client: 128.30.52.xxx, server:  
[www.xxx.com](http://www.xxx.com), request: “GET /rss/restaurants/sitemap HTTP/1.1”, upstream:  
“fastcgi://127.0.0.1:9000”, host: “[www.xxx.com](http://www.xxx.com)”

Thks a lot

Axel

Le 31 mai 2010 à 12:41, Maxim D. a écrit :

---

<div class="post-metadata">

**Author:** ![axel](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/6de8d8/32.png) [@axel](https://www.ruby-forum.com/u/axel)\
**Post date:** [May 31, 2010, 4:00pm UTC](https://www.ruby-forum.com/t/xml-files-truncated/189403/6 "2010-05-31T16:00:09Z")

</div>

Hi,

Actually I have these errors quite often, not only for big XML files,  
not only in the fastcgi\_temp folder. I have a general permission issue:

2010/05/31 15:48:07 [crit] 5253#0: \*47 open()  
“/usr/local/nginx/client\_body\_temp/0000000006” failed (13: Permission  
denied), client: 93.17.75.131, server: [cacti.xxx.com](http://cacti.xxx.com), request: “POST  
/templates\_import.php HTTP/1.1”, host: “[cacti.xxx.com](http://cacti.xxx.com)”, referrer:  
“[http://cacti.xxx.com/templates\_import.php](http://cacti.xxx.com/templates_import.php)”  
2010/05/31 15:48:16 [crit] 5257#0: \*67 open()  
“/usr/local/nginx/fastcgi\_temp/7/00/0000000007” failed (13: Permission  
denied) while reading upstream, client: 67.195.111.28, server:  
[www.xxx.com](http://www.xxx.com), request: “GET  
/restaurants-suivant-1-tri-ville-en-liste-tag=0-a-20-e HTTP/1.0”,  
upstream: “fastcgi://127.0.0.1:9000”, host: “[www.xxx.com](http://www.xxx.com)”  
2010/05/31 15:48:26 [error] 5257#0: \*76 open()  
“/var/www/xxx/prod/public/medias/galerie/restaurant-boulogne-billancourt-mon-bistrot-237\_1.jpg”  
failed (2: No such file or directory), client: 66.220.153.246, server:  
[www.xxx.com](http://www.xxx.com), request: “GET  
/medias/galerie/restaurant-boulogne-billancourt-mon-bistrot-237\_1.jpg  
HTTP/1.1”, host: “[www.xxx.com](http://www.xxx.com)”  
2010/05/31 15:48:57 [error] 5257#0: \*102 open()  
“/var/www/xxx/prod/public/videos/flvplayer.swf” failed (2: No such file  
or directory), client: 80.17.242.10, server: [www.xxx.com](http://www.xxx.com), request: “GET  
/videos/flvplayer.swf HTTP/1.1”, host: “[www.xxx.com](http://www.xxx.com)”, referrer:  
“[http://www.bigarrade.fr/](http://www.bigarrade.fr/)”  
2010/05/31 15:49:03 [crit] 5257#0: \*110 open()  
“/usr/local/nginx/client\_body\_temp/0000000008” failed (13: Permission  
denied), client: 93.17.75.131, server: [cacti.xxx.com](http://cacti.xxx.com), request: “POST  
/templates\_import.php HTTP/1.1”, host: “[cacti.xxx.com](http://cacti.xxx.com)”, referrer:  
“[http://cacti.xxx.com/templates\_import.php](http://cacti.xxx.com/templates_import.php)”  
2010/05/31 15:49:09 [crit] 5257#0: \*113 open()  
“/usr/local/nginx/client\_body\_temp/0000000009” failed (13: Permission  
denied), client: 93.17.75.131, server: [cacti.xxx.com](http://cacti.xxx.com), request: “POST  
/templates\_import.php HTTP/1.1”, host: “[cacti.xxx.com](http://cacti.xxx.com)”, referrer:  
“[http://cacti.xxx.com/templates\_import.php](http://cacti.xxx.com/templates_import.php)”

I’m checking the permissions again. Here’s what I have:

Nginx is running under www-data:  
root@xxx:/usr/local/nginx# ps aux | grep “nginx: worker process”  
www-data 5253 0.0 0.0 37288 3012 ? S 15:47 0:00 nginx:  
worker process  
www-data 5254 0.0 0.0 37180 2812 ? S 15:47 0:00 nginx:  
worker process  
www-data 5255 0.0 0.0 37248 2928 ? S 15:47 0:00 nginx:  
worker process  
www-data 5256 0.0 0.0 36948 2700 ? S 15:47 0:00 nginx:  
worker process  
www-data 5257 0.0 0.0 37288 2976 ? S 15:47 0:00 nginx:  
worker process

Client\_body\_temp, fastcgi\_temp and proxy\_temp are readable writable by  
www-data  
root@xxx:/usr/local/nginx# ls -l  
total 36  
drwxrwxr-- 2 www-data root 4096 2010-05-31 13:43 client\_body\_temp  
drwxrwxr-- 2 root root 4096 2010-05-11 16:05 conf  
drwxrwxr-- 12 www-data www-data 4096 2010-05-11 10:50 fastcgi\_temp  
drwxrwxr-- 2 root root 4096 2010-05-10 20:20 html  
drwxrwxr-- 2 root root 4096 2010-05-31 15:47 logs  
drwxrwxr-- 2 www-data www-data 4096 2010-05-10 18:22 proxy\_temp  
drwxrwxr-- 2 root root 4096 2010-05-12 00:00 sbin  
drwxrwxr-- 2 root root 4096 2010-05-31 15:22 sites-available  
drwxrwxr-- 2 root root 4096 2010-05-31 15:39 sites-enabled

I do not find any solution on google :-((  
Should I stop using custom users and go back to nobody?

Thks

Axel

Le 31 mai 2010 à 13:43, Maxim D. a écrit :

---

<div class="post-metadata">

**Author:** ![axel](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/6de8d8/32.png) [@axel](https://www.ruby-forum.com/u/axel)\
**Post date:** [May 31, 2010, 1:44pm UTC](https://www.ruby-forum.com/t/xml-files-truncated/189403/7 "2010-05-31T13:44:21Z")

</div>

Hello!

On Mon, May 31, 2010 at 01:20:06PM +0200, Axel wrote:

> nginx is running under www-data:www-data (it is maybe a bad idea  
> isn’t it?). I doublechecked filesystem permission and issued a  
> chown -R www-data: fastcgi\_temp && chmod -R 664 fastcgi\_temp.  
> This didn’t solve the issue. Here is some debug information  
> below.

[…]

> ## DEBUG LOG:
> 
> 2010/05/31 12:36:52 [crit] 22005#0: \*5 open()  
> “/usr/local/nginx/fastcgi\_temp/1/00/0000000001” failed (13:  
> Permission denied) while reading upstream, client: 93.17.75.xxx,

Check if www-data has at least execute access on “/usr”,  
“/usr/local”, “/usr/local/nginx”.

Maxim D.

---

<div class="post-metadata">

**Author:** ![axel](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/6de8d8/32.png) [@axel](https://www.ruby-forum.com/u/axel)\
**Post date:** [May 31, 2010, 4:10pm UTC](https://www.ruby-forum.com/t/xml-files-truncated/189403/8 "2010-05-31T16:10:06Z")

</div>

Do check your php daemon user/groups. Type  
ps aux | grep php

Best Regards,

Glen L.

---

<div class="post-metadata">

**Author:** ![axel](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/6de8d8/32.png) [@axel](https://www.ruby-forum.com/u/axel)\
**Post date:** [May 31, 2010, 4:07pm UTC](https://www.ruby-forum.com/t/xml-files-truncated/189403/9 "2010-05-31T16:07:10Z")

</div>

Pfff, even when I chmod -R 777 fastcgi\_temp errors keep appearing in  
logs

What am I missing out?

Thks

Le 31 mai 2010 à 13:43, Maxim D. a écrit :

---

<div class="post-metadata">

**Author:** ![axel](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/6de8d8/32.png) [@axel](https://www.ruby-forum.com/u/axel)\
**Post date:** [May 31, 2010, 4:19pm UTC](https://www.ruby-forum.com/t/xml-files-truncated/189403/10 "2010-05-31T16:19:28Z")

</div>

Hello!

On Mon, May 31, 2010 at 03:58:42PM +0200, Axel wrote:

> Actually I have these errors quite often, not only for big XML  
> files, not only in the fastcgi\_temp folder. I have a general  
> permission issue:

[…]

> > Check if www-data has at least execute access on “/usr”,  
> > “/usr/local”, “/usr/local/nginx”.

Did you done what you were said to?

Maxim D.

---

<div class="post-metadata">

**Author:** ![axel](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/6de8d8/32.png) [@axel](https://www.ruby-forum.com/u/axel)\
**Post date:** [May 31, 2010, 4:13pm UTC](https://www.ruby-forum.com/t/xml-files-truncated/189403/11 "2010-05-31T16:13:01Z")

</div>

Hi,

> Pfff, even when I chmod -R 777 fastcgi\_temp errors keep appearing in logs
> 
> What am I missing out?

The fact that permissions apply to the whole path.

You need at least +x bit on /usr, /usr/local and /usr/local/nginx.

Best regards,  
Piotr S. \< [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) \>

---

<div class="post-metadata">

**Author:** ![axel](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/6de8d8/32.png) [@axel](https://www.ruby-forum.com/u/axel)\
**Post date:** [May 31, 2010, 4:37pm UTC](https://www.ruby-forum.com/t/xml-files-truncated/189403/12 "2010-05-31T16:37:37Z")

</div>

Maxim,

My mistake: when I got your response, I checked permissions on /usr and  
/usr/local but not on /usr/local/nginx  
I added +x on nginx and it seems to be working now.

Thanks a lot !!

Axel

Le 31 mai 2010 à 16:18, Maxim D. a écrit :

---

<div class="post-metadata">

**Author:** ![axel](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/6de8d8/32.png) [@axel](https://www.ruby-forum.com/u/axel)\
**Post date:** [May 31, 2010, 4:57pm UTC](https://www.ruby-forum.com/t/xml-files-truncated/189403/13 "2010-05-31T16:57:31Z")

</div>

I was looking for documentation about this,  
thks for the tip

Axel

Le 31 mai 2010 à 16:40, Igor S. a écrit :

---

<div class="post-metadata">

**Author:** ![axel](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/6de8d8/32.png) [@axel](https://www.ruby-forum.com/u/axel)\
**Post date:** [May 31, 2010, 4:41pm UTC](https://www.ruby-forum.com/t/xml-files-truncated/189403/14 "2010-05-31T16:41:08Z")

</div>

On Mon, May 31, 2010 at 04:36:31PM +0200, Axel wrote:

> Maxim,
> 
> My mistake: when I got your response, I checked permissions on /usr and /usr/local but not on /usr/local/nginx  
> I added +x on nginx and it seems to be working now.
> 
> Thanks a lot !!
> 
> Axel

BTW, you may want to move temporary files directories in other place:

http {  
client\_body\_temp\_path /spool/nginx/client\_temp;  
fastcgi\_temp\_path /spool/nginx/fastcgi\_temp;  
proxy\_temp\_path /spool/nginx/proxy\_temp;

and set appropriate rights and user ownership only for these  
directories.

> > […]  
> > [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
> > [mailman.nginx.org archive](http://nginx.org/mailman/listinfo/nginx)
> 
> * * *
> 
> nginx mailing list  
> [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
> [mailman.nginx.org archive](http://nginx.org/mailman/listinfo/nginx)

–  
Igor S.

> **[Igor Sysoev](http://sysoev.ru/en/)**