NFS 0-byte problem

Hello all ,

We’re using nginx 1.1.6 ver. 5 server connected NetApp storage via NFS .
(/var/www )

Sometimes 1 or 2 server return cache size 0-byte and don’t answer http
request. Other 3 server working well .
If i restarted nginx or delete cache file , 0-byte problem resolve.

Example service conf :

upstream upstream.test.com {
server 1.1.1.1:80;
server 2.2.2.2:80 backup;
server 3.3.3.3:80 backup;
}

server {
listen 80;
server_name video.test.com;
access_log off;

    location / {
            proxy_pass         http://upstream.test.com;
            proxy_ignore_headers Expires Cache-Control Set-Cookie;
            proxy_cache_valid  200 30d;
            proxy_cache     $host;
            proxy_connect_timeout 5;
            proxy_read_timeout 10;
            proxy_next_upstream error timeout http_500 http_502

http_503 http_504 http_404;
set $maincache “main-a:”;
proxy_cache_key $maincache$host$uri;
proxy_redirect off;
proxy_set_header Host $proxy_host;
proxy_max_temp_file_size 0;
proxy_hide_header Cache-Control;
proxy_hide_header X-Powered-By;
proxy_hide_header X-AspNet-Version;
expires 30d;

    }

    location ~ /delete(/.*) {
            allow                   5.5.5.5;
            deny                    all;
            set $deluri $1;
            set $maincache  "main-a:";
            proxy_cache_purge       $host   $maincache$host$deluri;
    }

}

Where i’m mising ?

Thanks

Posted at Nginx Forum:

Is nginx accessing the NFS or just the upstream servers?

nginx caching on NFS if MISS via upstream.

Posted at Nginx Forum:

On Oct 28, 2011, at 7:06 PM, kernelheaders wrote:

nginx caching on NFS if MISS via upstream.

I wouldn’t recommend doing that. Cache should be on local “fast” disk.
You need atomic renames, etc, to work reliably.

–Brian

Brian A. Wrote:


nginx mailing list
[email protected]
nginx Info Page

We’re using 12 x 600 GB SAS 15K Hard drives on NetApp storage, so
storage I/O operations very fast. But my problem is not speed .

Posted at Nginx Forum:

Are they all using the same cache directoy/NFS mount point? Sharing a
cache directory seems not to be supported.

Posted at Nginx Forum:

2011/10/29 kernelheaders [email protected]

storage I/O operations very fast. But my problem is not speed .

  • what is your netapp appliance ? (just curious)

  • what are your mount options ?

On Oct 28, 2011, at 9:18 PM, kernelheaders wrote:

We’re using 12 x 600 GB SAS 15K Hard drives on NetApp storage, so
storage I/O operations very fast. But my problem is not speed .

The cache is expected to be used only by a single nginx instance.
Sharing it will not work. The cache is made up of the cache directory,
temp files, and shared memory.

If you really need shared cache as well, then look into using memc with
srcache modules.

–Brian

JackB Wrote:

Are they all using the same cache directoy/NFS
mount point? Sharing a cache directory seems not
to be supported.

Yeap , same cache and temp directory .

Posted at Nginx Forum:

Jérôme Loyet Wrote:


  • what is your netapp appliance ? (just curious)

  • what are your mount options ?


nginx mailing list
[email protected]
nginx Info Page

  • Storage FA2020
  • I try UDP , TCP a lot of version . But problem is not about mount.

Posted at Nginx Forum:

Brian A. Wrote:

The cache is expected to be used only by a single
nginx mailing list
[email protected]
nginx Info Page

Can you please explain why only one distance ? If you have same cachekey
and structure , cache work all intance even remote locations .
And system works well , sometimes and some servers occurs problem .

Posted at Nginx Forum:

Hello!

On Sat, Oct 29, 2011 at 09:59:43PM -0400, kernelheaders wrote:

and structure , cache work all intance even remote locations .
And system works well , sometimes and some servers occurs problem .

As Brian already wrote, nginx currently doesn’t allow sharing of
the same cache directory between several nginx instances. It
doesn’t expect cache files to appear or be changed by another
nginx instance. If you share cache directory between multiple
nginx instances - weird things will happen, and problem you see is
one of them.

Maxim D.