Acess log over nfs hanging

Hello all.
I’ve been using nginx as a reverse proxy and to write access.log files
named with variables. I am also using open log file cache.

It seems that when some processes are running in the nfs server, the
share
won’t allow writing for some time and I noticed all nginx workers in
status
D and not processing requests.

Are all access.log writes blocking? If my nfs server shutdown in an
unexpected way, will nginx stop proxying requests to the backend or
responses to the client?

Thanks in advance

On 02/07/14 20:28, Jader H. Silva wrote:

It seems that when some processes are running in the nfs server, the share won’t
allow writing for some time and I noticed all nginx workers in status D and not
processing requests.

I general it is a bad idea to write logs over NFS instead local HDD.

If you need central log store across many servers:

  • write logs locally
  • rotate as often as need (SIGUSR1)
  • copy logs to cental log server (rsync is handy for this, but other
    methods are
    possible)

Are all access.log writes blocking?

yes, blocking

If my nfs server shutdown in an unexpected
way, will nginx stop proxying requests to the backend or responses to the
client?

yes, will stop

On Tue, Feb 11, 2014 at 1:30 AM, Anton Y. [email protected]
wrote:

If you need central log store across many servers:

  • write logs locally
  • rotate as often as need (SIGUSR1)
  • copy logs to cental log server (rsync is handy for this, but other
    methods are possible)

yeah, try out: http://hekad.readthedocs.org/

it’s like syslog but can write to pretty much any backend store.

Hello!

On Thu, Feb 13, 2014 at 03:42:22PM -0200, Guilherme wrote:

Anton,

I already had the same issue logging to NFS, but I’m curious about why
nginx hang in some nfs failures. Log phase is the last, if there is no post
action, so why nginx stop responding in some NFS failures?

Because NFS failures by default result in infinite blocking of any
process trying to access NFS shares. And nginx worker processes
will be blocked as well and won’t be able to handle requests
anymore.

Do you think
that I can ease the situation tunning nfs client config, such as timeout
and retrans ?

Some hints can be found here:

http://mailman.nginx.org/pipermail/nginx/2014-February/042107.html

Using “soft” mount option and small timeouts may help a bit. But
I wouldn’t recommend using NFS if you want a service which is
expected to run independantly from your NFS server.


Maxim D.
http://nginx.org/

Anton,

I already had the same issue logging to NFS, but I’m curious about why
nginx hang in some nfs failures. Log phase is the last, if there is no
post
action, so why nginx stop responding in some NFS failures? Do you think
that I can ease the situation tunning nfs client config, such as timeout
and retrans ?

tks