Forum: NGINX shared memory zone "media" conflicts with already declared size 0

what seems to be the problem?

[emerg]: the size 52428800 of shared memory zone "media" conflicts with
already declared size 0 in /etc/nginx/conf.d/cache.conf:5
configuration file /etc/nginx/nginx.conf test failed

my configuration below

# nginx.conf
user nobody nogroup;
worker_processes 5;
error_log /var/log/nginx/error.log;
pid /var/run/nginx/nginx.pid;
worker_rlimit_nofile 8192;

events {
    use epoll;
    worker_connections  4096;
}

http {
    include base/std;
    include base/log_format;
    include base/gzip;
    include base/proxy_conf_real_ip;

    include /etc/nginx/conf.d/*.conf;
}

# base/std
include base/mime.types;
default_type application/octet-stream;

sendfile on;

tcp_nopush on;
tcp_nodelay off;

# base/log_format
log_format main '$remote_addr - $remote_user [$time_local] '
'"$request" $status  $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';

# base/gzip
gzip on;
gzip_http_version 1.0;
gzip_comp_level 2;
gzip_proxied any;
gzip_types text/plain text/css application/x-javascript text/xml
application/xml application/xml+rss text/javascript;

# base/proxy_conf_real_ip
include base/proxy_conf
proxy_set_header X-Real-IP $http_x_real_ip;

# base/proxy_conf
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
client_max_body_size 10m;
client_body_buffer_size 128k;
proxy_connect_timeout 90;
proxy_send_timeout 90;
proxy_read_timeout 90;
proxy_buffers 32 4k;

# conf.d/cache.conf
client_body_temp_path /mnt/client_temp;
proxy_temp_path /mnt/proxy_temp;
fastcgi_temp_path /mnt/fastcgi_temp;

proxy_cache_path /mnt/media levels=1:2 keys_zone=media:50m 
max_size=5000m;

server {
    listen 80;

    location / {
        proxy_pass http://media.server:8080;
        include extra/cache;
    }
}

# conf.d/ssl.conf
server {
    listen 443;

    ssl on;
    ssl_certificate ssl/ssl.crt;
    ssl_certificate_key ssl/ssl.key;

    ssl_session_timeout 5m;
    ssl_ciphers 
ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP;

    location / {
        proxy_pass https://media.server:4430;
        include extra/cache;
    }
}

# extra/cache
proxy_cache media;
proxy_cache_key $request_uri;
proxy_cache_valid 200 28d;
proxy_cache_valid 404 1m;
proxy_cache_valid 500 2m;
proxy_cache_use_stale error timeout invalid_header http_500 http_502
http_503 http_504;
proxy_ignore_headers Expires Cache-Control;
FileETag on;
expires max;

On Mon, Sep 28, 2009 at 06:15:50PM +0200, Tomasz Pajor wrote:

> what seems to be the problem?
> 
> [emerg]: the size 52428800 of shared memory zone "media" conflicts with 
> already declared size 0 in /etc/nginx/conf.d/cache.conf:5
> configuration file /etc/nginx/nginx.conf test failed

This may be caused if "proxy_cache media" is included before 
proxy_cache_path.

>
>> what seems to be the problem?
>>
>> [emerg]: the size 52428800 of shared memory zone "media" conflicts with 
>> already declared size 0 in /etc/nginx/conf.d/cache.conf:5
>> configuration file /etc/nginx/nginx.conf test failed
>>     
>
> This may be caused if "proxy_cache media" is included before proxy_cache_path.
>   
this is true,
in /etc/nginx/conf.d i have two files, cache.conf and ssl.conf, and what
it seems the ssl.conf is loaded first, shouldn't the files be loaded in
alphabetical order?

On Tue, Sep 29, 2009 at 10:57:02AM +0200, Tomasz Pajor wrote:

> this is true,
> in /etc/nginx/conf.d i have two files, cache.conf and ssl.conf, and what 
> it seems the ssl.conf is loaded first, shouldn't the files be loaded in 
> alphabetical order?

Currently, files are unsorted.
Probably, it should be changed to alphabetical order, but I'm not sure.

>> this is true,
>> in /etc/nginx/conf.d i have two files, cache.conf and ssl.conf, and what 
>> it seems the ssl.conf is loaded first, shouldn't the files be loaded in 
>> alphabetical order?
>>     
>
> Currently, files are unsorted.
> Probably, it should be changed to alphabetical order, but I'm not sure
I don't know, never the less the desired order can be achieved by adding
digits in front of the file name, 001-cache.conf, 002-ssl.conf fixed my
issue.

On Tue, Sep 29, 2009 at 11:41:15AM +0200, Tomasz Pajor wrote:

> >>>       
> issue.
Probably, names were reordered in the directory while renaming.

Hello!

On Tue, Sep 29, 2009 at 11:41:15AM +0200, Tomasz Pajor wrote:

> >>and what it seems the ssl.conf is loaded first, shouldn't the
> >>files be loaded in alphabetical order?
> >
> >Currently, files are unsorted.
> >Probably, it should be changed to alphabetical order, but I'm not sure
> I don't know, never the less the desired order can be achieved by
> adding digits in front of the file name, 001-cache.conf,
> 002-ssl.conf fixed my issue.

No, desired order can't be achieved by adding digits.  While using
globbed includes nginx loads them without any sorting, i.e. in
order how filesystem returns them.

You happened to get correct order after your renames, but things
may again go wild at any moment (most likely when you'll touch
something in this directory).

Currently the only solution is to include dependent files
explicitly.  Globs can be used *only* for completely independent
files (e.g. containing definitions of different server{} blocks).

Maxim Dounin

Hello!

On Tue, Sep 29, 2009 at 01:11:17PM +0400, Igor Sysoev wrote:

> > > This may be caused if "proxy_cache media" is included before proxy_cache_path.
> > >   
> > this is true,
> > in /etc/nginx/conf.d i have two files, cache.conf and ssl.conf, and what 
> > it seems the ssl.conf is loaded first, shouldn't the files be loaded in 
> > alphabetical order?
> 
> Currently, files are unsorted.
> Probably, it should be changed to alphabetical order, but I'm not sure.

BTW, have you seen any configurations that may seriously suffer
from changing this to alpabetical order?

As far as I understand performance drop will be noticeable
somewhere near 10k+ includes, while serious problems are unlikely
to happen before something like 100k+...

Maxim Dounin

Maxim Dounin wrote:
>>>>>> configuration file /etc/nginx/nginx.conf test failed
>>>       
> may again go wild at any moment (most likely when you'll touch 
> something in this directory).
>
> Currently the only solution is to include dependent files 
> explicitly.  Globs can be used *only* for completely independent 
> files (e.g. containing definitions of different server{} blocks)
So maybe it is a good idea to add alphabetic order for the wildcard 
include.

On Tuesday, September 29, 2009 at 13:32:10, Maxim Dounin wrote:

MD> Globs can be used *only* for completely independent files
MD> (e.g. containing definitions of different server{} blocks).

may be config test should generate warning "Undefined behavior"
if wildcard includes not containing completely independent files ?

may be only with -w command line switch,
analogue of perl "use warnings;" pragma:

$ perl -h | grep -- -w
  -w              enable many useful warnings (RECOMMENDED)

On Tue, Sep 29, 2009 at 03:15:21PM +0400, Maxim Dounin wrote:

> > > >> already declared size 0 in /etc/nginx/conf.d/cache.conf:5
> > Currently, files are unsorted.
> > Probably, it should be changed to alphabetical order, but I'm not sure.
> 
> BTW, have you seen any configurations that may seriously suffer 
> from changing this to alpabetical order?
> 
> As far as I understand performance drop will be noticeable 
> somewhere near 10k+ includes, while serious problems are unlikely 
> to happen before something like 100k+...

The single stopper now is Win32: FindFirstFile/FindNextFile may return
unordered files.

Hello!

On Tue, Sep 29, 2009 at 04:05:23PM +0300, Gena Makhomed wrote:

> On Tuesday, September 29, 2009 at 13:32:10, Maxim Dounin wrote:
> 
> MD> Globs can be used *only* for completely independent files
> MD> (e.g. containing definitions of different server{} blocks).
> 
> may be config test should generate warning "Undefined behavior"
> if wildcard includes not containing completely independent files ?

You never know if included chunks are dependent or not.  E.g. the
following lines are completely independent given that $var defined
somewhere before (assume each line in it's own include):

   rewrite  ^/blah$  /$var/blah;
   rewrite  ^/oops$  /$var/oops;

while these aren't:

   rewrite  ^/blah$  /$var/blah;
   set $var "something"
   rewrite  ^/oops$  /$var/oops;

Not even talking about more complex cases.

Maxim Dounin

SO cool ! I tested it and fix, Thanks!

Posted at Nginx Forum: 
http://forum.nginx.org/read.php?2,9716,235955#msg-235955

I am using nginx version: nginx/1.2.6 and facing a similar error:

Starting nginx: nginx: the configuration file /etc/nginx/nginx.conf 
syntax
is ok
nginx: [emerg] zero size shared memory zone "limit_per_ip"
nginx: configuration file /etc/nginx/nginx.conf test failed
invoke-rc.d: initscript nginx, action "start" failed.

Here's my nginx.conf file:

user www-data;
worker_processes 2;
pid /var/run/nginx.pid;

events {
  worker_connections 768;
  # multi_accept on;
}

http {

  # Cloudflare
  set_real_ip_from 204.93.240.0/24;
  set_real_ip_from 204.93.177.0/24;
  set_real_ip_from 199.27.128.0/21;
  set_real_ip_from 173.245.48.0/20;
  set_real_ip_from 103.22.200.0/22;
  set_real_ip_from 141.101.64.0/18;
  set_real_ip_from 108.162.192.0/18;
  set_real_ip_from 190.93.240.0/20;
  real_ip_header CF-Connecting-IP;

  sendfile on;
  tcp_nopush on;
  tcp_nodelay on;
  types_hash_max_size 2048;

  client_body_timeout 10;
  client_header_timeout 10;
  keepalive_timeout 10;
  send_timeout 10;
  #limit_zone limit_per_ip $binary_remote_addr 16m;

  server_tokens off;
  #charset utf-8;

  expires -1; #A negative time sets the Cache-Control header to no-cache

  client_max_body_size       10m;
  client_body_buffer_size    128k;

  # server_name_in_redirect off;

  include /etc/nginx/mime.types;
  default_type application/octet-stream;

  access_log /var/log/nginx/access.log;
  error_log /var/log/nginx/error.log;

  gzip on;
  gzip_disable "msie6";
  gzip_vary on; #you instruct proxies to store both a compressed and
uncompressed version of the content
  # gzip_proxied any;
  gzip_comp_level 6;
  # gzip_buffers 16 8k;
  gzip_http_version 1.1;
  gzip_types text/plain text/css application/json 
application/x-javascript
text/xml application/xml application/xml+rss text/javascript;

  include /etc/nginx/conf.d/*.conf;
  include /etc/nginx/sites-enabled/*;
}

Thanks for any help.

Posted at Nginx Forum: 
http://forum.nginx.org/read.php?2,9716,236021#msg-236021

On Fri, Feb 08, 2013 at 10:01:30AM -0500, atipico wrote:
> I am using nginx version: nginx/1.2.6 and facing a similar error:
>
> Starting nginx: nginx: the configuration file /etc/nginx/nginx.conf syntax
> is ok
> nginx: [emerg] zero size shared memory zone "limit_per_ip"
> nginx: configuration file /etc/nginx/nginx.conf test failed
> invoke-rc.d: initscript nginx, action "start" failed.
>
> Here's my nginx.conf file:
>
[...]
> http {
[...]
>   #limit_zone limit_per_ip $binary_remote_addr 16m;

The "limit_per_ip" zone, including its size of 16 megabytes,
was originally configured here.

>   include /etc/nginx/mime.types;
[...]
>   include /etc/nginx/conf.d/*.conf;
>   include /etc/nginx/sites-enabled/*;

At least one of the included files refers to this zone through
the "limit_conn" directive.  The diagnostics is rather limited
in this case, but the message tells you that the size of the
shared memory zone "limit_per_ip" is unknown (because the
corresponding directive is commented out).

JFYI, in modern versions of nginx the "limit_conn_zone" directive
should be used instead.  See http://nginx.org/r/limit_zone for
details.