Add -p option to override prefix

hi,

I attach a patch which add the -p option to nginx. This option
overrides the --prefix path and also the $NGX_CONF_PREFIX variable
which is used as prefix for “include” directives. So, all relative
path will be prefix with the value of this option.

I don’t know if it covers all cases but it seems ok in my env.

Why did I want to make such a patch ?
Because i’ll be running several instances of nginx on the same box,
with similar conf and similar directorys (/path/xx/conf,
/path/xx/docs, /path/xx/logs, /path/xx/cache, /path/xx/tmp, …). With
same conf, I can launch multiple instances of nginx with
/path/to/nginx -p /path/www, -p /patch/www2, …

I could run one instance of nginx with many “server” directives but
the box is mutualized between clients and clients doesn’t want to
share their instance with others …

You could then imagine many cases in which this option could be usefull.

++ jerome

This is interesting.

If I understood well this allows one to run isolated instances of nginx
listening on different IPs and also under different user accounts. Is
that correct?

At Tue, 24 Mar 2009 13:27:12 -0300,
“Juan Fco. Giordana” [email protected] wrote:

This is interesting.

If I understood well this allows one to run isolated instances of nginx
listening on different IPs and also under different user accounts. Is
that correct?

Why -c /path/to/different/nginx.conf is not a cool way?

I use -c /path/to/different/nginx.conf but with just -c all my path in
this file have to be absolute. I need to have the possibility to use
relative path not in --prefix.

2009/3/24 Kirill A. Korinskiy [email protected]:

On Tue, Mar 24, 2009 at 04:46:03PM +0100, J?r?me Loyet wrote:

/path/xx/docs, /path/xx/logs, /path/xx/cache, /path/xx/tmp, …). With
same conf, I can launch multiple instances of nginx with
/path/to/nginx -p /path/www, -p /patch/www2, …

I could run one instance of nginx with many “server” directives but
the box is mutualized between clients and clients doesn’t want to
share their instance with others …

You could then imagine many cases in which this option could be usefull.

I need to see how -p will affect on directives that use conf prefix
such as “include”, “ssl_certficate*”, and “auth_basic_user_file”.
Historically, nginx had the single prefix (not conf one), but this was
uncomfortably for Linux packages, where now the conf prefix is
/etc/nginx,
while default prefix is somewhere in other place.

Jérôme Loyet ha scritto:

hi,

I attach a patch which add the -p option to nginx. This option
overrides the --prefix path and also the $NGX_CONF_PREFIX variable
which is used as prefix for “include” directives. So, all relative
path will be prefix with the value of this option.

I don’t know if it covers all cases but it seems ok in my env.

I wrote a similar patch one year ago, I have attached it.
It is based on old nginx 0.5.x

It is used in production on http://unbit.it/, and Italian hosting
company, where a few accounts use nginx + mod_wsgi as application
server.

[…]

Regards Manlio

This is interesting.

If I understood well this allows one to run isolated instances of nginx
listening on different IPs and also under different user accounts. Is that
correct?

Jérôme Loyet wrote:

You could then imagine many cases in which this option could be usefull.

You can do whatever you want :slight_smile: Scope is only limited by your
imagination :slight_smile: (almost)

Here is an exemple of what our compagny needs and how I’ll make it. I
just warn that this has not been tested.

  • One box mutualized with several customers.
  • All architectures are the same: NET → nginx → apache(with php)
  • each customer has its own path and own conf file
  • each customer runs with a different user/group
  • each customer has an numerical ID (from 1)
  • each customer has its own prefix (/data/www/customers/$ID)
  • each customer has a frontend listening on *:(8000 + $ID)
  • each customer has a backend listening on *:(9000 + $ID)

The idea is to have a template file for nginx.conf. We’ll use m4 in
our case. This template will be something like this:

user M4_NGINX_USER M4_NGINX_GROUP;
error_log logs/errors;
pid logs/nginx.pid;
events {
workers 1024;
}

http {

mime types are common for all users

include /usr/local/nginx/conf/mime.types

proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_hide_header Set-Cookie;

proxy_temp_path temp/;

proxy_cache_path cache/ keys_zone=CACHE:10m;
proxy_cache CACHE;
proxy_cache_min_uses 1;
proxy_cache_use_stale error timeout invalid_header http_500 http_502
http_503 http_504;

proxy_cache_valid any M4_NGINX_DEFAULT_TIMEOUT;

server {
listen M4_NGINX_FRONTEND_PORT;

location / {
  proxy pass http://M4_NGINX_BACKEND_ADDRESS:M4_NGINX_BACKEND_PORT;
}

location ~* \.(gif|jpg|jpeg)$ {
  proxy pass http://M4_NGINX_BACKEND_ADDRESS:M4_NGINX_BACKEND_PORT;
  proxy_cache_valid M4_NGINX_IMAGES_TYPE M4_NGINX_IMAGES_TIMEOUT;
}

You can add some macros to generate more locations dynamicaly

the power of m4:

http://www.gnu.org/software/m4/manual/m4.html.gz

}
}

So this template is generated from few parameters:
M4_CUSTOMER_ID → 1
M4_NGINX_PREFIX → /data/www/customers/client/M4_CUSTOMER_ID
M4_NGINX_USER → client_ M4_CUSTOMER_ID
M4_NGINX_GROUP → client_ M4_CUSTOMER_ID
M4_NGINX_DEFAULT_TIMEOUT → 1m
M4_NGINX_FRONTEND_PORT → eval(8000 + M4_CUSTOMER_ID)
M4_NGINX_BACKEND_ADDRESS → 127.0.0.1
M4_NGINX_BACKEND_PORT → eval(9000 + M4_CUSTOMER_ID)
M4_NGINX_IMAGES_TYPE → any
M4_NGINX_IMAGES_TIMEOUT → 1d

Via a backoffice interface, the customer can set by his own some
parameters like (default timeout, images_types or images_timeout). The
backend then generate a temporary nginx conf file and:
test the temporary conf file
if the test is not valid {
report error to the user so that he can correct his settings
} else {
backup the old nginx.conf file
replace the nginx.conf file with the temporary one
send HUP signal to nginx
}

At startup each nginx has been launched with something like:
for prefix in /data/www/customers/*; do
test -d $prefix || continue
if /path/to/nginx -t -p $prefix/; then
echo “[info] nginx for customer basename $prefix correctly
launched”
else
echo “[ERROR] unable to launch nginx for customer basename $prefix” >/dev/stderr
fi
done

in this case I could have specified all my path abolutely as I have a
macro processor which can do the job. But I have other boxes as this
one but without a customer back office access. Each conf is done
manually so with relative path no (or much less) mistakes are
possible. We’re using this feature with apache (-d serverroot) for a
long time and it helps us a lot.

Hope this helps

++ Jerome

the previous patch (nginx-prefix-v2.patch) has to be applied after the
first one.

I attach here the full patch.

++ jerome

Le 25 mars 2009 21:16, Jérôme Loyet [email protected] a écrit :

2009/3/25 Igor S. [email protected]:

Because i’ll be running several instances of nginx on the same box,

I need to see how -p will affect on directives that use conf prefix
such as “include”, “ssl_certficate*”, and “auth_basic_user_file”.
Historically, nginx had the single prefix (not conf one), but this was
uncomfortably for Linux packages, where now the conf prefix is /etc/nginx,
while default prefix is somewhere in other place.

I made a new version of my patch. The patch still add the -p prefix but:
1- in auto/options, I’ve set the following variables:
+NGX_PREFIX_CONF_PATH=conf/nginx.conf
+NGX_PREFIX_ERROR_LOG_PATH=logs/error.log
+NGX_PREFIX_PID_PATH=logs/nginx.pid
+NGX_PREFIX_LOCK_PATH=logs/nginx.lock
+NGX_PREFIX_HTTP_CLIENT_TEMP_PATH=client_body_temp
+NGX_PREFIX_HTTP_PROXY_TEMP_PATH=proxy_temp
+NGX_PREFIX_HTTP_FASTCGI_TEMP_PATH=proxy_temp
+NGX_PREFIX_HTTP_LOG_PATH=logs/access.log

2- in auto/options I use them in remplacement of RAW string, for
exemple:

  •    NGX_CONF_PATH=$NGX_PREFIX/conf/nginx.conf
    
  •    NGX_CONF_PATH=$NGX_PREFIX/$NGX_PREFIX_CONF_PATH
    

3- 1) and 2) change nothing, except that default relative paths are in
variable now
4- In configure, I export those variables
5- in each source file in which one of thoses defined variable are
used, I changed it to
5a- if -p was not specified at runtime, nothing change. Use
NGX_PID_PATH and NGX_PREFIX_PID_PATH is not used at all.
5b- if -p was specified at runtime, so don’t use NGX_PID_PATH but “-p
parameter”/NGX_PREFIX_PID_PATH
6- If the -p parameter doesn’t finish with a ‘/’ char, then add it (I
didn’t take care of windows path yet)

I tried to make this patch clear and correct. It seams to work. But I
didn’t have time to test many features like (“ssl_certficate*”, and
"auth_basic_user_file). I’ll try that tonight or tomorow.

Hope this help.

++ Jerome

the previous patch (nginx-prefix-v2.patch) has to be applied after the
first one.

I attach here the full patch.

++ jerome

In my path I use ngx_snprintf with “%s%s” but it seems not to work, I
have overflows using this function.

I add tow logs to check the string content:
2009/03/26 11:56:34 [notice] 22943#0: pid file: ./logs/nginx.pid appq
2009/03/26 11:56:34 [notice] 22943#0: lock file: ./logs/nginx.lock Y

The function ngx_s(n)printf does not work with “%s%s”.
If I use sprintf instead, it works well. I was sure that ngx_sprintf
was an alias to sprintf it’s but not. Why are those kind of basic
functions have been recoded ? Is there a reason not to use sprintf ?

Hello!

On Thu, Mar 26, 2009 at 01:27:21PM +0100, Jérôme Loyet wrote:

I add tow logs to check the string content:
2009/03/26 11:56:34 [notice] 22943#0: pid file: ./logs/nginx.pid appq
2009/03/26 11:56:34 [notice] 22943#0: lock file: ./logs/nginx.lock Y

The function ngx_s(n)printf does not work with “%s%s”.

How do you log it? If you use “%s” while logging it won’t work -
since nginx strings aren’t normally null-terminated (and it looks
for me that your code doesn’t terminate them). You should
use %V instead, or %*s with explicitly specified length.

BTW, this may be an issue since in many places nginx assumes that
variables from configuration directives are null-terminated.
Probably you want to use ngx_snprintf("%s%s%Z") in your code
instead, this will produce null-terminated string (%Z expands to
‘\0’, see src/core/ngx_string.c for details).

If I use sprintf instead, it works well. I was sure that ngx_sprintf
was an alias to sprintf it’s but not. Why are those kind of basic
functions have been recoded ? Is there a reason not to use sprintf ?

There are at least two reasons:

  1. Standard sprintf() uses null-terminated strings and unable to
    correctly work with binary data. Also it wastes space and cpu
    time for terminating ‘\0’ where it isn’t needed.

  2. Standard sprintf() implementations tends to behave badly on
    out-of-memory conditions, since they allocate internal buffers for
    various reasons and usually call abort() if allocation fails.

Maxim D.

2009/3/26 Igor S. [email protected]:

have overflows using this function.
ngx_s(n)printf() does not add the trailing zero as nginx does not need
As to reasons not to use sprintf(), it has no formats even for
builtin types such as time_t, off_t, pid_t, rlim_t. And I do not want
to convert them to “long long”, since on 32-bit platforms %lld format
calls procedures just for division and module, where simple
CPU division/module opcode are enough. Besides, ngx_s(n)printf() supports
some nginx types such as ngx_str_t.

Thanks you for the answer. It makes sense now :slight_smile:
So here is the last version of the patch (for 0.7.46). ngx_sprintf are
handled correctly and I change a little something which was not great.

Before, I used the RUNTIME_PREFIX as a prefix for anything (pid, lock,
error_log, http log, include, …). It’s still the same but for the
include directive. Now there is a NGX_PREFIX_CONF_PREFIX in
auto/options which has the value “conf”. And when include are used,
path are relative to “RUNTIME_PREFIX/NGX_PREFIX_CONF_PREFIX” (instead
of . RUNTIME_PREFIX before). This makes the patch working with the
default nginx.conf file.

All include are relative to RUNTIME_PREFIX except for the include
directives (and others relative to conf) which are relative to
RUNTIME_PREFIX/conf by default.

I made few tests but not so much.

++ Jerome

On Thu, Mar 26, 2009 at 01:27:21PM +0100, J?r?me Loyet wrote:

I add tow logs to check the string content:
2009/03/26 11:56:34 [notice] 22943#0: pid file: ./logs/nginx.pid appq
2009/03/26 11:56:34 [notice] 22943#0: lock file: ./logs/nginx.lock Y

The function ngx_s(n)printf does not work with “%s%s”.
If I use sprintf instead, it works well. I was sure that ngx_sprintf
was an alias to sprintf it’s but not. Why are those kind of basic
functions have been recoded ? Is there a reason not to use sprintf ?

ngx_s(n)printf() does not add the trailing zero as nginx does not need
it in general. Therefore instead of

        ngx_snprintf(ccf->pid.data, ccf->pid.len, "%s%s",
                     cycle->root.data, NGX_PREFIX_PID_PATH);

you need

        ngx_sprintf(ccf->pid.data, "%V%s%Z",
                    &cycle->root, NGX_PREFIX_PID_PATH);

As to reasons not to use sprintf(), it has no formats even for
builtin types such as time_t, off_t, pid_t, rlim_t. And I do not want
to convert them to “long long”, since on 32-bit platforms %lld format
calls procedures just for division and module, where simple
CPU division/module opcode are enough. Besides, ngx_s(n)printf()
supports
some nginx types such as ngx_str_t.