Forum: NGINX nginx-0.7.44

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
Igor S. (Guest)
on 2009-03-23 15:47
(Received via mailing list)
Changes with nginx 0.7.44                                        23 Mar
2009

    *) Feature: the ngx_http_proxy_module preliminary cache support.

    *) Feature: the --with-pcre option in the configure.

    *) Feature: the "try_files" directive is now allowed on the server
       block level.

    *) Bugfix: the "try_files" directive handled incorrectly a query
string
       in a fallback parameter.

    *) Bugfix: the "try_files" directive might test incorrectly
directories.

    *) Bugfix: if there is the single server for given address:port
pair,
       then captures in regular expressions in a "server_name" directive
       did not work.
lhmwzy (Guest)
on 2009-03-23 15:57
(Received via mailing list)
Good work!

2009/3/23 Igor S. <removed_email_address@domain.invalid>:
Kevin W. (Guest)
on 2009-03-23 16:26
(Received via mailing list)
Nginx 0.7.44 for Windows is available for free download here:
http://cli.gs/744
--
Kevin W.




2009/3/23 Igor S. <removed_email_address@domain.invalid>:
Jérôme Loyet (Guest)
on 2009-03-23 16:48
(Received via mailing list)
2009/3/23 Igor S. <removed_email_address@domain.invalid>:
>       in a fallback parameter.
>
>    *) Bugfix: the "try_files" directive might test incorrectly directories.
>
>    *) Bugfix: if there is the single server for given address:port pair,
>       then captures in regular expressions in a "server_name" directive
>       did not work.
>
>
Hi Igor,

I have problems compiling this last version:

In file included from src/http/ngx_http_file_cache.c:10:
src/core/ngx_md5.h:18:17: md5.h: No such file or directory
In file included from src/http/ngx_http_file_cache.c:10:
src/core/ngx_md5.h:22: error: syntax error before "ngx_md5_t"
src/core/ngx_md5.h:22: warning: type defaults to `int' in declaration
of `ngx_md5_t'
src/core/ngx_md5.h:22: warning: data definition has no type or storage
class
src/http/ngx_http_file_cache.c: In function
`ngx_http_file_cache_create_key':
src/http/ngx_http_file_cache.c:94: error: syntax error before "md5"
src/http/ngx_http_file_cache.c:102: warning: implicit declaration of
function `MD5Init'
src/http/ngx_http_file_cache.c:102: error: `md5' undeclared (first use
in this function)
src/http/ngx_http_file_cache.c:102: error: (Each undeclared identifier
is reported only once
src/http/ngx_http_file_cache.c:102: error: for each function it appears
in.)
src/http/ngx_http_file_cache.c:112: warning: implicit declaration of
function `MD5Update'
src/http/ngx_http_file_cache.c:119: warning: implicit declaration of
function `MD5Final'
make[1]: *** [objs/src/http/ngx_http_file_cache.o] Error 1
make[1]: Leaving directory `/misc/distrib/BUILD/nginx-0.7.44'
make: *** [build] Error 2

I'm not using openssl and I don't have any md5.h file in my include
path. /usr/include/md5.h is valid on BSD but not on Linux. I have to
add the openssl library somehow in configure. I think there is a
missing check in the configure script.

The build is working correctly if I'm using
--openssl-dir=/usr/local/openssl-0.9.6.

++ Jerome
Jérôme Loyet (Guest)
on 2009-03-23 16:48
(Received via mailing list)
Le 23 mars 2009 15:37, Jérôme Loyet <removed_email_address@domain.invalid> a 
écrit :
>>    *) Bugfix: the "try_files" directive handled incorrectly a query string
>
> src/http/ngx_http_file_cache.c:94: error: syntax error before "md5"
> function `MD5Final'
> --openssl-dir=/usr/local/openssl-0.9.6.
Sorry I miss something: The build is working correctly if I'm using
--openssl-dir=/usr/local/openssl-0.9.6 AND --with-http_ssl_module
(only openssl-dir show that it founds the library but it doesn't use
it when compiling (no -I /usr/local/openssl-0.9.6/include))
Delta Y. (Guest)
on 2009-03-23 16:48
(Received via mailing list)
How to use the cache feature?
What kind of cache does it support?

2009/3/23 Igor S. <removed_email_address@domain.invalid>
Igor S. (Guest)
on 2009-03-23 16:58
(Received via mailing list)
On Mon, Mar 23, 2009 at 03:41:51PM +0100, J?r?me Loyet wrote:

> (only openssl-dir show that it founds the library but it doesn't use
> it when compiling (no -I /usr/local/openssl-0.9.6/include))

nginx's configure has no --openssl-dir= option.
Jérôme Loyet (Guest)
on 2009-03-23 17:01
(Received via mailing list)
2009/3/23 Igor S. <removed_email_address@domain.invalid>:
>> Sorry I miss something: The build is working correctly if I'm using
>> --openssl-dir=/usr/local/openssl-0.9.6 AND --with-http_ssl_module
>> (only openssl-dir show that it founds the library but it doesn't use
>> it when compiling (no -I /usr/local/openssl-0.9.6/include))
>
> nginx's configure has no --openssl-dir= option.

I meant --with-openssl= option. Sorry
Igor S. (Guest)
on 2009-03-23 17:09
(Received via mailing list)
On Mon, Mar 23, 2009 at 10:42:45PM +0800, Delta Y. wrote:

> How to use the cache feature?
> What kind of cache does it support?

Here is configuraiton example:

http {
     proxy_cache_path   /path/to/cache  levels=1:2
                        keys_zone=NAME:10m
                        inactive=5m     clean_time=2h00m;

     server {
         location / {
             proxy_pass    http://127.0.0.1;

             proxy_cache   NAME;

             proxy_cache_valid   200 302  1h;
             proxy_cache_valid   301      1d;
             proxy_cache_valid   any      1m;

             proxy_cache_min_uses  1;

             proxy_cache_use_stale   error  timeout invalid_header
http_500;
         }
     }

The cache currently ignores backend's Cache-Control, Expires, etc.
Igor S. (Guest)
on 2009-03-23 17:10
(Received via mailing list)
On Mon, Mar 23, 2009 at 03:55:54PM +0100, J?r?me Loyet wrote:

> >>
> >> Sorry I miss something: The build is working correctly if I'm using
> >> --openssl-dir=/usr/local/openssl-0.9.6 AND --with-http_ssl_module
> >> (only openssl-dir show that it founds the library but it doesn't use
> >> it when compiling (no -I /usr/local/openssl-0.9.6/include))
> >
> > nginx's configure has no --openssl-dir= option.
>
> I meant --with-openssl= option. Sorry

The --with-openssl= option should point to OpenSSL sources, but not
to inlcudes and binary libraries.
Jérôme Loyet (Guest)
on 2009-03-23 17:13
(Received via mailing list)
2009/3/23 Igor S. <removed_email_address@domain.invalid>:
>> >> > The build is working correctly if I'm using
>
> The --with-openssl= option should point to OpenSSL sources, but not
> to inlcudes and binary libraries.

OK thx for the answer.
Igor S. (Guest)
on 2009-03-23 17:25
(Received via mailing list)
On Mon, Mar 23, 2009 at 04:06:32PM +0100, J?r?me Loyet wrote:

> >> >> >
> >> I meant --with-openssl= option. Sorry
> >
> > The --with-openssl= option should point to OpenSSL sources, but not
> > to inlcudes and binary libraries.
>
> OK thx for the answer.

As your OpenSSL is set in non-standard place (I have just tested
building
on Debian with standard /usr/include/openssl/md5.h), you should use
something like this:

--with-cc-opt=-I/usr/local/openssl-0.9.6/include
Jérôme Loyet (Guest)
on 2009-03-23 17:34
(Received via mailing list)
2009/3/23 Igor S. <removed_email_address@domain.invalid>:
>> >> >> > add the openssl library somehow in configure. I think there is a
>> >> > nginx's configure has no --openssl-dir= option.
> something like this:
>
> --with-cc-opt=-I/usr/local/openssl-0.9.6/include

it's not working because of the #ifdef in md5.h:

#if (NGX_HAVE_OPENSSL_MD5_H)
#include <openssl/md5.h>
#else
#include <md5.h>
#endif

Has NGX_HAVE_OPENSSL_MD5_H is not defined, it looks for md5.h instead
of openssl/md5.h. And if I use
--with-cc-opt=-I/usr/local/openssl-0.9.6/include/openssl it won't work
because the function name are from OpenSSL lib and not from BSD kernel
(MD5_Init vs MD5Init).

I have to make configure to set NGX_HAVE_OPENSSL_MD5_H ... do you have
any clue ? I'm still searching

thx a lot
Igor S. (Guest)
on 2009-03-23 17:40
(Received via mailing list)
On Mon, Mar 23, 2009 at 04:27:53PM +0100, J?r?me Loyet wrote:

> >> >> >> > path. /usr/include/md5.h is valid on BSD but not on Linux. I have to
> >> >> >
> > on Debian with standard /usr/include/openssl/md5.h), you should use
> #endif
>
> Has NGX_HAVE_OPENSSL_MD5_H is not defined, it looks for md5.h instead
> of openssl/md5.h. And if I use
> --with-cc-opt=-I/usr/local/openssl-0.9.6/include/openssl it won't work
> because the function name are from OpenSSL lib and not from BSD kernel
> (MD5_Init vs MD5Init).
>
> I have to make configure to set NGX_HAVE_OPENSSL_MD5_H ... do you have
> any clue ? I'm still searching

-  --with-cc-opt=-I/usr/local/openssl-0.9.6/include/openssl
+  --with-cc-opt=-I/usr/local/openssl-0.9.6/include
Floren M. (Guest)
on 2009-03-23 18:10
(Received via mailing list)
Hi Igor,

>      proxy_cache_path   /path/to/cache  levels=1:2
>                         keys_zone=NAME:10m
>                         inactive=5m     clean_time=2h00m;
>
>      server {
>          location / {
>              proxy_pass    http://127.0.0.1;
>

Can we use the $scheme, instead http?

Thanks for the release.

Floren
Jérôme Loyet (Guest)
on 2009-03-23 18:13
(Received via mailing list)
2009/3/23 Igor S. <removed_email_address@domain.invalid>:
>> >> >> >
>> >> >> >> (only openssl-dir show that it founds the library but it doesn't use
>> >
>> #else
>> any clue ? I'm still searching
>
> -  --with-cc-opt=-I/usr/local/openssl-0.9.6/include/openssl
> +  --with-cc-opt=-I/usr/local/openssl-0.9.6/include

Here are configuration options:
./configure --with-http_addition_module --with-http_flv_module
--with-http_gzip_static_module --with-http_realip_module
--with-http_stub_status_module --with-http_sub_module
--with-cc-opt=-I/usr/local/openssl-0.9.6/include

And a part of the output:
[...]
checking for rsaref md5 library ... not found
checking for rsaref md library ... not found
checking for OpenSSL md5 crypto library ... not found
[...]
Configuration summary
  + using system PCRE library
  + OpenSSL library is not used
  + md5 library is not found
  + sha1 library is not used
  + using system zlib library

as OpenSSL md5 crypto library is not found, NGX_HAVE_OPENSSL_MD5_H is
not defined by configure. So ngx_md5.h will try to include <md5.h> and
not <openssl/md5.h>

#if (NGX_HAVE_OPENSSL_MD5_H)
#include <openssl/md5.h>
#else
#include <md5.h>
#endif

And make stops at compilation time when including ngx_md5.h from
ngx_http_file_cache.c

How to tell to configure script to lookup in /usr/local/openssl-0.9.6
for headers and lib from openssl ?
Igor S. (Guest)
on 2009-03-23 18:18
(Received via mailing list)
On Mon, Mar 23, 2009 at 05:03:40PM +0100, J?r?me Loyet wrote:

> >> >> >> > On Mon, Mar 23, 2009 at 03:41:51PM +0100, J?r?me Loyet wrote:
> >> >> >> >> --openssl-dir=/usr/local/openssl-0.9.6 AND --with-http_ssl_module
> >> >> OK thx for the answer.
> >> #include <openssl/md5.h>
> >> I have to make configure to set NGX_HAVE_OPENSSL_MD5_H ... do you have
>
>   + sha1 library is not used
>   + using system zlib library

What is in objs/autoconf.err under the line:
checking for OpenSSL md5 crypto library
?

> And make stops at compilation time when including ngx_md5.h from
> ngx_http_file_cache.c
>
> How to tell to configure script to lookup in /usr/local/openssl-0.9.6
> for headers and lib from openssl ?

This may be not include problem, but library's one.
Where is md5.h and libcrypto.so on your system:

/usr/local/openssl-0.9.6/include/openssl/md5.h
/usr/local/openssl-0.9.6/lib/libcrypto.so

?
Igor S. (Guest)
on 2009-03-23 18:20
(Received via mailing list)
On Mon, Mar 23, 2009 at 11:55:09AM -0400, Floren M. wrote:

> >      proxy_cache_path   /path/to/cache  levels=1:2
> >                         keys_zone=NAME:10m
> >                         inactive=5m     clean_time=2h00m;
> >
> >      server {
> >          location / {
> >              proxy_pass    http://127.0.0.1;
> >
>
> Can we use the $scheme, instead http?

You can, however, I do not understand how this relates to the caching.
Anton Y. (Guest)
on 2009-03-23 18:26
(Received via mailing list)
Floren M. wrote:
>>              proxy_pass    http://127.0.0.1;
>>
>
> Can we use the $scheme, instead http?
>
> Thanks for the release.
>

Why you want to use https:// between frontend and backend (which usually
connected via trusted
network where encryption is unnecessary).
Floren M. (Guest)
on 2009-03-23 19:01
(Received via mailing list)
Hi Anton,

> network where encryption is unnecessary).
Thanks for the clarification, I am not familiar with proxies.

Regards,

Floren
Igor S. (Guest)
on 2009-03-23 19:06
(Received via mailing list)
Attachment: patch.fastcgi.cache (0 Bytes)
On Mon, Mar 23, 2009 at 05:57:27PM +0300, Igor S. wrote:

>                         inactive=5m     clean_time=2h00m;
>
>              proxy_cache_min_uses  1;
>
>              proxy_cache_use_stale   error  timeout invalid_header http_500;
>          }
>      }
>
> The cache currently ignores backend's Cache-Control, Expires, etc.

The attached patch enables caching for FastCGI. The directives are the
same
as in proxy cache, just change proxy_ prefix to fastcgi_ one. Plus

fastcgi_cache_key  127.0.0.1:9000$request_uri;

to evaluate unique key in cache. In proxy the key is evalueyed intrnally
is something like

$schema$proxy_host:$proxy_port$uri$is_args$args
Jérôme Loyet (Guest)
on 2009-03-23 20:02
(Received via mailing list)
2009/3/23 Igor S. <removed_email_address@domain.invalid>:
>> >> >> >
>> >> >> >> >>
>> >> >> > to inlcudes and binary libraries.
>> >>
>> >> (MD5_Init vs MD5Init).
>> --with-http_stub_status_module --with-http_sub_module
>>   + OpenSSL library is not used
>> not <openssl/md5.h>
>> How to tell to configure script to lookup in /usr/local/openssl-0.9.6
>> for headers and lib from openssl ?
>
> This may be not include problem, but library's one.
> Where is md5.h and libcrypto.so on your system:
>
> /usr/local/openssl-0.9.6/include/openssl/md5.h
> /usr/local/openssl-0.9.6/lib/libcrypto.so
>
> ?

I found a small bug which caused my death on this :)

--- auto/feature.orig   Mon Mar 23 18:40:55 2009
+++ auto/feature        Mon Mar 23 18:41:11 2009
@@ -39,7 +39,7 @@


 ngx_test="$CC $CC_TEST_FLAGS $CC_AUX_FLAGS $ngx_feature_inc_path \
-          -o $NGX_AUTOTEST $NGX_AUTOTEST.c $NGX_LD_OPT
$ngx_feature_libs"
+          -o $NGX_AUTOTEST $NGX_AUTOTEST.c $NGX_CC_OPT $NGX_LD_OPT
$ngx_feature_libs"

 ngx_feature_inc_path=


In auto/feature the NGX_CC_OPT variable (from --with-cc-opt configure
option) was not used. So I could set whatever I wanted through
--with-cc-opt it would never be looked at.

now everything works fine. In my case I have to specify both
with-cc-opt and with-cc-opt:
--with-cc-opt=-I/usr/local/openssl-0.9.6/include
--with-ld-opt=-L/usr/local/openssl-0.9.6/lib

Does this patch sound good to you igor ?

++ Jerome
Igor S. (Guest)
on 2009-03-23 22:18
(Received via mailing list)
On Mon, Mar 23, 2009 at 06:51:50PM +0100, J?r?me Loyet wrote:

> >> >> #else
> >> >> any clue ? I'm still searching
> >> And a part of the output:
> >> š + using system zlib library
> >> #include <openssl/md5.h>
> > This may be not include problem, but library's one.
> +++ auto/feature        Mon Mar 23 18:41:11 2009
>
> In auto/feature the NGX_CC_OPT variable (from --with-cc-opt configure
> option) was not used. So I could set whatever I wanted through
> --with-cc-opt it would never be looked at.
>
> now everything works fine. In my case I have to specify both
> with-cc-opt and with-cc-opt:
> --with-cc-opt=-I/usr/local/openssl-0.9.6/include
> --with-ld-opt=-L/usr/local/openssl-0.9.6/lib
>
> Does this patch sound good to you igor ?

The $CC_TEST_FLAGS should have $NGX_CC_OPT.
What is in objs/autoconf.err ?
Jérôme Loyet (Guest)
on 2009-03-24 10:06
(Received via mailing list)
2009/3/23 Igor S. <removed_email_address@domain.invalid>:
>> >> >> #if (NGX_HAVE_OPENSSL_MD5_H)
>> >> >>
>> >> --with-cc-opt=-I/usr/local/openssl-0.9.6/include
>> >>   + md5 library is not found
>> >>
>> >> for headers and lib from openssl ?
>>
>>  ngx_feature_inc_path=
>>
>> Does this patch sound good to you igor ?
>
> The $CC_TEST_FLAGS should have $NGX_CC_OPT.
> What is in objs/autoconf.err ?

Hum ... it seems that yesterday I smoked some very good stuff. As I
tried many things I got confused myself ... I don't know why the hell
I found a bug there as it was working .... I'm very sorry to have you
spent some time on my problem which even didn't exist (except in my
smoked mind :p)

Thanks you
++ Jerome
Jérôme Loyet (Guest)
on 2009-03-24 18:16
(Received via mailing list)
2009/3/23 Igor S. <removed_email_address@domain.invalid>:
>                        inactive=5m     clean_time=2h00m;
>
>             proxy_cache_min_uses  1;
>
>             proxy_cache_use_stale   error  timeout invalid_header http_500;
>         }
>     }
>
> The cache currently ignores backend's Cache-Control, Expires, etc.

hi Igor,

Do I understand correctly the proxy_cache_use_stale directive:

proxy_cache_use_stale   error  timeout invalid_header http_500;

If cache is still valid {
  serve content from cache
} else {
  Ask fresh data to the backend
  if connection status IS IN  (proxy_cache_use_stale parameters) {
      serve content from cache
  } else {
      act as normal
  }
}

thx
++ jerome
Igor S. (Guest)
on 2009-03-24 18:26
(Received via mailing list)
On Tue, Mar 24, 2009 at 05:07:23PM +0100, J?r?me Loyet wrote:

> > š š š š š š š š š š š škeys_zone=NAME:10m
> > š š š š š š proxy_cache_valid š any š š š1m;
>
>   } else {
>       act as normal
>   }
> }

Yes.
Jérôme Loyet (Guest)
on 2009-03-24 18:35
(Received via mailing list)
2009/3/24 Igor S. <removed_email_address@domain.invalid>:
>> > http {
>> >             proxy_cache_valid   200 302  1h;
>>
>>   if connection status IS IN  (proxy_cache_use_stale parameters) {
>>       serve content from cache
>>   } else {
>>       act as normal
>>   }
>> }
>
> Yes.

Other question, how can I determine the maximum number of cache entrie
I have with a particular conf ?

proxy_cache_path  /data/nginx/cache  keys_zone=one:10m;

What is the size of keys on shared mem (X bytes). So that I can
extrapole the maximum number N of cache entries with YMo of shared
memory (N = (Y x 1024 x 1024) / X). (if there is nothing elese in the
shared mem and if the key size is constant)

Thx
++ jerome
Igor S. (Guest)
on 2009-03-24 18:47
(Received via mailing list)
On Tue, Mar 24, 2009 at 05:30:02PM +0100, J?r?me Loyet wrote:

> >> >
> >> >
> >> > The cache currently ignores backend's Cache-Control, Expires, etc.
> >> š Ask fresh data to the backend
> I have with a particular conf ?
>
> proxy_cache_path  /data/nginx/cache  keys_zone=one:10m;
>
> What is the size of keys on shared mem (X bytes). So that I can
> extrapole the maximum number N of cache entries with YMo of shared
> memory (N = (Y x 1024 x 1024) / X). (if there is nothing elese in the
> shared mem and if the key size is constant)

The key size is 128 bytes.
Jérôme Loyet (Guest)
on 2009-03-24 20:32
(Received via mailing list)
2009/3/24 Igor S. <removed_email_address@domain.invalid>:
>> >> >
>> >> >
>> >> >     }
>> >>   serve content from cache
>>
> The key size is 128 bytes.
Thanks for the answer.

Is there a way to know the effective used size in the SHM segement ?

Another question about proxy_cache_valid. Let say I set it to 10m. I
do some hits and the requests are cached well. Now I update my conf
and set proxy_cache_valid to 30s and send a HUP signal to nginx. How
is already cached files timeout handled at restart or reload ?

Is the cache going to be mark as stale 30s after ?
Is the cache going to stay in the cache until its 10m timeout is
reached (and then be recached with a 30s timeout) ?

Thanks you very much. This cache module for proxy is just what we were
waiting for :)

++ Jerome
Igor S. (Guest)
on 2009-03-25 19:27
(Received via mailing list)
On Tue, Mar 24, 2009 at 07:22:03PM +0100, J?r?me Loyet wrote:

> > The key size is 128 bytes.
>
> Thanks for the answer.
>
> Is there a way to know the effective used size in the SHM segement ?

No, if shared memory is overflowed you will see in error_log the
message:
"ngx_slab_alloc(): failed". The problem is that currently no way to see
what memory zone is overflowed: no name and subsystem (proxy cache,
ssl_session_cache, limit requests, etc.) I'm going to fix this.

> Another question about proxy_cache_valid. Let say I set it to 10m. I
> do some hits and the requests are cached well. Now I update my conf
> and set proxy_cache_valid to 30s and send a HUP signal to nginx. How
> is already cached files timeout handled at restart or reload ?
>
> Is the cache going to be mark as stale 30s after ?
> Is the cache going to stay in the cache until its 10m timeout is
> reached (and then be recached with a 30s timeout) ?

The second. The valid time is stored in both memory and file as
now + proxy_cache_valid. Therefore, a response will be valid
for 10m after its creation, and then will be recached for 30s.
Johan Bergström (Guest)
on 2009-03-25 19:44
(Received via mailing list)
On Mar 25, 2009, at 18:16 , Igor S. wrote:

>>>> memory (N = (Y x 1024 x 1024) / X). (if there is nothing elese in
> message:
> "ngx_slab_alloc(): failed". The problem is that currently no way to
> see
> what memory zone is overflowed: no name and subsystem (proxy cache,
> ssl_session_cache, limit requests, etc.) I'm going to fix this.

Would it be possible to somehow add this into the status module, so
it's possible to graph both disk allocation and other stats such as
hits/misses?
Igor S. (Guest)
on 2009-03-26 09:01
(Received via mailing list)
On Wed, Mar 25, 2009 at 06:36:38PM +0100, Johan Bergstr?m wrote:

> >>>>
> >>Is there a way to know the effective used size in the SHM segement ?
> hits/misses?
Only when the full featured status module will be introduced.
This topic is locked and can not be replied to.