Support for 3rd party zlib libraries

addis_a · July 29, 2014, 10:46pm

Hello,
I recently came across a modified version of zlib with code contributed
by
Intel [1] that makes use of modern CPU instructions to increase
performance. In testing, the performance gains seemed substantial,
however
when I tried to use this version with nginx, the following alert types
appeared in the error_log on gzip requests:

2014/07/30 05:20:29 [alert] 22285#0: *739837 gzip filter failed to use
preallocated memory: 65536 of 65520 while sending to client
2014/07/30 05:40:42 [alert] 29462#0: *230460 gzip filter failed to use
preallocated memory: 1040 of 4112

It appears nginx is pre-allocating a buffer based on the original zlib
memory usage patterns, however the Intel optimized version has slightly
higher memory requirements due to padding for SSE functions etc.

Is there a chance this version could be supported by nginx, or a
configuration option made available to control the allocation size?

Thanks.

[1] GitHub - intel/zlib

indigoi · July 29, 2014, 11:23pm

Hello!

On Tue, Jul 29, 2014 at 1:46 PM, Richard S. wrote:

I recently came across a modified version of zlib with code contributed by
Intel [1] that makes use of modern CPU instructions to increase performance.
In testing, the performance gains seemed substantial, however when I tried
to use this version with nginx, the following alert types appeared in the
error_log on gzip requests:

[…]

Is there a chance this version could be supported by nginx, or a
configuration option made available to control the allocation size?

Well, I used to write a patch to enable IPP zlib (8.0) support in
NGINX (enabled by ./configure --with-ipp-zlib), just for your
reference:

HG changeset patch

User Yichun Z. [email protected]

Date 1406668777 25200

Tue Jul 29 14:19:37 2014 -0700

Node ID 2a54efe7a747af2f70cb8af0cff62910d6b84a7f

Parent c038cc33739bbfab2ed50819191298471f22d233

Gzip: added support for IPP zlib 8.0.

This feature can now be enabled by ./configure --with-zlib-ipp.

diff -r c038cc33739b -r 2a54efe7a747 auto/lib/zlib/conf
— a/auto/lib/zlib/conf Fri Jul 25 14:43:29 2014 -0700
+++ b/auto/lib/zlib/conf Tue Jul 29 14:19:37 2014 -0700
@@ -6,6 +6,15 @@
if [ $ZLIB != NONE ]; then
CORE_INCS=“$CORE_INCS $ZLIB”

if [ “$ZLIB_IPP” = YES ]; then
+cat << END

+$0: error: option --with-zlib-ipp conflicts with --with-zlib=.
+
+END

```
   exit 1
```
fi
case “$NGX_CC_NAME” in
```
   msvc* | owc* | bcc)
```

@@ -53,18 +62,26 @@ else
ngx_feature_incs=“#include <zlib.h>”
ngx_feature_path=
ngx_feature_libs=“-lz”
+
ngx_feature_test=“z_stream z; deflate(&z, Z_NO_FLUSH)”
. auto/feature

     if [ $ngx_found = yes ]; then
         CORE_LIBS="$CORE_LIBS $ngx_feature_libs"

```
       ZLIB=YES
```

```
       if [ "$ZLIB_IPP" = YES ]; then
```

           have=NGX_HAVE_ZLIB_IPP . auto/have

```
           ZLIB=IPP
```
```
       else
```
```
           ZLIB=YES
```
```
       fi
```
```
       ngx_found=no
   fi
```
fi

if [ $ZLIB != YES ]; then

if [ $ZLIB != YES -a $ZLIB != IPP ]; then
cat << END

$0: error: the HTTP gzip module requires the zlib library.
diff -r c038cc33739b -r 2a54efe7a747 auto/options
— a/auto/options Fri Jul 25 14:43:29 2014 -0700
+++ b/auto/options Tue Jul 29 14:19:37 2014 -0700
@@ -133,6 +133,7 @@ SHA1_OPT=
SHA1_ASM=NO

USE_ZLIB=NO
+ZLIB_IPP=NO
ZLIB=NONE
ZLIB_OPT=
ZLIB_ASM=NO
@@ -299,6 +300,7 @@ use the "–without-http_limit_conn_modu
–with-sha1-opt=*) SHA1_OPT=“$value” ;;
–with-sha1-asm) SHA1_ASM=YES ;;

   --with-zlib-ipp)                 ZLIB_IPP=YES               ;;
   --with-zlib=*)                   ZLIB="$value"              ;;
   --with-zlib-opt=*)               ZLIB_OPT="$value"          ;;
   --with-zlib-asm=*)               ZLIB_ASM="$value"          ;;

diff -r c038cc33739b -r 2a54efe7a747 auto/summary
— a/auto/summary Fri Jul 25 14:43:29 2014 -0700
+++ b/auto/summary Tue Jul 29 14:19:37 2014 -0700
@@ -65,6 +65,7 @@ esac

case $ZLIB in
YES) echo " + using system zlib library" ;;

IPP) echo " + using IPP zlib library" ;;
NONE) echo " + zlib library is not used" ;;
*) echo " + using zlib library: $ZLIB" ;;
esac
diff -r c038cc33739b -r 2a54efe7a747
src/http/modules/ngx_http_gzip_filter_module.c
— a/src/http/modules/ngx_http_gzip_filter_module.c Fri Jul 25
14:43:29 2014 -0700
+++ b/src/http/modules/ngx_http_gzip_filter_module.c Tue Jul 29
14:19:37 2014 -0700
@@ -521,7 +521,18 @@ ngx_http_gzip_filter_memory(ngx_http_req
- *) 5920 bytes on amd64 and sparc64
  */

+#if NGX_HAVE_ZLIB_IPP

/* Below is from deflate.c in ipp-samples.8.0.0.005 */
if (wbits == 8) {
```
   wbits = 9;
```
}
ctx->allocated = 8192 + 5 * (1 << (memlevel + 6)) + (1 << (wbits +
1))

                + (1 << (wbits + 2)) + (1 << (memlevel + 9));

+#else
ctx->allocated = 8192 + (1 << (wbits + 2)) + (1 << (memlevel + 9));
+#endif
}

indigoi · July 30, 2014, 12:48am

Well, I used to write a patch to enable IPP zlib (8.0) support in
NGINX (enabled by ./configure --with-ipp-zlib), just for your
reference:

Thank you for the patch. This solves the issue with streamed responses,
however when the “if (r->headers_out.content_length_n > 0)” branch is
taken, eg with static content, I still receive the 2nd alert type below.
The loop which reduces wbits and memlevel seems like it may need
adjusting,
I will try to figure out the exact values but for now I commented it out
to
avoid the alerts at the expense of some wasted memory.

014/07/29 17:40:00 [alert] 22854#0: *15 gzip filter failed to use
preallocated memory: 65536 of 34800

indigoi · July 30, 2014, 1:06am

Hello!

On Tue, Jul 29, 2014 at 3:47 PM, Richard S. wrote:

Thank you for the patch. This solves the issue with streamed responses,
however when the “if (r->headers_out.content_length_n > 0)” branch is taken,
eg with static content, I still receive the 2nd alert type below.

Oh, we should probably skip that condition altogether for IPP zlib.
The formula is accurate and was copied directly from the IPP zlib
source code. Try this additional patch:

diff -r 2a54efe7a747 src/http/modules/ngx_http_gzip_filter_module.c
— a/src/http/modules/ngx_http_gzip_filter_module.c Tue Jul 29
14:19:37 2014 -0700
+++ b/src/http/modules/ngx_http_gzip_filter_module.c Tue Jul 29
16:06:03 2014 -0700
@@ -492,6 +492,7 @@ ngx_http_gzip_filter_memory(ngx_http_req
wbits = conf->wbits;
memlevel = conf->memlevel;

+#if !NGX_HAVE_ZLIB_IPP
if (r->headers_out.content_length_n > 0) {

     /* the actual zlib window size is smaller by 262 bytes */

@@ -505,6 +506,7 @@ ngx_http_gzip_filter_memory(ngx_http_req
memlevel = 1;
}
}
+#endif

 ctx->wbits = wbits;
 ctx->memlevel = memlevel;

indigoi · July 30, 2014, 1:10am

Hey Yichun,

Oh, we should probably skip that condition altogether for IPP zlib.
The formula is accurate and was copied directly from the IPP zlib
source code. Try this additional patch:

Just to make this clear, the zlib library that Richard is referring to
is a fork of standard zlib (like ours), not IPP zlib.

Best regards,
Piotr S.

indigoi · July 30, 2014, 2:06am

Hello!

On Tue, Jul 29, 2014 at 04:06:11PM -0700, Yichun Z. (agentzh) wrote:

 if (r->headers_out.content_length_n > 0) {

     /* the actual zlib window size is smaller by 262 bytes */

Skipping this block is a bad idea - it means that small
responses will allocate many unneeded memory.

(And, as already pointed out by Piotr, the original question isn’t
about IPP zlib.)

–
Maxim D.
http://nginx.org/

indigoi · July 30, 2014, 4:17am

Hello!

On Tue, Jul 29, 2014 at 10:46:03PM +0200, Richard S. wrote:

preallocated memory: 1040 of 4112
[1] GitHub - intel/zlib
This version indeed uses larger allocations (much larger in some
cases), and also uses different sizes which confuses nginx code
when it tries to allocate 8k for deflate state.

Quick hack to do correct preallocations for this zlib version:

— a/src/http/modules/ngx_http_gzip_filter_module.c Wed Jul 09
12:27:15 2014 -0700
+++ b/src/http/modules/ngx_http_gzip_filter_module.c Wed Jul 30
05:57:50 2014 +0400
@@ -521,7 +521,16 @@ ngx_http_gzip_filter_memory(ngx_http_req
* *) 5920 bytes on amd64 and sparc64
*/

+#if 0
ctx->allocated = 8192 + (1 << (wbits + 2)) + (1 << (memlevel + 9));
+#endif
+

if (conf->level == 1) {
```
   wbits = 13;
```
}
ctx->allocated = 8192 + 16 + (1 << (wbits + 2))

                + (1 << (ngx_max(memlevel, 8) + 8)) + (1 <<

(memlevel + 8));
}

@@ -987,7 +996,7 @@ ngx_http_gzip_filter_alloc(void *opaque,

 alloc = items * size;

if (alloc % 512 != 0 && alloc < 8192) {

if (alloc % 512 != 0 && alloc < 8192 && items == 1) {

/*
 * The zlib deflate_state allocation, it takes about 6K,

This is expected to work with normal zlib as well, but will be
suboptimal from memory usage point of view.

Note well that alerts logged aren’t really serious - nginx is able
to handle such cases properly, and will fallback to normal pool
allocations instead of using preallocated block. It’s just not
something it expects to ever happen, so it logs alerts to make
sure it’s noticed. With different zlib libraries out there, we
probably want to just silence the alert, keeping preallocation
tuned for normal zlib.

–
Maxim D.
http://nginx.org/

indigoi · July 30, 2014, 1:14am

Hello!

On Tue, Jul 29, 2014 at 4:09 PM, Piotr S. wrote:

Just to make this clear, the zlib library that Richard is referring to
is a fork of standard zlib (like ours), not IPP zlib.

Okay, I see. Thank you for pointing that out

Regards,
-agentzh