Forum: NGINX RFC: Feasibility of a "dynamic module loader" built in to nginx?

It seems (at least to me) that the past year has seen explosive growth
in nginx usage and third party module development.

I believe that for package management, as well as module adoption, it
would be a definite move in the right direction to be able to load
modules up on demand instead of having to custom compile nginx with
the modules you want.

That allows for module development to be done completely independently
of nginx, modules to be added/removed, etc, without needing to
recompile nginx every time.

I find that the majority of people would most likely say this is a
good idea, a few may think it is not necessary, but everyone can
benefit, and for those who don't, they can still maintain their nginx
however they want, by recompiling each time.

I don't expect much overhead being added, as the modules would still
only be loaded when the master process (or perhaps child processes?)
startup.

Ideally it would be as simple as

load /usr/lib/nginx/mod_drizzle.so;

Or something in the configuration. This would allow plugins to
flourish without complex build instructions or having to touch the
nginx core; module developers could be informed to make version checks
and issue a notice when trying to startup, something along the lines
of "mod_drizzle is not compatible with this version of nginx (=>
0.8.43 needed)" or something of that nature.

I do not know if nginx could support this or not, it seems like it
should be able to, if it is able to accept modules at compile time,
just some more magic could be added to dynamically load something else
instead.

I'm sure some people would offer to help sponsor this if needed...

As usual, this is open for discussion, I'd really like to hear
Igor/Maxim's take, or what the FRiCKLE guys or agentzh would have to
say about how useful this would be or how easy they think it could be
accomplished.

Personally I have not played with many third party modules; I have
passenger being built as part of my build script but I only use it on
one machine, all the other ones issue an annoying notice (that I've
been trying to get removed) - but being able to just install a module
and then enable it quickly to play with it, that would allow me to try
out all these third party modules out there.

The future seems to be moving to ultimate modularity. Having to
recompile the core each time for a third party module seems archaic
still. This should allow for module developers to focus a lot more on
purely releasing updates to modules, and not instructions on compiling
core with them, etc.

Thanks,
mike

Hello!

On Thu, Aug 26, 2010 at 04:14:10AM -0700, Michael Shadle wrote:

> recompile nginx every time.
[...]

This was already discussed several times.  Just to name a few
reasons why no:

1. Using dynamically linked libraries implies speed penalty.

2. It is likely to cause problems like DLL hell.  Even with small
number of carefully selected dynamic libraries nginx currently use
(e.g. openssl) - there are periodically reported problems.

3. It will require to maintain ABI, not something we currently do.
E.g. right now ngx_http_request_s structure changes depending on:
whether pcre library used, http cache enabled, gzip support
enabled, stub status compiled in.  Not even talking about local
patches and changes between versions.  So things is much more
complex than "0.8.43 needed".

4. Most of us are happy enough with static linkage.  Especially
keeping in mind that binary may be upgraded with zero downtime.

Maxim Dounin

On Thu, Aug 26, 2010 at 7:14 PM, Michael Shadle <mike503@gmail.com> 
wrote:
> I believe that for package management, as well as module adoption, it
> would be a definite move in the right direction to be able to load
> modules up on demand instead of having to custom compile nginx with
> the modules you want.
>

I totally agree here :) We wrote 15 nginx C modules in the last year
(just one or two are opensourced yet) and we found rpm packaging
especially hard in a corporation environment. For every different kind
of business, we have to provide a variation of nginx rpm package just
because they use a different set of modules. (Yeah, we could compile
all those modules into a giant nginx but it hurts performance.)

> That allows for module development to be done completely independently
> of nginx, modules to be added/removed, etc, without needing to
> recompile nginx every time.
>

I personally spent a lot of CPU cycles on recompiling nginx on my
laptop during nginx development. Fortunately I have a fast machine,
but still it's far from ideal :)

> I find that the majority of people would most likely say this is a
> good idea, a few may think it is not necessary, but everyone can
> benefit, and for those who don't, they can still maintain their nginx
> however they want, by recompiling each time.
>

Having more choices is always a Good Thing (TM) ;)

> I don't expect much overhead being added, as the modules would still
> only be loaded when the master process (or perhaps child processes?)
> startup.
>

chaoslawful and I have talked about the implementation details several
times and dynamic loader support for the nginx core is one of our TODO
items. And yeah, it also means that it's very likely that we'll
maintain our own patches to the core or even our own fork of nginx
(given the historic records in the nginx world) :)

> Ideally it would be as simple as
>
> load /usr/lib/nginx/mod_drizzle.so;
>

Cool, and by that way, we can re-arrange the order of nginx modules
just in the config file! Like so:

   load /path/to/ngx_rds_json.so
   load /path/to/ngx_srcache.so
   load /path/to/ngx_lua.so

or make it even more elegant and more portable for other OS's:

   use ngx_rds_json;
   use ngx_srcache;
   use ngx_lua;

And we can control the search paths for the module .so files from the
environment. Quite perl-ish, isn't it? ;)

> Or something in the configuration. This would allow plugins to
> flourish without complex build instructions or having to touch the
> nginx core; module developers could be informed to make version checks
> and issue a notice when trying to startup, something along the lines
> of "mod_drizzle is not compatible with this version of nginx (=>
> 0.8.43 needed)" or something of that nature.
>

*nod*

There we go!

Cheers,
-agentzh

On Fri, Aug 27, 2010 at 2:03 PM, agentzh <agentzh@gmail.com> wrote:
>
> I totally agree here :) We wrote 15 nginx C modules in the last year
> (just one or two are opensourced yet)

Sorry, typo here. It should have been "just one or two are not
opensourced yet" :P

Cheers,
-agentzh

Hi,
I was thinking about this about two weeks, but (as Maxim already pointed
out) currently there is no ABI, which means that you would need to build
modules against your target nginx version on your target operating 
system.

This renders this idea viable only for 3 use cases that I can think of 
right
now:
1) operating system's pre-compiled binary packages,
2) corporate environments / clusters with staging server(s),
3) closed-source modules.

Having this probably wouldn't hurt, but it won't be useful for as many
people as you would hope to.

Best regards,
Piotr Sikora < piotr.sikora@frickle.com >

Hi,

What might work just as well, and wouldn't suffer the API / performance
issues would be making it easier to compile optional modules into the
source without recompiling the whole Nginx source code.

When I'm doing module development, I just build the object code of the
module I'm writing, and combined it with the pre-compiled object files
of all the other parts of Nginx, thereby typically taking less than a
second to compile (I'm assuming most developers do this too).

It would be nice to be able to have an alternative build script that
makes it easier to do this in a customizable way, allowing the reuse of
existing object code if it's available, compiling it if it's not and
forcing re-compiling if so desired.

It could be set up to accept the normal configuration options, then
automatically work out which object files needed to be re-generated and
which ones could use the pre-compiled versions, and could be extended to
include options and 3rd party modules too.

It wouldn't be particularly difficult to do this, but I'm really busy
with other stuff at the moment. If someone wants to write such a script,
I'll add it to the ngx_devel_kit
(http://github.com/simpl-it/ngx_devel_kit).  Otherwise I'll write it at
some point later on, but I can't give any timelines as of yet.

Cheers,

Marcus.