Access key module

In October, Mykola Grechukh announced his new ngx_http_accesskey_module
to the Russian listserver.[1] The module lets you give access keys to
specific IP addresses, which is useful for restricting downloads to
certain clients. I packaged his patch into a stand-alone module and
translated the documentation. You can find installation instructions and
usage notes on the wiki:

http://wiki.codemongers.com/NginxHttpAccessKeyModule

This is now the 10th third-party, open-source module that I know of.
Cool!

Evan

  1. http://article.gmane.org/gmane.comp.web.nginx.russian/13650

On Sat, 2007-12-29 at 22:19 -0500, Evan M. wrote:

In October, Mykola Grechukh announced his new ngx_http_accesskey_module
to the Russian listserver.[1] The module lets you give access keys to
specific IP addresses, which is useful for restricting downloads to
certain clients. I packaged his patch into a stand-alone module and
translated the documentation. You can find installation instructions and
usage notes on the wiki:

http://wiki.codemongers.com/NginxHttpAccessKeyModule

This is now the 10th third-party, open-source module that I know of. Cool!

And this makes me wonder if we want to implement some sort of “clearing
house” for 3rd party modules. Specifically, I think the following
would be useful:

  1. A secure, centralized place (RCS system or even just tarballs) to
    download from (wiki is too open: even links to outside source code make
    me nervous since they could be altered by a malicious editor to lead to
    poisoned source code).

  2. Issue tracking. Might be handy for module authors to not have to
    maintain their own Trac (or whatever) instance.

  3. Perhaps adding some feature to the Nginx build process that allowed
    3rd party modules to be easily downloaded and installed (from site
    listed in item 1) via configure/make flags (or is this just wishful
    thinking?).

Anyway, ideas welcome. I’m happy to setup and maintain whatever is
decided upon.

Regards,
Cliff

Cliff W. ha scritto:

This is now the 10th third-party, open-source module that I know of. Cool!

And this makes me wonder if we want to implement some sort of “clearing
house” for 3rd party modules. Specifically, I think the following
would be useful:

  1. A secure, centralized place (RCS system or even just tarballs) to
    download from (wiki is too open: even links to outside source code make
    me nervous since they could be altered by a malicious editor to lead to
    poisoned source code).

Each developer should sign his module source distribution with PGP.

Moreover, each developer should have his key signed by a “master key”
(Igor?) so that nginx core can verify the external module during
configuration.

But none of the open source projects I know with support to external
modules do such a thing… (signing and verification is done at
“Distribution” level, like Debian).

Adding a RCS system can be too hard to handle, altough a distribuited
revision system can help (and some of them permits to sign each
revision).

  1. Issue tracking. Might be handy for module authors to not have to
    maintain their own Trac (or whatever) instance.

Yes, this can be very helpful.
But it is always possible to use Google Code or some other projects
hosting.

  1. Perhaps adding some feature to the Nginx build process that allowed
    3rd party modules to be easily downloaded and installed (from site
    listed in item 1) via configure/make flags (or is this just wishful
    thinking?).

I don’t think this is a priority.
It would help, however, if nginx is more “friendly” with external
modules.

Actually the configure script command line is fixed; I would like, as an
example, to add an hook to the configure script so that for mod_wsgi it
is be possible to specify the Python interpreter path to use via
–with-python option.

Anyway, ideas welcome. I’m happy to setup and maintain whatever is
decided upon.

Regards,
Cliff

Thanks Manlio P.

On Sun, 2007-12-30 at 16:41 +0100, Manlio P. wrote:

Cliff W. ha scritto:

  1. A secure, centralized place (RCS system or even just tarballs) to
    download from (wiki is too open: even links to outside source code make
    me nervous since they could be altered by a malicious editor to lead to
    poisoned source code).

Each developer should sign his module source distribution with PGP.

But if this is linked to from the wiki, then it does little good. All a
malicious person needs to do is sign their poisoned source with PGP as
well, either on the wiki or on a fake module page they substitute for
the real one.

Moreover, each developer should have his key signed by a “master key”
(Igor?) so that nginx core can verify the external module during
configuration.

This would be better, but seems more complicated than need be (at least
initially). All we really need for the moment is a place where not just
any anonymous user who creates a wiki account can change where source
code is available from.

But none of the open source projects I know with support to external
modules do such a thing… (signing and verification is done at
“Distribution” level, like Debian).

Sure, but a lot of people are using source rather than distro tools to
install Nginx, so I don’t think counting on this is safe. Plus, a lot
of OSS projects do exactly this: usually it’s a “contrib” directory in
the source distribution or RCS tree. With Nginx we don’t have the main
source tree hosted in RCS, so it’s a bit different, but the concept is
similar.

This actually brings up another thought: perhaps we should start an RCS
tree for Nginx. We don’t actually need Igor to use it (I’m sure he has
his own tools and methods and I don’t want to interrupt that): diffs can
be generated automatically from the tarball Igor provides. Of course,
we lose revision comments (assuming there are any), but we would at
least have a public history and a place to provide downstream patches
against (i.e. adding the ability to automatically pull 3rd party modules
into the build process).

Adding a RCS system can be too hard to handle, altough a distribuited
revision system can help (and some of them permits to sign each revision).

Not sure why this would be hard. Have current module developers vote on
a preferred RCS and use it. And I agree that a distributed system (such
as git) would be a good choice.

  1. Issue tracking. Might be handy for module authors to not have to
    maintain their own Trac (or whatever) instance.

Yes, this can be very helpful.
But it is always possible to use Google Code or some other projects hosting.

Sure, but it doesn’t solve the problem of creating a trusted code
repository. Remember, the problem isn’t how people are storing their
code: it’s how others get to that code. If they follow a link from a
wiki, then there’s the potential that they are not going to a legitimate
site.

  1. Perhaps adding some feature to the Nginx build process that allowed
    3rd party modules to be easily downloaded and installed (from site
    listed in item 1) via configure/make flags (or is this just wishful
    thinking?).

I don’t think this is a priority.

Well, nothing is a priority until it’s a problem =) We’ve gone from
having zero 3rd party modules to nearly a dozen in just a few months.
It would be nice to get something in place while it’s still a manageable
task.

Regards,
Cliff

Hi,

On Sun, Dec 30, 2007 at 03:48:12PM -0800, Cliff W. wrote:

This actually brings up another thought: perhaps we should start an RCS
tree for Nginx. We don’t actually need Igor to use it (I’m sure he has
his own tools and methods and I don’t want to interrupt that): diffs can
be generated automatically from the tarball Igor provides. Of course,
we lose revision comments (assuming there are any), but we would at
least have a public history and a place to provide downstream patches
against (i.e. adding the ability to automatically pull 3rd party modules
into the build process).

Will http://git.localdomain.pl/ do? It has a few disadvantages:

  • unofficial (I just put it online without asking anybody)
  • a bit messy (though the vanilla branches are clean, the commits there
    have wrong dates and the older ones have wrong authors too), but
    nothing that couldn’t be cleaned up

As for its advantages:

  • it’s already there
  • can sign tags with pgp keys
  • distributed

Adding a RCS system can be too hard to handle, altough a distribuited
revision system can help (and some of them permits to sign each revision).

Not sure why this would be hard. Have current module developers vote on
a preferred RCS and use it. And I agree that a distributed system (such
as git) would be a good choice.

My vote goes to git :slight_smile: OTOH, Keeping trees from two OSS systems in sync
should be doable. I can host mercurial or bazaar or whatever too, if
needed.

BTW, how do you suggest to keep the official repo up to date with
contrib modules? Igor’s releases I can keep track of, but this won’t
scale to contrib modules. I can either give the developers push access
(which I’d rather not do), or set up a mailing list for sending
lkml-style “please pull” requests. Periodic updates maybe?

Sure, but it doesn’t solve the problem of creating a trusted code
repository. Remember, the problem isn’t how people are storing their
code: it’s how others get to that code. If they follow a link from a
wiki, then there’s the potential that they are not going to a legitimate
site.

Does the current nginx wiki offer different access levels, i.e. only
“foo” can edit the “Downloads” page? We could then split the downloads
into official+contrib (editable by the select few) and the unofficial
modules (IMO editable by anyone registered).

  1. Perhaps adding some feature to the Nginx build process that allowed
    3rd party modules to be easily downloaded and installed (from site
    listed in item 1) via configure/make flags (or is this just wishful
    thinking?).

Or perhaps simply distribute the contrib modules together with the main
nginx archive (or as e.g. nginx-contrib-$VERSION.tar.gz). The modules
tend to have negligible download sizes, so why bother writing a script
to save a few KB?

I don’t think this is a priority.

Well, nothing is a priority until it’s a problem =) We’ve gone from
having zero 3rd party modules to nearly a dozen in just a few months.
It would be nice to get something in place while it’s still a manageable
task.

I think that an official repository (with vanilla and all the contrib
modules in one tree), with some nice bug tracker (trac indeed would be
fine) would help. This could also become a place for collecting
development documentation (i.e. internal APIs, ways to do things
nginx-style, common coding pitfalls, FAQs etc.) which is too technical
to put it on the main wiki.

The “Guide to Nginx Module Development” is a great document but I
think that the community could extend it even further, e.g. I’m
currently up to date with nginx’s shared memory support (which needs
getting used to) and load balancer interface, Manlio with his mod_wsgi
could probably write a lot of interesting stuff etc.

Best regards,
Grzegorz N.

Cliff W. ha scritto:

[…]
This would be better, but seems more complicated than need be (at least
initially). All we really need for the moment is a place where not just
any anonymous user who creates a wiki account can change where source
code is available from.

Ah, ok.
What about a Trac instance with a strict access policy?

[…]

This actually brings up another thought: perhaps we should start an RCS
tree for Nginx. We don’t actually need Igor to use it (I’m sure he has
his own tools and methods and I don’t want to interrupt that): diffs can
be generated automatically from the tarball Igor provides.

Latest versions of Subversion include the svnsync tool, so Igor can just
use it to create a public copy of his repository, keeping it in sync.

Then from this public SVN repository, each developer can create a copy
using his favorite RCS.

Later we may discuss about using a shared repository.

[…]

Manlio P.

Grzegorz N. ha scritto:

[…]

BTW, how do you suggest to keep the official repo up to date with
contrib modules? Igor’s releases I can keep track of, but this won’t
scale to contrib modules. I can either give the developers push access
(which I’d rather not do), or set up a mailing list for sending
lkml-style “please pull” requests. Periodic updates maybe?

I think that each external module should be kept separate from nginx
“core”.

An example is my mod_wsgi module, where I provide support for several
nginx version using patches.

[…]

I think that an official repository (with vanilla and all the contrib
modules in one tree), with some nice bug tracker (trac indeed would be
fine) would help. This could also become a place for collecting
development documentation (i.e. internal APIs, ways to do things
nginx-style, common coding pitfalls, FAQs etc.) which is too technical
to put it on the main wiki.

This would be a great thing, but we need the collaboration of Igor,
otherwise I would consider it just a fork of nginx.

The “Guide to Nginx Module Development” is a great document but I
think that the community could extend it even further, e.g. I’m
currently up to date with nginx’s shared memory support (which needs
getting used to) and load balancer interface, Manlio with his mod_wsgi
could probably write a lot of interesting stuff etc.

Yes, with mod_wsgi I have learned a lot of nginx internals ;-).

Unfortunately I’m not very good at writing documentation.
However I can start to write single documentation pages for each of the
nginx “public” headers, and some documentation page for nginx
“astractions” and main concepts (events, process channels, buffer
chains, modules and configuration).

Best regards,
Grzegorz N.

Regards Manlio P.

On Mon, Dec 31, 2007 at 12:55:23PM +0100, Manlio P. wrote:

I think that each external module should be kept separate from nginx “core”.

An example is my mod_wsgi module, where I provide support for several
nginx version using patches.

I think that it might be useful to keep an official contrib module list.
If nothing else, it should increase confidence in these modules.

The directory layout could look e.g. like:
src/ (nginx source)
contrib/
mod_wsgi-0.5/
mod_wsgi-0.6/
upstream_fair/
whatever/

We could/should also keep some metadata about the modules (required
nginx version, required extra libraries, etc.) in an easily parsable
format for some future tool to include them in the build process (maybe
patch-o-matic style but simpler).

I’m not going to fight about putting them in a single tree (though I’d
like it), but I think they should all be accessible from a single place
with a unified interface. IMO this makes them look more polished than a
bunch of links (an svn here, a tarball there etc.). However, this would
require a certain amount of coordination between module authors.

I think that an official repository (with vanilla and all the contrib
modules in one tree), with some nice bug tracker (trac indeed would be
fine) would help. This could also become a place for collecting
development documentation (i.e. internal APIs, ways to do things
nginx-style, common coding pitfalls, FAQs etc.) which is too technical
to put it on the main wiki.

This would be a great thing, but we need the collaboration of Igor,
otherwise I would consider it just a fork of nginx.

Yes, Igor’s blessing would be neccessary for the project to be deemed
official. However, if we aggregated some third-party modules and
provided a common interface to them, I’d be reluctant to call it a fork.
A frontend or “community edition” maybe. Realistically, we cannot expect
Igor to maintain the external modules, so the level of support we’d
need would be “lack of open hostility” :wink: and accepting bug reports
from the tracker. Of course, his input into the documentation would be
invaluable.

The “Guide to Nginx Module Development” is a great document but I
think that the community could extend it even further, e.g. I’m
currently up to date with nginx’s shared memory support (which needs
getting used to) and load balancer interface, Manlio with his mod_wsgi
could probably write a lot of interesting stuff etc.

Yes, with mod_wsgi I have learned a lot of nginx internals ;-).

I bet :slight_smile: upstream_fair is a pretty small module but I had to dig through
almost all the nginx code to get it up to shape.

Unfortunately I’m not very good at writing documentation.

  • Hi, I’m Greg, and I’m a bad writer too.
  • (all together) Hi, Greg!

However I can start to write single documentation pages for each of the
nginx “public” headers, and some documentation page for nginx
“astractions” and main concepts (events, process channels, buffer
chains, modules and configuration).

I think that documenting headers is less useful than the bigger picture.
This is IMHO a major deficiency in many OSS projects, which document
every class/function via doxygen and call it a day. E.g. to use shared
memory in nginx, you need not only to allocate a segment and use it, but
also wire it up in such a way that subsequent reloads don’t trash it
or cause leaks etc. Even the best documentation for a single function
won’t cut it (again, IMHO). I’d like the documentation to be
task-oriented, i.e. “How do I write a handler?”, “When and how should I
use rbtrees?”, etc. It could then serve as a sequel to the Guide.

Best regards,
Grzegorz N.

Grzegorz N. ha scritto:

src/ (nginx source)

Well, metadata is not required IMHO.

Since we have a full copy of the nginx sources, we can just modify the
auto configuration scripts, so that contrib modules became first class
modules (–with-wsgi-module, --with-upstream-fair-module, and so on).

I’m not going to fight about putting them in a single tree (though I’d
like it), but I think they should all be accessible from a single place
with a unified interface. IMO this makes them look more polished than a
bunch of links (an svn here, a tarball there etc.). However, this would
require a certain amount of coordination between module authors.

Keeping all in a single tree can be a good thing, but it will need a
rethink about how each developer handle its module.

[…]

or cause leaks etc. Even the best documentation for a single function
won’t cut it (again, IMHO). I’d like the documentation to be
task-oriented, i.e. “How do I write a handler?”, “When and how should I
use rbtrees?”, etc. It could then serve as a sequel to the Guide.

But I don’t want to just write reference documentation.
I want to use a bottom-up method, starting with documenting each header
files, and then integrating all the documentation.

This is, IMHO, more easy.

Best regards,
Grzegorz N.

Regards and happy new year Manlio P.

Grzegorz N. wrote:

src/ (nginx source)
contrib/
mod_wsgi-0.5/
mod_wsgi-0.6/
upstream_fair/
whatever/

We could/should also keep some metadata about the modules (required
nginx version, required extra libraries, etc.) in an easily parsable
format for some future tool to include them in the build process (maybe
patch-o-matic style but simpler).

I’m not sure we’d need metadata. Modules should be guaranteed to work
with the Nginx version they ship with, and they can do their own library
checking in their config scripts. (E.g. my Circle GIF module checks
for ImageMagick during ./configure.)

I’m not going to fight about putting them in a single tree (though I’d
like it), but I think they should all be accessible from a single place
with a unified interface. IMO this makes them look more polished than a
bunch of links (an svn here, a tarball there etc.). However, this would
require a certain amount of coordination between module authors.

I too would prefer that the modules sit in a single tree. Perhaps a good
model is the Linux kernel: they have hundreds of modules written and
maintained by different people, but each one is approved by a number of
people before shipping. I personally think that most non-Igor modules
should sit in the main tree (src/http/modules), and the --add-module
mechanism should be used only for one-off or proprietary modules that
are privately maintained.

Yes, Igor’s blessing would be neccessary for the project to be deemed
official. However, if we aggregated some third-party modules and
provided a common interface to them, I’d be reluctant to call it a fork.
A frontend or “community edition” maybe. Realistically, we cannot expect
Igor to maintain the external modules, so the level of support we’d
need would be “lack of open hostility” :wink: and accepting bug reports
from the tracker. Of course, his input into the documentation would be
invaluable.

We should talk to Igor about his attitude toward third-party modules. I
imagine he just doesn’t have time to inspect, test, and integrate them.
If it makes things easier, I’d be willing to consolidate our modules
into a “community edition” and work with Igor to get changes accepted
upstream.

almost all the nginx code to get it up to shape.

I think that documenting headers is less useful than the bigger picture.
This is IMHO a major deficiency in many OSS projects, which document
every class/function via doxygen and call it a day. E.g. to use shared
memory in nginx, you need not only to allocate a segment and use it, but
also wire it up in such a way that subsequent reloads don’t trash it
or cause leaks etc. Even the best documentation for a single function
won’t cut it (again, IMHO). I’d like the documentation to be
task-oriented, i.e. “How do I write a handler?”, “When and how should I
use rbtrees?”, etc. It could then serve as a sequel to the Guide.

Yes, it’s time we had something more flexible than the Guide. I would
like both a low-level API reference and a high-level document more like
the Guide. For the low-level stuff, I think something like javadoc
parsed from the code would work best. (Another task for the “community
edition”?) For the high-level stuff, I rather like how the Django book
does it:

http://www.djangobook.com/en/1.0/

Each chapter has an individual author, but then anyone can leave
comments in the margins. I’d prefer this model to a wiki because
high-level guides are difficult to write, and their authors deserve
credit for their work.

Evan

On Mon, Dec 31, 2007 at 03:47:13PM +0100, Manlio P. wrote:

Grzegorz N. ha scritto:

I’m not going to fight about putting them in a single tree (though I’d
like it), but I think they should all be accessible from a single place
with a unified interface. IMO this makes them look more polished than a
bunch of links (an svn here, a tarball there etc.). However, this would
require a certain amount of coordination between module authors.

Keeping all in a single tree can be a good thing, but it will need a
rethink about how each developer handle its module.

Well, the modules might be in separate trees but should present a single
interface for browsing, download, documentation and reporting bugs. This
will be rather simple when all developers agree on a single RCS system,
set up some bridge between them or hell freezes over :wink:

Maybe we should choose an RCS that maybe even nobody uses personally,
but has a decent web interface and can import/export repositories to all
common systems.

What I’m aiming for (I guess) is a central place for nginx third-party
modules, with one-click download of the whole pack, maybe some
coordinated release schedule etc.

But I don’t want to just write reference documentation.
I want to use a bottom-up method, starting with documenting each header
files, and then integrating all the documentation.

This is, IMHO, more easy.

It seems I misunderstood you. I thought you want to just document the
API. IMO a top-down approach is more informative. The low-level details
can be quickly found by reading the source, while the big picture
requires more time to “sink in”.

Of course, as this is all OSS, the worst that happens if we disagree is
that we end up meeting in the middle :wink:

Best regards,
Grzegorz N.

Evan M. ha scritto:

I too would prefer that the modules sit in a single tree. Perhaps a good
model is the Linux kernel: they have hundreds of modules written and
maintained by different people, but each one is approved by a number of
people before shipping.

+1

I personally think that most non-Igor modules
should sit in the main tree (src/http/modules), and the --add-module
mechanism should be used only for one-off or proprietary modules that
are privately maintained.

+1

[…]

We should talk to Igor about his attitude toward third-party modules. I
imagine he just doesn’t have time to inspect, test, and integrate them.
If it makes things easier, I’d be willing to consolidate our modules
into a “community edition” and work with Igor to get changes accepted
upstream.

Ok.

[…]

Yes, it’s time we had something more flexible than the Guide. I would
like both a low-level API reference and a high-level document more like
the Guide. For the low-level stuff, I think something like javadoc
parsed from the code would work best. (Another task for the “community
edition”?)

I think that it is better to keep the documentation outside the
sources.

For the high-level stuff, I rather like how the Django book
does it:

http://www.djangobook.com/en/1.0/

Each chapter has an individual author, but then anyone can leave
comments in the margins. I’d prefer this model to a wiki because
high-level guides are difficult to write, and their authors deserve
credit for their work.

+1.
The documentation should be edited using an RCS (I hate to edit
documents using browsers), but comments should be easy to add.

Evan

Manlio P.

Grzegorz N. ha scritto:

rethink about how each developer handle its module.

Well, the modules might be in separate trees but should present a single
interface for browsing, download, documentation and reporting bugs. This
will be rather simple when all developers agree on a single RCS system,
set up some bridge between them or hell freezes over :wink:

It’s hard :).
I use Mercurial, you use Git, Adrian P. (the author of fancy index)
use Bazaar.

By the way: in the Third-Party HTTP modules list we should add the
developer’s names.

Maybe we should choose an RCS that maybe even nobody uses personally,
but has a decent web interface and can import/export repositories to all
common systems.

What I’m aiming for (I guess) is a central place for nginx third-party
modules, with one-click download of the whole pack, maybe some
coordinated release schedule etc.

For a central place it is not really required to have a shared RCS
repository.

We can also just add a bug traker, restricted wiki pages with download
links and a mailing list.

Maybe it is possible to write a wrapper for the nginx configuration that
will be able to download external required modules (I have written, as
an example, a wrapper for the nginx binary so that one can use the
command line instead of a configuration file), configure them with an
unified interface, and then call the configure script.

External modules that requires user configurable variables can use the
OS environment, and the configuration wrapper will setup the environment
for them.

[…]
It seems I misunderstood you. I thought you want to just document the
API. IMO a top-down approach is more informative. The low-level details
can be quickly found by reading the source, while the big picture
requires more time to “sink in”.

Of course, as this is all OSS, the worst that happens if we disagree is
that we end up meeting in the middle :wink:

Ok :wink:

Best regards,
Grzegorz N.

Regards Manlio P.

On Mon, Dec 31, 2007 at 11:00:33AM -0500, Evan M. wrote:

I’m not sure we’d need metadata. Modules should be guaranteed to work
with the Nginx version they ship with, and they can do their own library
checking in their config scripts. (E.g. my Circle GIF module checks
for ImageMagick during ./configure.)

OK, I’m outnumbered :wink: If the modules are to be kept in a single tree,
it would indeed be pointless. I rather thought of separately distributed
modules, but I like this way more.

I too would prefer that the modules sit in a single tree. Perhaps a good
model is the Linux kernel: they have hundreds of modules written and
maintained by different people, but each one is approved by a number of
people before shipping. I personally think that most non-Igor modules
should sit in the main tree (src/http/modules), and the --add-module
mechanism should be used only for one-off or proprietary modules that
are privately maintained.

I don’t think we (as the community) have enough resources to ensure
quality review of each other’s code, so the modules will probably remain
more or less individual creations. As for putting external modules in
the main tree, this will make them look rather like a part of nginx
itself, which has both advantages and disadvantages, IMO. If we want to
distribute a pack of nginx modules, maybe we should keep them in a
structure parallel to original source, e.g. contrib-src/http/modules. Or
maybe not, just brainstorming :slight_smile:

We should talk to Igor about his attitude toward third-party modules. I
imagine he just doesn’t have time to inspect, test, and integrate them.
If it makes things easier, I’d be willing to consolidate our modules
into a “community edition” and work with Igor to get changes accepted
upstream.

I’m not really sure about it. Nginx is still mostly a one-man project
and by integrating the modules into nginx core Igor would implicitly
gain responsibility for them. Although I’m all for integrating more
modules into core nginx, this should be (IMO) purely Igor’s decision.
OTOH, I see the community edition as a wrapper adding stuff, which Igor
does not neccessarily want to see upstream, but which the community
finds valuable.

Yes, it’s time we had something more flexible than the Guide. I would
like both a low-level API reference and a high-level document more like
the Guide. For the low-level stuff, I think something like javadoc
parsed from the code would work best. (Another task for the “community
edition”?) For the high-level stuff, I rather like how the Django book
does it:

Well, the code is clean and rather self-documenting but comments are
pretty scarce, so an automated tool will probably extract just the
function prototypes. I think that a hand-written document with links to
the source (lxr for example) will do just fine.

IMO, the nginx source is pretty easy to follow at the low level (what
does this function do), but the big picture is way harder (where is X
handled), especially given the event-driven model and abundance of
callbacks. Thus (again, IMO) we should concentrate at the high level
description.

http://www.djangobook.com/en/1.0/

Each chapter has an individual author, but then anyone can leave
comments in the margins. I’d prefer this model to a wiki because
high-level guides are difficult to write, and their authors deserve
credit for their work.

I fully agree.

Best regards,
Grzegorz N.

El Mon, 31 Dec 2007 17:49:42 +0100
Manlio P. [email protected] escribió:

hell freezes over :wink:

It’s hard :).
I use Mercurial, you use Git, Adrian P. (the author of fancy
index) use Bazaar.

Personally it is not a big deal for me having to use any other RCS than
Bazaar – at least while the chosen one is “reasonable” (I do not want
to go back to CVS, gosh!). I use Bazaar because when I started using
distributed SCMs it was less crude than others regarding usability. I
have only used Mercurial and Git occasionally but they feel comfortable
in their latest incarnations.

We can also just add a bug traker, restricted wiki pages with
download links and a mailing list.

I definitely agree that a Trac-like thing would be useful, especially
regarding bug tracking – and you get a source browser and an integrated
wiki for free!

Maybe it is possible to write a wrapper for the nginx configuration
that will be able to download external required modules (I have
written, as an example, a wrapper for the nginx binary so that one
can use the command line instead of a configuration file), configure
them with an unified interface, and then call the configure script.

External modules that requires user configurable variables can use
the OS environment, and the configuration wrapper will setup the
environment for them.

This seems useful, and maybe would encourage packagers to add a set of
third-party modules to prepackaged versions of Nginx. Currently I am
maintaining a custom Gentoo ebuild which adds the fancyindex module as
an option. It would be very nice if people which builds directly from
source could mix-and-match modules.

Even having this tool, I believe the future if Nginx modules is loading
them dynamically via dlopen(), as other webservers do. Maybe Igor has
some roadmap regarding this point…

Best regards, and happy 2008,

Grzegorz N. wrote:

quality review of each other’s code, so the modules will probably remain
more or less individual creations. As for putting external modules in
the main tree, this will make them look rather like a part of nginx
itself, which has both advantages and disadvantages, IMO. If we want to
distribute a pack of nginx modules, maybe we should keep them in a
structure parallel to original source, e.g. contrib-src/http/modules. Or
maybe not, just brainstorming :slight_smile:

I think we’re talking about three separate problems:

  1. Distributing community modules to end-users.

  2. Marking community modules as “stable.”

  3. Distributing community modules/patches to developers.

We can achieve #1 with a community edition of some sort. I hesitate to
do it because modules often require patches to the nginx core (modifying
data structures, etc.), and this may cause some confusion among
end-users especially when bugs arise, then it might need its own mailing
list, and things get hairy. There’s also the question of what goes in
the community edition and who decides (the malicious code problem).

#2, marking modules as “stable,” does require some work; either Igor
needs to review them and accept them into the official nginx download,
or we need some sort of peer-review process so that outsiders can trust
the community edition. I do think it’s essential that modules be able to
graduate to “first-class” status, but we will need Igor’s cooperation. I
am curious to hear about Igor’s willingness to integrate third-party
modules.

For #3, distributing modules and patches among developers, I again like
the Linux kernel model: everyone keeps their own copy of the source, and
posts the changes they make to a mailing list. To get updates, you just
pull from developers you trust.

#1 and #2 are not trivial problems. My inclination is to solve #3 first
and build a proper developer community; then the solutions to #1 and #2
might become more obvious. It might be that one developer’s branch
becomes the de-facto “community edition (testing)”, and another
developer’s branch becomes the de-facto “community edition (stable)”. In
any event, we’ll get a clearer idea of how many people are interested in
testing and integrating.

BTW, I vote for git. :slight_smile:

OTOH, I see the community edition as a wrapper adding stuff, which Igor
does not neccessarily want to see upstream, but which the community
finds valuable.

Well, Igor could delegate responsibility for individual modules to their
original authors. When questions or bugs come up, then that person is
responsible for dealing with them. If a module is causing problems and
that person is delinquent in addressing issues, Igor can just rip the
module out until a new owner is found. But it again depends on whether
he’s willing to move beyond a one-man project.

function prototypes. I think that a hand-written document with links to
the source (lxr for example) will do just fine.

I don’t entirely agree. Some functions do a lot of work and I would have
appreciated a sentence or two in summary. However, in these cases
perhaps refactoring the code would be more useful than commenting it. I
do agree that we should start with an organized list of function
prototypes and constants, and spend most of our energies on high-level
descriptions.

comments in the margins. I’d prefer this model to a wiki because
high-level guides are difficult to write, and their authors deserve
credit for their work.

I fully agree.

(What’s aol?)

Evan

On Mon, Dec 31, 2007 at 03:13:12PM -0500, Evan M. wrote:

I think we’re talking about three separate problems:

  1. Distributing community modules to end-users.

  2. Marking community modules as “stable.”

  3. Distributing community modules/patches to developers.

(snip interesting analysis)

I agree completely, the Linux model works on a huge scale and it should
work on a smaller one, too.

So in the short term, we’d just have to set up Trac somewhere (I can put
one up if needed).

Well, Igor could delegate responsibility for individual modules to their
original authors. When questions or bugs come up, then that person is
responsible for dealing with them. If a module is causing problems and
that person is delinquent in addressing issues, Igor can just rip the
module out until a new owner is found. But it again depends on whether
he’s willing to move beyond a one-man project.

True. IMO Igor should simply state his opinion (when he comes back from
vacation), no point in speculating.

I don’t entirely agree. Some functions do a lot of work and I would have
appreciated a sentence or two in summary. However, in these cases
perhaps refactoring the code would be more useful than commenting it. I
do agree that we should start with an organized list of function
prototypes and constants, and spend most of our energies on high-level
descriptions.

My statement was quite general and there certainly are functions which
might use better comments (ngx_init_cycle() comes to mind), but on the
whole, the low-level code is mostly self explanatory. As for documenting
the API, at this stage a reference documentation could consist of the
headers themselves (Igor mentioned that the API will be stable by 1.0
but some things are bound to change by then).

(What’s aol?)

Me Too - Wikipedia :slight_smile:

Best regards,
Grzegorz N.

Manlio P. wrote:

By the way: in the Third-Party HTTP modules list we should add the
developer’s names.

Done.

Evan

On Mon, 2007-12-31 at 17:33 +0100, Manlio P. wrote:

The documentation should be edited using an RCS (I hate to edit
documents using browsers), but comments should be easy to add.

Not an RCS, but at least gets you out of the browser:

https://addons.mozilla.org/en-US/firefox/addon/4125

Cliff

Cliff W. ha scritto:

On Mon, 2007-12-31 at 17:33 +0100, Manlio P. wrote:

The documentation should be edited using an RCS (I hate to edit
documents using browsers), but comments should be easy to add.

Not an RCS, but at least gets you out of the browser:

https://addons.mozilla.org/en-US/firefox/addon/4125

I have it installed, but it does not solve the problem at the origin
;-).

Cliff

Manlio P.