Forum: NGINX history of Nginx

Hi.

The next week I will present a talk about my WSGI module for Nginx at
Pycon Due - http://www.pycon.it/ (an Italian conference dedicated to
Python), and I would like to talk about the history of Nginx.

Igor, can you please provide some informations like:
- when did you started Nginx
- why did you started Nginx
- a rationale about the choice of the architecture
- a quick review of the various features timeline
   (as an example, did nginx was multiprocess from the start?)
- know limitations with the current architecture
- when did you understood that nginx was going to be a success?


For the maintainers of the english wiki:
- when did you started the project?



Thanks  Manlio Perillo

Hi Manlio,

On Son 04.05.2008 12:21, Manlio Perillo wrote:
> Hi.

[snipp]

> For the maintainers of the english wiki:
> - when did you started the project?

At the end of April 2006 I have started to translate the
src/core/nginx.c from the source due the fact that there wasn't any doc
for this even in Russia.

In the first May weeks, exactly on 06.05.2006 on the nginx.net was the
first time that the html pages has seen the world ;-)

At this time the netcraft place was 23/22 with 52,092/56,828 counted
hosts, of course some of them are virtualhosts.

In September 2006 the wiki was setuped by Cliff Wells.

At this time netcraft shows it on place 18 with 87,172 counted hosts.

Now nginx is on place 8 with 1,018,503 counted hosts ;-)

This is the history from my point please correct me if I'am wrong.

Cheers

Aleks

On Sun, May 04, 2008 at 12:21:13PM +0200, Manlio Perillo wrote:

> The next week I will present a talk about my WSGI module for Nginx at 
> Pycon Due - http://www.pycon.it/ (an Italian conference dedicated to 
> Python), and I would like to talk about the history of Nginx.
> 
> Igor, can you please provide some informations like:
> - when did you started Nginx
> - why did you started Nginx
> - a rationale about the choice of the architecture

In spring 2001 I had written Apache mod_accel that is enhanced 
replacement
of mod_proxy. But it was clear that Apache has low scalability.
I had read http://kegel.com/c10k.html, had investigated existent servers
httpd, boa, etc. and had decided that I need something like these 
servers,
but with SSI, proxy, and cache support. Also it should has flexible
configuration like Apache and supports online upgrade and quick log 
rotation.
I planed it as portable server including native Win32 support (not
Cygwin). Initially I plan to use master process and single worker with
several threads.

> - a quick review of the various features timeline
>   (as an example, did nginx was multiprocess from the start?)

Spring 2002 - first drafts

  Fall 2003 - working code, kqueue, select, poll, /dev/poll
              master process and single worker
              partially working threads

Spring 2004 - epoll, rtsig
              gzip
              master process and several workers
              using in production on several sites

Summer 2004 - SSL, POP3/IMAP proxy
              nginx runs on www.rambler.ru

 04.10.2004 - 0.0.1 public release (BTW, it was 47 Sputnik anniversary)

    01.2005 - FastCGI

    05.2005 - SSI; proxy and FastCGI use common upstream module

    11.2005 - memcached, map modules

    01.2006 - embedded perl

    04.2006 - PUT/DELETE

    05.2006 - upstream blocks

    09.2006 - event ports

    07.2007 - proxy/fastcgi_store

    09.2007 - open file cache

    11.2007 - proxy_pass variable support, DNS resolver

> - know limitations with the current architecture

Embed interpreters are limited to non-blocking operations.
Blocking on disk operations.
It's difficult to write complex modules those interact with many
sources and destinations simultaneously.

> - when did you understood that nginx was going to be a success?

When I've got working code, fall 2003, year before public release.

Igor Sysoev ha scritto:
> 
> In spring 2001 I had written Apache mod_accel that is enhanced replacement
> of mod_proxy. But it was clear that Apache has low scalability.
> I had read http://kegel.com/c10k.html, had investigated existent servers
> httpd, boa, etc. and had decided that I need something like these servers,
> but with SSI, proxy, and cache support. Also it should has flexible
> configuration like Apache and supports online upgrade and quick log rotation. 
> I planed it as portable server including native Win32 support (not
> Cygwin). Initially I plan to use master process and single worker with
> several threads.
> 

What are the difference between the Apache and Nginx multiprocess model?

Nginx used a master process and all the worker call accept on the same
sockets.

But what about Apache?

> [...]
>> - know limitations with the current architecture
> 
> Embed interpreters are limited to non-blocking operations.
> [...]
> It's difficult to write complex modules those interact with many
> sources and destinations simultaneously.
> 


Do you have considered corutines?
There are some libraries that implement coroutines in C, using POSIX
setcontext, longjmp or custom ASM code.

As an example one librery with a nice API:
http://xmailserver.org/pcl.html

>> - when did you understood that nginx was going to be a success?
> 
> When I've got working code, fall 2003, year before public release.
> 


Thanks   Manlio Perillo

On Wed, May 07, 2008 at 12:32:38PM +0200, Manlio Perillo wrote:
> What are the difference between the Apache and Nginx multiprocess model?
> 
> Nginx used a master process and all the worker call accept on the same 
> sockets.
> 
> But what about Apache?

I feel sort of qualified to answer :)

Apache uses a 1:1 relation between processed requests and execution
threads (where "threads" may well be standard Unix processes, and that
was actually the only way before Apache 2.0; read "threads or processes"
when you see "threads" further down).

OK, not exactly 1:1, as Apache keeps a pool of spare threads and the
threads themselves live much longer than connections.

Since Apache 2.0 the details of process model are delegated to a module
called "MPM" (multiprocessing model, IIRC). The most popular are
mpm_prefork (simple fork()) and mpm_worker (two-tiered architecture with
a bunch of processes, each having a number of pthreads). More exotic 
MPMs
include e.g. mpm_perchild (workers running under different uids, passing
FDs between each other) or mpm_event.

The Apache model has an advantage of simplicity as e.g. nothing prevents
you from blocking or sleeping in handlers but in incurs a much bigger
overhead, especially regarding memory usage.

As for accepting connections, it works more or less the same in Apache
(threads sleeping on a single accept mutex).

> Do you have considered corutines?
> There are some libraries that implement coroutines in C, using POSIX 
> setcontext, longjmp or custom ASM code.
> 
> As an example one librery with a nice API:
> http://xmailserver.org/pcl.html

Me personally, I find coroutines hard to wrap my mind around and I
actually prefer the nginx "horde of callbacks" model.

Best regards,
 Grzegorz Nosek

Grzegorz Nosek ha scritto:
> On Wed, May 07, 2008 at 12:32:38PM +0200, Manlio Perillo wrote:
>> What are the difference between the Apache and Nginx multiprocess model?
>>
>> Nginx used a master process and all the worker call accept on the same 
>> sockets.
>>
>> But what about Apache?
> 
> I feel sort of qualified to answer :)
> 

 > [...]

Ok, thanks

 > [...]
 >
>> Do you have considered corutines?
>> There are some libraries that implement coroutines in C, using POSIX 
>> setcontext, longjmp or custom ASM code.
>>
>> As an example one librery with a nice API:
>> http://xmailserver.org/pcl.html
> 
> Me personally, I find coroutines hard to wrap my mind around and I
> actually prefer the nginx "horde of callbacks" model.
> 

The problem is that you need to write a program with this model in mind.
With coroutines you can support an application almost unchanged.


I'm planning to ad support to greenlets (a coroutine implementation for
Python) in the WSGI module for Nginx.

> Best regards,
>  Grzegorz Nosek
> 


Regards  Manlio Perillo

Igor Sysoev wrote:

> Blocking on disk operations.

Did you perhaps consider a solution with AIO and userland caching?

On Wed, May 07, 2008 at 04:28:57PM +0200, Valery Kholodkov wrote:

> Igor Sysoev wrote:
> 
> > Blocking on disk operations.
> 
> Did you perhaps consider a solution with AIO and userland caching?

I'm going to add disk AIO, but I do not see much value of userland data
caching: it duplicates memory usage (kernel+user) and add copy operation
(writev) as compared to a sendfile.

As an aside, I've always been a bit curious about the name.

Cliff

On Wed, May 07, 2008 at 12:32:38PM +0200, Manlio Perillo wrote:

> >>- a rationale about the choice of the architecture
> >several threads.
> >
> 
> What are the difference between the Apache and Nginx multiprocess model?
> 
> Nginx used a master process and all the worker call accept on the same 
> sockets.
> 
> But what about Apache?

Grzegorz's answer is right.

> There are some libraries that implement coroutines in C, using POSIX 
> setcontext, longjmp or custom ASM code.

The corutines is almost the same as user-land threads, so every corutine
requires its own stack. It's not easy to say how much stack space should 
be
reserved. Now it's not a problem in 64-bit world, but in 32-bit world it
was simply impossible to reserve, for example, 40000 128K stacks -
it takes 5G address space.

And also it much easy for me to write and to debug FSM.

On Wed, May 07, 2008 at 12:32:59PM -0700, Cliff Wells wrote:

> As an aside, I've always been a bit curious about the name.  

The base for name was NG letters those sounds like en-gee in English
(I at least I think so :). X is simply fine letter. But ngx was already
used many times. There were some variants - ngnx, nginx and nginex.
nginx seemed the better for me and I had look it in Google and
found the only quote symbol.