Forum: NGINX Nginx - Google Summer of Code ideas

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
C. (Guest)
on 2009-03-10 14:04
(Received via mailing list)
Hi everyone,

I'm organizing some google summer of code ideas for our organization and
  nginx could play a very interesting and key role in one of them.  So
two quick questions.

1) Would anyone be willing to mentor a student?
2) What are some other ideas that we could possibly add to the list to
make it exciting?

We have about a day or so to collect ideas and add stuff to the
application if it makes sense. I'm on irc if anyone wants to say hi and
talk about ideas.

Cheers,

./Christopher



codestr0m - irc.freenode.net #osunix / #gsoc / #nginx
mike (Guest)
on 2009-03-10 20:12
(Received via mailing list)
oh, i've got a nice little wishlist going ;)

features:
* mod_svn
* mogilefs integration:
** engine yard would pay a bounty for this (as they've said)
** basically just needs to interface with a tracker to say "where is
this file based on this URI" and then dynamically proxy to the
mogstored server. the dynamic proxy piece is either complete now or
would be a very small tweak I am thinking

using nginx as a layer 7 / gzip / ssl capable proxy:
I've used it this way in the past but when upstreams went down it was
not very graceful. The only idea I had was to trigger a command to
remove the upstream from the pool until it came back again.
* dynamic upstream management, basic healthchecking should fix that, I
think.
* external alert when upstream goes down (or could also trigger a
command) - might not be required with above
* fair scheduler (might not need it) but what about weighted least
connections? (emulate lvs functionality)

authentication:
* SPNEGO/Kerberos/etc. integration - (I'm paying someone $400 w/
RentACoder already)
* someone on IRC mentioned adding in digest support

logging:
* ability to error_log off; or related in any area (server, location,
etc) - i don't think it supports this right now
* allowing for error_log to be overridden anywhere would allow for
debugging based not on debug_connection or a global error_log foo
debug; but would allow for specific debugging based JUST on the host
you want
* perhaps a different debug output, sometimes it's a bit too much.
like a more simplified debug log (look how lighttpd does it - it
allows you to figure out how it's handling the file and such) - right
now ngnix's can be overkill
* have a way to log statistics (simple byte and request counter) per
Host: header, right now flatfile parsing and stuff for that basic info
is not the funnest

management:
* cluster manager (manage multiple nginx instances, like zeus' web
interface)
mike (Guest)
on 2009-03-10 20:26
(Received via mailing list)
should also note

On Tue, Mar 10, 2009 at 11:01 AM, mike <removed_email_address@domain.invalid> 
wrote:
> oh, i've got a nice little wishlist going ;)

>> 1) Would anyone be willing to mentor a student?

igor (of course), maxim, valery all seem to be pretty good hackers.
but probably don't have time.

this would be neat because we'd have more nginx module/code hackers
trained for future bug fixing/features/modules...

>> 2) What are some other ideas that we could possibly add to the list to
>> make it exciting?

well, if we do look into doing adaptive process spawning and need an
nginx module to communicate how many backends are in use (to
fcgiwrap/spawn-fcgi/whatever) there needs to be a module done on the
nginx side to handle that.
C. (Guest)
on 2009-03-10 21:07
(Received via mailing list)
mike wrote:
> this would be neat because we'd have more nginx module/code hackers
> trained for future bug fixing/features/modules...

Hi Mike

yes exactly my point!

I have a rather lengthy write-up on some ideas about dtrace,
communication layer and near real time algorithm adjustments for
balancing a large cluster.

If you or any of the possible mentors are on irc I think this will 1)
make it easier to balance mentoring and 2) speed up communication for
planning.  Also feel free to email me off list so we can speed up
planning and cause less list noise.

./Christopher
Valery K. (Guest)
on 2009-03-10 21:49
(Received via mailing list)
C. wrote:
>
> Hi everyone,
>
> I'm organizing some google summer of code ideas for our organization and
>  nginx could play a very interesting and key role in one of them.  So
> two quick questions.
>
> 1) Would anyone be willing to mentor a student?

Depends on how many hours per week you need for this. I could handle 5
to 10 hours I think.
C. (Guest)
on 2009-03-10 22:16
(Received via mailing list)
Valery K. wrote:
> Depends on how many hours per week you need for this. I could handle 5
> to 10 hours I think.

I think that's more than enough.  The key is a couple things like any
project..

1) Having a very detailed plan to avoid questions in the first place
2) Selecting students who are qualified
3) Being on irc, but letting them know not to expect instant replies..
If more than one mentor is around it can balance out well like any
normal dev channel.

#osunix is what I'm using for all gsoc stuff now and currently very
quiet, but some students may pop in and ask questions.  It's also a good
chance to get to know them and bring them into the community a bit more.
  This way the gsoc not only goes well, but hopefully they stay as
contributors.

./C
Valery K. (Guest)
on 2009-03-10 22:36
(Received via mailing list)
C. wrote:
>>
>> Depends on how many hours per week you need for this. I could handle 5
>> to 10 hours I think.
>
> I think that's more than enough.  The key is a couple things like any
> project..
>
> 1) Having a very detailed plan to avoid questions in the first place
> 2) Selecting students who are qualified

Be more specific: plan for what and qualified for what?

> 3) Being on irc, but letting them know not to expect instant replies..
> If more than one mentor is around it can balance out well like any
> normal dev channel.

Well, I have been on irc for a while, but then I forgot to restore irc
account in my client because of general laziness. After your mail I'm
back there.

But generally I'm available per email. I'm trying to respond to every
question and fortunately there are not much of them. If someone feels I
don't respond -- alert me in maillist or per phone.
Manlio P. (Guest)
on 2009-03-11 20:03
(Received via mailing list)
mike ha scritto:
> oh, i've got a nice little wishlist going ;)
>
> features:
> * mod_svn

The only idea of implementing mod_svn from scratch in Nginx is crazy :).

> * mogilefs integration:
 > [...]
> * dynamic upstream management, basic healthchecking should fix that, I think.
> * external alert when upstream goes down (or could also trigger a
> command) - might not be required with above

This does not need to be done with Nginx.
Just use a pre existing healthchecking software with each of the
upstream servers.

 > [...]
> authentication:
> * SPNEGO/Kerberos/etc. integration - (I'm paying someone $400 w/
> RentACoder already)
> * someone on IRC mentioned adding in digest support
>

If someone is interested in sponsoring digest auth support, I should be
able to implement it.
I have already implemented a rather complete HTTP Digest Authentication
support in my Python WSGI framework.

> [...]
> * have a way to log statistics (simple byte and request counter) per
> Host: header, right now flatfile parsing and stuff for that basic info
> is not the funnest
>

Why not use a separate log parser?

> management:
> * cluster manager (manage multiple nginx instances, like zeus' web interface)
>

Isn't it possible to use some existing tool?

 > [...]


Manlio P.
mike (Guest)
on 2009-03-11 22:26
(Received via mailing list)
Sorry for the messy email I'm in a lab with a crappy keyboard that is
too hard to use properly and the mouse is basically useless :P

On Wed, Mar 11, 2009 at 10:50 AM, Manlio P.
<removed_email_address@domain.invalid> wrote:
> mike ha scritto:
>>
>> oh, i've got a nice little wishlist going ;)
>>
>> features:
>> * mod_svn
>
> The only idea of implementing mod_svn from scratch in Nginx is crazy :).
>
why is that? isn't it just DAV? it just needs to support some more
OPTIONS commands or something? (I could be off my rocker, I thought I
read that somewhere)

> servers.
and what, have an /etc/nginx/upstreams.conf file that i manually
update and kill -HUP nginx or whatever appropriate signal to reload
every time i notice an upstream going up or down?

another issue is nginx is not 'smart' so the healthchecking would need
to be more than just tcp port 80 is open... i guess that's where
external things come in to play. but simplifying the software stack
would be amazing, and it could be an -optional- module in nginx :)


> to implement it.
> I have already implemented a rather complete HTTP Digest Authentication
> support in my Python WSGI framework.

I don't care as much about digest myself but it was something someone
brought up. This is open source software too. Doesn't anyone do
anything anymore just for the ego boost? :) I know I would be if I
knew some C!

>
>> [...]
>> * have a way to log statistics (simple byte and request counter) per
>> Host: header, right now flatfile parsing and stuff for that basic info
>> is not the funnest
>>
>
> Why not use a separate log parser?

i am right now. but for basic statistics... seems like major overkill,
especially when you start looking at gigs of data from multiple
webservers and having to parse/merge the data

>
>> management:
>> * cluster manager (manage multiple nginx instances, like zeus' web
>> interface)
>>
>
> Isn't it possible to use some existing tool?
>
>> [...]

like cpanel? etc? all of those suck. have you ever used zeus's? it has
all the capabilities of the webserver avaialble. it needs to be a tool
designed around nginx's capabilities and such. not just "setup foo.com
under /home/foo/bar/web"
Johan Bergström (Guest)
on 2009-03-11 23:46
(Received via mailing list)
Hi,

On Mar 10, 2009, at 12:50 , C. wrote:
> make it exciting?
Here's two ideas/suggestions from my wish list:

1: Nginx should keepalive connections against proxied backends
This could/should apply to all kinds of backends ranging from http to
FastCGI. While digging into backend unification, why not enable
memcached as a "standard" backend instead of a third party module (of
course compile-time option)? I recall seeing proof of concepts on this
at this mailing list.

2: Nginx proxy store should be purge:able
The most fundamental action for a proper proxy cache. One could dig
deeper into this and enable more advanced rules for invalidation by
looking at ncache or other proxy servers such as varnish. I'm not
suggesting Nginx to act as a "full fledged" proxy - only cover the
basic needs.
qyb (Guest)
on 2009-03-12 02:31
(Received via mailing list)
Here's my wish:

 Implement "init master" hook for module development
Maxim D. (Guest)
on 2009-03-12 03:15
(Received via mailing list)
Hello!

On Wed, Mar 11, 2009 at 10:36:41PM +0100, Johan Bergström wrote:

>>
>> 1) Would anyone be willing to mentor a student?
>> 2) What are some other ideas that we could possibly add to the list to
>> make it exciting?
>
> Here's two ideas/suggestions from my wish list:
>
> 1: Nginx should keepalive connections against proxied backends
> This could/should apply to all kinds of backends ranging from http to
> FastCGI. While digging into backend unification, why not enable

Keepalive to memcached backends works perfectly with
ngx_http_upstream_keepalive module.

Making it work with http & fastcgi backends requires some
non-trivial modifications in proxy/fastcgi/upstream modules and
nginx core.  I have some work-in-progress for this, but it's not
yet complete.

> memcached as a "standard" backend instead of a third party module (of
> course compile-time option)? I recall seeing proof of concepts on this
> at this mailing list.

Memcached *is* standard backend, not a "third party module".  And
it shares most of it's code with http backends (proxy_pass) and
fastcgi backends (fastcgi_pass).

Maxim D.
daniel (Guest)
on 2009-03-12 12:15
(Received via mailing list)
I would like to see Digest Authentication in Nginx.

C. さんは書きました:
张立冰 (Guest)
on 2009-03-12 12:34
(Received via mailing list)
Digest Authentication?

I have Implemented a simple token
module<http://www.libing.name/2009/03/11/nginx-token-modu...,
used for http authentication with backend memcached. Maybe that is
helpful
for you.
Huy Phan (Guest)
on 2009-03-12 14:26
(Received via mailing list)
张立冰 <zhang.libing@...> writes:

>
>
> Digest Authentication?
>  
> I have Implemented a simple token module, used for http authentication with
backend memcached. Maybe that is helpful for you.
>

Hi zhang,
really interesting to see your modules, the idea is exactly what im
trying to do
these 2 weeks.
What I see after checking your code is that :
1. If the token is invalid : return 403, otherwise 404.
2. When the request is valid (the token is exist), it returns 404, and
then
internal redirect to the 'real' place => If someone knows the url of
real place,
he can access without any authentication.

I had develop a module similar like that ( in fact I see that we have
80% same
code ), but the process is a little different. Can you take a look at
these
threads and we can work on it together :)

http://thread.gmane.org/gmane.comp.web.nginx.english/10008
http://thread.gmane.org/gmane.comp.web.nginx.english/10379
张立冰 (Guest)
on 2009-03-12 15:36
(Received via mailing list)
Attachment: 360.gif (0 Bytes)
Hi,Huy Phan.

 Wow, we did the same job. [?]

2. When the request is valid (the token is exist), it returns 404, and
then
> internal redirect to the 'real' place => If someone knows the url of real
> place,
> he can access without any authentication.
>

Yes, I hook the http status 404, and redirect to the 'real' place at the
backend. BUT this place must be placed at intranet or the same server,
just
like 127.0.0.1.And that is the trust environment. ^_^

I have placed a simple conf at this page.(
http://www.libing.name/2009/03/11/nginx-token-module.html).

http://thread.gmane.org/gmane.comp.web.nginx.english/10008
> http://thread.gmane.org/gmane.comp.web.nginx.english/10379
>

I have checked those two posts.It seems you want a module to do the
access
check job for media files.(/v/empty.flv?token=1234).
And I think there is no need memcached to store the check token. Maybe
you
can work with http access key
module<http://www.nginx-community.org/NginxHttpAccessKeyModule>
and mod_parsed_vars <http://hg.mperillo.ath.cx/nginx/mod_parsed_vars>to
generate dynamic token with COOKIE/GET/POST vars. Just like
sessionid.And
that is betteeeer than work with memcached.
张立冰 (Guest)
on 2009-03-12 15:39
(Received via mailing list)
Attachment: 360.gif (0 Bytes)
And here is a post about this topic.
http://www.libing.name/2008/11/23/nginx-applicatio...
I'm sorry, it is in Chinese. But I think the script at this page will
helpful for you.

2009/3/12 张立冰 <removed_email_address@domain.invalid>
Huy Phan (Guest)
on 2009-03-12 16:56
(Received via mailing list)
张立冰 <zhang.libing@...> writes:

> I have checked those two posts.It seems you want a module to do the access
check job for media files.(/v/empty.flv?token=1234).And I think there is
no need
memcached to store the check token. Maybe you can work with http access
key
module and mod_parsed_vars to generate dynamic token with
COOKIE/GET/POST vars.
Just like sessionid.And that is betteeeer than work with memcached.
>

Hi 张立冰,

Actually, this is the second phase of our flow, the 1st phase is that :
i. user request to play a media
ii. we insert a value for a random key ( generated by us ) to memcached.
iii. we give the key to user.

And you've already known the second phase :)
What I mean here is that, memcached is a kind of "must-have-thing".

As you can see in my previous posts, I've almost done the code. It just
needs some small tweaks to be perfect :). So if you have time, I can
share
the code and we can work on it.
mike (Guest)
on 2009-03-12 18:25
(Received via mailing list)
that is not exactly the same, is it? (although it is very useful)
C. (Guest)
on 2009-03-12 21:00
(Received via mailing list)
As mentioned on IRC.

1) sticky sessions/session affinity
2) ajp13 connector for java web servers


./C
mike (Guest)
on 2009-03-12 21:44
(Received via mailing list)
1) i still do not get this need. use central session management. many
options for it...
mike (Guest)
on 2009-03-12 22:29
(Received via mailing list)
oh and if Grzegorz decides on making his fastcgi-to-anything adapter
perhaps it could handle the ajp13 stuff too? and could technically
replace the need for php-fpm?

Grzegorz and Andrei should get together. Maybe the mod_wsgi guy too.
Look at some sort of fastcgi-to-anything adapter so that any webserver
can benefit as long as it speaks fastcgi. I'm going to vote for CGI
first (since php-fpm exists) ;)
Michael Baudino (Guest)
on 2009-03-13 02:07
(Received via mailing list)
C. wrote:
>
> We have about a day or so to collect ideas and add stuff to the
> application if it makes sense. I'm on irc if anyone wants to say hi and
> talk about ideas.
>
> Cheers,
>
> ./Christopher
>
>

Hi C.

Participating in the SoC is a definitively good idea.

What about implementing async disk IO ?
Merlin (Guest)
on 2009-03-13 03:48
(Received via mailing list)
On Thu, Mar 12, 2009 at 4:51 PM, Michael Baudino <
removed_email_address@domain.invalid> wrote:

>
What's wrong with aio_write() and aio_read()?

- Merlin
Maxim D. (Guest)
on 2009-03-13 13:53
(Received via mailing list)
Hello!

On Thu, Mar 12, 2009 at 06:40:13PM -0700, Merlin wrote:

> > Michael Baudino
> >
> >
> >
> What's wrong with aio_write() and aio_read()?

Well, you really want to hear?  There is a couple of issues:

1. There is no good standard method of notification.  Per POSIX
it's uses signals to notify process about completed operations.
Under FreeBSD notifications is possible over kqueue, and probably
other OSes have something too, but it's anyway will require some
non-trivial porting.

2. It's usually implemented as a thread pool within OS kernel (at
least FreeBSD and Linux implementations AFAIK), and usually have
some limits administrator should be aware of (including maximum
number of io requests system may queue).

3. It's not (yet) supported by nginx.

Personally I think that AIO interface is wierd and it would be
much better to support O_NONBLOCK on normal file descriptors.  But
it's not likely to happen in the near future. :)

So currently AIO is the only disk async IO interface we have, and
it would be good thing to implement it's support in nginx.

Maxim D.
Valery K. (Guest)
on 2009-03-13 16:33
(Received via mailing list)
----- "Maxim D." <removed_email_address@domain.invalid> wrote:

> > >
>
> 1. There is no good standard method of notification.  Per POSIX
> it's uses signals to notify process about completed operations.
> Under FreeBSD notifications is possible over kqueue, and probably
> other OSes have something too, but it's anyway will require some
> non-trivial porting.

Here I disagree: linux kernel supports notification via eventfd since
2.6.18. eventfd syscall is available as of glibc 2.8, which is a part of
e.g. ubuntu intrepid distribution.

Personally I've tested this interface and it looks fine to me.

> 2. It's usually implemented as a thread pool within OS kernel (at
> least FreeBSD and Linux implementations AFAIK), and usually have
> some limits administrator should be aware of (including maximum
> number of io requests system may queue).

It is unclear whether kernel thread pool is a disadvantage or not. I
don't think it is reasonable to use disk AIO as a primary mean to serve
files, I think it won't make server faster than while using sendfile,
but AIO could be serious boost for large out-of-cache files, because it
eliminates blocking.

With such considerations kernel thread pool doesn't seem to be a big
hassle.
Marcus C. (Guest)
on 2009-03-13 17:07
(Received via mailing list)
Hi,
> but AIO could be serious boost for large out-of-cache files, because it eliminates 
blocking.
>
As part of a project I'm working on, I'm going to implement a generic,
embeddable, lightweight, disk-and-memory object caching engine, that
would use disk AIO for the disk operations.  It would basically be
similar to the role of memcached, but could be embedded (as well as
accessed via sockets) and would include disk caches too (if you want to
use that feature).

This could be used both for page caching and for storing other data
objects, and could be optimized to store the most frequently used
objects in memory.

For this, I'll be using AIO for the disk accesses.  I don't know when
I'll be doing this particular feature (some time within the next few
months probably), but if no-one else has already done so, I'm happy to
write some AIO code for Nginx in the process.

> Here I disagree: linux kernel supports notification via eventfd since 2.6.18. eventfd 
syscall is available as of glibc 2.8, which is a part of e.g. ubuntu intrepid 
distribution.
>
>
Valery, have you done performance comparisons between eventfd and
aio_read() etc?  Which one fared better in your experience?

Cheers,

Marcus.
Maxim D. (Guest)
on 2009-03-13 17:23
(Received via mailing list)
Hello!

On Fri, Mar 13, 2009 at 02:21:14PM +0000, Valery K. wrote:

> > > > Hi C.
> > > What's wrong with aio_write() and aio_read()?
>
> Personally I've tested this interface and it looks fine to me.

Disagree with what?  I've said there are interfaces, but they
aren't standard.  You've just added one more example of
non-standard interface.

> > 2. It's usually implemented as a thread pool within OS kernel (at
> > least FreeBSD and Linux implementations AFAIK), and usually have
> > some limits administrator should be aware of (including maximum
> > number of io requests system may queue).
>
> It is unclear whether kernel thread pool is a disadvantage or not. I don't think it is 
reasonable to use disk AIO as a primary mean to serve files, I think it won't make server 
faster than while using sendfile, but AIO could be serious boost for large out-of-cache 
files, because it eliminates blocking.
>
> With such considerations kernel thread pool doesn't seem to be a big hassle.

Yes.  I've mentioned it just to show that AIO interface isn't
perfect and should be used with care.

I actually think that AIO support in nginx would be really great.
I've just here to say that it's not trivial task.

Maxim D.

p.s. As discussed some time ago on russian mailing list, probably
the best way will be to combine sendfile() with SF_NODISKIO and
AIO reading of 1 byte to place some file pages to cache (if
sendfile() returns EBUSY).  Just mentioning it here to make sure
this idea (Igor's one AFAIR) won't be lost if someone finally
start digging into AIO support.
Valery K. (Guest)
on 2009-03-14 10:47
(Received via mailing list)
----- "Maxim D." <removed_email_address@domain.invalid> wrote:

> aren't standard.  You've just added one more example of
> non-standard interface.

I disagree that there is no good interface. I think they are all as best
as they could be at the moment. After all kqueue notification of AIO
completion isn't POSIX compliant.

> because it eliminates blocking.
> >
> > With such considerations kernel thread pool doesn't seem to be a big
> hassle.
>
> Yes.  I've mentioned it just to show that AIO interface isn't
> perfect and should be used with care.
>
> I actually think that AIO support in nginx would be really great.
> I've just here to say that it's not trivial task.

I can imagine.

>
> Maxim D.
>
> p.s. As discussed some time ago on russian mailing list, probably
> the best way will be to combine sendfile() with SF_NODISKIO and
> AIO reading of 1 byte to place some file pages to cache (if
> sendfile() returns EBUSY).  Just mentioning it here to make sure
> this idea (Igor's one AFAIR) won't be lost if someone finally
> start digging into AIO support.

I remember it.
Maxim D. (Guest)
on 2009-03-14 11:18
(Received via mailing list)
Hello!

On Fri, Mar 13, 2009 at 03:36:44PM +0000, Valery K. wrote:

> > > Personally I've tested this interface and it looks fine to me.
> >
> > Disagree with what?  I've said there are interfaces, but they
> > aren't standard.  You've just added one more example of
> > non-standard interface.
>
> I disagree that there is no good interface. I think they are all as best as they could 
be at the moment. After all kqueue notification of AIO completion isn't POSIX compliant.

So it looks like you disagree with yourself, since I've never said
that there is no good interface.  :)

I've said quite a different thing: standard interface as defined
by POSIX isn't good.

FreeBSD interface with AIO notifications via kqueue is good,
eventfd under linux probably is good too.  But they are both
non-standard (and non-portable).  So nginx have to support many
notification interfaces for various OSes.

Maxim D.
Valery K. (Guest)
on 2009-03-14 12:03
(Received via mailing list)
----- "Maxim D." <removed_email_address@domain.invalid> wrote:

> > > >
> >
> > I disagree that there is no good interface. I think they are all as
> best as they could be at the moment. After all kqueue notification of
> AIO completion isn't POSIX compliant.
>
> So it looks like you disagree with yourself, since I've never said
> that there is no good interface.  :)
>
> I've said quite a different thing: standard interface as defined
> by POSIX isn't good.

The only drawback I see is that it doesn't include queuing interface,
like epoll. This damages the whole idea.

> FreeBSD interface with AIO notifications via kqueue is good,
> eventfd under linux probably is good too.  But they are both
> non-standard (and non-portable).  So nginx have to support many
> notification interfaces for various OSes.

Well, nginx has to support several asynchronous socket IO interfaces.
And since AIO interfaces are parallel to socket IO, this doesn't make
things significantly more complicated.

Actually, I have a draft of patch of AIO support for nginx with kqueue
notifications, but I don't have FreeBSD box at the moment :(

I may send it to anyone who is interested.
Michael Baudino (Guest)
on 2009-03-14 13:14
(Received via mailing list)
Valery K. wrote:
> Actually, I have a draft of patch of AIO support for nginx with kqueue notifications, 
but I don't have FreeBSD box at the moment :(
>
> I may send it to anyone who is interested.
>

Hello,

Actually, I am interested in such a patch (or "draft of a patch").
I can't promise I'll work on it as my schedule if far too busy already,
but I'd love to see what's going on with it.

Would you post it on this mailing list (as a new thread, maybe, we're
hijacking the Google SoC here), or in the brand new forum ;)
Merlin (Guest)
on 2009-03-14 14:08
(Received via mailing list)
I agree; I am interested and will probably tinker with it, but I can't
promise anything more than that.  A new thread here would be perfect.

On Fri, Mar 13, 2009 at 11:04 AM, Michael Baudino <
Valery K. (Guest)
on 2009-03-14 14:09
(Received via mailing list)
Michael Baudino wrote:
> but I'd love to see what's going on with it.
>
> Would you post it on this mailing list (as a new thread, maybe, we're
> hijacking the Google SoC here), or in the brand new forum ;)
>

Here is what I have.
Manlio P. (Guest)
on 2009-03-14 17:11
(Received via mailing list)
mike ha scritto:
> [...]
>>>
>>> features:
>>> * mod_svn
>> The only idea of implementing mod_svn from scratch in Nginx is crazy :).
>>
> why is that? isn't it just DAV? it just needs to support some more
> OPTIONS commands or something? (I could be off my rocker, I thought I
> read that somewhere)
>

Its because DAV is not simple to implement.

> and what, have an /etc/nginx/upstreams.conf file that i manually
> update and kill -HUP nginx or whatever appropriate signal to reload
> every time i notice an upstream going up or down?
>

Right.

> another issue is nginx is not 'smart' so the healthchecking would need
> to be more than just tcp port 80 is open... i guess that's where
> external things come in to play. but simplifying the software stack
> would be amazing, and it could be an -optional- module in nginx :)
>

If this requires some internal rewrite of Nginx, then I'm not sure it is
a good idea.

 > [...]
>
Just use an efficient log parser; one, as an example, that is able to
parse multiple file at a time.


 > [...]


Regards  Manlio
mike (Guest)
on 2009-03-14 19:12
(Received via mailing list)
I'd like to say first - during brainstorming there are no bad ideas :)

On Sat, Mar 14, 2009 at 8:00 AM, Manlio P.
<removed_email_address@domain.invalid> wrote:

> Its because DAV is not simple to implement.

But nginx already does DAV. I use it for mogilefs without a problem.

What needs to be determined is what is missing to make this happen.
Obviously there are some limitations, but I am sure it can be done.

>>> This does not need to be done with Nginx.
>>> Just use a pre existing healthchecking software with each of the upstream
>>> servers.
>>
>> and what, have an /etc/nginx/upstreams.conf file that i manually
>> update and kill -HUP nginx or whatever appropriate signal to reload
>> every time i notice an upstream going up or down?
>>
>
> Right.

That just seems .... messy. Although if the avahi/zeroconf idea gets
mixed in this might become a moot point as I understand it it would be
discovering and adding/removing automagically.

> If this requires some internal rewrite of Nginx, then I'm not sure it is a
> good idea.

That's why it's thrown out there as an idea.

I'm sure it can be done with a third party module. After all it's just
issuing an HTTP request to an upstream and getting back a response,
and then removing it from the pool.

upstream backend  {
  server backend1.example.com weight=5 check=tcp;
  server backend2.example.com:8080 check=response
url=http://foo.com/health.php expect="hello world";
  server unix:/tmp/backend3;
}

etc?

check=tcp is simple, check=response would expect a plaintext response
back.

Or, those attributes could be cleaned up or put into it's own type of
block:

response foo {
   url http://foo.com/health.php
   expect "hello world";
}

block, and check=response response=@foo;

If would need to be smart enough to pick out the host piece so it can
issue the proper Host: when connecting to the internal IPs of the
servers.
Manlio P. (Guest)
on 2009-03-22 01:48
(Received via mailing list)
mike ha scritto:

Sorry for the late response.

> I'd like to say first - during brainstorming there are no bad ideas :)
>
> On Sat, Mar 14, 2009 at 8:00 AM, Manlio P.
> <removed_email_address@domain.invalid> wrote:
>
>> Its because DAV is not simple to implement.
>
> But nginx already does DAV. I use it for mogilefs without a problem.
>

Nginx implements the "simple" part of DAV.


 > [...]


Regards  Manlio
mike (Guest)
on 2009-03-22 02:23
(Received via mailing list)
On Sat, Mar 21, 2009 at 4:43 PM, Manlio P.
<removed_email_address@domain.invalid> wrote:

> Nginx implements the "simple" part of DAV.

If it implemented full DAV there might not be a request for mod_svn
then right? :)

So... that's the request. Make it support more of DAV. It can be an
optional extension to enable (I think dav already is anyway)

:)
This topic is locked and can not be replied to.