Re: Is it possible to monitor the fair proxy balancer?

Server NGINX
1

Hi,

On Sun, June 29, 2008 19:57, Grzegorz N. wrote:

On sob, cze 28, 2008 at 11:36:51 +0200, Brice F. wrote:

I think we could all (I mean mainly module authors) benefits from an
nginx
API (like the configuration system) to declare centrally
counters/variables of each module that in turn could be displayed by the
status module. If I have some spare time soon, I’ll try to produce an
nginx patch to implement this.

I have attached an experimental patchset for pluggable status reports
for nginx HTTP modules plus initial upstream_fair support.

Many thanks for doing the hard work. It looks like good from my quick
glance. I’ll add this to the upload progress module (if there are some
monitoring needs there, but at least showing debug information could be
great).

The patchset is rather ugly because it requires adding a field to every
HTTP
module definition, thus breaking 3rd party modules. However, the
required changes are trivial (as you can see in the patch).

Yes, I’m fine with this, although I envisioned something reverse ie
modules declaring fields/counters (kind of a mib if you see what I mean)
to the “monitoring engine”, and the status module displaying this
information. But your scheme has the big advantage to be simple, and the
displaying of the information is up to the module.

I considered adding support for all modules (not only HTTP ones) and
changing the ngx_module_t structure instead (which has some room to
spare and would only require a few macro changes) but I think it’s up to
Igor to decide.

Please have a look and let me know what you think.

Yes, that’s a good work. I hope Igor could accept such kind of patch to
be
part of nginx.

2

Hi,

On pon, cze 30, 2008 at 09:23:30 +0200, Brice F. wrote:

Many thanks for doing the hard work. It looks like good from my quick
glance. I’ll add this to the upload progress module (if there are some
monitoring needs there, but at least showing debug information could be
great).

Thanks :slight_smile: Not really hard, just tedious copy&paste. The real meat of the
patch is only a few lines.

As for monitoring the upload progress module, off the top of my head:

  • number of concurrent uploads
  • speed + total size of every upload

:slight_smile:

The patchset is rather ugly because it requires adding a field to every
HTTP
module definition, thus breaking 3rd party modules. However, the
required changes are trivial (as you can see in the patch).

Yes, I’m fine with this, although I envisioned something reverse ie
modules declaring fields/counters (kind of a mib if you see what I mean)
to the “monitoring engine”, and the status module displaying this

I know what you wanted to do. Maybe it is a good idea but it would have
to allow registering/unregistering variables on the fly. E.g.
upstream_fair keeps a tree of upstreams, where every element lives
almost but not exactly as long as an nginx cycle (they get created and
destroyed with the first request of the new cycle).

information. But your scheme has the big advantage to be simple, and the
displaying of the information is up to the module.

Your approach would lead to more rigid status reports, which would be
easier to parse (and could eventually enable selective status reports
for ultra low-overhead monitoring, e.g.
/status/upstream_fair/mongrel_cluster1/10.0.0.1/nreq). If the free-form
approach gets into nginx, I think it would be nice to create a standard
format of the reports, and maybe even some Perl/Ruby/whatever glue to
parse it in a uniform way.

I considered adding support for all modules (not only HTTP ones) and
changing the ngx_module_t structure instead (which has some room to
spare and would only require a few macro changes) but I think it’s up to
Igor to decide.

Please have a look and let me know what you think.

Yes, that’s a good work. I hope Igor could accept such kind of patch to be
part of nginx.

So, Igor, what’s your opinion? :slight_smile: The patch can be made less intrusive
by hooking into ngx_module_t and using e.g. spare_hook0 plus new macro
(e.g. NGX_MODULE_V2_PADDING with one fewer zero). Then, only modules
which do implement the ->report_status hook would have to be changed.

However, then I’d probably also change the signature not to take an
ngx_http_request_t*, but only an ngx_pool_t* for allocation (to get rid
of HTTP-specific stuff).

Best regards,
Grzegorz N.