Per-Request Unique Identifier (Log Correlation)

There is information available to log by the fastcgi backend which is
not available to nginx, and vice versa (e.g., $body_bytes_sent). I need
some way to match these two logs together after the fact. This
necessarily implies a quasi-GUID guaranteed to be unique to each
request. Preferably it should be something generated by nginx and fed
to backend through a fastcgi_param (or for those using proxy, through a
request header).

One idea is to combine these:

(a) nginx worker $pid
(b) a timestamp (<=1s res.)
© a request counter internal to each nginx worker process.

Together, these should together fulfill all the foregoing requirements
in real-world practice. (Information leakage is not a concern, since
this will only be used internally. Efficiency is obviously paramount.)
However as far as I can tell, there is no such thing as ©, nor
anything else which can be used to uniquely identify each request.

(Yes there is other stuff which should be added to theoretically
guarantee uniqueness. I am aware of the difficulty of finding a
portable high-performance solution for that, e.g. from maildir
discussions.)

Any chance to get a monotonically increasing counter of requests since
worker process start? I am guessing this would take about 6 lines of
code; but like the old legend about Tesla, the trick is knowing where to
put those 6 lines! I expect many people could use this to sync logs
across nginx frontend and fastcgi or proxy backend.

Performance impact of initializing unsigned long long (>= 64-bit)
counter on worker start, ++ on each request, and keeping 8 more bytes in
memory per process is negligible to both cycles and processor cache
usage (unless there are any internal thread issues I don’t know about).
I don’t know the impact of exposing that as a $worker_request_counter
variable; but I assume nginx handles config variables with very high
efficiency. For a worker serving 1,000 req./s, uint64_t or equivalent
would overflow after about 584 million years; I think most people reload
their nginx workers more often than that.

Obviously all of the above is kind of a kludge, and any theoretically
elegant solution would be best.

Thanks,

helen

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,147866,147866#msg-147866

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs