Pallissard, Matthew wrote:
On 2020-03-06T17:02:14, Howard Chu wrote:
> Howard Chu wrote:
>> Just some initial thoughts on what a new logging daemon should do for us:
>> The primary goal - we want to use a binary message format with as few format
conversions as possible between log
>> sender and log processor.
> One other concern - what do we do about debug output to stderr? If we keep that,
> then we still have message formatting overhead inside slapd itself, and we're
> potentially managing two redundant sets of message formats.
I'm not sure if you're toying with the idea of removing output to stdout/stderr
but I figure I'll share my 2cents anyway.
We make *heavy* use of openldap in containers and having the output go to stdout and
stderr is *super* helpful when debugging issues in vanilla docker or kubernetes.
Not only does stdout allow you to use native tools such as `journalctl` or `kubectl` out
of the box, log aggregation is a completely solved problem in this workflow and is trivial
That being said, persisting logs to disk in a binary format has its merits. Personally,
I don't think plain text logs scale all that well.
Plaintext logs scale horribly. When you have servers processing hundreds of thousands to
millions of queries/sec,
you have to rotate log files pretty frequently otherwise your disks get filled. The
majority of the text associated
with each message is constant and redundant as well. (Which also means the logfiles are
highly compressible, but
that still requires CPU to accomplish.)
We may be able to tolerate keeping the stderr messaging intact. When it's turned on it
probably won't be worse than
a 10% performance hit. https://www.openldap.org/lists/openldap-devel/201511/msg00000.html
And writing to stderr is still several times faster than syslog() at the moment.
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/