So, thinking along the lines of a binary log format for StatsLog, to avoid the syslog overhead, the first obvious idea was to just dump timestamped BER messages into a logfile, and have a separate postprocessor command produce human readable text.
Thinking further about this, we could just dump PCAP format - and use tcpdump, WireShark, or whatever favorite tool to render the log. If all we want is a timestamped log of messages that slapd has processed, we can do that right now and won't have to invest any effort at all into postprocessors.
That still leaves a question of what to do with Debug messages that also go to syslog - it's easier to identify problems if the error message appears somewhere close to the log of the original request. So we'd need a tool to interleave these in order, if we had to pull messages both from the binary log and from syslog. Or, we could define a new custom packet type for these Debug/diagnostic messages, and just spit them out into the PCAP file too. This might require us to write a custom parser plugin for WireShark or whatever, to render these messages. That's still not a big deal, compared to inventing our own entire log postprocessing framework.
Thoughts, suggestions?
--On Wednesday, April 25, 2018 2:02 AM +0100 Howard Chu hyc@symas.com wrote:
That still leaves a question of what to do with Debug messages that also go to syslog - it's easier to identify problems if the error message appears somewhere close to the log of the original request. So we'd need a tool to interleave these in order, if we had to pull messages both from the binary log and from syslog. Or, we could define a new custom packet type for these Debug/diagnostic messages, and just spit them out into the PCAP file too. This might require us to write a custom parser plugin for WireShark or whatever, to render these messages. That's still not a big deal, compared to inventing our own entire log postprocessing framework.
I like the idea of adding a custom packet type for the debug/diagnostic messages and then developing the custom parser plugin for Wireshark etc. Then everything could be handled within a single system.
--Quanah
--
Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
I share the opinion of Quanah. I feel that the adding of a custom packet type is a sensible idea. Is there any visualization of the debug/diagnostic messages currently? Would Wireshark be capable of providing such a visualization? It is not clear to me how these debug messages are normally used. Would a developer typically perform a manual inspection of the syslog to diagnose problems?
Cheers, Andy
-----Original Message----- From: openldap-devel [mailto:openldap-devel-bounces@openldap.org] On Behalf Of Quanah Gibson-Mount Sent: 25 April 2018 15:59 To: Howard Chu; OpenLDAP-devel@openldap.org Subject: Re: Binary log format
--On Wednesday, April 25, 2018 2:02 AM +0100 Howard Chu hyc@symas.com wrote:
That still leaves a question of what to do with Debug messages that also go to syslog - it's easier to identify problems if the error message appears somewhere close to the log of the original request. So we'd need a tool to interleave these in order, if we had to pull messages both from the binary log and from syslog. Or, we could define a new custom packet type for these Debug/diagnostic messages, and just spit them out into the PCAP file too. This might require us to write a custom parser plugin for WireShark or whatever, to render these messages. That's still not a big deal, compared to inventing our own entire log postprocessing framework.
I like the idea of adding a custom packet type for the debug/diagnostic messages and then developing the custom parser plugin for Wireshark etc. Then everything could be handled within a single system.
--Quanah
--
Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
-
Howard Chu -
Lawrence, Andy -
Quanah Gibson-Mount