Dear Team,
I'm using CentOS 5, some machines with OpenLDAP 2.3.43-12, some with 2.4.23-15, on both 32 and 64-bit architectures.
I am puzzled by the lack of logging I see.
In all cases, I am using slapd.conf OpenLDAP configuration.
I am running slapd with the option -l local0: # ps auxwww | grep slap[d] ldap 17866 0.0 1.7 242104 9000 ? Ssl 15:54 0:00 /usr/sbin/slapd -h ldap:/// -u ldap -l local0
With this in /etc/syslog.conf: local0.info -/var/log/ldap
and this: # grep loglevel /etc/openldap/slapd.conf loglevel -1
I get nothing logged when I restart OpenLDAP, nor when I search (and get output).
However, if I change the /etc/syslog.conf entry to: local0.* -/var/log/ldap
I get plenty of output into /var/log/ldap.
QUESTIONS:
1. What should I expect to be logged at info priority with loglevel -1? 2. Does anyone have any suggestions of what I may be missing here?
Am 19.09.2011 08:47, schrieb Nick Urbanik:
Dear Team,
I'm using CentOS 5, some machines with OpenLDAP 2.3.43-12, some with 2.4.23-15, on both 32 and 64-bit architectures.
I am puzzled by the lack of logging I see.
In all cases, I am using slapd.conf OpenLDAP configuration.
I am running slapd with the option -l local0: # ps auxwww | grep slap[d] ldap 17866 0.0 1.7 242104 9000 ? Ssl 15:54 0:00 /usr/sbin/slapd -h ldap:/// -u ldap -l local0
With this in /etc/syslog.conf: local0.info -/var/log/ldap
and this: # grep loglevel /etc/openldap/slapd.conf loglevel -1
I get nothing logged when I restart OpenLDAP, nor when I search (and get output).
However, if I change the /etc/syslog.conf entry to: local0.* -/var/log/ldap
I get plenty of output into /var/log/ldap.
QUESTIONS:
- What should I expect to be logged at info priority with loglevel -1?
- Does anyone have any suggestions of what I may be missing here?
Hi,
as far as I know, OpenLDAP doesn't really use syslog priorities. You have to set at least 'debug' and then control what you want to log with the loglevel setting in slapd.conf.
Regards, Christian Manal
Dear Christian,
On 19/09/11 09:52 +0200, Christian Manal wrote:
QUESTIONS:
- What should I expect to be logged at info priority with loglevel -1?
- Does anyone have any suggestions of what I may be missing here?
Hi,
as far as I know, OpenLDAP doesn't really use syslog priorities. You have to set at least 'debug' and then control what you want to log with the loglevel setting in slapd.conf.
With production machines, setting syslog to log debug priority for loglevel stats caused a massive write load to the disk, with excessive detail written at each client connection, making logging impractical to use at that level.
Are you sure this is correct? If so, then no logging is practical with OpenLDAP, which would be rather sad.
Am 19.09.2011 10:57, schrieb Nick Urbanik:
Dear Christian,
On 19/09/11 09:52 +0200, Christian Manal wrote:
QUESTIONS:
- What should I expect to be logged at info priority with loglevel -1?
- Does anyone have any suggestions of what I may be missing here?
Hi,
as far as I know, OpenLDAP doesn't really use syslog priorities. You have to set at least 'debug' and then control what you want to log with the loglevel setting in slapd.conf.
With production machines, setting syslog to log debug priority for loglevel stats caused a massive write load to the disk, with excessive detail written at each client connection, making logging impractical to use at that level.
Are you sure this is correct? If so, then no logging is practical with OpenLDAP, which would be rather sad.
Not 100%. Since OpenLDAP has it's own facility on my boxes (local4), this hasn't been an issue for me so far.
Regards, Christian Manal
Nick Urbanik writes:
On 19/09/11 09:52 +0200, Christian Manal wrote:
as far as I know, OpenLDAP doesn't really use syslog priorities. You have to set at least 'debug' (...)
Right. man slapd, options -l and -s, says slapd by default uses syslog local4.debug.
and then control what you want to log with the loglevel setting in slapd.conf.
Actually, normally control it by doing nothing. Stay with the default, 'stats' (256). Nick's 'loglevel -1' does indeed request a massive amount of logging. Don't do that unless you need to debug slapd.
Dear Hallvard,
On 19/09/11 11:32 +0200, Hallvard B Furuseth wrote:
Nick Urbanik writes:
On 19/09/11 09:52 +0200, Christian Manal wrote:
as far as I know, OpenLDAP doesn't really use syslog priorities. You have to set at least 'debug' (...)
Right. man slapd, options -l and -s, says slapd by default uses syslog local4.debug.
Yes, I used -l local0 as we're using local4 for another purpose.
and then control what you want to log with the loglevel setting in slapd.conf.
That is what we do.
Actually, normally control it by doing nothing. Stay with the default, 'stats' (256).
What I have said is that with *only* loglevel stats, in our production servers, the write load is excessive, sometimes filling up the hard disk in four days. The disk I/O from the logging of stats at debug priority is excessive. Of course, only OpenLDAP was logging at debug level.
Perhaps our production OpenLDAP servers work harder than the servers everyone else here on the list manages? It's possible: we are a large ISP, and we provision all our services with OpenLDAP.
If what you say is correct, then OpenLDAP has a bug, which I will file.
Nick's 'loglevel -1' does indeed request a massive amount of logging. Don't do that unless you need to debug slapd.
I temporarily set it to -1 in our non-production, .dev environment to examine this problem of no output at info syslog priority.
On Tue, 20 Sep 2011, Nick Urbanik wrote:
What I have said is that with *only* loglevel stats, in our production servers, the write load is excessive, sometimes filling up the hard
They can be large. We rotate (including compression) nightly. Fortunately they do compress very well...
disk in four days. The disk I/O from the logging of stats at debug priority is excessive. Of course, only OpenLDAP was logging at debug level.
At a minimum, make sure that your syslog daemon isn't flushing on each write. (With some daemons, -/file/name disables this.) Actually, we do remote syslog (@loghost) so the UDP is send-and-forget from our live servers, so there's no disk I/O at all.
(Note that none of the above is particular to OpenLDAP; we do this with most of our services.)
Perhaps our production OpenLDAP servers work harder than the servers everyone else here on the list manages? It's possible: we are a large ISP, and we provision all our services with OpenLDAP.
There are some pretty large installations watching the list...
Dear Aaron,
On 19/09/11 19:40 -0400, Aaron Richton wrote:
On Tue, 20 Sep 2011, Nick Urbanik wrote:
What I have said is that with *only* loglevel stats, in our production servers, the write load is excessive, sometimes filling up the hard
They can be large. We rotate (including compression) nightly. Fortunately they do compress very well...
...but compressing 25 GB files in itself takes away resources from answering the large number of LDAP queries.
disk in four days. The disk I/O from the logging of stats at debug priority is excessive. Of course, only OpenLDAP was logging at debug level.
At a minimum, make sure that your syslog daemon isn't flushing on each write. (With some daemons, -/file/name disables this.)
Naturally, we do that.
Actually, we do remote syslog (@loghost) so the UDP is send-and-forget from our live servers, so there's no disk I/O at all.
Each member of our cluster of four LDAP slaves in Sydney generates about 25 GB per day with loglevel set to 'stats'. The cluster of three in Melbourne are less busy, but still fairly prolific. Imagine sending 1 additional terabyte of data to our syslog servers every ten days, and you need money to pay for the additional infrastructure.
(Note that none of the above is particular to OpenLDAP; we do this with most of our services.)
Agreed; basic system administration.
Perhaps our production OpenLDAP servers work harder than the servers everyone else here on the list manages? It's possible: we are a large ISP, and we provision all our services with OpenLDAP.
There are some pretty large installations watching the list...
But clearly most are smaller than our installation.
I've raised bug http://www.openldap.org/its/index.cgi/Incoming?id=7046
Dear Christian,
On 19/09/11 11:13 +0200, Christian Manal wrote:
Am 19.09.2011 10:57, schrieb Nick Urbanik:
Dear Christian,
On 19/09/11 09:52 +0200, Christian Manal wrote:
QUESTIONS:
- What should I expect to be logged at info priority with loglevel -1?
- Does anyone have any suggestions of what I may be missing here?
Hi,
as far as I know, OpenLDAP doesn't really use syslog priorities. You have to set at least 'debug' and then control what you want to log with the loglevel setting in slapd.conf.
With production machines, setting syslog to log debug priority for loglevel stats caused a massive write load to the disk, with excessive detail written at each client connection, making logging impractical to use at that level.
Are you sure this is correct? If so, then no logging is practical with OpenLDAP, which would be rather sad.
Not 100%. Since OpenLDAP has it's own facility on my boxes (local4), this hasn't been an issue for me so far.
We have OpenLDAP logging to its own facility, with nothing but loglevel 'stats', we get in the order of 25 GB of log file from OpenLDAP alone.
I guess it's not an issue for you because you have fewer clients than we do (large and very busy clusters of mail servers, RADIUS servers, DHCP servers). For us it's a bug: 7046.
Nick Urbanik wrote:
Dear Aaron,
On 19/09/11 19:40 -0400, Aaron Richton wrote:
On Tue, 20 Sep 2011, Nick Urbanik wrote:
What I have said is that with *only* loglevel stats, in our production servers, the write load is excessive, sometimes filling up the hard
They can be large. We rotate (including compression) nightly. Fortunately they do compress very well...
...but compressing 25 GB files in itself takes away resources from answering the large number of LDAP queries.
disk in four days. The disk I/O from the logging of stats at debug priority is excessive. Of course, only OpenLDAP was logging at debug level.
At a minimum, make sure that your syslog daemon isn't flushing on each write. (With some daemons, -/file/name disables this.)
Naturally, we do that.
Actually, we do remote syslog (@loghost) so the UDP is send-and-forget from our live servers, so there's no disk I/O at all.
Each member of our cluster of four LDAP slaves in Sydney generates about 25 GB per day with loglevel set to 'stats'. The cluster of three in Melbourne are less busy, but still fairly prolific. Imagine sending 1 additional terabyte of data to our syslog servers every ten days, and you need money to pay for the additional infrastructure.
(Note that none of the above is particular to OpenLDAP; we do this with most of our services.)
Agreed; basic system administration.
Perhaps our production OpenLDAP servers work harder than the servers everyone else here on the list manages? It's possible: we are a large ISP, and we provision all our services with OpenLDAP.
There are some pretty large installations watching the list...
But clearly most are smaller than our installation.
I've raised bug http://www.openldap.org/its/index.cgi/Incoming?id=7046
Obviously since OpenLDAP's behavior here is "by design" it is not a bug. Just a note - there have been at least two attempts to overhaul the Debug/Syslog support in recent years. Neither made visible progress. At this point the infrastructure exists to log individual messages at distinct priority levels, but nobody has been interested in taking the time to sift through all of the existing Debug messages and assign them specific priorities. If you would like to see your bug report make any progress, most likely you are going to need to come up with the person with that time and motivation.
Dear Howard,
On 19/09/11 19:15 -0700, Howard Chu wrote:
Obviously since OpenLDAP's behavior here is "by design" it is not a bug. Just a note - there have been at least two attempts to overhaul the Debug/Syslog support in recent years. Neither made visible progress. At this point the infrastructure exists to log individual messages at distinct priority levels, but nobody has been interested in taking the time to sift through all of the existing Debug messages and assign them specific priorities. If you would like to see your bug report make any progress, most likely you are going to need to come up with the person with that time and motivation.
That person would be me. I would be happy to do the work if the prospect exists of me working towards getting the changes incorporated into the main OpenLDAP code base. This change would be very useful to us.
Nick Urbanik wrote:
Dear Howard,
On 19/09/11 19:15 -0700, Howard Chu wrote:
Obviously since OpenLDAP's behavior here is "by design" it is not a bug. Just a note - there have been at least two attempts to overhaul the Debug/Syslog support in recent years. Neither made visible progress. At this point the infrastructure exists to log individual messages at distinct priority levels, but nobody has been interested in taking the time to sift through all of the existing Debug messages and assign them specific priorities. If you would like to see your bug report make any progress, most likely you are going to need to come up with the person with that time and motivation.
That person would be me. I would be happy to do the work if the prospect exists of me working towards getting the changes incorporated into the main OpenLDAP code base. This change would be very useful to us.
Great. Then I suggest you start with http://www.openldap.org/devel/ and then read include/ldap_log.h in the source. Then rewrite the desired Debug invocations in the slapd source, and forward your patch as a followup to your ITS.
Howard Chu writes:
Nick Urbanik wrote:
That person would be me. I would be happy to do the work if the prospect exists of me working towards getting the changes incorporated into the main OpenLDAP code base. This change would be very useful to us.
Great. Then I suggest you start with http://www.openldap.org/devel/ and then read include/ldap_log.h in the source. Then rewrite the desired Debug invocations in the slapd source, and forward your patch as a followup to your ITS.
Hold on. Been there, done that: Google(NEW_LOGGING site:openldap.org). The code was committed in #ifdefs, but in the end it was deleted again.
So many things could be improved with slapd logging, but don't get carried away and do a lot of work which might be rejected. E.g. design and code something which can live alongside the old Statslog()/Debug(), or which those can be converted to with a script, and then improve on it later. That belongs in the -devel list or your ITS though, not in -technical.
openldap-technical@openldap.org
-
Aaron Richton -
Christian Manal -
Hallvard B Furuseth -
Howard Chu -
Nick Urbanik