I'm running some 2.4.23 servers, and I've encountered some slowdown on loglevel other than 0. Even 256 (stats; recommended) impacts about a 4x slowdown on queries. Logging is to syslog.
Running ldapsearch slows from 0.005-0.010 seconds to about 0.030-0.040 seconds; and that includes loading the binary. That's from localhost to remove potential DNS lookups.
I stumbled across this when logging was wrong, and the slowdown was 100x.
I'm aware that 2.4.23 isn't the latest version. I'm also quite happy, for now, to run loglevel 0.
I'm wondering if this is the expected behaviour, given that it's the recommended configuration. Or should I go dig to find the slowdown?
(I did check the indexes, and db_stats, etc. All seems fine.)
I apologise for the disclaimer, Berend
CONFIDENTIALITY NOTICE The contents of and attachments to this e-mail are intended for the addressee only, and may contain the confidential information of Argility (Proprietary) Limited and/or its subsidiaries. Any review, use or dissemination thereof by anyone other than the intended addressee is prohibited. If you are not the intended addressee please notify the writer immediately and destroy the e-mail. Argility (Proprietary) Limited and its subsidiaries distance themselves from and accept no liability for unauthorised use of their e-mail facilities or e-mails sent other than strictly for business purposes.
How does your syslog config look like ? Did you prefix the filename with the magic - symbol in order to ask for async I/Os on that file?
E.g.
*.info;mail.none;authpriv.none;cron.none -/var/log/messages
mail.* -/var/log/maillog
local4.* -/var/log/slapd.log
Instead of
*.info;mail.none;authpriv.none;cron.none /var/log/messages
mail.* /var/log/maillog
local4.* /var/log/slapd.log
++Cyrille
Man syslog.conf Regular File Typically messages are logged to real files. The file has to be specified with full pathname, beginning with a slash ??/??.
You may prefix each entry with the minus ??-?? sign to omit syncing the file after every logging. Note that you might lose information if the system crashes right behind a write attempt. Nevertheless this might give you back some performance, especially if you run programs that use logging in a very verbose manner. -----Original Message----- From: openldap-technical-bounces@OpenLDAP.org [mailto:openldap-technical-bounces@OpenLDAP.org] On Behalf Of Berend De Schouwer Sent: Wednesday, June 13, 2012 12:28 PM To: openldap-technical@openldap.org Subject: loglevel expected performance impact
I'm running some 2.4.23 servers, and I've encountered some slowdown on loglevel other than 0. Even 256 (stats; recommended) impacts about a 4x slowdown on queries. Logging is to syslog.
Running ldapsearch slows from 0.005-0.010 seconds to about 0.030-0.040 seconds; and that includes loading the binary. That's from localhost to remove potential DNS lookups.
I stumbled across this when logging was wrong, and the slowdown was 100x.
I'm aware that 2.4.23 isn't the latest version. I'm also quite happy, for now, to run loglevel 0.
I'm wondering if this is the expected behaviour, given that it's the recommended configuration. Or should I go dig to find the slowdown?
(I did check the indexes, and db_stats, etc. All seems fine.)
I apologise for the disclaimer, Berend
CONFIDENTIALITY NOTICE The contents of and attachments to this e-mail are intended for the addressee only, and may contain the confidential information of Argility (Proprietary) Limited and/or its subsidiaries. Any review, use or dissemination thereof by anyone other than the intended addressee is prohibited. If you are not the intended addressee please notify the writer immediately and destroy the e-mail. Argility (Proprietary) Limited and its subsidiaries distance themselves from and accept no liability for unauthorised use of their e-mail facilities or e-mails sent other than strictly for business purposes.
If we talk about syslog .. SuSE (opensuse/SLES) writes local4 in /var/log/localmessages and /var/log/messages!! Best way here to write messages in separate file is:
#### part from syslog-ng.conf #####
filter f_ldap { program(slapd);};
#change original lines: filter f_local { facility(local0, local1, local2, local3, local4, local5, local6, local7) and not filter(f_ldap); }; filter f_messages { not facility(news, mail) and not filter(f_iptables) and not filter(f_ldap); };
# # LDAP in separate file # destination ldaplog { file("/var/log/ldap.log" owner(ldap) group(ldap)); }; log { source(src); filter(f_ldap); destination(ldaplog); };
### end ######
Additionally there is a sync option for syslg-ng. It can be set in the options { ...} line.
BTW a few other thoughts, who may influence the speed if you turn on logging...: * How many ldap requests per second you get on LDAP? - you can check this via monitor db ~# ldapsearch ... -b"cn=Search,cn=Operations,cn=Monitor" monitorCounter ~# ldapsearch ... -b"cn=Uptime,cn=Time,cn=Monitor" monitoredInfo then you can calculate the average of your req/s ...
* How is sized your system (especially CPU, RAM, disks, raid, controller, partitions, ..) * database size * Do you use shm_key (takes away IO from your disk)?
We log (level 256) since a short time too, with about (max) 1200 ldap req/sec and the system and IO is mostly idle :-)
Kindly regards Meike
2012/6/13 Maucci, Cyrille cyrille.maucci@hp.com:
How does your syslog config look like ? Did you prefix the filename with the magic - symbol in order to ask for async I/Os on that file?
E.g.
*.info;mail.none;authpriv.none;cron.none -/var/log/messages
mail.* -/var/log/maillog
local4.* -/var/log/slapd.log
Instead of
*.info;mail.none;authpriv.none;cron.none /var/log/messages
mail.* /var/log/maillog
local4.* /var/log/slapd.log
++Cyrille
Man syslog.conf Regular File Typically messages are logged to real files. The file has to be specified with full pathname, beginning with a slash ??/??.
You may prefix each entry with the minus ??-?? sign to omit syncing the file after every logging. Note that you might lose information if the system crashes right behind a write attempt. Nevertheless this might give you back some performance, especially if you run programs that use logging in a very verbose manner. -----Original Message----- From: openldap-technical-bounces@OpenLDAP.org [mailto:openldap-technical-bounces@OpenLDAP.org] On Behalf Of Berend De Schouwer Sent: Wednesday, June 13, 2012 12:28 PM To: openldap-technical@openldap.org Subject: loglevel expected performance impact
I'm running some 2.4.23 servers, and I've encountered some slowdown on loglevel other than 0. Even 256 (stats; recommended) impacts about a 4x slowdown on queries. Logging is to syslog.
Running ldapsearch slows from 0.005-0.010 seconds to about 0.030-0.040 seconds; and that includes loading the binary. That's from localhost to remove potential DNS lookups.
I stumbled across this when logging was wrong, and the slowdown was 100x.
I'm aware that 2.4.23 isn't the latest version. I'm also quite happy, for now, to run loglevel 0.
I'm wondering if this is the expected behaviour, given that it's the recommended configuration. Or should I go dig to find the slowdown?
(I did check the indexes, and db_stats, etc. All seems fine.)
I apologise for the disclaimer, Berend
CONFIDENTIALITY NOTICE The contents of and attachments to this e-mail are intended for the addressee only, and may contain the confidential information of Argility (Proprietary) Limited and/or its subsidiaries. Any review, use or dissemination thereof by anyone other than the intended addressee is prohibited. If you are not the intended addressee please notify the writer immediately and destroy the e-mail. Argility (Proprietary) Limited and its subsidiaries distance themselves from and accept no liability for unauthorised use of their e-mail facilities or e-mails sent other than strictly for business purposes.
On Wed, 2012-06-13 at 14:23 +0100, Maucci, Cyrille wrote:
How does your syslog config look like ?
The - did make an appreciable difference, thanks.
I apologise for the disclaimer again, Berend
CONFIDENTIALITY NOTICE The contents of and attachments to this e-mail are intended for the addressee only, and may contain the confidential information of Argility (Proprietary) Limited and/or its subsidiaries. Any review, use or dissemination thereof by anyone other than the intended addressee is prohibited. If you are not the intended addressee please notify the writer immediately and destroy the e-mail. Argility (Proprietary) Limited and its subsidiaries distance themselves from and accept no liability for unauthorised use of their e-mail facilities or e-mails sent other than strictly for business purposes.
openldap-technical@openldap.org