--On Thursday, March 18, 2010 3:47 PM -0400 "Beuerlein, Edward" Edward_Beuerlein@cable.comcast.com wrote:
Hi Quanah, Thanks for replying! Please let us know what other info you might need to help us figure out these large increases in memory usage.
Here's the info you requested:
I did a dump of the database and then ran this on it(please let me know if that isn't what you wanted):
# cat auth01.031610.ldif |grep dn |wc -l 60753
So you have 60,753 entries. Just to be sure, I would have used grep ^dn: to be exact. ;)
set_cachesize 0 268435456 1
You have 256MB allocated to BDB
69M cn.bdb 26M dn2id.bdb 3.7M entryCSN.bdb 2.2M entryUUID.bdb 160K gidNumber.bdb 2.1G id2entry.bdb 840K ipHostNumber.bdb 44K memberNisNetgroup.bdb 304K memberUid.bdb 3.1M nisNetgroupTriple.bdb 2.9M objectClass.bdb 8.0K ou.bdb 8.0K sudoUser.bdb 3.0M uid.bdb 364K uidNumber.bdb 792M uniqueMember.bdb 2.9G total
Your BDB database is 2.9GB.
olcDbCacheSize 1000 olcDbCacheFree 1 olcDbDNcacheSize 0 olcDBIDLcacheSize 1000
You're only allowing the first 1000 entries to be cached in OpenLDAP, and your IdlCache is quite small as well. dncachesize is fine (unlimited).
While none of this explains your memory spikes, your server is definitely poorly tuned. It may be that your massive discrepancy between BDB cache allocation and actual BDB size may be causing the problem (in looking at the bdb error messages in your log).
I would highly advise the following changes.
For DB_CONFIG
set_cachesize 4 0 1
This will increase the BDB cache to 4GB (which can hold your 2.9GB DB easily)
For cn=config:
olcDbCacheSize 80000 olcDbCacheFree 1000 olcDBIDLcacheSize 240000
This will allow all entries to be cached in OpenLDAP, free up a reasonable number if you exceed the cache, and allow a decent IDL size.
I forgot to ask which slapd backend you are using (hdb or bdb), and which version of BDB you are using (and if it is fully patched) which is also useful information.
You may also wish to read over http://wiki.zimbra.com/index.php?title=OpenLDAP_Performance_Tuning_6.0, I think you guys have some experience with Zimbra... ;)
--Quanah
--
Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration
Thanks for the suggestions...
Bdb, BerkelyDB-4.7.25, yes, fully patched.
MAT
On 3/18/10 1:56 PM, "Quanah Gibson-Mount" quanah@zimbra.com wrote:
--On Thursday, March 18, 2010 3:47 PM -0400 "Beuerlein, Edward" Edward_Beuerlein@cable.comcast.com wrote:
Hi Quanah, Thanks for replying! Please let us know what other info you might need to help us figure out these large increases in memory usage.
Here's the info you requested:
I did a dump of the database and then ran this on it(please let me know if that isn't what you wanted):
# cat auth01.031610.ldif |grep dn |wc -l 60753
So you have 60,753 entries. Just to be sure, I would have used grep ^dn: to be exact. ;)
set_cachesize 0 268435456 1
You have 256MB allocated to BDB
69M cn.bdb 26M dn2id.bdb 3.7M entryCSN.bdb 2.2M entryUUID.bdb 160K gidNumber.bdb 2.1G id2entry.bdb 840K ipHostNumber.bdb 44K memberNisNetgroup.bdb 304K memberUid.bdb 3.1M nisNetgroupTriple.bdb 2.9M objectClass.bdb 8.0K ou.bdb 8.0K sudoUser.bdb 3.0M uid.bdb 364K uidNumber.bdb 792M uniqueMember.bdb 2.9G total
Your BDB database is 2.9GB.
olcDbCacheSize 1000 olcDbCacheFree 1 olcDbDNcacheSize 0 olcDBIDLcacheSize 1000
You're only allowing the first 1000 entries to be cached in OpenLDAP, and your IdlCache is quite small as well. dncachesize is fine (unlimited).
While none of this explains your memory spikes, your server is definitely poorly tuned. It may be that your massive discrepancy between BDB cache allocation and actual BDB size may be causing the problem (in looking at the bdb error messages in your log).
I would highly advise the following changes.
For DB_CONFIG
set_cachesize 4 0 1
This will increase the BDB cache to 4GB (which can hold your 2.9GB DB easily)
For cn=config:
olcDbCacheSize 80000 olcDbCacheFree 1000 olcDBIDLcacheSize 240000
This will allow all entries to be cached in OpenLDAP, free up a reasonable number if you exceed the cache, and allow a decent IDL size.
I forgot to ask which slapd backend you are using (hdb or bdb), and which version of BDB you are using (and if it is fully patched) which is also useful information.
You may also wish to read over http://wiki.zimbra.com/index.php?title=OpenLDAP_Performance_Tuning_6.0, I think you guys have some experience with Zimbra... ;)
--Quanah
--
Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc
Zimbra :: the leader in open source messaging and collaboration
openldap-software@openldap.org
-
Mathew Rowley -
Quanah Gibson-Mount