--On Thursday, March 18, 2010 3:47 PM -0400 "Beuerlein, Edward"
<Edward_Beuerlein@cable.comcast.com> wrote:

> Hi Quanah,
> Thanks for replying!  Please let us know what other info you might need
> to help us figure out these large increases in memory usage.
>
> Here's the info you requested:
>
> I did a dump of the database and then ran this on it(please let me know
> if that isn't what you wanted):
>
># cat auth01.031610.ldif |grep dn |wc -l
> 60753

So you have 60,753 entries.  Just to be sure, I would have used grep ^dn:
to be exact. ;)

> set_cachesize 0 268435456 1

You have 256MB allocated to BDB

> 69M     cn.bdb
> 26M     dn2id.bdb
> 3.7M    entryCSN.bdb
> 2.2M    entryUUID.bdb
> 160K    gidNumber.bdb
> 2.1G    id2entry.bdb
> 840K    ipHostNumber.bdb
> 44K     memberNisNetgroup.bdb
> 304K    memberUid.bdb
> 3.1M    nisNetgroupTriple.bdb
> 2.9M    objectClass.bdb
> 8.0K    ou.bdb
> 8.0K    sudoUser.bdb
> 3.0M    uid.bdb
> 364K    uidNumber.bdb
> 792M    uniqueMember.bdb
> 2.9G    total

Your BDB database is 2.9GB.

> olcDbCacheSize 1000
> olcDbCacheFree 1
> olcDbDNcacheSize 0
> olcDBIDLcacheSize 1000

You're only allowing the first 1000 entries to be cached in OpenLDAP, and
your IdlCache is quite small as well.  dncachesize is fine (unlimited).

While none of this explains your memory spikes, your server is definitely
poorly tuned.  It may be that your massive discrepancy between BDB cache
allocation and actual BDB size may be causing the problem (in looking at
the bdb error messages in your log).

I would highly advise the following changes.

For DB_CONFIG

set_cachesize 4 0 1

This will increase the BDB cache to 4GB (which can hold your 2.9GB DB
easily)

For cn=config:

olcDbCacheSize 80000
olcDbCacheFree 1000
olcDBIDLcacheSize 240000

This will allow all entries to be cached in OpenLDAP, free up a reasonable
number if you exceed the cache, and allow a decent IDL size.

I forgot to ask which slapd backend you are using (hdb or bdb), and which
version of BDB you are using (and if it is fully patched) which is also
useful information.

You may also wish to read over
<http://wiki.zimbra.com/index.php?title=OpenLDAP_Performance_Tuning_6.0>, I
think you guys have some experience with Zimbra... ;)

--Quanah

--

Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra ::  the leader in open source messaging and collaboration