Quanah,
I just used tcmalloc but I still with the same behavior. slapd doesn't have limit for the cache and then it cannot run large databases.
My DB entrance is not large. One example is :
dn: uid=1219843773896\2B554184XXXXXX,ou=REPOSITORY,ou=CONTENT,o=domain,c=country creationdate: 200808271329Z localelist: en,fr weblogin: +554184XXXXXX expirationdate: 200810161329Z profileid: 0 mail: +554184XXXXXX/type=PLMN@<domain> uid: 1219843773896+554184XXXXXX objectClass: commonuser objectClass: user accountStatus: 1 webpassword: 0000 cn: +554184XXXXXX optionvalues:: AAEAAAcBAgEAAQEBAQEBAAAAAQEBAAEBAQEAAQAAAAAAAAMAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA== optionstates:: AAAAAAABAwAAAAAAAAABAAAAAQIAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA== isintercepted: FALSE
dn: pnnumber=\2B554184XXXXXX,uid=1219843773896\2B554184XXXXXX,ou=REPOSITORY,ou =CONTENT,o=domain,c=country subpnid: 0 pntype: 2 pncaps: 7 objectClass: phoneinfo pnnumber: +554184XXXXXX
dn: submxid=0,uid=1219843773896\2B554184XXXXXX,ou=REPOSITORY,ou=CONTENT,o=domain,c=country secretcode: 0000 fraudaccessnb: 0 mailMessageStore: ./Maildir maillogin: +554184XXXXXX/type=PLMN@<domain> objectClass: commonmailbox objectClass: courierinfo submxid: 0 mailQuota: 1048576 mailpassword: <secret> homeDirectory: /var/mms/mail/0/00/0000000000/0
See above I have 3 DN's.
In 64 bits systems with more memory and small dn's this could be hidden but with the time all physical memory will be used.
So, I made an ldapsearch as (searching a single DN):
time ldapsearch -LLL -x -D "cn=admin,ou=CONTENT,o=domain,c=country" -w secret -b "ou=CONTENT,o=domain,c=country" -H ldap://10.142.15.42:389 'pnnumber=*'| egrep -e '^pnnumber*' |wc -l 1000003
real 6m56.321s user 0m25.617s sys 0m8.392s
So just one of the 3 DNs this DB has. Since the filesize is :
1.9G dn2id.bdb 3.1G id2entry.bdb 29M maillogin.bdb 3.3M objectClass.bdb 28M pnnumber.bdb 1.3M submxid.bdb 84M uid.bdb
I have part of size load into cache(only for pnnnumber).
But slapd doesn't respect cache limits and load all into memory. It also never release it.
The entrances are not large and caches in the order of 1000 entrances should not make slapd process stay with almost 1GB of memory.
Also once it occupies this memory it never release anymore.
I use lsof and the tcmalloc is being used.
Regards,
Rodrigo.
--- On Thu, 1/22/09, Quanah Gibson-Mount quanah@zimbra.com wrote:
From: Quanah Gibson-Mount quanah@zimbra.com Subject: Re: (ITS#5860) slapd memeory leak under openldap 2.4 To: rlvcosta@yahoo.com, openldap-its@openldap.org Date: Thursday, January 22, 2009, 4:04 PM --On Thursday, January 22, 2009 10:01 AM -0800 Rodrigo Costa rlvcosta@yahoo.com wrote:
Quanah,
Just a correction about the man, it was incorrect
since my man was
getting from cache(and so from 2.3 version). Now it is
ok.
I would like a little more directions about how can be
compiled openldap
2.4.13 with tcmalloc. Just need to include a lib path?
Some configure
parameter in openldap?
I use tcmalloc myself, so basically you go and build google-perf-tools, and then you preload the library before starting slapd. I have a little startup script for slapd:
#!/bin/bash ulimit -n 32768 ulimit -c unlimited export LD_PRELOAD=/opt/zimbra/tcmalloc/lib/libtcmalloc.so exec /opt/zimbra/openldap/sbin/slapd "$@"
--Quanah
--
Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc
Zimbra :: the leader in open source messaging and collaboration