Howard,
I have the always the same DB_CONFIG file as below :
set_cachesize 0 52428800 1
set_flags DB_LOG_AUTOREMOVE
set_lg_regionmax 1048576 set_lg_max 10485760 set_lg_bsize 2097152
set_tmp_dir /var/tmp
I just removed the comment lines so it fits better on screen.
About versions I have :
[root@brtldp12 yum.repos.d]# /usr/libexec/slapd -V @(#) $OpenLDAP: slapd 2.4.13 (Jan 22 2009 02:28:12) $ root@brtldp12:/root/openldap-2.4.13/servers/slapd
[root@brtldp12 yum.repos.d]# db_verify -V Berkeley DB 4.7.25: (May 15, 2008)
About the man pages I have :
[root@brtldp12 yum.repos.d]# ll /usr/man/man8/slapd.8.gz -rw-r--r-- 1 root root 4392 Jan 22 03:28 /usr/man/man8/slapd.8.gz
[root@brtldp12 yum.repos.d]# ll /usr/man/man5/slapd-bdb.5.gz -rw-r--r-- 1 root root 4089 Jan 22 03:28 /usr/man/man5/slapd-bdb.5.gz
I was expecting to find information about dncachesize in slapd-bdb but there isn't nothing there.
Where at config.c I have :
"( OLcfgDbAt:1.5 NAME 'olcDbDirtyRead' " "DESC 'Allow reads of uncommitted data' " "SYNTAX OMsBoolean SINGLE-VALUE )", NULL, NULL }, { "dncachesize", "size", 2, 2, 0, ARG_UINT|ARG_OFFSET, (void *)offsetof(struct bdb_info, bi_cache.c_eimax), "( OLcfgDbAt:1.12 NAME 'olcDbDNcacheSize' " "DESC 'DN cache size' " "SYNTAX OMsInteger SINGLE-VALUE )", NULL, NULL }
So the dncachesize is supposed to be in the code.
In any case slapd never respects memory limitation and consumes it without boundaries. It also never releases any memory.
Then after sometime the process cannot allocate more memory and crashes.
Best Regards,
Rodrigo.
--- On Thu, 1/22/09, Howard Chu hyc@symas.com wrote:
From: Howard Chu hyc@symas.com Subject: Re: (ITS#5860) slapd memeory leak under openldap 2.4 To: rlvcosta@yahoo.com Cc: openldap-its@openldap.org Date: Thursday, January 22, 2009, 1:35 PM rlvcosta@yahoo.com wrote:
See that I have 2 DBs and in each DB area in
slapd.conf, read without error by slapd as seen above, I have the cachesize, idlcachesize and dncachesize specified with small values just to monitor the memory usage by slapd.
Unfortunately even with this configuration I still
having slapd consuming memory without release it. So if a new entrance is queried by some LDAP client, slapd will consume more memory. Since my DB dn's are bigger than 3GB then soon or later the slapd will crash.
See my DB information :
7.2G dn2id.bdb 12G id2entry.bdb 110M maillogin.bdb 3.3M objectClass.bdb 108M pnnumber.bdb 1.3M submxid.bdb 323M uid.bdb
See my dn2id is much bigger than yours and maybe you
just did not saw this issue happenig since your dn can be loaded in your memory and you in a 64 bits environment can use more than 3GB per process.
But in any case looks like there is a problem in slapd
that do not respect the cache limitations imposed.
In this way we cannot use slapd for large databases.
Please see my configuration above and let me know if a made a mistake(I hope so) since dncachesize is not documented at man pages.
You're mistaken, dncachesize is in the 2.4 manpages. You seem to not be reading the documentation that came with the software release you're using.
Just as example, with the DB I loaded for tests with 1
million entrances, with the cache sizes definitions as in the slapd.conf described above and making a ldapsearch on all entrances from the CONTENT DB, in the end I have :
PID USER PR NI VIRT RES SHR S %CPU %MEM
TIME+ COMMAND
3189 ldap 15 0 842m 724m 67m S 99 6.1
7:00.48 slapd
See the slapd process already consumed 842m and I just
read 1 million entrances, even dncachesize is defined to be 1000.
I do not see anyway to control the memory usage by
slapd.
Is this "disrespect" to slapd.conf
directives a possible problem?
Looks like slapd is behaving normally so far, though it's hard to tell without also seeing your DB_CONFIG settings. Test again with your full database and report the results.
-- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/