From slapd.conf/cn=config: a) cachesize setting b) idlcachesize setting c) dncachesize setting
cachesize 750000 dncachesize 750000 idlcachesize 2250000
Thanks and best regards
Your settings here don't make a lot of sense.
I would try cachesize 1500000 dncachesize 0 (or leave it unset) idlcachesize 3000000
Yes, I know this values are very low in my configuration, but if I have problems with the memory, should I increase the caches that use this memory?
In the last thread , you confirmed, that if I increasing the DB cachesize, the system is faster running out of memory. http://www.openldap.org/lists/openldap-technical/201211/msg00171.html ("ch_realloc means the system ran out of memory. Increasing the DB_CONFIG cachesize will run you out of memory more quickly.")
To get the slapd a little bit more stable, I reduced this values (caches) to the posted in last message. Before i had: cachesize 1000000 dncachesize 1000000 idlcachesize 3000000
Your current idlcachesize in particular may be part of the issue in addition to your glibc setting.
But the problem exist on the test system with tcmalloc too.
So are 24GByte RAM for 1.500.000 DNs and 6GByte DB are not enough? During all time (especially the crash), there was enough physical (unallocated) memory free. But "slapd/malloc" tried to allocate more and more (as you can see in the post from 2013/01/12), but did not use it, no matter if glibc or tcmalloc I hope I interpreted it right?!
Thanks Meike
Btw.: I use a "special version" of openldap. Because of limited index slot of 65535, I changed
in openldap-2.4.33.orig/include/ldap_pvt_thread.h from # define LDAP_PVT_THREAD_STACK_SIZE ( 1 * 1024 * 1024 * sizeof(void *) ) to # define LDAP_PVT_THREAD_STACK_SIZE ( 2 * 1024 * 1024 * sizeof(void *) )
and in openldap-2.4.33/servers/slapd/back-bdb/idl.h from #define BDB_IDL_LOGN 16 /* DB_SIZE is 2^16, UM_SIZE is 2^17 */ to #define BDB_IDL_LOGN 17 /* DB_SIZE is 2^17, UM_SIZE is 2^18 */
as Howard it recommended http://www.openldap.org/lists/openldap-technical/201101/msg00095.html
Maybe this could be the problem??
My changes are available under: https://build.opensuse.org/package/show?expand=0&package=openldap2&p...
--On Tuesday, January 15, 2013 2:09 PM +0100 Meike Stone meike.stone@googlemail.com wrote:
Btw.: I use a "special version" of openldap. Because of limited index slot of 65535, I changed
in openldap-2.4.33.orig/include/ldap_pvt_thread.h from # define LDAP_PVT_THREAD_STACK_SIZE ( 1 * 1024 * 1024 * sizeof(void *) ) to # define LDAP_PVT_THREAD_STACK_SIZE ( 2 * 1024 * 1024 * sizeof(void *) )
and in openldap-2.4.33/servers/slapd/back-bdb/idl.h from # define BDB_IDL_LOGN 16 /* DB_SIZE is 2^16, UM_SIZE is 2^17 */ to # define BDB_IDL_LOGN 17 /* DB_SIZE is 2^17, UM_SIZE is 2^18 */
as Howard it recommended http://www.openldap.org/lists/openldap-technical/201101/msg00095.html
Maybe this could be the problem??
My changes are available under: https://build.opensuse.org/package/show?expand=0&package=openldap2&p... t=home%3Ameikestone
Yes, that would significantly increase memory usage. I have only ever done the *second* modification (BDB_IDL_LOGN) to fix the IDL issues. I've run that way for years.
Try dropping the modification to LDAP_PVT_THREAD_STACK_SIZE. It has never been necessary for any of my customers, many of whom have DBs with 6 to 10 million entries.
--Quanah
--
Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration
openldap-technical@openldap.org