pcache crashes after 2.4.25 upgrade; complains of unclean shutdowns and missing DB_CONFIGs after every restart
After upgrading from 2.4.24 to 2.4.25 my pcache overlay using hdb backend first failed to start with messages complaining about a missing suffix. Now that I've added one I'm getting the following message:
hdb_db_open: database "dc=example,dc=com": unclean shutdown detected; attempting recovery. hdb_db_open: warning - no DB_CONFIG file found in directory /var/lib/ldap: (14).#012Expect poor performance for suffix "dc=example,dc=com". slapd starting bdb(dc=example,dc=com): PANIC: fatal region error detected; run recovery last message repeated 3 times
But DB_CONFIG does exist and has proper permissions and the unclean shutdown message repeats after every restart. I've never had to add the suffix to the pcache backend database before and the documentation examples (http://www.openldap.org/doc/admin24/proxycache.html) never mention it either. Is this a misconfiguration or do I need to file a bug?
PCACHE OVERLAY:
dn: olcOverlay={0}pcache objectClass: olcOverlayConfig objectClass: olcPcacheConfig olcOverlay: {0}pcache olcPcache: hdb 100000 3 1000 100 olcPcacheAttrset: 0 uid userPassword uidNumber gidNumber cn homeDirectory logi nShell gecos description memberUid uniqueMember objectClass olcPcacheAttrset: 1 cn automountInformation olcPcacheAttrset: 2 cn mail olcPcacheTemplate: "(|(cn=)(mail=)(sn=))" 2 7200 0 0 0 olcPcacheTemplate: "(&(objectClass=)(|(cn=)(cn=)(cn=)))" 1 3600 600 0 0 olcPcacheTemplate: "(&(objectClass=)(|(cn=)(cn=)))" 1 3600 600 0 0 olcPcacheTemplate: "(&(objectClass=)(|(cn=)(gidNumber=)))" 1 3600 600 0 0 olcPcacheTemplate: "(&(objectClass=)(gidNumber=))" 0 1800 0 0 0 olcPcacheTemplate: "(&(objectClass=)(uidNumber=))" 0 1800 0 0 0 olcPcacheTemplate: "(&(objectClass=)(uniqueMember=))" 0 1800 900 0 0 olcPcacheTemplate: "(&(objectClass=)(memberUid=))" 0 1800 900 0 0 olcPcacheTemplate: "(objectClass=)" 0 1800 0 0 0 olcPcacheTemplate: "(&(objectClass=))" 0 1800 0 0 0 olcPcacheTemplate: "(&(objectClass=)(cn=))" 0 1800 0 0 0 olcPcacheTemplate: "(&(objectClass=)(uid=))" 0 1800 0 0 0 olcPcacheTemplate: "(&(objectClass=)(|(memberUid=)(uniqueMember=)))" 0 1800 0 0 0 olcPcachePosition: tail olcPcacheMaxQueries: 10000 olcPcachePersist: FALSE olcPcacheValidate: FALSE olcPcacheOffline: FALSE
HDB backend:
dn: olcDatabase={0}hdb objectClass: olcHdbConfig objectClass: olcPcacheDatabase olcDatabase: {0}hdb olcDbDirectory: /var/lib/ldap olcDbCacheSize: 1000 olcDbNoSync: FALSE olcDbDirtyRead: FALSE olcDbIDLcacheSize: 3000 olcDbLinearIndex: FALSE olcDbMode: 0600 olcDbSearchStack: 16 olcDbShmKey: 0 olcDbCacheFree: 1 olcDbDNcacheSize: 0 createTimestamp: 20110225174535Z olcDbConfig: {0}set_cachesize 0 10485760 1 olcDbConfig: {1}set_lg_regionmax 262144 olcDbConfig: {2}set_lg_bsize 2097152 olcDbConfig: {3}set_flags DB_LOG_AUTOREMOVE olcDbConfig: {4}set_lg_max 10485760 olcDbIndex: objectClass eq olcDbIndex: cn eq,subinitial olcDbIndex: uid eq,subinitial olcDbIndex: uidNumber eq olcDbIndex: gidNumber eq olcDbIndex: userPassword eq olcDbIndex: description eq olcDbIndex: loginShell eq olcDbIndex: homeDirectory eq olcDbIndex: memberUid eq olcDbIndex: gecos eq olcDbIndex: automountInformation eq olcDbIndex: uniqueMember eq olcDbIndex: mail eq,subinitial olcDbIndex: sn eq,subinitial olcDbIndex: givenName eq,subinitial olcDbIndex: member eq olcDbIndex: nisMapName eq olcDbIndex: pcacheQueryID eq olcSuffix: dc=example,dc=com
--On April 5, 2011 5:01:50 PM -0400 Tyler Gates tgates81@gmail.com wrote:
After upgrading from 2.4.24 to 2.4.25 my pcache overlay using hdb backend first failed to start with messages complaining about a missing suffix. Now that I've added one I'm getting the following message:
Nothing should ever crash openldap. Make sure you built with debugging symbols (-g to gcc), no optimization (-O0), get a backtrace of a crash via gdb, and file an ITS for it.
Thanks!
--Quanah
It doesn't actually 'crash' but is unresponsive. Submitted ITS 6891: http://www.openldap.org/its/index.cgi/Incoming?selectid=6891;statetype=-1
On 04/05/2011 09:55 PM, Quanah Gibson-Mount wrote:
--On April 5, 2011 5:01:50 PM -0400 Tyler Gates tgates81@gmail.com wrote:
After upgrading from 2.4.24 to 2.4.25 my pcache overlay using hdb backend first failed to start with messages complaining about a missing suffix. Now that I've added one I'm getting the following message:
Nothing should ever crash openldap. Make sure you built with debugging symbols (-g to gcc), no optimization (-O0), get a backtrace of a crash via gdb, and file an ITS for it.
Thanks!
--Quanah
One of the recommended ways we learned via this list (and other sources) to determine the the bdb cache size is to get the size of the entire db using this: "du -ch /var/lib/ldap/*.bdb". Which we have done and seems to be working good for the initial settings for our ldap rebuild. We have "set_cachesize 0 4153344 1" in our DB_CONFIG and the command returns 4.1M. We are going to put a simple monitor script in place to keep an eye on the db size so we will have an indication that the cache size may need to be increased as the size of the db grows.
Does anyone have any recommendations for a threshold to use for increasing the cache setting? In other other words, if the db size grows 10% larger than the cache setting, is that a good time to let the admins know it is time to raise the cachesize? Just looking for suggestions on something to use as a trigger in a script.
--On Thursday, April 07, 2011 3:01 PM -0600 ldap@mm.st wrote:
One of the recommended ways we learned via this list (and other sources) to determine the the bdb cache size is to get the size of the entire db using this: "du -ch /var/lib/ldap/*.bdb". Which we have done and seems to be working good for the initial settings for our ldap rebuild. We have "set_cachesize 0 4153344 1" in our DB_CONFIG and the command returns 4.1M. We are going to put a simple monitor script in place to keep an eye on the db size so we will have an indication that the cache size may need to be increased as the size of the db grows.
Does anyone have any recommendations for a threshold to use for increasing the cache setting? In other other words, if the db size grows 10% larger than the cache setting, is that a good time to let the admins know it is time to raise the cachesize? Just looking for suggestions on something to use as a trigger in a script.
There are two ways in which the DB cachesize is used.
a) When running slapadd to load a database. In this case, it will be fastest if the BDB cachesize matches the size of the entire db (du -ch *.bdb)
b) When slapd is running. In this case, it really only needs (id2entry.bdb + dn2id.bdb) size. This is always smaller than du -ch *.bdb
So if you tune your cache for (a) it will account for (b) with some growth. Personally, if you have plenty of RAM on the box, I'd just set your cache size to be a bit larger than you are now, say 50MB. It seems you have quite the small database if it is only 4.1MB in size.
Otherwise, yes, build a script to monitor the cache, and set an alert when it gets close to whatever threshhold you deem critical. If it is a slowly growing database, then perhaps 5%. If it grows quickly, then maybe 20%.
--Quanah
--
Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration
openldap-technical@openldap.org
-
ldap＠mm.st -
Quanah Gibson-Mount -
Tyler Gates