openldap-bugs

openldap-bugs@openldap.org

1 participants
20967 discussions

Re: (ITS#7636) slapd crash when multi-master replication (syncrepl) enabled
by kb9vqf＠pearsoncomputing.net 02 Jul '13

02 Jul '13

Since my report above seems to have been somewhat mangled by the submission process, here is a link to a plain-text version of the same report: http://www.ceet.niu.edu/pastebin/openldap_bug_report_7636.txt

1 0

(ITS#7636) slapd crash when multi-master replication (syncrepl) enabled
by kb9vqf＠pearsoncomputing.net 02 Jul '13

02 Jul '13

Full_Name: Timothy Pearson Version: 2.4.31 OS: Debian Wheezy URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (131.156.2.26) The setup: Multi-master syncrepl on two servers Identical hardware and software between servers Self-signed TLS using common (private) CA certificate The problem: slapd on one server crashes repeatably within a minute of slapd starting on the other server. slapd works reliably if and only if the other server is not running a slapd process. Backtrace (does not change appreciably from crash to crash): Program received signal SIGABRT, Aborted. [Switching to Thread 0x7fffe356c700 (LWP 10433)] 0x00007ffff5a32475 in *__GI_raise (sig=<optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64 64 ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory. (gdb) bt #0 0x00007ffff5a32475 in *__GI_raise (sig=<optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64 #1 0x00007ffff5a356f0 in *__GI_abort () at abort.c:92 #2 0x00007ffff5a6d52b in __libc_message (do_abort=<optimized out>, fmt=<optimized out>) at ../sysdeps/unix/sysv/linux/libc_fatal.c:189 #3 0x00007ffff5a76d76 in malloc_printerr (action=3, str=0x7ffff5b4f170 "munmap_chunk(): invalid pointer", ptr=<optimized out>) at malloc.c:6283 #4 0x00007ffff63d214a in slapi_op_search_callback (op=0x7fffe3569fd0, rs=<optimized out>, prc=<optimized out>) at ../../../../../servers/slapd/slapi/slapi_overlay.c:313 #5 slapi_op_search_callback (op=0x7fffe3569fd0, rs=<optimized out>, prc=<optimized out>) at ../../../../../servers/slapd/slapi/slapi_overlay.c:296 #6 0x00007ffff63d304f in slapi_op_func (rs=0x7fffe3569f60, op=<optimized out>) at ../../../../../servers/slapd/slapi/slapi_overlay.c:631 #7 slapi_op_func (op=0x7fffe3569fd0, rs=0x7fffe3569f60) at ../../../../../servers/slapd/slapi/slapi_overlay.c:556 #8 0x00005555555ff18a in overlay_op_walk (op=op@entry=0x7fffe3569fd0, rs=0x7fffe3569f60, which=op_search, oi=0x5555559e57f0, on=0x5555559e4510) at ../../../../servers/slapd/backover.c:661 #9 0x00005555555ff31b in over_op_func (op=0x7fffe3569fd0, rs=<optimized out>, which=<optimized out>) at ../../../../servers/slapd/backover.c:723 #10 0x00007ffff1c4870e in syncprov_findbase (op=op@entry=0x555555dd8560, fc=fc@entry=0x7fffe356a280) at ../../../../../servers/slapd/overlays/syncprov.c:453 #11 0x00007ffff1c4b1ef in syncprov_op_search (op=0x555555dd8560, rs=0x7fffe356ba50) at ../../../../../servers/slapd/overlays/syncprov.c:2465 #12 0x00005555555ff18a in overlay_op_walk (op=op@entry=0x555555dd8560, rs=rs@entry=0x7fffe356ba50, which=which@entry=op_search, oi=0x5555559e57f0, on=0x5555559e15e0) at ../../../../servers/slapd/backover.c:661 #13 0x00007ffff63d3086 in slapi_op_func (rs=0x7fffe356ba50, op=<optimized out>) at ../../../../../servers/slapd/slapi/slapi_overlay.c:647 #14 slapi_op_func (op=0x555555dd8560, rs=0x7fffe356ba50) at ../../../../../servers/slapd/slapi/slapi_overlay.c:556 #15 0x00005555555ff18a in overlay_op_walk (op=op@entry=0x555555dd8560, rs=0x7fffe356ba50, which=op_search, oi=0x5555559e57f0, on=0x5555559e4510) at ../../../../servers/slapd/backover.c:661 #16 0x00005555555ff31b in over_op_func (op=0x555555dd8560, rs=<optimized out>, which=<optimized out>) at ../../../../servers/slapd/backover.c:723 #17 0x0000555555594641 in fe_op_search (op=0x555555dd8560, rs=0x7fffe356ba50) at ../../../../servers/slapd/search.c:402 #18 0x0000555555593f06 in do_search (op=0x555555dd8560, rs=0x7fffe356ba50) at ../../../../servers/slapd/search.c:247 #19 0x0000555555591961 in connection_operation (ctx=ctx@entry=0x7fffe356bba0, arg_v=arg_v@entry=0x555555dd8560) at ../../../../servers/slapd/connection.c:1150 #20 0x0000555555591c84 in connection_read_thread (ctx=0x7fffe356bba0, argv=<optimized out>) at ../../../../servers/slapd/connection.c:1286 #21 0x00007ffff7b9dff3 in ?? () from /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2 #22 0x00007ffff5d90b50 in start_thread (arg=<optimized out>) at pthread_create.c:304 #23 0x00007ffff5adaa7d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112 #24 0x0000000000000000 in ?? () Relevant configuration files: olcDatabase={0}config.ldif: dn: olcDatabase={0}config objectClass: olcDatabaseConfig olcDatabase: {0}config olcAccess: {0}to * by group/groupOfNames/member.exact="cn=realmadmins,ou=groups,ou=core,ou=realm,dc=<redacted>" write by dn.base="uid=ldapadmin,ou=users,ou=core,ou=realm,dc=<redacted>" write by dn.base="cn=admin,dc=<redacted>" write by sockurl.regex="^ldapi:///$" write by dynacl/aci write by * none olcAddContentAcl: TRUE olcLastMod: TRUE olcMaxDerefDepth: 15 olcReadOnly: FALSE olcRootDN: cn=config olcRootPW:: REDACTED olcSyncUseSubentry: FALSE olcMonitoring: FALSE structuralObjectClass: olcDatabaseConfig creatorsName: cn=config createTimestamp: 20130702170316Z olcSyncrepl: {0}rid=001 provider=ldaps://ldap002.<redacted>/ binddn="cn=admin,dc=<redacted>" bindmethod=simple credentials="ldapadmin001" searchbase="cn=config" type=refreshAndPersist retry="5 5 300 5 600 +" timeout=1 tls_reqcert=never tls_cacert="/etc/trinity/ldap/tde-ca/anchors/tdeca.pem"lcSyncrepl: {1}rid=002 provider=ldaps://ldap003.<redacted>/ binddn="cn=admin,dc=<redacted>" bindmethod=simple credentials="ldapadmin001" searchbase="cn=config" type=refreshAndPersist retry="5 5 300 5 600 +" timeout=1 tls_reqcert=never tls_cacert="/etc/trinity/ldap/tde-ca/anchors/tdeca.pem" olcMirrorMode: TRUE entryCSN: 20130702180604.682031Z#000000#002#000000 modifiersName: modifyTimestamp: 20130702180604Z olcDatabase={1}hdb.ldif: dn: olcDatabase={1}hdb objectClass: olcDatabaseConfig objectClass: olcHdbConfig olcDatabase: {1}hdb olcDbDirectory: /var/lib/ldap olcSuffix: dc=<redacted> olcAccess: {0}to attrs=userPassword,shadowLastChange,krb5Key,krb5PrincipalName,krb5KeyVersionNumber,krb5MaxLife,krb5MaxRenew,krb5KDCFlags,privateRootCertificateKey by group/groupOfNames/member.exact="cn=realmadmins,ou=groups,ou=core,ou=realm,dc=<redacted>" write by dn.base="uid=ldapadmin,ou=users,ou=core,ou=realm,dc=<redacted>" by sockurl.regex="^ldapi:///$" write by anonymous auth by self write by * none olcAccess: {1}to dn.base="" by * read olcAccess: {2}to * by group/groupOfNames/member.exact="cn=realmadmins,ou=groups,ou=core,ou=realm,dc=<redacted>" write by dn.base="uid=ldapadmin,ou=users,ou=core,ou=realm,dc=<redacted>" write by sockurl.regex="^ldapi:///$" write by dynacl/aci write olcAddContentAcl: FALSE olcLastMod: TRUE olcMaxDerefDepth: 15 olcReadOnly: FALSE olcRootDN: cn=admin,dc=<redacted> olcRootPW:: REDACTED olcMonitoring: TRUE olcDbCacheSize: 1000 olcDbCheckpoint: 512 30 olcDbConfig: {0}set_cachesize 0 67108864 1 olcDbConfig: {1}set_lg_regionmax 262144 olcDbConfig: {2}set_lg_bsize 2097152 olcDbNoSync: FALSE olcDbDirtyRead: FALSE olcDbIDLcacheSize: 0 olcDbIndex: objectClass eq olcDbIndex: krb5PrincipalName eq,pres olcDbIndex: cn eq,pres,subinitial olcDbIndex: mail eq,pres olcDbIndex: uid pres,eq olcDbIndex: uidNumber eq olcDbIndex: gidNumber eq olcDbLinearIndex: FALSE olcDbMode: 0600 olcDbSearchStack: 16 olcDbShmKey: 0 olcDbCacheFree: 1 olcDbDNcacheSize: 0 olcPlugin: postoperation /opt/trinity/lib/slapi-acl-manager.so plugin_init admingroup-dn:=cn=realmadmins,ou=groups,ou=core,ou=realm,dc=<redacted> realm:=CEET.NIU.EDU aclfile:=/etc/heimdal-kdc/kadmind.acl builtinadmin:=admin structuralObjectClass: olcHdbConfig creatorsName: cn=config createTimestamp: 20130702170316Z olcSyncrepl: {0}rid=001 provider=ldaps://ldap002.<redacted>/ binddn="cn=admin,dc=<redacted>" bindmethod=simple credentials="ldapadmin001" searchbase="dc=<redacted>" type=refreshAndPersist retry="5 5 300 5" timeout=1 tls_reqcert=never tls_cacert="/etc/trinity/ldap/tde-ca/anchors/tdeca.pem" olcSyncrepl: {1}rid=002 provider=ldaps://ldap003.<redacted>/ binddn="cn=admin,dc=<redacted>" bindmethod=simple credentials="ldapadmin001" searchbase="dc=<redacted>" type=refreshAndPersist retry="5 5 300 5" timeout=1 tls_reqcert=never tls_cacert="/etc/trinity/ldap/tde-ca/anchors/tdeca.pem" olcMirrorMode: TRUE entryCSN: 20130702170511.039863Z#000000#002#000000 modifiersName: modifyTimestamp: 20130702170511Z olcDatabase={0}config/olcOverlay={0}syncprov.ldif: dn: olcOverlay={0}syncprov objectClass: olcOverlayConfig objectClass: olcSyncProvConfig olcOverlay: {0}syncprov structuralObjectClass: olcSyncProvConfig entryUUID: 14f25934-7785-1032-93bd-0fe5d581c3b6 creatorsName: createTimestamp: 20130702170337Z entryCSN: 20130702170337.631532Z#000000#002#000000 modifiersName: modifyTimestamp: 20130702170337Z olcDatabase={1}hdb/olcOverlay={0}syncprov.ldif: dn: olcOverlay={0}syncprov objectClass: olcOverlayConfig objectClass: olcSyncProvConfig olcOverlay: {0}syncprov structuralObjectClass: olcSyncProvConfig entryUUID: bfa2e530-778d-1032-9b41-e7480f731418 creatorsName: createTimestamp: 20130702180539Z entryCSN: 20130702180539.975057Z#000000#002#000000 modifiersName: modifyTimestamp: 20130702180539Z

1 0

(ITS#7635) lmdb race condition in read vs write txns
by hyc＠OpenLDAP.org 01 Jul '13

01 Jul '13

Full_Name: Howard Chu Version: OS: URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (84.127.144.82) Submitted by: hyc It's possible for a write txn to update a meta page while a reader is creating its snapshot of the meta page. In most cases this situation is harmless, but if the update raises the mm_last_pg field it may later cause an assert if the reader still has the old value of mm_last_pg, but accesses a DB whose pages are beyond the old value. A fix has been tested and will be committed shortly.

1 0

RE: (ITS#7634) openldap 2.4.24 hangs while attempting to shutdown
by madhan.kumar＠alcatel-lucent.com 01 Jul '13

01 Jul '13

Thanks Micheal. -----Original Message----- From: Michael Ströder [mailto:michael@stroeder.com] Sent: Monday, July 01, 2013 1:49 PM To: KUMAR, Madhan (Madhan); openldap-its(a)openldap.org Subject: Re: (ITS#7634) openldap 2.4.24 hangs while attempting to shutdown madhan.kumar(a)alcatel-lucent.com wrote: > We have an issue of openldap 2.4.24 hanging. This is similar to the issue > mentioned in the link: > http://www.openldap.org/lists/openldap-technical/201208/msg00119.html If you read the follow-up in the ML archive Howard says this was fixed in 2.4.25. So could you please try out recent release 2.4.35? Ciao, Michael.

1 0

Re: (ITS#7634) openldap 2.4.24 hangs while attempting to shutdown
by michael＠stroeder.com 01 Jul '13

01 Jul '13

madhan.kumar(a)alcatel-lucent.com wrote: > We have an issue of openldap 2.4.24 hanging. This is similar to the issue > mentioned in the link: > http://www.openldap.org/lists/openldap-technical/201208/msg00119.html If you read the follow-up in the ML archive Howard says this was fixed in 2.4.25. So could you please try out recent release 2.4.35? Ciao, Michael.

1 0

(ITS#7634) openldap 2.4.24 hangs while attempting to shutdown
by madhan.kumar＠alcatel-lucent.com 30 Jun '13

30 Jun '13

Full_Name: Madhan Kumar Version: 2.4.24 OS: linux URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (135.245.192.4) We have an issue of openldap 2.4.24 hanging. This is similar to the issue mentioned in the link: http://www.openldap.org/lists/openldap-technical/201208/msg00119.html When I checked the openldap logs I could see that the listeners are shown as busy continuously and an automatic internal shutdown is initiated to recover the ldap process. Unfortunately in our case the automatic internal shutdown is stuck and hence the issue. daemon: epoll: listen=7 active_threads=0 tvp=zero daemon: epoll: listen=8 busy daemon: epoll: listen=7 active_threads=0 tvp=zero daemon: epoll: listen=8 busy daemon: epoll: listen=7 active_threads=0 tvp=zero daemon: epoll: listen=8 busy daemon: epoll: listen=7 active_threads=0 tvp=zero daemon: epoll: listen=8 busy daemon: epoll: listen=7 active_threads=0 tvp=zero daemon: epoll: listen=8 busy daemon: epoll: listen=7 active_threads=0 tvp=zero daemon: epoll: listen=8 busy daemon: epoll: listen=7 active_threads=0 tvp=zero daemon: epoll: listen=8 busy daemon: shutdown requested and initiated. daemon: closing 7 daemon: closing 8 We need to get an confirmation that this issue is being fixed in openldap 2.4.25 so that we can upgrade our openldap version. Please confirm.

1 0

RE: (ITS#7633) Slapd hangs on hdb write lock
by ck＠cksoft.de 28 Jun '13

28 Jun '13

Hi Dusan, On Fri, 28 Jun 2013, dusan.fric(a)t-systems.sk wrote: > Hi Chu, > > thanks for your quick response. We will try to contact Oracle to advice here. > > However, what is your suggestion here ' try different settings for the deadlock detector' exactly? > > BTW - certainly, we are patched DB 4.7.25 (all 4 patches)... In case you can update to 2.4.35 you might want to test the mdb backend with your workload. Especially if you can easily reproduce the problem. Greetings Christian > > Regards, > Dusan > > -----Original Message----- > From: Howard Chu [mailto:hyc@symas.com] > Sent: 26. j??na 2013 13:32 > To: Fric, Dusan; openldap-its(a)openldap.org > Subject: Re: (ITS#7633) Slapd hangs on hdb write lock > > dusan.fric(a)t-systems.sk wrote: >> Full_Name: Dusan Fric >> Version: 2.4.32 >> OS: RHEL 5.7 x64 >> URL: ftp://ftp.openldap.org/incoming/ >> Submission from: (NULL) (88.212.40.139) >> >> >> We are experiencing frequent hangs in slapd. Once hung we usualy >> cannot continue to connect until we kill -9 the slapd process and >> restart it. The directory is used for 2 applications as user eDir and >> we are using it in production over 1 month - we have noted the busier >> the directory becomes the more often it hangs (now twice per week). >> >> We have installed identical configurations on 3 environments, each has >> only one single server (no replication). There are 30k entries in the >> directory (production). >> >> We are running: >> >> RHEL 5.7 x64 (VMWare with NFS mountpoints) OpenLDAP 2.4.32 Berkeley DB >> 4.8.30 (4.7.25) >> >> We were starting with DB 4.8.30, after downgrade to version 4.7.25 >> (according to >> ITS#7378 - >> https://www.openldap.org/its/index.cgi/Incoming?id=7378;selectid=7378) >> we are facing the same issue. >> >> The problem occurs when two requests simultaneously try to update >> attributes of a record and in most cases of the same DN value. >> We can easily reproduce it with a java test program running 2 threads >> each connecting to the ldap server and updating the record for a >> particular DN value. >> It need not be the same DN value but DN values which reside on the >> same BDB page. > > Thanks for the traces, but the information you've provided shows that this is a BDB problem, not an OpenLDAP bug. You'll have to contact Oracle for help. > The BDB deadlock detector is supposed to handle these situations. > > For example in your 4.8.30 lock status there are only 2 conflicting transactions 80000028 and 80000029 and they are only contending on a single page. This is one of the most elementary deadlock situations. > > You might try different settings for the deadlock detector, but ultimately this appears to be a BDB bug and the choice of detector shouldn't matter. > >> Configuration >> >> olc cn=config (part): >> dn: cn=config >> objectClass: olcGlobal >> cn: config >> olcConcurrency: 0 >> olcConnMaxPending: 100 >> olcConnMaxPendingAuth: 1000 >> olcSockbufMaxIncoming: 262143 >> olcSockbufMaxIncomingAuth: 16777215 >> olcThreads: 32 >> >> olc DB hdb config (part): >> dn: olcDatabase={1}hdb,cn=config >> objectClass: olcHdbConfig >> objectClass: olcDatabaseConfig >> olcDatabase: {1}hdb >> olcDbCacheSize: 80000 >> olcDbCheckpoint: 128 5 >> olcDbConfig: {0}set_cachesize 0 268435456 1 >> olcDbConfig: {1}set_lg_max 10485760 >> olcDbConfig: {2}set_lg_bsize 2097152 >> olcDbConfig: {3}set_lg_dir /pkg/openldap/dblog >> olcDbConfig: {4}set_lg_regionmax 262144 >> olcDbConfig: {5}set_lk_detect DB_LOCK_EXPIRE >> olcDbConfig: {6}set_flags DB_TXN_NOSYNC >> olcDbDirtyRead: FALSE >> olcDbDNcacheSize: 0 >> olcDbIDLcacheSize: 240000 >> olcDbIndex: default eq,sub >> olcDbIndex: objectClass eq >> olcDbIndex: cn eq,sub >> olcDbIndex: uid eq,sub >> olcDbIndex: mail pres,eq >> olcDbIndex: sn eq,sub >> olcDbIndex: member eq >> olcDbLinearIndex: FALSE >> olcDbMode: 0600 >> olcDbNoSync: TRUE >> olcDbSearchStack: 20 >> olcDbShmKey: 0 >> olcLastMod: TRUE >> olcMaxDerefDepth: 15 >> olcMonitoring: TRUE >> olcReadOnly: FALSE >> >> We have managed to collect db_stat lock information, which indicates >> the same issue with DB write locks on both DB versions. >> >> db_stat -C ol (4.8.30) >> =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= >> Lock REGINFO information: >> Lock Region type >> 5 Region ID >> /apps/DECCLASA-1/data/openldap/__db.005 Region name >> 0x2ab41cdc3000 Region address >> 0x2ab41cdc3138 Region primary address >> 0 Region maximum allocation >> 0 Region allocated >> Region allocations: 3006 allocations, 0 failures, 0 frees, 1 longest >> REGION_JOIN_OK Region flags >> =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= >> Locks grouped by lockers: >> Locker Mode Count Status ----------------- Object --------------- >> 1 dd= 0 locks held 1 write locks 0 pid/thread 30034/47555767609232 >> 1 READ 1 HELD id2entry.bdb handle 0 >> 2 dd= 0 locks held 0 write locks 0 pid/thread 30034/47555767609232 >> 3 dd= 0 locks held 1 write locks 0 pid/thread 30034/47555767609232 >> 3 READ 1 HELD dn2id.bdb handle 0 >> 4 dd= 0 locks held 0 write locks 0 pid/thread 30034/1147971904 >> 5 dd= 0 locks held 0 write locks 0 pid/thread 30034/47555767609232 >> 6 dd= 0 locks held 0 write locks 0 pid/thread 30034/1139579200 >> 7 dd= 0 locks held 0 write locks 0 pid/thread 30034/1147971904 >> 8 dd= 0 locks held 0 write locks 0 pid/thread 30034/1131186496 >> 9 dd= 0 locks held 1 write locks 0 pid/thread 30034/1122793792 >> 9 READ 1 HELD objectClass.bdb handle 0 >> a dd= 0 locks held 0 write locks 0 pid/thread 30034/1122793792 >> b dd= 0 locks held 0 write locks 0 pid/thread 30034/1122793792 >> c dd= 0 locks held 1 write locks 0 pid/thread 30034/1122793792 >> c READ 1 HELD uid.bdb handle 0 >> d dd= 0 locks held 0 write locks 0 pid/thread 30034/1122793792 >> e dd= 0 locks held 0 write locks 0 pid/thread 30034/1122793792 >> f dd= 0 locks held 0 write locks 0 pid/thread 30034/1122793792 >> 10 dd= 0 locks held 1 write locks 0 pid/thread 30034/1147971904 >> 10 READ 1 HELD member.bdb handle 0 >> 11 dd= 0 locks held 0 write locks 0 pid/thread 30034/1147971904 >> 12 dd= 0 locks held 0 write locks 0 pid/thread 30034/1147971904 >> 13 dd= 0 locks held 1 write locks 0 pid/thread 30034/1147971904 >> 13 READ 1 HELD mail.bdb handle 0 >> 14 dd= 0 locks held 0 write locks 0 pid/thread 30034/1147971904 >> 15 dd= 0 locks held 0 write locks 0 pid/thread 30034/1147971904 >> 16 dd= 0 locks held 0 write locks 0 pid/thread 30034/1147971904 >> 17 dd= 0 locks held 0 write locks 0 pid/thread 30034/1147971904 >> 18 dd= 0 locks held 1 write locks 0 pid/thread 30034/1147971904 >> 18 READ 1 HELD sn.bdb handle 0 >> 19 dd= 0 locks held 0 write locks 0 pid/thread 30034/1147971904 >> 1a dd= 0 locks held 0 write locks 0 pid/thread 30034/1147971904 >> 1b dd= 0 locks held 1 write locks 0 pid/thread 30034/1147971904 >> 1b READ 1 HELD cn.bdb handle 0 >> 1c dd= 0 locks held 0 write locks 0 pid/thread 30034/1147971904 >> 1d dd= 0 locks held 0 write locks 0 pid/thread 30034/1147971904 >> 1e dd= 0 locks held 0 write locks 0 pid/thread 30034/1122793792 >> 1f dd= 0 locks held 0 write locks 0 pid/thread 30034/1131186496 >> 80000003 dd= 0 locks held 0 write locks 0 pid/thread 30034/47555767609232 >> 80000004 dd= 0 locks held 0 write locks 0 pid/thread 30034/1139579200 >> 80000005 dd= 0 locks held 0 write locks 0 pid/thread 30034/1147971904 >> 80000006 dd= 0 locks held 0 write locks 0 pid/thread 30034/1131186496 >> 80000007 dd= 0 locks held 0 write locks 0 pid/thread 30034/1122793792 >> 80000026 dd= 0 locks held 1 write locks 0 pid/thread 30034/1122793792 >> 80000026 READ 1 HELD 0x5e948 len: 9 data: 0x560x0500000000000000 >> 80000027 dd= 0 locks held 1 write locks 0 pid/thread 30034/1131186496 >> 80000027 READ 1 HELD 0x36490 len: 9 data: 0x370000000000000000 >> 80000028 dd= 0 locks held 1 write locks 0 pid/thread 30034/1131186496 >> 80000028 WRITE 1 WAIT mail.bdb page 3 >> 80000028 READ 1 HELD mail.bdb page 3 >> 80000029 dd= 0 locks held 1 write locks 0 pid/thread 30034/1122793792 >> 80000029 WRITE 1 WAIT mail.bdb page 3 >> 80000029 READ 1 HELD mail.bdb page 3 >> =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= >> Locks grouped by object: >> Locker Mode Count Status ----------------- Object --------------- >> 80000027 READ 1 HELD 0x36490 len: 9 data: 0x370000000000000000 >> >> 1b READ 1 HELD cn.bdb handle 0 >> >> 10 READ 1 HELD member.bdb handle 0 >> >> 9 READ 1 HELD objectClass.bdb handle 0 >> >> c READ 1 HELD uid.bdb handle 0 >> >> 80000026 READ 1 HELD 0x5e948 len: 9 data: 0x560x0500000000000000 >> >> 1 READ 1 HELD id2entry.bdb handle 0 >> >> 18 READ 1 HELD sn.bdb handle 0 >> >> 3 READ 1 HELD dn2id.bdb handle 0 >> >> 80000029 READ 1 HELD mail.bdb page 3 >> 80000028 READ 1 HELD mail.bdb page 3 >> 80000029 WRITE 1 WAIT mail.bdb page 3 >> 80000028 WRITE 1 WAIT mail.bdb page 3 >> >> 13 READ 1 HELD mail.bdb handle 0 >> >> db_stat -C ol (4.7.25) >> =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= >> Lock REGINFO information: >> Lock Region type >> 5 Region ID >> /apps/DECCLASA-1/data/openldap/__db.005 Region name >> 0x2b4a8a577000 Original region address >> 0x2b4a8a577000 Region address >> 0x2b4a8a577138 Region primary address >> 0 Region maximum allocation >> 0 Region allocated >> Region allocations: 3006 allocations, 0 failures, 0 frees, 1 longest >> REGION_JOIN_OK Region flags >> =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= >> Locks grouped by lockers: >> Locker Mode Count Status ----------------- Object --------------- >> 1 dd= 0 locks held 1 write locks 0 pid/thread 32447/47463753631632 >> 1 READ 1 HELD id2entry.bdb handle 0 >> 2 dd= 0 locks held 0 write locks 0 pid/thread 32447/47463753631632 >> 3 dd= 0 locks held 1 write locks 0 pid/thread 32447/47463753631632 >> 3 READ 1 HELD dn2id.bdb handle 0 >> 4 dd= 0 locks held 0 write locks 0 pid/thread 32447/1114126656 >> 5 dd= 0 locks held 0 write locks 0 pid/thread 32447/47463753631632 >> 6 dd= 0 locks held 0 write locks 0 pid/thread 32447/1097341248 >> 7 dd= 0 locks held 1 write locks 0 pid/thread 32447/1097341248 >> 7 READ 1 HELD objectClass.bdb handle 0 >> 8 dd= 0 locks held 0 write locks 0 pid/thread 32447/1097341248 >> 9 dd= 0 locks held 0 write locks 0 pid/thread 32447/1097341248 >> a dd= 0 locks held 0 write locks 0 pid/thread 32447/1097341248 >> b dd= 0 locks held 0 write locks 0 pid/thread 32447/1097341248 >> c dd= 0 locks held 1 write locks 0 pid/thread 32447/1097341248 >> c READ 1 HELD cn.bdb handle 0 >> d dd= 0 locks held 0 write locks 0 pid/thread 32447/1097341248 >> e dd= 0 locks held 0 write locks 0 pid/thread 32447/1097341248 >> f dd= 0 locks held 1 write locks 0 pid/thread 32447/1097341248 >> f READ 1 HELD uid.bdb handle 0 >> 10 dd= 0 locks held 0 write locks 0 pid/thread 32447/1097341248 >> 11 dd= 0 locks held 0 write locks 0 pid/thread 32447/1097341248 >> 12 dd= 0 locks held 1 write locks 0 pid/thread 32447/1097341248 >> 12 READ 1 HELD member.bdb handle 0 >> 13 dd= 0 locks held 0 write locks 0 pid/thread 32447/1097341248 >> 14 dd= 0 locks held 0 write locks 0 pid/thread 32447/1097341248 >> 15 dd= 0 locks held 1 write locks 0 pid/thread 32447/1105733952 >> 15 READ 1 HELD mail.bdb handle 0 >> 16 dd= 0 locks held 0 write locks 0 pid/thread 32447/1105733952 >> 17 dd= 0 locks held 0 write locks 0 pid/thread 32447/1105733952 >> 18 dd= 0 locks held 0 write locks 0 pid/thread 32447/1105733952 >> 19 dd= 0 locks held 0 write locks 0 pid/thread 32447/1105733952 >> 1a dd= 0 locks held 1 write locks 0 pid/thread 32447/1105733952 >> 1a READ 1 HELD sn.bdb handle 0 >> 1b dd= 0 locks held 0 write locks 0 pid/thread 32447/1105733952 >> 1c dd= 0 locks held 0 write locks 0 pid/thread 32447/1105733952 >> 1d dd= 0 locks held 0 write locks 0 pid/thread 32447/1105733952 >> 1e dd= 0 locks held 0 write locks 0 pid/thread 32447/1105733952 >> 1f dd= 0 locks held 0 write locks 0 pid/thread 32447/1097341248 >> 80000003 dd= 0 locks held 0 write locks 0 pid/thread 32447/47463753631632 >> 80000004 dd= 0 locks held 0 write locks 0 pid/thread 32447/1097341248 >> 80000008 dd= 0 locks held 0 write locks 0 pid/thread 32447/1105733952 >> 80000038 dd= 0 locks held 0 write locks 0 pid/thread 32447/1114126656 >> 800000a9 dd= 0 locks held 1 write locks 0 pid/thread 32447/1097341248 >> 800000a9 READ 1 HELD 0x2d228 len: 9 data: 0x370000000000000000 >> 800000aa dd= 0 locks held 2 write locks 0 pid/thread 32447/1097341248 >> 800000aa WRITE 1 WAIT mail.bdb page 3 >> 800000aa READ 1 HELD mail.bdb page 1 >> 800000aa READ 1 HELD mail.bdb page 3 >> 800000ab dd= 0 locks held 1 write locks 0 pid/thread 32447/1105733952 >> 800000ab READ 1 HELD 0x43d58 len: 9 data: 0x540x0500000000000000 >> 800000ac dd= 0 locks held 2 write locks 0 pid/thread 32447/1105733952 >> 800000ac WRITE 1 WAIT mail.bdb page 3 >> 800000ac READ 1 HELD mail.bdb page 1 >> 800000ac READ 1 HELD mail.bdb page 3 >> 800000b1 dd= 0 locks held 0 write locks 0 pid/thread 32447/1122519360 >> 800000b2 dd= 0 locks held 0 write locks 0 pid/thread 32447/1130912064 >> =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= >> Locks grouped by object: >> Locker Mode Count Status ----------------- Object --------------- >> 800000a9 READ 1 HELD 0x2d228 len: 9 data: 0x370000000000000000 >> >> 1 READ 1 HELD id2entry.bdb handle 0 >> >> 3 READ 1 HELD dn2id.bdb handle 0 >> >> 7 READ 1 HELD objectClass.bdb handle 0 >> >> f READ 1 HELD uid.bdb handle 0 >> >> 1a READ 1 HELD sn.bdb handle 0 >> >> 800000aa READ 1 HELD mail.bdb page 3 >> 800000ac READ 1 HELD mail.bdb page 3 >> 800000aa WRITE 1 WAIT mail.bdb page 3 >> 800000ac WRITE 1 WAIT mail.bdb page 3 >> >> c READ 1 HELD cn.bdb handle 0 >> >> 15 READ 1 HELD mail.bdb handle 0 >> >> 800000aa READ 1 HELD mail.bdb page 1 >> 800000ac READ 1 HELD mail.bdb page 1 >> >> 12 READ 1 HELD member.bdb handle 0 >> >> 800000ab READ 1 HELD 0x43d58 len: 9 data: 0x540x0500000000000000 >> >> >> We have also collected the backtrace of threads for both DB versions >> which I have uploaded to: >> https://dl.dropboxusercontent.com/u/92115703/dusan_gdb_threads_4.8.30_ >> 20130625.txt >> >> https://dl.dropboxusercontent.com/u/92115703/dusan_gdb_threads_4.7.25_ >> 20130625.txt >> >> > > > -- > -- Howard Chu > CTO, Symas Corp. http://www.symas.com > Director, Highland Sun http://highlandsun.com/hyc/ > Chief Architect, OpenLDAP http://www.openldap.org/project/ > > > -- Christian Kratzer CK Software GmbH Email: ck(a)cksoft.de Wildberger Weg 24/2 Phone: +49 7032 893 997 - 0 D-71126 Gaeufelden Fax: +49 7032 893 997 - 9 HRB 245288, Amtsgericht Stuttgart Web: http://www.cksoft.de/ Geschaeftsfuehrer: Christian Kratzer

1 0

RE: (ITS#7633) Slapd hangs on hdb write lock
by dusan.fric＠t-systems.sk 28 Jun '13

28 Jun '13

Hi Chu, thanks for your quick response. We will try to contact Oracle to advice here. However, what is your suggestion here ' try different settings for the deadlock detector' exactly? BTW - certainly, we are patched DB 4.7.25 (all 4 patches)... Regards, Dusan -----Original Message----- From: Howard Chu [mailto:hyc@symas.com] Sent: 26. júna 2013 13:32 To: Fric, Dusan; openldap-its(a)openldap.org Subject: Re: (ITS#7633) Slapd hangs on hdb write lock dusan.fric(a)t-systems.sk wrote: > Full_Name: Dusan Fric > Version: 2.4.32 > OS: RHEL 5.7 x64 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (88.212.40.139) > > > We are experiencing frequent hangs in slapd. Once hung we usualy > cannot continue to connect until we kill -9 the slapd process and > restart it. The directory is used for 2 applications as user eDir and > we are using it in production over 1 month - we have noted the busier > the directory becomes the more often it hangs (now twice per week). > > We have installed identical configurations on 3 environments, each has > only one single server (no replication). There are 30k entries in the > directory (production). > > We are running: > > RHEL 5.7 x64 (VMWare with NFS mountpoints) OpenLDAP 2.4.32 Berkeley DB > 4.8.30 (4.7.25) > > We were starting with DB 4.8.30, after downgrade to version 4.7.25 > (according to > ITS#7378 - > https://www.openldap.org/its/index.cgi/Incoming?id=7378;selectid=7378) > we are facing the same issue. > > The problem occurs when two requests simultaneously try to update > attributes of a record and in most cases of the same DN value. > We can easily reproduce it with a java test program running 2 threads > each connecting to the ldap server and updating the record for a > particular DN value. > It need not be the same DN value but DN values which reside on the > same BDB page. Thanks for the traces, but the information you've provided shows that this is a BDB problem, not an OpenLDAP bug. You'll have to contact Oracle for help. The BDB deadlock detector is supposed to handle these situations. For example in your 4.8.30 lock status there are only 2 conflicting transactions 80000028 and 80000029 and they are only contending on a single page. This is one of the most elementary deadlock situations. You might try different settings for the deadlock detector, but ultimately this appears to be a BDB bug and the choice of detector shouldn't matter. > Configuration > > olc cn=config (part): > dn: cn=config > objectClass: olcGlobal > cn: config > olcConcurrency: 0 > olcConnMaxPending: 100 > olcConnMaxPendingAuth: 1000 > olcSockbufMaxIncoming: 262143 > olcSockbufMaxIncomingAuth: 16777215 > olcThreads: 32 > > olc DB hdb config (part): > dn: olcDatabase={1}hdb,cn=config > objectClass: olcHdbConfig > objectClass: olcDatabaseConfig > olcDatabase: {1}hdb > olcDbCacheSize: 80000 > olcDbCheckpoint: 128 5 > olcDbConfig: {0}set_cachesize 0 268435456 1 > olcDbConfig: {1}set_lg_max 10485760 > olcDbConfig: {2}set_lg_bsize 2097152 > olcDbConfig: {3}set_lg_dir /pkg/openldap/dblog > olcDbConfig: {4}set_lg_regionmax 262144 > olcDbConfig: {5}set_lk_detect DB_LOCK_EXPIRE > olcDbConfig: {6}set_flags DB_TXN_NOSYNC > olcDbDirtyRead: FALSE > olcDbDNcacheSize: 0 > olcDbIDLcacheSize: 240000 > olcDbIndex: default eq,sub > olcDbIndex: objectClass eq > olcDbIndex: cn eq,sub > olcDbIndex: uid eq,sub > olcDbIndex: mail pres,eq > olcDbIndex: sn eq,sub > olcDbIndex: member eq > olcDbLinearIndex: FALSE > olcDbMode: 0600 > olcDbNoSync: TRUE > olcDbSearchStack: 20 > olcDbShmKey: 0 > olcLastMod: TRUE > olcMaxDerefDepth: 15 > olcMonitoring: TRUE > olcReadOnly: FALSE > > We have managed to collect db_stat lock information, which indicates > the same issue with DB write locks on both DB versions. > > db_stat -C ol (4.8.30) > =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= > Lock REGINFO information: > Lock Region type > 5 Region ID > /apps/DECCLASA-1/data/openldap/__db.005 Region name > 0x2ab41cdc3000 Region address > 0x2ab41cdc3138 Region primary address > 0 Region maximum allocation > 0 Region allocated > Region allocations: 3006 allocations, 0 failures, 0 frees, 1 longest > REGION_JOIN_OK Region flags > =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= > Locks grouped by lockers: > Locker Mode Count Status ----------------- Object --------------- > 1 dd= 0 locks held 1 write locks 0 pid/thread 30034/47555767609232 > 1 READ 1 HELD id2entry.bdb handle 0 > 2 dd= 0 locks held 0 write locks 0 pid/thread 30034/47555767609232 > 3 dd= 0 locks held 1 write locks 0 pid/thread 30034/47555767609232 > 3 READ 1 HELD dn2id.bdb handle 0 > 4 dd= 0 locks held 0 write locks 0 pid/thread 30034/1147971904 > 5 dd= 0 locks held 0 write locks 0 pid/thread 30034/47555767609232 > 6 dd= 0 locks held 0 write locks 0 pid/thread 30034/1139579200 > 7 dd= 0 locks held 0 write locks 0 pid/thread 30034/1147971904 > 8 dd= 0 locks held 0 write locks 0 pid/thread 30034/1131186496 > 9 dd= 0 locks held 1 write locks 0 pid/thread 30034/1122793792 > 9 READ 1 HELD objectClass.bdb handle 0 > a dd= 0 locks held 0 write locks 0 pid/thread 30034/1122793792 > b dd= 0 locks held 0 write locks 0 pid/thread 30034/1122793792 > c dd= 0 locks held 1 write locks 0 pid/thread 30034/1122793792 > c READ 1 HELD uid.bdb handle 0 > d dd= 0 locks held 0 write locks 0 pid/thread 30034/1122793792 > e dd= 0 locks held 0 write locks 0 pid/thread 30034/1122793792 > f dd= 0 locks held 0 write locks 0 pid/thread 30034/1122793792 > 10 dd= 0 locks held 1 write locks 0 pid/thread 30034/1147971904 > 10 READ 1 HELD member.bdb handle 0 > 11 dd= 0 locks held 0 write locks 0 pid/thread 30034/1147971904 > 12 dd= 0 locks held 0 write locks 0 pid/thread 30034/1147971904 > 13 dd= 0 locks held 1 write locks 0 pid/thread 30034/1147971904 > 13 READ 1 HELD mail.bdb handle 0 > 14 dd= 0 locks held 0 write locks 0 pid/thread 30034/1147971904 > 15 dd= 0 locks held 0 write locks 0 pid/thread 30034/1147971904 > 16 dd= 0 locks held 0 write locks 0 pid/thread 30034/1147971904 > 17 dd= 0 locks held 0 write locks 0 pid/thread 30034/1147971904 > 18 dd= 0 locks held 1 write locks 0 pid/thread 30034/1147971904 > 18 READ 1 HELD sn.bdb handle 0 > 19 dd= 0 locks held 0 write locks 0 pid/thread 30034/1147971904 > 1a dd= 0 locks held 0 write locks 0 pid/thread 30034/1147971904 > 1b dd= 0 locks held 1 write locks 0 pid/thread 30034/1147971904 > 1b READ 1 HELD cn.bdb handle 0 > 1c dd= 0 locks held 0 write locks 0 pid/thread 30034/1147971904 > 1d dd= 0 locks held 0 write locks 0 pid/thread 30034/1147971904 > 1e dd= 0 locks held 0 write locks 0 pid/thread 30034/1122793792 > 1f dd= 0 locks held 0 write locks 0 pid/thread 30034/1131186496 > 80000003 dd= 0 locks held 0 write locks 0 pid/thread 30034/47555767609232 > 80000004 dd= 0 locks held 0 write locks 0 pid/thread 30034/1139579200 > 80000005 dd= 0 locks held 0 write locks 0 pid/thread 30034/1147971904 > 80000006 dd= 0 locks held 0 write locks 0 pid/thread 30034/1131186496 > 80000007 dd= 0 locks held 0 write locks 0 pid/thread 30034/1122793792 > 80000026 dd= 0 locks held 1 write locks 0 pid/thread 30034/1122793792 > 80000026 READ 1 HELD 0x5e948 len: 9 data: 0x560x0500000000000000 > 80000027 dd= 0 locks held 1 write locks 0 pid/thread 30034/1131186496 > 80000027 READ 1 HELD 0x36490 len: 9 data: 0x370000000000000000 > 80000028 dd= 0 locks held 1 write locks 0 pid/thread 30034/1131186496 > 80000028 WRITE 1 WAIT mail.bdb page 3 > 80000028 READ 1 HELD mail.bdb page 3 > 80000029 dd= 0 locks held 1 write locks 0 pid/thread 30034/1122793792 > 80000029 WRITE 1 WAIT mail.bdb page 3 > 80000029 READ 1 HELD mail.bdb page 3 > =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= > Locks grouped by object: > Locker Mode Count Status ----------------- Object --------------- > 80000027 READ 1 HELD 0x36490 len: 9 data: 0x370000000000000000 > > 1b READ 1 HELD cn.bdb handle 0 > > 10 READ 1 HELD member.bdb handle 0 > > 9 READ 1 HELD objectClass.bdb handle 0 > > c READ 1 HELD uid.bdb handle 0 > > 80000026 READ 1 HELD 0x5e948 len: 9 data: 0x560x0500000000000000 > > 1 READ 1 HELD id2entry.bdb handle 0 > > 18 READ 1 HELD sn.bdb handle 0 > > 3 READ 1 HELD dn2id.bdb handle 0 > > 80000029 READ 1 HELD mail.bdb page 3 > 80000028 READ 1 HELD mail.bdb page 3 > 80000029 WRITE 1 WAIT mail.bdb page 3 > 80000028 WRITE 1 WAIT mail.bdb page 3 > > 13 READ 1 HELD mail.bdb handle 0 > > db_stat -C ol (4.7.25) > =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= > Lock REGINFO information: > Lock Region type > 5 Region ID > /apps/DECCLASA-1/data/openldap/__db.005 Region name > 0x2b4a8a577000 Original region address > 0x2b4a8a577000 Region address > 0x2b4a8a577138 Region primary address > 0 Region maximum allocation > 0 Region allocated > Region allocations: 3006 allocations, 0 failures, 0 frees, 1 longest > REGION_JOIN_OK Region flags > =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= > Locks grouped by lockers: > Locker Mode Count Status ----------------- Object --------------- > 1 dd= 0 locks held 1 write locks 0 pid/thread 32447/47463753631632 > 1 READ 1 HELD id2entry.bdb handle 0 > 2 dd= 0 locks held 0 write locks 0 pid/thread 32447/47463753631632 > 3 dd= 0 locks held 1 write locks 0 pid/thread 32447/47463753631632 > 3 READ 1 HELD dn2id.bdb handle 0 > 4 dd= 0 locks held 0 write locks 0 pid/thread 32447/1114126656 > 5 dd= 0 locks held 0 write locks 0 pid/thread 32447/47463753631632 > 6 dd= 0 locks held 0 write locks 0 pid/thread 32447/1097341248 > 7 dd= 0 locks held 1 write locks 0 pid/thread 32447/1097341248 > 7 READ 1 HELD objectClass.bdb handle 0 > 8 dd= 0 locks held 0 write locks 0 pid/thread 32447/1097341248 > 9 dd= 0 locks held 0 write locks 0 pid/thread 32447/1097341248 > a dd= 0 locks held 0 write locks 0 pid/thread 32447/1097341248 > b dd= 0 locks held 0 write locks 0 pid/thread 32447/1097341248 > c dd= 0 locks held 1 write locks 0 pid/thread 32447/1097341248 > c READ 1 HELD cn.bdb handle 0 > d dd= 0 locks held 0 write locks 0 pid/thread 32447/1097341248 > e dd= 0 locks held 0 write locks 0 pid/thread 32447/1097341248 > f dd= 0 locks held 1 write locks 0 pid/thread 32447/1097341248 > f READ 1 HELD uid.bdb handle 0 > 10 dd= 0 locks held 0 write locks 0 pid/thread 32447/1097341248 > 11 dd= 0 locks held 0 write locks 0 pid/thread 32447/1097341248 > 12 dd= 0 locks held 1 write locks 0 pid/thread 32447/1097341248 > 12 READ 1 HELD member.bdb handle 0 > 13 dd= 0 locks held 0 write locks 0 pid/thread 32447/1097341248 > 14 dd= 0 locks held 0 write locks 0 pid/thread 32447/1097341248 > 15 dd= 0 locks held 1 write locks 0 pid/thread 32447/1105733952 > 15 READ 1 HELD mail.bdb handle 0 > 16 dd= 0 locks held 0 write locks 0 pid/thread 32447/1105733952 > 17 dd= 0 locks held 0 write locks 0 pid/thread 32447/1105733952 > 18 dd= 0 locks held 0 write locks 0 pid/thread 32447/1105733952 > 19 dd= 0 locks held 0 write locks 0 pid/thread 32447/1105733952 > 1a dd= 0 locks held 1 write locks 0 pid/thread 32447/1105733952 > 1a READ 1 HELD sn.bdb handle 0 > 1b dd= 0 locks held 0 write locks 0 pid/thread 32447/1105733952 > 1c dd= 0 locks held 0 write locks 0 pid/thread 32447/1105733952 > 1d dd= 0 locks held 0 write locks 0 pid/thread 32447/1105733952 > 1e dd= 0 locks held 0 write locks 0 pid/thread 32447/1105733952 > 1f dd= 0 locks held 0 write locks 0 pid/thread 32447/1097341248 > 80000003 dd= 0 locks held 0 write locks 0 pid/thread 32447/47463753631632 > 80000004 dd= 0 locks held 0 write locks 0 pid/thread 32447/1097341248 > 80000008 dd= 0 locks held 0 write locks 0 pid/thread 32447/1105733952 > 80000038 dd= 0 locks held 0 write locks 0 pid/thread 32447/1114126656 > 800000a9 dd= 0 locks held 1 write locks 0 pid/thread 32447/1097341248 > 800000a9 READ 1 HELD 0x2d228 len: 9 data: 0x370000000000000000 > 800000aa dd= 0 locks held 2 write locks 0 pid/thread 32447/1097341248 > 800000aa WRITE 1 WAIT mail.bdb page 3 > 800000aa READ 1 HELD mail.bdb page 1 > 800000aa READ 1 HELD mail.bdb page 3 > 800000ab dd= 0 locks held 1 write locks 0 pid/thread 32447/1105733952 > 800000ab READ 1 HELD 0x43d58 len: 9 data: 0x540x0500000000000000 > 800000ac dd= 0 locks held 2 write locks 0 pid/thread 32447/1105733952 > 800000ac WRITE 1 WAIT mail.bdb page 3 > 800000ac READ 1 HELD mail.bdb page 1 > 800000ac READ 1 HELD mail.bdb page 3 > 800000b1 dd= 0 locks held 0 write locks 0 pid/thread 32447/1122519360 > 800000b2 dd= 0 locks held 0 write locks 0 pid/thread 32447/1130912064 > =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= > Locks grouped by object: > Locker Mode Count Status ----------------- Object --------------- > 800000a9 READ 1 HELD 0x2d228 len: 9 data: 0x370000000000000000 > > 1 READ 1 HELD id2entry.bdb handle 0 > > 3 READ 1 HELD dn2id.bdb handle 0 > > 7 READ 1 HELD objectClass.bdb handle 0 > > f READ 1 HELD uid.bdb handle 0 > > 1a READ 1 HELD sn.bdb handle 0 > > 800000aa READ 1 HELD mail.bdb page 3 > 800000ac READ 1 HELD mail.bdb page 3 > 800000aa WRITE 1 WAIT mail.bdb page 3 > 800000ac WRITE 1 WAIT mail.bdb page 3 > > c READ 1 HELD cn.bdb handle 0 > > 15 READ 1 HELD mail.bdb handle 0 > > 800000aa READ 1 HELD mail.bdb page 1 > 800000ac READ 1 HELD mail.bdb page 1 > > 12 READ 1 HELD member.bdb handle 0 > > 800000ab READ 1 HELD 0x43d58 len: 9 data: 0x540x0500000000000000 > > > We have also collected the backtrace of threads for both DB versions > which I have uploaded to: > https://dl.dropboxusercontent.com/u/92115703/dusan_gdb_threads_4.8.30_ > 20130625.txt > > https://dl.dropboxusercontent.com/u/92115703/dusan_gdb_threads_4.7.25_ > 20130625.txt > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7377) Poor libmdb error handling
by h.b.furuseth＠usit.uio.no 26 Jun '13

26 Jun '13

I wrote: > at > least MDB_KEYEXIST can happen after touching pages, which affects > the txn size even if the change "does nothing". Only when updating subDBs, I think. -- Hallvard

1 0

Re: (ITS#7377) Poor libmdb error handling
by h.b.furuseth＠usit.uio.no 26 Jun '13

26 Jun '13

Howard Chu writes: > h.b.furuseth(a)usit.uio.no wrote: >> Changes from a failed liblmdb operation can be visible. Sometimes >> the resulting DB will also be inconsistent. The operation should >> invalidate the affected transaction/cursors or revert the change. > > The transaction should be invalidated. Nothing more can be done > once any error occurs in a transaction. That would break back-mdb for MDB_NOTFOUND/MDB_KEYEXIST. And at least MDB_KEYEXIST can happen after touching pages, which affects the txn size even if the change "does nothing". I expected to generally invalidate except in necessary cases, and from then on make more cases "gentle" at our convenience without spending much time on it. >> put(MDB_MULTIPLE) can write some of the provided items and then >> fail, but be able to keep the txn usable. (...) > > No, "partial success" is not allowed. A transaction is atomic, all > or nothing. It would be atomic like a partial OS write(). It'd be up to the user what to do next. Not that I care myself if it is supported. >> A failed txn could still grow the txn size, unless MDB "un-touches" >> pages which memcmp says have not changed, rewinds me_pglast and >> me_pghead, etc. Which seems a lot of work, likely not worth the >> trouble. > > pghead/pglast should not persist into the environment until a successful > commit. Until then any error should just discard them. Sorry, I meant a failed _operation_ could grow them and thus affect the committed freelist size. >> I think cursors need a C_INVALID flag, so future ops cannot give a >> surprising success result instead of failing: get(MDB_NEXT/MDB_PREV) >> take ~C_INITIALIZED to mean (re)start from the beginning/end. > > Good idea. ...or C_VALID so a single OR/AND can set/clear them both. >> Don't know how often reverting is a relevant option. (...) > > Reverting is irrelevant. Once a txn is aborted no changes are visible. True, but I meant while the txn is still live and usable. Hopefully it's only relevant for "nice to have" cases. -- Hallvard

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs