I'm running OpenLDAP 2.4.17 with 3 servers configured in multi-master mode (server 1 replicates to server 2 and server 3, server 2 replicates to server 1 and server 3, and server 3 replicates to server 1 and server 2). While executing shell scripts to test replication, all 3 slapd instances began returning error code 32 to all requests. No information could be retrieved from any of the 3 LDAP servers using ldapsearch, Apache Directory Studio, LDAP Administrator tool, or the LDAP Browser tool. The slapd log records with debug level = sync contained numerous "findbase failed! 32" errors. Starting and restarting the slapd instances still resulted in error code 32 being returned for any LDAP request. I had to restore the LDAP databases to be able to view/update records in any of the 3 LDAP databases.
Server 1 configuration snippet: overlay syncprov syncprov-checkpoint 100 15 syncprov-sessionlog 5000
syncrepl rid=001 provider=ldap://localhost:3892 type=refreshAndPersist retry="60 60 300 +" searchbase="dc=authentx" scope=sub schemachecking=off bindmethod=simple binddn="cn=xroot,dc=authentx" credentials="980170contact"
syncrepl rid=002 provider=ldap://localhost:3893 type=refreshAndPersist retry="60 60 300 +" searchbase="dc=authentx" scope=sub schemachecking=off bindmethod=simple binddn="cn=xroot,dc=authentx" credentials="980170contact"
mirrormode on
Server 2 configuration snippet: overlay syncprov syncprov-checkpoint 100 15 syncprov-sessionlog 5000
syncrepl rid=001 provider=ldap://localhost:3891 type=refreshAndPersist retry="60 60 300 +" searchbase="dc=authentx" scope=sub schemachecking=off bindmethod=simple binddn="cn=xroot,dc=authentx" credentials="980170contact"
syncrepl rid=002 provider=ldap://localhost:3893 type=refreshAndPersist retry="60 60 300 +" searchbase="dc=authentx" scope=sub schemachecking=off bindmethod=simple binddn="cn=xroot,dc=authentx" credentials="980170contact"
mirrormode on
Server 3 configuration snippet: overlay syncprov syncprov-checkpoint 100 15 syncprov-sessionlog 5000
syncrepl rid=001 provider=ldap://localhost:3891 type=refreshAndPersist retry="60 60 300 +" searchbase="dc=authentx" scope=sub schemachecking=off bindmethod=simple binddn="cn=xroot,dc=authentx" credentials="980170contact"
syncrepl rid=002 provider=ldap://localhost:3892 type=refreshAndPersist retry="60 60 300 +" searchbase="dc=authentx" scope=sub schemachecking=off bindmethod=simple binddn="cn=xroot,dc=authentx" credentials="980170contact"
mirrormode on
Snippets from log file from server 1 around the slapd error 32 problem time: syncprov_sendresp: to=002, cookie=rid=001,sid=001,csn=20090811143737.466654Z#000000#001#000000 syncprov_sendresp: to=003, cookie=rid=001,sid=001,csn=20090811143737.466654Z#000000#001#000000 slap_queue_csn: queing 0xa2452d98 20090811143737.567781Z#000000#001#000000 syncprov_sendresp: to=002, cookie=rid=001,sid=001,csn=20090811143737.567781Z#000000#001#000000 slap_graduate_commit_csn: removing 0xa17098a8 20090811143737.567781Z#000000#001#000000 syncprov_sendresp: to=003, cookie=rid=001,sid=001,csn=20090811143737.567781Z#000000#001#000000 findbase failed! 32 findbase failed! 32 findbase failed! 32 findbase failed! 32 findbase failed! 32 findbase failed! 32 findbase failed! 32 findbase failed! 32 findbase failed! 32 findbase failed! 32 findbase failed! 32 findbase failed! 32 findbase failed! 32 findbase failed! 32 findbase failed! 32 findbase failed! 32 connection_input: conn=456 deferring operation: binding findbase failed! 32 findbase failed! 32 findbase failed! 32 findbase failed! 32 findbase failed! 32 findbase failed! 32 connection_input: conn=461 deferring operation: binding
Snippets from log file from server 2 around the slapd error 32 problem time: slap_queue_csn: queing 0xa1709520 20090811143737.567781Z#000000#001#000000 slap_graduate_commit_csn: removing 0xa1709650 20090811143737.567781Z#000000#001#000000 syncrepl_updateCookie: rid=001 be_modify failed (32) do_syncrepl: rid=001 rc 32 retrying (59 retries left) syncprov_sendresp: to=003, cookie=rid=002,sid=002,csn=20090811143737.567781Z#000000#001#000000 do_syncrep2: cookie=rid=002,sid=003,csn=20090811143737.567781Z#000000#001#000000 syncrepl_entry: rid=002 LDAP_RES_SEARCH_ENTRY(LDAP_SYNC_DELETE) syncrepl_entry: rid=002 be_search (32) syncrepl_entry: rid=002 (null) slap_queue_csn: queing 0xa1714320 20090811143737.567781Z#000000#001#000000 slap_graduate_commit_csn: removing 0xa1716558 20090811143737.567781Z#000000#001#000000 do_syncrepl: rid=002 rc 32 retrying (59 retries left) findbase failed! 32 do_syncrep2: rid=001 LDAP_RES_SEARCH_RESULT do_syncrep2: rid=001 LDAP_RES_SEARCH_RESULT (32) No such object do_syncrep2: rid=001 (32) No such object do_syncrepl: rid=001 rc -2 retrying (59 retries left) do_syncrep2: rid=002 LDAP_RES_SEARCH_RESULT do_syncrep2: rid=002 LDAP_RES_SEARCH_RESULT (32) No such object do_syncrep2: rid=002 (32) No such object do_syncrepl: rid=002 rc -2 retrying (59 retries left) findbase failed! 32
Snippets from log file from server 3 around the slapd error 32 problem time: slap_graduate_commit_csn: removing 0x835e498 20090811143737.466654Z#000000#001#000000 do_syncrep2: cookie=rid=001,sid=001,csn=20090811143737.567781Z#000000#001#000000 syncrepl_entry: rid=001 LDAP_RES_SEARCH_ENTRY(LDAP_SYNC_DELETE) syncrepl_entry: rid=001 be_search (0) syncrepl_entry: rid=001 dc=authentx slap_queue_csn: queing 0xa160ee80 20090811143737.567781Z#000000#001#000000 syncprov_matchops: skipping original sid 001 slap_graduate_commit_csn: removing 0xa16134f8 20090811143737.567781Z#000000#001#000000 syncrepl_entry: rid=001 be_delete dc=authentx (0) slap_queue_csn: queing 0xa160ee80 20090811143737.567781Z#000000#001#000000 slap_graduate_commit_csn: removing 0xa16134f8 20090811143737.567781Z#000000#001#000000 syncrepl_updateCookie: rid=001 be_modify failed (32) do_syncrepl: rid=001 rc 32 retrying (59 retries left) syncprov_sendresp: to=002, cookie=rid=002,sid=003,csn=20090811143737.567781Z#000000#001#000000 do_syncrep2: cookie=rid=002,sid=002,csn=20090811143737.567781Z#000000#001#000000 syncrepl_entry: rid=002 LDAP_RES_SEARCH_ENTRY(LDAP_SYNC_DELETE) syncrepl_entry: rid=002 be_search (32) syncrepl_entry: rid=002 (null) slap_queue_csn: queing 0x8338688 20090811143737.567781Z#000000#001#000000 slap_graduate_commit_csn: removing 0x8360318 20090811143737.567781Z#000000#001#000000 do_syncrepl: rid=002 rc 32 retrying (59 retries left) do_syncrep2: rid=002 LDAP_RES_SEARCH_RESULT do_syncrep2: rid=002 LDAP_RES_SEARCH_RESULT (32) No such object do_syncrep2: rid=002 (32) No such object do_syncrepl: rid=002 rc -2 retrying (59 retries left) findbase failed! 32
I have been unable to reproduce this problem. Any suggestions on how to prevent this problem from occurring and how to recover when this problem occurs?
Thanks, Barry Colston