Hello All,
Running OpenLDAP version 2.4.58 on Ubuntu 20.04 in multicaster mode.
Replication Configuration:
olcSyncrepl: {0}rid=101 provider=ldaps://<host1>:10636 binddn="<binddn>" bindm ethod=simple credentials=<creds> searchbase="dc=company,dc=com" type =refreshAndPersist interval=00:00:00:30 retry="5 5 60 +" timeout=1 keepalive= "240:10:30" olcSyncrepl: {1}rid=102 provider=ldaps://<host2>:10636 binddn="<binddn>" bindm ethod=simple credentials=<creds> searchbase="dc=company,dc=com" type =refreshAndPersist interval=00:00:00:30 retry="5 5 60 +" timeout=1 keepalive= "240:10:30" olcSyncrepl: {2}rid=103 provider=ldaps://<host3>:10636 binddn="<binddn>" bindm ethod=simple credentials=<creds> searchbase="dc=company,dc=com" type =refreshAndPersist interval=00:00:00:30 retry="5 5 60 +" timeout=1 keepalive= "240:10:30" olcSyncrepl: {3}rid=104 provider=ldaps://<host4>:636 binddn="<binddn>" bindmethod=si mple credentials=<creds> searchbase="dc=company,dc=com" type=refresh AndPersist interval=00:00:00:30 retry="5 5 60 +" timeout=1 keepalive="240:10: 30" olcMirrorMode: TRUE
# {1}syncprov, {2}mdb, config dn: olcOverlay={1}syncprov,olcDatabase={2}mdb,cn=config objectClass: olcOverlayConfig objectClass: olcSyncProvConfig olcOverlay: {1}syncprov olcSpCheckpoint: 100 10
Under normal conditions, all operations perform normally in terms of replication consistency. However, when performing a stream of “ADD” operations eventually producer/consumer go out of sync where consumer thinks it has latest state than producer. All the write operations including ADD are pointed to a producer node (not load balancing the writes). Introduced 30 second delay to “ADD” operations (for the sake of testing) and experiencing the same behavior. The issue is manifesting after adding around 4k entries (sometimes even sooner).
Here are the log snippets.
May 4 23:06:47 openldap-service-0 slapd[199]: do_syncrep2: rid=102 LDAP_RES_SEARCH_RESULT (53) Server is unwilling to perform
May 4 23:06:47 openldap-service-0 slapd[199]: do_syncrep2: rid=102 (53) Server is unwilling to perform
May 4 23:06:37 openldap-service-1 slapd[199]: conn=1046 op=1 SEARCH RESULT tag=101 err=53 nentries=0 text=consumer state is newer than provider!
Checked for the clock synchronization and verified that all instances have NTP enabled. Wondering where and what I should be looking for.
Thanks in advance.
--On Friday, May 6, 2022 9:55 AM -0700 Rallavagu Kon rallavagu@gmail.com wrote:
Hello All,
Running OpenLDAP version 2.4.58 on Ubuntu 20.04 in multicaster mode.
Under normal conditions, all operations perform normally in terms of replication consistency. However, when performing a stream of "ADD" operations eventually producer/consumer go out of sync where consumer thinks it has latest state than producer. All the write operations including ADD are pointed to a producer node (not load balancing the writes). Introduced 30 second delay to "ADD" operations (for the sake of testing) and experiencing the same behavior. The issue is manifesting after adding around 4k entries (sometimes even sooner).
I would note that OpenLDAP 2.4 series is no longer supported and that the OpenLDAP 2.5 series has numerous replication related changes. I'd strongly advise deploying OpenLDAP 2.5 and seeing if this resolves the issue.
If you are unable to package OpenLDAP 2.5 yourself, you may want to use the freely available from Symas or the LTB project.
Symas: https://repo.symas.com/soldap2.5/
LTB: https://ltb-project.org/download.html
Regards, Quanah
On May 6, 2022, at 09:22, Quanah Gibson-Mount quanah@fast-mail.org wrote:
--On Friday, May 6, 2022 9:55 AM -0700 Rallavagu Kon rallavagu@gmail.com wrote:
Hello All,
Running OpenLDAP version 2.4.58 on Ubuntu 20.04 in multicaster mode.
Under normal conditions, all operations perform normally in terms of replication consistency. However, when performing a stream of "ADD" operations eventually producer/consumer go out of sync where consumer thinks it has latest state than producer. All the write operations including ADD are pointed to a producer node (not load balancing the writes). Introduced 30 second delay to "ADD" operations (for the sake of testing) and experiencing the same behavior. The issue is manifesting after adding around 4k entries (sometimes even sooner).
I would note that OpenLDAP 2.4 series is no longer supported and that the OpenLDAP 2.5 series has numerous replication related changes. I'd strongly advise deploying OpenLDAP 2.5 and seeing if this resolves the issue.
If you are unable to package OpenLDAP 2.5 yourself, you may want to use the freely available from Symas or the LTB project.
Symas: https://repo.symas.com/soldap2.5/
LTB: https://ltb-project.org/download.html
Regards, Quanah
Upgraded to 2.5.12 and modified N-Way multi producer replication to “mirror mode”. Replication working fine after these changes. However, the monitoring scripts that were using ldapsearch are not able to read “contextCSN” attribute anymore. Hence, the monitoring is broken. However, slapcat output shows contextCSN attribute. Wondering if there is anything we need to look into.
Thanks in advance.
--On Friday, May 20, 2022 10:43 AM -0700 Rallavagu Kon rallavagu@gmail.com wrote:
Upgraded to 2.5.12 and modified N-Way multi producer replication to "mirror mode". Replication working fine after these changes. However, the monitoring scripts that were using ldapsearch are not able to read "contextCSN" attribute anymore. Hence, the monitoring is broken. However, slapcat output shows contextCSN attribute. Wondering if there is anything we need to look into.
You would need to examine your ACLs to see why.
--Quanah
openldap-technical@openldap.org
-
Quanah Gibson-Mount -
Rallavagu Kon