https://bugs.openldap.org/show_bug.cgi?id=10652

Issue ID: 10652 Summary: MPR: concurrent delete and modify of the same entry leave providers permanently divergent Product: OpenLDAP Version: 2.6.16 Hardware: All OS: All Status: UNCONFIRMED Keywords: needs_review Severity: normal Priority: --- Component: slapd Assignee: bugs@openldap.org Reporter: dns.spiros@gmail.com Target Milestone: ---

Created attachment 1239 --> https://bugs.openldap.org/attachment.cgi?id=1239&action=edit Attached is a patch, to be applied to the OpenLDAP repository using `git-am`, which contains the reproducer. Details are provided in the comment.

In a multi-provider setup, if one provider deletes an entry while another provider modifies the same entry, the providers can end up in different final states: on some of them the entry is gone, on others it exists. The outcome depends only on the order in which each provider receives the two changes, not on their CSNs. Afterwards every provider reports the same contextCSN, so the difference is never resolved.

Configuration:

- 3 providers, serverID 1, 2 and 3, each one replicating from the other two - back-mdb, plain syncrepl (no accesslog/delta-syncrepl), type=refreshAndPersist, "multiprovider on" - syncprov with "syncprov-sessionlog 2000000" and "syncprov-checkpoint 1000 10" - no other overlays

Steps:

1. Create cn=G3,ou=groups,dc=example,dc=com (groupOfNames, member U0) and wait until it is identical on all three providers. 2. Isolate provider 1 from providers 2 and 3. The established syncrepl connections stay open but no traffic passes (as with an iptables DROP on established sessions), so no refresh happens when the partition heals: the queued persist messages are just delivered. 3. On provider 1, delete G3 (entryCSN ...#001). 4. About one second later, on provider 2, modify G3 adding member U3 (entryCSN ...#002, newer than the delete). 5. Heal the partition and wait (120 seconds in my test).

Expected result:

The same outcome for G3 on all providers, whichever outcome the replication model chooses.

Actual result (same on 2.6.10, 2.6.16 and 2.7.2):

- Provider 1, which has already deleted G3, receives the modify from provider 2 and applies it as an add, so G3 is recreated with provider 2's entryCSN:

syncrepl_entry: rid=002 be_add cn=G3,ou=groups,dc=example,dc=com (0)

- Providers 2 and 3, which already have the modify, then receive the delete from provider 1 and apply it, although its CSN is older than the entryCSN of the entry being removed:

syncrepl_entry: rid=001 be_delete cn=G3,ou=groups,dc=example,dc=com (0)

Later copies of the same delete forwarded by the other provider are dropped:

do_syncrep2: rid=003 CSN too old, ignoring 20261028084456.552508Z#000000#001#000000 (cn=G3,ou=groups,dc=example,dc=com)

- Final state: G3 exists on provider 1 and does not exist on providers 2 and 3. contextCSN is identical on all three, so nothing triggers a further resync.

The two decisions that produce the divergence seem to be: - a replicated delete is applied without comparing its CSN with the entryCSN of the local entry; - a replicated modify for an entry that no longer exists locally is turned into an add, recreating the entry.

Notes:

- This is not ITS#9282 (deleted entry resurrected after one provider was down): here both changes are made concurrently on running providers and both are replicated. - With the same test, if the partition closes the connections instead of holding them (so the consumers refresh on reconnect), a different problem appears: writes made on provider 1 during the partition are never received by providers 2 and 3 ("not new enough, ignored" during the refresh). I am reporting that separately, as it may be related to ITS#10026. - I first saw this on a production-like cluster (3 providers and 1 read-only consumer, mTLS, SASL EXTERNAL, memberof and refint overlays); the setup above is the minimal one that still reproduces it.