This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools.
--0-1015574222-1491391973=:71829 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8BIT
Thanks for the patch to provide a test script that just shows the same thing.
I see two possible solutions:
1) replacing the same attribute twice in the same modify LDIF is illegal (as it was in older releases) 2) the accesslog/syncrepl needs to record the final result, not every bit of the change.
I believe the easiest solution is #1. However, #2 would be proper if this kind of abuse is legal.
Consider the following deck:
dn: cn=Oh Noes,ou=People,dc=example,dc=com changetype: modify delete: sn sn: George - add: sn sn: Thomas - delete: sn sn: Thomas - add: sn sn: George -
Is that legal? Well, it works and results in an entry in the accesslog that contains:
reqMod: sn:- George reqMod: sn:+ Thomas reqMod: sn:- Thomas reqMod: sn:+ George
and has no issues in the replication.
Therefore, why does this deck:
dn: cn=Oh Noes,ou=People,dc=example,dc=com changetype: modify replace: sn sn: Thomas - replace: sn sn: George -
which creates an accesslog entry that contains:
reqMod: sn:= Thomas reqMod: sn:= George
Interestingly, this morning when I was performing this, the replica ate that accesslog entry and put both of those sn's into the LDAP entry on the replica (instead of the normal barking in the logs that it failed). Perhaps because I had done the former test with the delete/add/delete/add sequence?
At this point, I'm thinking the accesslog should probably generate the minimal changes necessary to get to the right state. That would mean the former test should produce:
reqMod: sn:- George reqMod: sn:+ George
and the latter:
reqMod:sn:= George
Either that, or the syncrepl processing of what accesslog is sending needs some serious investigation. I'll see what I can find for time to look into that serious investigation.
- Frank
On Tue, 4 Apr 2017 at 1:54pm, Ondřej Kuzník wrote:
The attached patch updates the testsuite to reproduce the issue: ftp://ftp.openldap.org/incoming/Ondrej-Kuznik-20170404-ITS-6545-testcase.patch
The issue is that there is information lost when creating the log entry:
- the ordering of values is not guaranteed by LDAP, even though this is
rarely an issue with OpenLDAP with main backends (unless some overlay interferes)
- there is no way to record where one modification ends and another
begins
The obvious way to fix this is to record that a modification has ended. This is technically a change to the accesslog format but delta syncrepl will recover by falling back to plain syncrepl (which it already does in the reported case).
We could also record the order modifications as well (implementing the X-ORDERED trait on the attribute), but that would break most consumers of this format.