Greetings.
The slapo-memberof(5) manpage mentions limitations on when it can be used, in the context of replication. The current text is very confusing, and possibly not self-consistent.
This message might be more appropriate as an ITS PR, but I'm sending it here first, partly in case I've got the wrong end of the stick, and partly as google-bait for the benefit of anyone as puzzled as I was.
The text currently says:
The memberof overlay may be used with any backend that provides full read-write functionality, but it is mainly intended for use with local storage backends. The maintenance operations it performs are internal to the server on which the overlay is configured and are never replicated. Replica servers should be configured with their own instances of the memberOf overlay if it is desired to maintain these memberOf attributes on the replicas.
Fine -- that seems to me to say very clearly that the memberof attribute is OK to use across replicas, as long as each replica has its own memberof overlay.
However, immediately after that, the text says:
Note that slapo-memberOf is not compatible with syncrepl based replication, and should not be used in a replicated environment. An alternative is to use slapo-dynlist to emulate slapo-memberOf behavior.
This seems to flatly contradict (my understanding of) the first part of the paragraph.
So which is it?
A 2009 list thread [1] seems to suggest that you can sort of get away with it, but only just. However another list thread (2015) [2] describes an apparently successful setup using simple syncrepl, with memberof in the provider and consumer, and where the memberof attribute is automatically excluded from the replication (there’s explicit advice not to exclude it using exattrs=memberof in the syncrepl setup). The latter posting refers to ITS issue 7400 [3], ‘Memberof and Syncrepl incompatibility’, which last had activity in 2012, but which still appears to be open. The latter issue notes in passing that ‘memberof must be configured on each replica’, which implies that this is a feasible setup.
**However**: closed ITS 8613 (2017) [4] says baldly that
The slapo-memberOf overlay is not safe to use in a replicated environment due to the way in which replication occurs.
It also explains why, and gives a (compressed but complete) suggestion of how one might use the dynlist overlay to do something similar. A 2018 list message [5] says in passing
Unfortunately, there is no defined standard for the “memberOf” functionality (it’s a MS hack) and so there’s nothing that details how it should or shouldn’t behave with replication.
Going with the most recent statement, therefore, it seems that this it's simply not possible to use memberOf in a replicated environment, unless (rather precariously, I'd have thought) you completely avoid 'refresh' mode in replication, as mentioned in [4].
I suggest adjusting the text in the slapo-memberof manpage. If the third sentence (‘Replica servers should...’) were simply removed, that would remove most of the contradiction. In the following sentence (‘Note that...’), the sentence first seems to suggest that it's only syncrepl replication that the overlay is incompatible with (ie, that there are other replication mechanisms which will work), but it goes on to state that it's incompatible with _all_ replication methods. Which is it?
Finally, given the aside in [5], it might be worth indicating in the manpage that memberOf should only be used when other considerations force it, and should (by the sound of it) be avoided otherwise.
Have I misunderstood something?
Best wishes,
Norman
[1] https://www.openldap.org/lists/openldap-software/200910/msg00037.html [2] https://www.openldap.org/lists/openldap-technical/201505/msg00127.html [3] https://www.openldap.org/its/index.cgi/Software%20Bugs?id=7400;selectid=7400 [4] http://www.openldap.org/its/index.cgi/?findid=8613 [5] https://www.openldap.org/lists/openldap-technical/201809/msg00099.html
On 9/9/19 4:06 PM, Norman Gray wrote:
The slapo-memberof(5) manpage mentions limitations on when it can be used, in the context of replication. The current text is very confusing, and possibly not self-consistent. [..] The text currently says:
The memberof overlay may be used with any backend that provides full read-write functionality, [..]
Fine -- that seems to me to say very clearly that the memberof attribute is OK to use across replicas, as long as each replica has its own memberof overlay.
Yes.
However, immediately after that, the text says:
Note that slapo-memberOf is not compatible with syncrepl based replication, and should not be used in a replicated environment. An alternative is to use slapo-dynlist to emulate slapo-memberOf behavior.
This seems to flatly contradict (my understanding of) the first part of the paragraph.
The problem is that in syncrepl refresh phase entries can be replicated in any order. So if a group entry comes in before the member entries are present you will see some warnings in the log and the entries may not be consistent.
See ITS#8613 for details:
https://www.openldap.org/its/index.cgi/?findid=8613
Ciao, Michael.
P.S.: Personally I can't see a good reason why memberOf attribute is not replicated just like any other operational attribute.
Michael, hello.
On 9 Sep 2019, at 16:16, Michael Ströder wrote:
On 9/9/19 4:06 PM, Norman Gray wrote:
However, immediately after that, the text says:
Note that slapo-memberOf is not compatible with syncrepl based replication, and should not be used in a replicated environment. An alternative is to use slapo-dynlist to emulate slapo-memberOf behavior.
This seems to flatly contradict (my understanding of) the first part of the paragraph.
The problem is that in syncrepl refresh phase entries can be replicated in any order. So if a group entry comes in before the member entries are present you will see some warnings in the log and the entries may not be consistent.
See ITS#8613 for details:
Indeed: I quoted ITS#8613 which says 'slapo-memberOf overlay is not safe to use in a replicated environment', because of the ordering issue that you've mentioned here.
It seems that one can use the memberof overlay if-and-only-if one has separately done something to deal with the ordering (which would be tricky) or one has some means of completely avoiding 'refresh' mode in the sync (which would be tricky). In practice, that looks very much like a 'no', and the contradiction in the manpage still seems to be present.
For what it's worth, the manpage text looks as if the 'Note that...' sentence were added at some point (possibly in response to ITS#8613?) without amending the text immediately before it, so that the result is a little garbled.
Best wishes,
Norman
openldap-technical@openldap.org
-
Michael Ströder -
Norman Gray