Still talking to myself...
Howard Chu wrote:
> So continuing the discussion of what to do with syncrepl and
> 1) the provider must be told about all of the sources of changes
> living within its context. possible sources are
> a) local changes
> b) changes received via syncrepl
> 2) every source of changes must have a unique sid.
> a) if it's a syncprov, then it's configured explicitly there
> b) if it's a syncrepl consumer pulling from elsewhere, it uses the
> remote server's sid.
The olcServerID config attribute has been added for configuring these
IDs. It is a global config keyword, not associated with a particular
provider. A single serverID can be configured, for simple static setups.
Or you can configure a list of serverIDs and corresponding URLs, to
allow a single configuration to be replicated across a pool of servers.
When a consumer is configured for multimaster (mirrormode; looks like we'll
have to make "multimaster" a synonym) the serverID will also be sent
explicitly in the sync cookie. By default (single master) it is not sent on
its own. When the serverID is sent, the provider will use it to additionally
filter updates. I.e., changes with an entryCSN whose SID matches the
consumer's SID are assumed to have originated at the consumer and won't be
sent back again. This avoids one wasted network transaction. (The consumer
would just ignore the update anyway.)
> 3) the provider must aggregate all of the cookies for each of these
> change sources and send them to consumers pulling from it.
The consumer now checks to see if it's a subordinate DB; if so it will
perform its contextCSN updates through the parent DB. If a syncprov
overlay is present it will get a chance to see the contextCSN update.
This behavior can be checked using test033, by enabling glue and syncprov on
the superior DB and leaving syncprov commented out on the other DBs.
> There's a desire to be able to configure multiple change
> the same context though. E.g., mirrormode is defined to only work with
> two servers mirroring each other, it would be nice to be able to
> extend this to additional failover servers.
I had in mind to extend the slap_bindconf structure to accomodate a list of
URLs. If the first connection failed, each would be immediately tried in
turn, before returning a failure to the caller. But for the moment I'm
leaving that alone.
I've modified the consumer to allow multiple syncrepl
the same backend. Corresponding changes are still needed in the
provider. The contextCSN attribute is now multi-valued, allowing a CSN
per SID to be tracked. Modifies to the contextCSN must be done with
specific Delete/Add instead of Replace.
Both the consumer and provider have been updated accordingly.
There's no restriction on how this gets used - a consumer can
multiple providers that master disjoint subtrees of the context, or they
can overlap partially or fully. As long as each provider has a unique
SID their multiple contextCSNs will be tracked properly.
We still need more tests for the various scenarios...
> From half-multi-master we can go all the way to multi- if we add
> collision detection and conflict resolution. There's a pretty simple
> way to handle collision detection - we just need to pass the entry's
> old entryCSN along with the rest of the modification info. On the
> consumer we check and see if the oldEntryCSN matches the consumer
> entry's current entryCSN. If they match, there is no collision. If
> they don't match, we need to resolve the conflict.
Aside from allowing us to log that a conflict occurred, keeping the
oldCSN around doesn't seem to buy us much. Since the conflict resolution
is still determined solely by the current entryCSN, I'm dropping this
idea. All we need to check is if the incoming mod's entryCSN is <= the
current entryCSN and drop the change if so.
The oldCSN would still be needed to support multimaster with delta-syncrepl.
That would also allow us to do change-level conflict resolution with
delta-syncrepl instead of just entry-level as the current code does. But at
this point I'm not interested in adding that support.
It shouldn't be a major problem, we still use the op counter if
resolution is too low and multiple updates occur in the same timeslice.
The op counter is also important because it's possible for the system clock
to run backwards in certain situations...
Not all of these changes are checked in yet, but they'll be
coming in soon.
The code is in place; we just need some more test configurations...
So - no one can say OpenLDAP doesn't have multimaster replication any more.
-- Howard Chu
Chief Architect, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc
Chief Architect, OpenLDAP http://www.openldap.org/project/