The Fedora folks have some comments defending their support of multi-master replication here http://directory.fedora.redhat.com/wiki/MMRConsideredHarmful
For the most part there's nothing noteworthy. Just wanted to address their point "There is no way for any LDAP loosely coupled replication to guarantee "read your writes" consistency." In fact, as we've discussed here a few times in the past, this not true.
We can guarantee "read your writes" consistency with the help of a tweaked chaining overlay. I.e., 1) the client issues a write request to a slave 2) the slave chains the request to the master 3) when the chained write succeeds, the slave performs the write locally 4) the response is returned to the client
Assuming the client continues talking to the same slave, it will always see consistent data.
It's a form of two-phase commit but a very special case. Since it only requires waiting for one remote operation to succeed it's a lot more efficient than managing generalized N-server two-phase commit.
As a further optimization, the chained request can carry a control "don't loop back to me" telling the master not to propagate this update back to the initiating slave. It only saves a small bit of bandwidth, since with syncrepl, the subsequent redundant update would be idempotent.
(The first part of this, I know, is just preaching to the choir... the second part might be of more interest to developers.)
I note that my I-D focuses on the lost of ACID properties at the master and with directory updaters, but does say: It is noted that X.500 replication (shadowing) model allows for transient inconsistencies to exist between the master and shadow copies of directory information. As applications which update information operate upon the master copy, any inconsistencies in shadow copies are not evident to these applications.
I said "master copy" not "directly with the master". That is, the application can specify dontUseCopy to ensure it reads its (and any subsequent) updates.
In LDAP, the application can simply read its updates through use of the post-read entry control (RFC 4527). And soon, the LDAP dontUseCopy control mechanism will be standardized.
Now, if a client reads without using one of these two mechanisms, then, of course, it risks getting an old entry. That's X.500 (and LDAP).
The service can try to prevent an updater from getting an old entry using one (or a combination) of approaches...
We can guarantee "read your writes" consistency with the help of a tweaked chaining overlay. I.e.,
- the client issues a write request to a slave
- the slave chains the request to the master
- when the chained write succeeds, the slave performs the write locally
- the response is returned to the client
There is a problem with this approach. The write at the master could succeed but fail at the slave because of inconsistencies between the slave and the master (as measured at the times each update is to be applied) caused by replication lag (or other issues). Another approach would be to include a post-read control to the chained operation and just sync the entry to that. But replication lag can causes problems in this approach as well (consider renamed entries). One possible approach for dealing with replication lag issues is to use an assertion control to ensure modify is being applied to proper entry.
Another approach is to simply chain reads of any updater. This approach suffers from disconnect issues (unless the service has a long memory).
-- Kurt
Kurt D. Zeilenga wrote:
We can guarantee "read your writes" consistency with the help of a tweaked chaining overlay. I.e.,
- the client issues a write request to a slave
- the slave chains the request to the master
- when the chained write succeeds, the slave performs the write locally
- the response is returned to the client
Another approach is to simply chain reads of any updater. This approach suffers from disconnect issues (unless the service has a long memory).
Ah, yes... We could simply maintain a queue of recent updates. Accesses to any entry whose DN is in the queue get chained back to the master. Once the corresponding update is received from the master the DN can be dropped from the queue. Of course chaining reads would place additional load on the master.
-
Howard Chu -
Kurt D. Zeilenga