Hi,
I need to set up a backup OpenLDAP cluster and I am looking for advices on the best solutions to achieve it.
The situation is: * A mirror mode cluster on one geographical site * A mirror mode cluster on another geographical site (backup site) * I would prefer not to impact configuration of main cluster * I think that full multi-master on different geographical site is not the best solution (but maybe I am totally wrong on this point)
The backup site must be in sync with the main site.
Here are my two ideas: * Configure a syncrepl client to main cluster on each node of the backup cluster. Question is: will not be conflicts as each node of the backup cluster is already synced with the other backup node? * Configure a LDAP proxy (back-ldap) to backup cluster, with syncrepl client to main cluster. But would back-ldap be able to write operational attributes to backup cluster?
I imagine that some of you already have such needs, could you share your experience?
Thanks,
Clément.
Sorry, but currently I strongly not recommended multi-master. We have a big troubles in the same case, but most high load.
I am currently working on the two major issues: - syncrepl seems unreliable, survives no more than 3-5 min under a stress test (ITS#7968). - fix of ITS#7904 does not affect syncrepl, thereby MDB_MAP_FULL may occur in case of LMDB backed (no ITS now).
But another couple seem be forever: - replication is significantly slowdown if are more than 3 nodes. - replication may never be completed until all ldap-write activities are stopped.
More over, just RTFM http://www.openldap.org/doc/admin24/guide.html 18.2.2.3. Arguments against Multi-Master replication - If connectivity with a provider is lost because of a network partition, then "automatic failover" can just compound the problem. - Typically, a particular machine cannot distinguish between losing contact with a peer because that peer crashed, or because the network link has failed. - If a network is partitioned and multiple clients start writing to each of the "masters" then reconciliation will be a pain; it may be best to simply deny writes to the clients that are partitioned from the single provider.
Leonid.
2014-10-17 20:23 GMT+04:00 Clément OUDOT clem.oudot@gmail.com:
Hi,
I need to set up a backup OpenLDAP cluster and I am looking for advices on the best solutions to achieve it.
The situation is:
- A mirror mode cluster on one geographical site
- A mirror mode cluster on another geographical site (backup site)
- I would prefer not to impact configuration of main cluster
- I think that full multi-master on different geographical site is not the
best solution (but maybe I am totally wrong on this point)
The backup site must be in sync with the main site.
Here are my two ideas:
- Configure a syncrepl client to main cluster on each node of the backup
cluster. Question is: will not be conflicts as each node of the backup cluster is already synced with the other backup node?
- Configure a LDAP proxy (back-ldap) to backup cluster, with syncrepl client
to main cluster. But would back-ldap be able to write operational attributes to backup cluster?
I imagine that some of you already have such needs, could you share your experience?
Thanks,
Clément.
--On Saturday, October 18, 2014 9:11 PM +0400 Леонид Юрьев leo@yuriev.ru wrote:
Sorry, but currently I strongly not recommended multi-master. We have a big troubles in the same case, but most high load.
I am currently working on the two major issues:
- syncrepl seems unreliable, survives no more than 3-5 min under a
stress test (ITS#7968).
- fix of ITS#7904 does not affect syncrepl, thereby MDB_MAP_FULL may
occur in case of LMDB backed (no ITS now).
But another couple seem be forever:
- replication is significantly slowdown if are more than 3 nodes.
- replication may never be completed until all ldap-write activities
are stopped.
Test cases? ITS reports?
Also, I personally use delta-syncrepl vs straight syncrepl.
--Quanah
--
Quanah Gibson-Mount Server Architect Zimbra, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration
Hello,
I currently have several problems with MMR with lots of modification of data made in the Ldap : - ITS 7830 (http://www.openldap.org/its/index.cgi/Incoming?id=7830;selectid=7830) --> MDB_MAP_FULL with lots of modifications and read access - http://www.openldap.org/lists/openldap-technical/201408/msg00152.html --> Pb with Modify in big groups. I reproduce in MDB but not in hdb (perhaps is it a problem with the OS and the management of its memory) - http://www.openldap.org/lists/openldap-technical/201409/msg00165.html --> Delay with ADD and DEL synchronisation with at the end no DEL on the replica - http://www.openldap.org/lists/openldap-technical/201409/msg00171.html --> Full sync of an empty ldap with a full ldap - modified entry on the full ldap are not replicated on the empty (greater than snapshot)
I cannot for the moment test with 2.4.40 but i'll try it soon.
-----Message d'origine----- De : openldap-technical [mailto:openldap-technical-bounces@openldap.org] De la part de Quanah Gibson-Mount Envoyé : lundi 20 octobre 2014 21:34 À : Леонид Юрьев Cc : openldap-technical@openldap.org Objet : Re: Synchronizing two mirror mode clusters
--On Saturday, October 18, 2014 9:11 PM +0400 Леонид Юрьев leo@yuriev.ru wrote:
Sorry, but currently I strongly not recommended multi-master. We have a big troubles in the same case, but most high load.
I am currently working on the two major issues:
- syncrepl seems unreliable, survives no more than 3-5 min under a
stress test (ITS#7968).
- fix of ITS#7904 does not affect syncrepl, thereby MDB_MAP_FULL may
occur in case of LMDB backed (no ITS now).
But another couple seem be forever:
- replication is significantly slowdown if are more than 3 nodes.
- replication may never be completed until all ldap-write activities
are stopped.
Test cases? ITS reports?
Also, I personally use delta-syncrepl vs straight syncrepl.
--Quanah
--
Quanah Gibson-Mount Server Architect Zimbra, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration
I assume that ITS7830 (MDB_MAP_FULL with lots of modifications and read access) is the same as ITS7904, and would be solved by a 'dreamcatcher' feature.
2014-10-22 16:26 GMT+04:00 Meunier, Antonin antonin.meunier@cgi.com:
Hello,
I currently have several problems with MMR with lots of modification of data made in the Ldap :
- ITS 7830 (http://www.openldap.org/its/index.cgi/Incoming?id=7830;selectid=7830) --> MDB_MAP_FULL with lots of modifications and read access
- http://www.openldap.org/lists/openldap-technical/201408/msg00152.html --> Pb with Modify in big groups. I reproduce in MDB but not in hdb (perhaps is it a problem with the OS and the management of its memory)
- http://www.openldap.org/lists/openldap-technical/201409/msg00165.html --> Delay with ADD and DEL synchronisation with at the end no DEL on the replica
- http://www.openldap.org/lists/openldap-technical/201409/msg00171.html --> Full sync of an empty ldap with a full ldap - modified entry on the full ldap are not replicated on the empty (greater than snapshot)
I cannot for the moment test with 2.4.40 but i'll try it soon.
-----Message d'origine----- De : openldap-technical [mailto:openldap-technical-bounces@openldap.org] De la part de Quanah Gibson-Mount Envoyé : lundi 20 octobre 2014 21:34 À : Леонид Юрьев Cc : openldap-technical@openldap.org Objet : Re: Synchronizing two mirror mode clusters
--On Saturday, October 18, 2014 9:11 PM +0400 Леонид Юрьев leo@yuriev.ru wrote:
Sorry, but currently I strongly not recommended multi-master. We have a big troubles in the same case, but most high load.
I am currently working on the two major issues:
- syncrepl seems unreliable, survives no more than 3-5 min under a
stress test (ITS#7968).
- fix of ITS#7904 does not affect syncrepl, thereby MDB_MAP_FULL may
occur in case of LMDB backed (no ITS now).
But another couple seem be forever:
- replication is significantly slowdown if are more than 3 nodes.
- replication may never be completed until all ldap-write activities
are stopped.
Test cases? ITS reports?
Also, I personally use delta-syncrepl vs straight syncrepl.
--Quanah
--
Quanah Gibson-Mount Server Architect Zimbra, Inc.
Zimbra :: the leader in open source messaging and collaboration
Леонид Юрьев wrote:
I assume that ITS7830 (MDB_MAP_FULL with lots of modifications and read access) is the same as ITS7904, and would be solved by a 'dreamcatcher' feature.
Would be best to have an actual reproducible test case so we don't have to assume anything.
2014-10-22 16:26 GMT+04:00 Meunier, Antonin antonin.meunier@cgi.com:
Hello,
I currently have several problems with MMR with lots of modification of data made in the Ldap :
- ITS 7830 (http://www.openldap.org/its/index.cgi/Incoming?id=7830;selectid=7830) --> MDB_MAP_FULL with lots of modifications and read access
- http://www.openldap.org/lists/openldap-technical/201408/msg00152.html --> Pb with Modify in big groups. I reproduce in MDB but not in hdb (perhaps is it a problem with the OS and the management of its memory)
There was already a reply to this, but you never followed up with any feedback.
- http://www.openldap.org/lists/openldap-technical/201409/msg00165.html --> Delay with ADD and DEL synchronisation with at the end no DEL on the replica
- http://www.openldap.org/lists/openldap-technical/201409/msg00171.html --> Full sync of an empty ldap with a full ldap - modified entry on the full ldap are not replicated on the empty (greater than snapshot)
I cannot for the moment test with 2.4.40 but i'll try it soon.
-----Message d'origine----- De : openldap-technical [mailto:openldap-technical-bounces@openldap.org] De la part de Quanah Gibson-Mount Envoyé : lundi 20 octobre 2014 21:34 À : Леонид Юрьев Cc : openldap-technical@openldap.org Objet : Re: Synchronizing two mirror mode clusters
--On Saturday, October 18, 2014 9:11 PM +0400 Леонид Юрьев leo@yuriev.ru wrote:
Sorry, but currently I strongly not recommended multi-master. We have a big troubles in the same case, but most high load.
I am currently working on the two major issues:
- syncrepl seems unreliable, survives no more than 3-5 min under a
stress test (ITS#7968).
- fix of ITS#7904 does not affect syncrepl, thereby MDB_MAP_FULL may
occur in case of LMDB backed (no ITS now).
But another couple seem be forever:
- replication is significantly slowdown if are more than 3 nodes.
- replication may never be completed until all ldap-write activities
are stopped.
Test cases? ITS reports?
Also, I personally use delta-syncrepl vs straight syncrepl.
--Quanah
Clément OUDOT wrote:
Hi,
I need to set up a backup OpenLDAP cluster and I am looking for advices on the best solutions to achieve it.
The situation is:
- A mirror mode cluster on one geographical site
- A mirror mode cluster on another geographical site (backup site)
- I would prefer not to impact configuration of main cluster
- I think that full multi-master on different geographical site is not the
best solution (but maybe I am totally wrong on this point)
The backup site must be in sync with the main site.
Here are my two ideas:
- Configure a syncrepl client to main cluster on each node of the backup
cluster. Question is: will not be conflicts as each node of the backup cluster is already synced with the other backup node?
If you are actually using mirrormode, where only one server receives writes at a time, there can be no conflict.
- Configure a LDAP proxy (back-ldap) to backup cluster, with syncrepl client
to main cluster. But would back-ldap be able to write operational attributes to backup cluster?
Configure back-ldap with appropriate credentials, and configure updatedn on the target server, then yes, it can write operational attributes.
I imagine that some of you already have such needs, could you share your experience?
Thanks,
Clément.
openldap-technical@openldap.org
-
Clément OUDOT -
Howard Chu -
Meunier, Antonin -
Quanah Gibson-Mount -
Леонид Юрьев