Hi,
Is it possible, or even sane to consider adding read-only consumers to a MMR setup? If so, any recommendations, pitfalls or gotchas? Or should I simply re-start from scratch -- I'm still not in production.
Just for the record, I'm using 2.4.46 on Debian/Stretch 9.5. Both mirrors are backends to an HA haproxy server.
Thanks, jf
--On Wednesday, October 17, 2018 5:52 PM -0400 Jean-Francois Malouin Jean-Francois.Malouin@bic.mni.mcgill.ca wrote:
Hi,
Is it possible, or even sane to consider adding read-only consumers to a MMR setup? If so, any recommendations, pitfalls or gotchas? Or should I simply re-start from scratch -- I'm still not in production.
I've had setups with 2-node MMR on the front, and read only consumers. It works just fine, as long as any given consumer only points to one master. Theoretically, it's supposed to work so that consumers can point to more than one master in an MMR setup, but my experience didn't match that (http://www.openldap.org/its/index.cgi/?findid=8373).
--Quanah
--
Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
On 10/18/18 1:41 AM, Quanah Gibson-Mount wrote:
I've had setups with 2-node MMR on the front, and read only consumers. It works just fine, as long as any given consumer only points to one master. Theoretically, it's supposed to work so that consumers can point to more than one master in an MMR setup, but my experience didn't match that (http://www.openldap.org/its/index.cgi/?findid=8373).
There's no config in ITS#8373. But you mention accesslog DB. Does that mean your setup used delta-syncrepl? If yes, does that issue also apply to normal syncrepl?
Ciao, Michael.
On Thu, Oct 18, 2018 at 10:10:33AM +0200, Michael Ströder wrote:
On 10/18/18 1:41 AM, Quanah Gibson-Mount wrote:
I've had setups with 2-node MMR on the front, and read only consumers. It works just fine, as long as any given consumer only points to one master. Theoretically, it's supposed to work so that consumers can point to more than one master in an MMR setup, but my experience didn't match that (http://www.openldap.org/its/index.cgi/?findid=8373).
There's no config in ITS#8373. But you mention accesslog DB. Does that mean your setup used delta-syncrepl? If yes, does that issue also apply to normal syncrepl?
I can see cn=config is being replicated in that ticket. It makes more sense that the 'unwilling to perform' issue would be limited to replicating the config (which can be more picky about its modifications) rather than regular databases.
That kind of setup is being used in a significant number of deployments and by Linus' law we'd have more reports like this one.
At this point replication of cn=config is broken and unsupported. However, we see miltiple Producers using MMR and Consumers added in using PCR (formerly Master/Slave Replication) frequently. We recommend this kind of setup, particularly in geographically diverse architectures. It greatly reduces the amount of data traversing the WAN and reduces downtime for maintenance and upgrades. Delta-Syncrepl only further reduces the network load. Jason TruppSymas
Pardon typos. Sent from smartphone. -------- Original message --------From: Ondřej Kuzník ondra@mistotebe.net Date: 10/18/18 5:44 AM (GMT-06:00) To: Michael Ströder michael@stroeder.com Cc: Quanah Gibson-Mount quanah@symas.com, OpenLDAP Tehnical openldap-technical@openldap.org Subject: Re: Adding read-only consumers to a Mirror Mode Replication setup? On Thu, Oct 18, 2018 at 10:10:33AM +0200, Michael Ströder wrote:
On 10/18/18 1:41 AM, Quanah Gibson-Mount wrote:
I've had setups with 2-node MMR on the front, and read only consumers. It works just fine, as long as any given consumer only points to one master. Theoretically, it's supposed to work so that consumers can point to more than one master in an MMR setup, but my experience didn't match that (http://www.openldap.org/its/index.cgi/?findid=8373).
There's no config in ITS#8373. But you mention accesslog DB. Does that mean your setup used delta-syncrepl? If yes, does that issue also apply to normal syncrepl?
I can see cn=config is being replicated in that ticket. It makes more sense that the 'unwilling to perform' issue would be limited to replicating the config (which can be more picky about its modifications) rather than regular databases.
That kind of setup is being used in a significant number of deployments and by Linus' law we'd have more reports like this one.
Jason Trupp wrote:
At this point replication of cn=config is broken and unsupported.
Eh? Not true.
-------- Original message -------- From: Ondřej Kuzník ondra@mistotebe.net Date: 10/18/18 5:44 AM (GMT-06:00) To: Michael Ströder michael@stroeder.com Cc: Quanah Gibson-Mount quanah@symas.com, OpenLDAP Tehnical openldap-technical@openldap.org Subject: Re: Adding read-only consumers to a Mirror Mode Replication setup?
On Thu, Oct 18, 2018 at 10:10:33AM +0200, Michael Ströder wrote:
On 10/18/18 1:41 AM, Quanah Gibson-Mount wrote:
I've had setups with 2-node MMR on the front, and read only consumers. It works just fine, as long as any given consumer only points to one master. Theoretically, it's supposed to work so that consumers can point to more than one master in an MMR setup, but my experience didn't match that (http://www.openldap.org/its/index.cgi/?findid=8373).
There's no config in ITS#8373. But you mention accesslog DB. Does that mean your setup used delta-syncrepl? If yes, does that issue also apply to normal syncrepl?
I can see cn=config is being replicated in that ticket. It makes more sense that the 'unwilling to perform' issue would be limited to replicating the config (which can be more picky about its modifications) rather than regular databases.
That kind of setup is being used in a significant number of deployments and by Linus' law we'd have more reports like this one.
-- Ondřej Kuzník Senior Software Engineer Symas Corporation http://www.symas.com Packaged, certified, and supported LDAP solutions powered by OpenLDAP
--On Thursday, October 18, 2018 4:35 PM +0100 Howard Chu hyc@symas.com wrote:
Jason Trupp wrote:
At this point replication of cn=config is broken and unsupported.
Eh? Not true.
Sure it is. We have an ITS about this (ITS#8286). I've asked for review on my proposed fix several times.
--Quanah
--
Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
--On Thursday, October 18, 2018 11:10 AM +0200 Michael Ströder michael@stroeder.com wrote:
On 10/18/18 1:41 AM, Quanah Gibson-Mount wrote:
I've had setups with 2-node MMR on the front, and read only consumers. It works just fine, as long as any given consumer only points to one master. Theoretically, it's supposed to work so that consumers can point to more than one master in an MMR setup, but my experience didn't match that (http://www.openldap.org/its/index.cgi/?findid=8373).
There's no config in ITS#8373. But you mention accesslog DB. Does that mean your setup used delta-syncrepl?
Yes. It was simply a standard Zimbra MMR deployment.
If yes, does that issue also apply to normal syncrepl?
No idea. I generally consider "normal" syncrepl unsafe as using it can lead to data loss. I use delta-syncrepl exclusively because of this, with a focus on eliminating all scnearios in which it might fall back to "normal" syncrepl. (See http://www.openldap.org/its/index.cgi/?findid=8125 for example).
--Quanah
--
Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
On 10/18/18 3:24 PM, Quanah Gibson-Mount wrote:
I generally consider "normal" syncrepl unsafe as using it can lead to data loss. I use delta-syncrepl exclusively because of this, with a focus on eliminating all scnearios in which it might fall back to "normal" syncrepl. (See http://www.openldap.org/its/index.cgi/?findid=8125 for example).
Hmm, in Æ-DIR I exclude some attributes available only at the provider from replication to the consumers (e.g. OTP seeds). I don't want to have ACLs with val.regex on 'reqMod' to achieve this. Thus I use normal syncrepl replication.
Ciao, Michael.
--On Thursday, October 18, 2018 7:24 AM -0700 Quanah Gibson-Mount quanah@symas.com wrote:
--On Thursday, October 18, 2018 11:10 AM +0200 Michael Ströder michael@stroeder.com wrote:
On 10/18/18 1:41 AM, Quanah Gibson-Mount wrote:
I've had setups with 2-node MMR on the front, and read only consumers. It works just fine, as long as any given consumer only points to one master. Theoretically, it's supposed to work so that consumers can point to more than one master in an MMR setup, but my experience didn't match that (http://www.openldap.org/its/index.cgi/?findid=8373).
There's no config in ITS#8373. But you mention accesslog DB. Does that mean your setup used delta-syncrepl?
Yes. It was simply a standard Zimbra MMR deployment.
So out of curiousity, I deployed this configuration again (2 MMR nodes with delta-syncrepl, 1 consumer) and it no longer manifests. So it appears one of the many fixes to replication in the nearly 3 years since I filed the bug resolved the issue. I've closed the ITS.
So generally, I'd say then that it should be OK to deploy in that way.
--Quanah
--
Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
* Quanah Gibson-Mount quanah@symas.com [20181018 09:26]:
--On Thursday, October 18, 2018 11:10 AM +0200 Michael Ströder michael@stroeder.com wrote:
On 10/18/18 1:41 AM, Quanah Gibson-Mount wrote:
I've had setups with 2-node MMR on the front, and read only consumers. It works just fine, as long as any given consumer only points to one master. Theoretically, it's supposed to work so that consumers can point to more than one master in an MMR setup, but my experience didn't match that (http://www.openldap.org/its/index.cgi/?findid=8373).
There's no config in ITS#8373. But you mention accesslog DB. Does that mean your setup used delta-syncrepl?
Yes. It was simply a standard Zimbra MMR deployment.
If yes, does that issue also apply to normal syncrepl?
No idea. I generally consider "normal" syncrepl unsafe as using it can lead to data loss. I use delta-syncrepl exclusively because of this, with a focus on eliminating all scnearios in which it might fall back to "normal" syncrepl. (See http://www.openldap.org/its/index.cgi/?findid=8125 for example).
--Quanah
I'd like to follow up on this discussion, sorry if I've taken so long to reply.
First, I'd like to thank all that have commented so far, much appreciated.
I have a few more questions because some sort of un-easiness about my setup has crept up in view of the different comments that have been written so far.
Right now I have 2 Debian Stretch 9.5 servers running 2.4.46 from the stretch backports. Servers are in a MMR setup, using syncrepl for replication (NOT delta-syncrepl), with a LMDB backend.
The intent is to use the directory as a users authentication repository for a 100+ workstations-- with what I said above, would such a setup considered safe? Am I asking for trouble down the road with version 2.4.46?
Finally, should I rather consider the LTB project for Debian OpenLDAP as been mentioned in some other threads rather than using the Debian backports? I'm a bit reluctant to roll my own packaging from source.
Sorry for the very naive questions, I'm still fairly new to OpenLDAP!
thanks! jf
--
Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
On 10/23/18 8:44 PM, Jean-Francois Malouin wrote:
Right now I have 2 Debian Stretch 9.5 servers running 2.4.46 from the stretch backports. Servers are in a MMR setup, using syncrepl for replication (NOT delta-syncrepl), with a LMDB backend.
The intent is to use the directory as a users authentication repository for a 100+ workstations-- with what I said above, would such a setup considered safe? Am I asking for trouble down the road with version 2.4.46?
It should work.
Finally, should I rather consider the LTB project for Debian OpenLDAP as been mentioned in some other threads rather than using the Debian backports? I'm a bit reluctant to roll my own packaging from source.
The recommendation for LTB builds have two reasons:
1. At some times Debian packages were far behind OpenLDAP's releases while LTB package updates are most times published a couple of days after an OpenLDAP release.
2. Debian, and only Debian, links OpenLDAP with GNUTLS because they have some old licensing paranoia regarding OpenSSL. This caused trouble in the past. Forgot the details, not sure about the current state.
Bear in mind on Debian: The GNUTLS wrapper in OpenLDAP does not return TLS related error messages as diagnostic message to the client. So if cert validation fails at the client side the only message you see is "Server Down". People then look for connection problems and do not get the idea to look after cert config error. The OpenSSL wrapper returns a text message from the OpenSSL libs as diagnostic message.
Sorry for the very naive questions, I'm still fairly new to OpenLDAP!
Your questions are not naive. You're welcome asking here.
Ciao, Michael.
Hi,
* Michael Ströder michael@stroeder.com [20181024 03:26]:
On 10/23/18 8:44 PM, Jean-Francois Malouin wrote:
<snip>
Finally, should I rather consider the LTB project for Debian OpenLDAP as been mentioned in some other threads rather than using the Debian backports? I'm a bit reluctant to roll my own packaging from source.
The recommendation for LTB builds have two reasons:
- At some times Debian packages were far behind OpenLDAP's releases
while LTB package updates are most times published a couple of days after an OpenLDAP release.
- Debian, and only Debian, links OpenLDAP with GNUTLS because they have
some old licensing paranoia regarding OpenSSL. This caused trouble in the past. Forgot the details, not sure about the current state.
Bear in mind on Debian: The GNUTLS wrapper in OpenLDAP does not return TLS related error messages as diagnostic message to the client. So if cert validation fails at the client side the only message you see is "Server Down". People then look for connection problems and do not get the idea to look after cert config error. The OpenSSL wrapper returns a text message from the OpenSSL libs as diagnostic message.
The GnuTLS stuff I'm well aware of, and infuriated at it as I've been at the receiving end of it a few times too many! Just for that, if I had known at the time, would have been reason enough to try the LTB builds!
Sorry for the very naive questions, I'm still fairly new to OpenLDAP!
Your questions are not naive. You're welcome asking here.
Ciao, Michael.
Again, thank you for your comments.
regards, jf
openldap-technical@openldap.org
-
Howard Chu -
Jason Trupp -
Jean-Francois Malouin -
Michael Ströder -
Ondřej Kuzník -
Quanah Gibson-Mount