Hello,
I would like to use the N-way mutlimaster mode and a virtual IP to manage failover for applications. The virtual IP will be configured trough keepalived.
To work with N-way mutlimaster, we must start OpenLDAP process on the LDAP URI defined in cn=config olcServerID parameter. So we can't use ldap://* to start the service. On the machine running the virtual IP, I can of course listen on this IP by adding an LDAP URI in the start command: slapd -h ldap://master1.example.com ldap://virtual.example.com
But this command will not work it the virtual IP is not set on the node as OpenLDAP refuses to start on an unknown IP.
Does anyone already face this issue and found a solution?
Clément.
2018-01-15 10:38 GMT+04:00 Clément OUDOT clem.oudot@gmail.com:
Hello,
I would like to use the N-way mutlimaster mode and a virtual IP to manage failover for applications. The virtual IP will be configured trough keepalived.
hello,
I have exactly this configuration and I didn't have to do anything particular
ldap is started on each node with "slapd -h ldap://" and it works
you can also try to add ~~~ net.ipv4.ip_nonlocal_bind = 1 ~~~
in /etc/sysctl.conf to allow bind on non-existent(-yet) address
regards, Jephté
To work with N-way mutlimaster, we must start OpenLDAP process on the LDAP URI defined in cn=config olcServerID parameter. So we can't use ldap://* to start the service. On the machine running the virtual IP, I can of course listen on this IP by adding an LDAP URI in the start command: slapd -h ldap://master1.example.com ldap://virtual.example.com
But this command will not work it the virtual IP is not set on the node as OpenLDAP refuses to start on an unknown IP.
Does anyone already face this issue and found a solution?
Clément.
Jephte Clain wrote:
2018-01-15 10:38 GMT+04:00 Clément OUDOT clem.oudot@gmail.com:
I would like to use the N-way mutlimaster mode and a virtual IP to manage failover for applications. The virtual IP will be configured trough keepalived.
I have exactly this configuration and I didn't have to do anything particular
ldap is started on each node with "slapd -h ldap://" and it works
As Clément wrote he has multiple values configured for olcServerID in cn=config (aka serverID in slapd.conf). So he can't just use "slapd -h ldap://" because the parameter value -h is used by slapd to determine the actual server ID to be used.
Well, that's one of the reasons why I avoid to set multiple serverID values. And of course that means that you cannot use a replicated cn=config.
Ciao, Michael.
--On Monday, January 15, 2018 10:05 AM +0100 Michael Ströder michael@stroeder.com wrote:
Well, that's one of the reasons why I avoid to set multiple serverID values. And of course that means that you cannot use a replicated cn=config.
Unique server IDs are required when using MMR, period. It is not isolated to the cn=config replication case.
--Quanah
--
Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
Quanah Gibson-Mount wrote:
--On Monday, January 15, 2018 10:05 AM +0100 Michael Ströder michael@stroeder.com wrote:
Well, that's one of the reasons why I avoid to set multiple serverID values. And of course that means that you cannot use a replicated cn=config.
Unique server IDs are required when using MMR, period. It is not isolated to the cn=config replication case.
Yes, but when you're replicating cn=config you must list all serverIDs in each config. Otherwise, you only need to set one serverID per config. That was Michael's point.
--On Monday, January 15, 2018 4:35 PM +0000 Howard Chu hyc@symas.com wrote:
Quanah Gibson-Mount wrote:
--On Monday, January 15, 2018 10:05 AM +0100 Michael Ströder michael@stroeder.com wrote:
Well, that's one of the reasons why I avoid to set multiple serverID values. And of course that means that you cannot use a replicated cn=config.
Unique server IDs are required when using MMR, period. It is not isolated to the cn=config replication case.
Yes, but when you're replicating cn=config you must list all serverIDs in each config. Otherwise, you only need to set one serverID per config. That was Michael's point.
Ah, right... multi-valued in that case (I read the "multiple serverIDs" as meaning only using a single serverID value). ;)
--Quanah
--
Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
2018-01-15 10:05 GMT+01:00 Michael Ströder michael@stroeder.com:
Jephte Clain wrote:
2018-01-15 10:38 GMT+04:00 Clément OUDOT clem.oudot@gmail.com:
I would like to use the N-way mutlimaster mode and a virtual IP to manage failover for applications. The virtual IP will be configured trough keepalived.
I have exactly this configuration and I didn't have to do anything particular
ldap is started on each node with "slapd -h ldap://" and it works
As Clément wrote he has multiple values configured for olcServerID in cn=config (aka serverID in slapd.conf). So he can't just use "slapd -h ldap://" because the parameter value -h is used by slapd to determine the actual server ID to be used.
Well, that's one of the reasons why I avoid to set multiple serverID values. And of course that means that you cannot use a replicated cn=config.
Does that mean that we can set MMR replication for data (not cn=config) with only one value in olcServerID? In this case we don't have the constraint to start the daemon on a dedicated LDAP URI? Would we still be able to define all olcSyncrepl statements in the database or do we need to set only those not concerning the current node?
Clément.
Clément OUDOT wrote:
2018-01-15 10:05 GMT+01:00 Michael Ströder michael@stroeder.com:
As Clément wrote he has multiple values configured for olcServerID in cn=config (aka serverID in slapd.conf). So he can't just use "slapd -h ldap://" because the parameter value -h is used by slapd to determine the actual server ID to be used.
Well, that's one of the reasons why I avoid to set multiple serverID values. And of course that means that you cannot use a replicated cn=config.
Does that mean that we can set MMR replication for data (not cn=config) with only one value in olcServerID? In this case we don't have the constraint to start the daemon on a dedicated LDAP URI?
Yes.
Would we still be able to define all olcSyncrepl statements in the database or do we need to set only those not concerning the current node?
Note sure about that. Give it a try.
In my setup I'm avoiding this because
1. it's easy with ansible/puppet/name-your-favourite-config-mgmt-here
2. my monitoring check looks whether all provider replicas are reachable and I was too lazy to sort out the local replica in the check.
Ciao, Michael.
On Mon, Jan 15, 2018 at 18:16:57 +0100, Clément OUDOT wrote:
2018-01-15 10:05 GMT+01:00 Michael Ströder michael@stroeder.com:
Jephte Clain wrote: As Clément wrote he has multiple values configured for olcServerID in cn=config (aka serverID in slapd.conf). So he can't just use "slapd -h ldap://" because the parameter value -h is used by slapd to determine the actual server ID to be used.
Well, that's one of the reasons why I avoid to set multiple serverID values. And of course that means that you cannot use a replicated cn=config.
Does that mean that we can set MMR replication for data (not cn=config) with only one value in olcServerID? In this case we don't have the constraint to start the daemon on a dedicated LDAP URI? Would we still be able to define all olcSyncrepl statements in the database or do we need to set only those not concerning the current node?
We do MMR with service IP's, using identical slapd.conf's (with multi-valued serverID) and -h ldap://, but we use the proper system hostnames for syncrepl providers.
Geert
2018-01-16 0:20 GMT+04:00 Geert Hendrickx geert@hendrickx.be:
On Mon, Jan 15, 2018 at 18:16:57 +0100, Clément OUDOT wrote:
2018-01-15 10:05 GMT+01:00 Michael Ströder michael@stroeder.com:
Jephte Clain wrote: As Clément wrote he has multiple values configured for olcServerID in cn=config (aka serverID in slapd.conf). So he can't just use "slapd -h ldap://" because the parameter value -h is used by slapd to determine the actual server ID to be used.
Well, that's one of the reasons why I avoid to set multiple serverID values. And of course that means that you cannot use a replicated
cn=config.
Does that mean that we can set MMR replication for data (not cn=config) with only one value in olcServerID? In this case we don't have the constraint to start the daemon on a dedicated LDAP URI? Would we still be able to define all olcSyncrepl statements in the database or do we need to set only those not concerning the current node?
We do MMR with service IP's, using identical slapd.conf's (with multi-valued serverID) and -h ldap://, but we use the proper system hostnames for syncrepl providers.
hello,
that's what I meant earlier when I said I did nothing special:
all nodes run with -h ldap:// and I have as many olcServerID as defined nodes my cn=config is also replicated between all nodes
for example, I have: ~~~ dn: cn=config ... olcServerID: 1 ldap://ldapm3.univ.run/ olcServerID: 2 ldap://ldapm4.univ.run/
dn: olcDatabase={0}config,cn=config ... olcSyncrepl: {0}... rid=0 provider="ldap://ldapm3.univ.run/" ... olcSyncrepl: {1}... rid=3 provider="ldap://ldapm4.univ.run/" ... olcUpdateRef: ldap://ldapm3.univ.run/ olcUpdateRef: ldap://ldapm4.univ.run/ olcMirrorMode: TRUE
# and for each suffix something like this: dn: olcDatabase={1}mdb,cn=config ... olcSuffix: dc=univ-reunion,dc=fr olcSyncrepl: {0}... rid=2 provider="ldap://ldapm3.univ.run/" ... olcSyncrepl: {1}... rid=5 provider="ldap://ldapm4.univ.run/" ... olcUpdateRef: ldap://ldapm3.univ.run/ olcUpdateRef: ldap://ldapm4.univ.run/ olcMirrorMode: TRUE ~~~
It has worked that way for several years now... Is it ok or am I just asking for trouble?
btw, rid is generated by a script for each new database, nothing funny to see here :-)
best regards, Jephté Clain
Geert-- geert.hendrickx.be :: geert@hendrickx.be :: PGP: 0xC4BB9E9F This e-mail was composed using 100% recycled spam messages!
On Tue, Jan 16, 2018 at 10:24:42AM +0400, Jephte Clain wrote:
all nodes run with -h ldap:// and I have as many olcServerID as defined nodes my cn=config is also replicated between all nodes
for example, I have:
olcServerID: 1 ldap://ldapm3.univ.run/ olcServerID: 2 ldap://ldapm4.univ.run/
olcSyncrepl: {0}... rid=0 provider="ldap://ldapm3.univ.run/" ... olcSyncrepl: {1}... rid=3 provider="ldap://ldapm4.univ.run/" ...
There is an important variation of this problem when running cloud machines in services like AWS. The actual machines have IPv4 addresses in private address space - usually 10.x.x.x - but if you need to access them from outside their own availability zone (datacentre) you must use a different (public) IP address. The machines usually have hostnames of the form ip-10-x-x-x and there is no way to resolve those globally.
So, we might have one server in each of three zones and need to set up MMR between them:
Zone A: Server IP: 10.21.0.1 Public IP: 134.82.17.63 Default hostname: ip-10-21-0-1
Zone B: Server IP: 10.22.0.71 Public IP: 193.17.65.123 Default hostname: ip-10-22-0-71
Zone C: Server IP: 10.35.0.5 Public IP: 7.45.81.52 Default hostname: ip-10-35-0-5
Not too much of a problem if each server has a separate config, but if we want replicated configs it gets harder.
The first problem is that the private IP ranges used by the three zones may overlap. Indeed, there are cases where the three servers could have identical local IPs (and thus identical default hostnames).
Even without that, we have to cope with the fact that the servers are on private addresses but have to contact each other using public addresses.
One option would be to set new hostnames on the servers and use those to key into the appropriate config values. The hostnames would need to be resolvable to public IP addresses so that the MMR config would work properly. Does anything check that the hostname actually resolves to an IP address bound to the server I wonder? Even if OpenLDAP is OK with that, it might break something else running on the machine.
Another option would be to use the new hostnames in the -h URL list, but as has already been mentioned this will fail because the server cannot bind to the public IP.
What is best practice here? I am leaning towards *not* replicating config because it makes some updates really hard (think changing the max size of an MDB database without stopping all the servers at the same time). I still like the idea that the config should be identical on all servers during normal operation though.
[ Note that I am not considering the OP's virtual IP problem here, as load-balancing/failover in this environment would be done separately ]
Andrew
--On Wednesday, January 24, 2018 11:59 AM +0000 Andrew Findlay andrew.findlay@skills-1st.co.uk wrote:
What is best practice here? I am leaning towards *not* replicating config because it makes some updates really hard (think changing the max size of an MDB database without stopping all the servers at the same time). I still like the idea that the config should be identical on all servers during normal operation though.
You can change the MDB maxsize at any time, including while slapd is running. So that scenario at least doesn't compute. ;)
--Quanah
--
Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
On Wed, Jan 24, 2018 at 06:37:43AM -0800, Quanah Gibson-Mount wrote:
You can change the MDB maxsize at any time, including while slapd is running. So that scenario at least doesn't compute. ;)
I would still rather do such risky / load-inducing changes on one replica at a time. Anyway, running with fully-replicated config makes it even more important to have a good solution to the problem I described.
Andrew
--On Wednesday, January 24, 2018 3:06 PM +0000 Andrew Findlay andrew.findlay@skills-1st.co.uk wrote:
On Wed, Jan 24, 2018 at 06:37:43AM -0800, Quanah Gibson-Mount wrote:
You can change the MDB maxsize at any time, including while slapd is running. So that scenario at least doesn't compute. ;)
I would still rather do such risky / load-inducing changes on one replica at a time. Anyway, running with fully-replicated config makes it even more important to have a good solution to the problem I described.
Please ellaborate on what is risky (or load inducing) about changing the maxsize. I've had a script for years that has automatically adjusted the maxsize as necessary based on the size of the DB vs maxsize, etc, via cn=config. This script has been deployed on servers throughout the world via my former job @ Zimbra. Changing the maxsize has never been a problem.
You have 3 basic scenarios:
#1: You decrease the maxsize to < size of the DB #2: You set the maxsize = to the size of the DB #3: You increase the maxsize > size of the DB
In scenario #1, lmdb refuses to set the maxsize to something less than the size of the DB. It will make it equal to the size of the DB (Leading to scenario #2)
In scenario #2, the server will not accept any new growth (write ops will be rejected). Monitoring should alert you to that, but the system will recover once you adjust the maxsize to something useful.
In scenario #3, the maxsize is increased.
I've not seen any load related issues for such an operation, so it's not clear to me what you mean by that, either.
The other problem is definitely interesting, and I'm not clear on a good solution for it.
--Quanah
--
Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
On Wed, Jan 24, 2018 at 09:27:20AM -0800, Quanah Gibson-Mount wrote:
Please ellaborate on what is risky (or load inducing) about changing the maxsize.
It is entirely my assumption - wrong it would appear, which is good news! I was thinking of this as something like grow/shrink ops on filesystems.
I've had a script for years that has automatically adjusted the maxsize as necessary based on the size of the DB vs maxsize, etc, via cn=config. This script has been deployed on servers throughout the world via my former job @ Zimbra. Changing the maxsize has never been a problem.
You have 3 basic scenarios:
#1: You decrease the maxsize to < size of the DB #2: You set the maxsize = to the size of the DB #3: You increase the maxsize > size of the DB
In scenario #1, lmdb refuses to set the maxsize to something less than the size of the DB. It will make it equal to the size of the DB (Leading to scenario #2)
Is 'size of the DB' the current length of the DB file, or the amount of disk space actually used by it?
In scenario #2, the server will not accept any new growth (write ops will be rejected). Monitoring should alert you to that, but the system will recover once you adjust the maxsize to something useful.
In scenario #3, the maxsize is increased.
I've not seen any load related issues for such an operation, so it's not clear to me what you mean by that, either.
Just to be clear on this, is it OK to change maxsize in the config file while slapd is down or does this only work with LDAP-based config changes?
The other problem is definitely interesting, and I'm not clear on a good solution for it.
How about adding a flag to slapd to specify the server ID so that it can go the other way through the config, convert ID to FQDN, and drop that FQDN from the set of replication sources?
Andrew
--On Wednesday, January 24, 2018 5:39 PM +0000 Andrew Findlay andrew.findlay@skills-1st.co.uk wrote:
In scenario #1, lmdb refuses to set the maxsize to something less than the size of the DB. It will make it equal to the size of the DB (Leading to scenario #2)
Is 'size of the DB' the current length of the DB file, or the amount of disk space actually used by it?
It's the amount of disk space used (DB+freespace table, etc)
In scenario #2, the server will not accept any new growth (write ops will be rejected). Monitoring should alert you to that, but the system will recover once you adjust the maxsize to something useful.
In scenario #3, the maxsize is increased.
I've not seen any load related issues for such an operation, so it's not clear to me what you mean by that, either.
Just to be clear on this, is it OK to change maxsize in the config file while slapd is down or does this only work with LDAP-based config changes?
If you change the maxsize while slapd is down, it won't get applied until you start slapd. If you're using cn=config, and change it while slapd's running, the change is immediate. My deployments used cn=config, and the script would just modify the maxsize while slapd was running.
The other problem is definitely interesting, and I'm not clear on a good solution for it.
How about adding a flag to slapd to specify the server ID so that it can go the other way through the config, convert ID to FQDN, and drop that FQDN from the set of replication sources?
I'll discuss with Howard, and see. I hate seeing more options added to slapd, but it may be the only option (no pun intended!) for this scenario. ;) The cloud was fairly nascent when this was all designed, so this case wasn't really a consideration at that point in time. If you were to pass in the server ID, I think you could get rid of the multi-valued serverID bit entirely, since every server would know who it was already.
--Quanah
--
Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
On Wed, Jan 24, 2018 at 09:43:58AM -0800, Quanah Gibson-Mount wrote:
How about adding a flag to slapd to specify the server ID so that it can go the other way through the config, convert ID to FQDN, and drop that FQDN from the set of replication sources?
I'll discuss with Howard, and see. I hate seeing more options added to slapd, but it may be the only option (no pun intended!) for this scenario. ;) The cloud was fairly nascent when this was all designed, so this case wasn't really a consideration at that point in time. If you were to pass in the server ID, I think you could get rid of the multi-valued serverID bit entirely, since every server would know who it was already.
I think you would still need that. Maybe I have missed something else here, but how does slapd avoid making a syncrepl connection to itself in the replicated-config scenario? I was assuming that it just ignores syncrepl clauses where the provider URL matches its own hostname. If it is done by IP then more thought will be required.
The multi-valued attribute is neat in the case of servers with simple IP<->hostname mappings so we would probably want to keep it for that.
Andrew
Andrew Findlay wrote:
On Wed, Jan 24, 2018 at 09:43:58AM -0800, Quanah Gibson-Mount wrote:
How about adding a flag to slapd to specify the server ID so that it can go the other way through the config, convert ID to FQDN, and drop that FQDN from the set of replication sources?
I'll discuss with Howard, and see. I hate seeing more options added to slapd, but it may be the only option (no pun intended!) for this scenario. ;) The cloud was fairly nascent when this was all designed, so this case wasn't really a consideration at that point in time. If you were to pass in the server ID, I think you could get rid of the multi-valued serverID bit entirely, since every server would know who it was already.
I think you would still need that. Maybe I have missed something else here, but how does slapd avoid making a syncrepl connection to itself in the replicated-config scenario?
Usually I avoid this by config management vars.
I was assuming that it just ignores syncrepl clauses where the provider URL matches its own hostname. If it is done by IP then more thought will be required.
My recommendation for multi-homed scenarios and/or rapidly changing IP addresses is simply to avoid multi-valued serverID config.
The multi-valued attribute is neat in the case of servers with simple IP<->hostname mappings so we would probably want to keep it for that.
It's also useful for a client to ask slapd for the current effective serverID of a provider replica.
E.g. in Æ-DIR there's a script which sends notification about added user entry with further information to the user. If I have n replicas I don't want n e-mails to be sent out. So the script searches new entries created locally by using serverID part in entryCSN assertion value. Would be cumbersome to sort out the right serverID from a multi-valued serverID set.
So yet another idea for a feature request was having a separate operational attribute in cn=config or cn=monitor reporting the current effective serverID.
Ciao, Michael.
Andrew Findlay wrote:
There is an important variation of this problem when running cloud machines in services like AWS. The actual machines have IPv4 addresses in private address space - usually 10.x.x.x - but if you need to access them from outside their own availability zone (datacentre) you must use a different (public) IP address. The machines usually have hostnames of the form ip-10-x-x-x and there is no way to resolve those globally.
So, we might have one server in each of three zones and need to set up MMR between them:
Zone A: Server IP: 10.21.0.1 Public IP: 134.82.17.63 Default hostname: ip-10-21-0-1
Zone B: Server IP: 10.22.0.71 Public IP: 193.17.65.123 Default hostname: ip-10-22-0-71
Zone C: Server IP: 10.35.0.5 Public IP: 7.45.81.52 Default hostname: ip-10-35-0-5
Not too much of a problem if each server has a separate config, but if we want replicated configs it gets harder.
The first problem is that the private IP ranges used by the three zones may overlap. Indeed, there are cases where the three servers could have identical local IPs (and thus identical default hostnames).
I'd recommend to solely use the public IP addresses and avoid to use the cloud's system DNS. Add DNS RRs for the public IP addresses to a DNS namespace you control.
Even without that, we have to cope with the fact that the servers are on private addresses but have to contact each other using public addresses.
One option would be to set new hostnames on the servers and use those to key into the appropriate config values. The hostnames would need to be resolvable to public IP addresses so that the MMR config would work properly. Does anything check that the hostname actually resolves to an IP address bound to the server I wonder? Even if OpenLDAP is OK with that, it might break something else running on the machine.
These issues are the reason why the ansible role for installing Æ-DIR servers internally strictly distinguishs between system's hostname (canonical FQDN) and the service's FQDNs for accessing the slapd instances. In ansible the system's hostnames more less match what you would add as so-called inventory hostname and I use this as a key to look up the appropriate individual service FQDN specified by host_vars. This gets provisioned to slapd.conf, web2ldap config, and even OpenSSL config snippets to produce CSRs with all appropriate FQDNs put into subjectAltName, etc.
Another option would be to use the new hostnames in the -h URL list, but as has already been mentioned this will fail because the server cannot bind to the public IP.
Despite the system's hostname not resolvable is there another constraint why you cannot use -h <public-IP> or just the need for the FQDN?
Ciao, Michael.
--On Wednesday, January 24, 2018 6:40 PM +0000 Andrew Findlay andrew.findlay@skills-1st.co.uk wrote:
I'll discuss with Howard, and see. I hate seeing more options added to slapd, but it may be the only option (no pun intended!) for this scenario. ;) The cloud was fairly nascent when this was all designed, so this case wasn't really a consideration at that point in time. If you were to pass in the server ID, I think you could get rid of the multi-valued serverID bit entirely, since every server would know who it was already.
I think you would still need that. Maybe I have missed something else here, but how does slapd avoid making a syncrepl connection to itself in the replicated-config scenario? I was assuming that it just ignores syncrepl clauses where the provider URL matches its own hostname. If it is done by IP then more thought will be required.
I'm not saying to get rid of it entirely, I'm just noting that with it passed in as an argument, there shouldn't be a need to have a multivalued setting in the configuration. In fact, no serverID setting in the config at all. Even if it connected to itself, it would discard all traffic that came from itself (by checking the serverID value in the entryCSN, which is how loops are currently avoided when a change originating on a master makes its way back from another master). Might add some overhead though.
--Quanah
--
Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
openldap-technical@openldap.org
-
Andrew Findlay -
Clément OUDOT -
Geert Hendrickx -
Howard Chu -
Jephte Clain -
Michael Ströder -
Quanah Gibson-Mount