hi to all,
is it somehow possible to finde the provider in "syncrepl" of a consumer via DNS SRV-records. If I have several providers with lloadd in front of it and the consumers are only contacting the loadbalancer, the it would be nice to use the SRV-Records of the DNS. I could then set up two loadbalancer with a different priority, so if one failed the consumer would switch to the second loadbalancer. Just an idea :-). Because using just one loadbalancer it will be a single point of failure.
Thanks
Stefan
On Thu, Jan 19, 2023 at 11:48:45AM +0100, Stefan Kania wrote:
hi to all,
is it somehow possible to finde the provider in "syncrepl" of a consumer via DNS SRV-records. If I have several providers with lloadd in front of it and the consumers are only contacting the loadbalancer, the it would be nice to use the SRV-Records of the DNS. I could then set up two loadbalancer with a different priority, so if one failed the consumer would switch to the second loadbalancer. Just an idea :-). Because using just one loadbalancer it will be a single point of failure.
Hi Stefan, or you can use libldap's fallback capabilities where a URI can be in the form of "uri1,uri2,uri3,...", and using that scheme in syncrepl's provider="" option, you don't even need a load balancer for that.
Regards,
Hi Ondřej,
I know, that I can put more then one uri to the "provider"-entry of syncrepl. The questions comes up wile installing and testing lloadd. So it not a special lloadd-thing. In general I would like to know if I could use the SRV-records. That would make changing an OpenLDAP-Server (and it's DNS-name) easy. I only need to change the SRV-record and don't have to modify each ldap-server. Like in sssd and kerberos.
Stefan
Am 19.01.23 um 14:01 schrieb Ondřej Kuzník:
On Thu, Jan 19, 2023 at 11:48:45AM +0100, Stefan Kania wrote:
hi to all,
is it somehow possible to finde the provider in "syncrepl" of a consumer via DNS SRV-records. If I have several providers with lloadd in front of it and the consumers are only contacting the loadbalancer, the it would be nice to use the SRV-Records of the DNS. I could then set up two loadbalancer with a different priority, so if one failed the consumer would switch to the second loadbalancer. Just an idea :-). Because using just one loadbalancer it will be a single point of failure.
Hi Stefan, or you can use libldap's fallback capabilities where a URI can be in the form of "uri1,uri2,uri3,...", and using that scheme in syncrepl's provider="" option, you don't even need a load balancer for that.
Regards,
On Thu, Jan 19, 2023 at 04:39:26PM +0100, Stefan Kania wrote:
Hi Ondřej,
I know, that I can put more then one uri to the "provider"-entry of syncrepl. The questions comes up wile installing and testing lloadd. So it not a special lloadd-thing. In general I would like to know if I could use the SRV-records. That would make changing an OpenLDAP-Server (and it's DNS-name) easy. I only need to change the SRV-record and don't have to modify each ldap-server. Like in sssd and kerberos.
Hi Stefan, unlike back-ldap etc., lloadd opens a pool of persistent connections to configured servers ahead of time. Using SRV records as a configuration source has been on the wishlist however there is no way of getting notified that the record has been changed. On top of that, it has been found much easier to maintain a separate manager process (a human with a checklist, ansible, some kind of provisioning tool, consul/nomad triggered script, k8s operator, ...) that updates its configuration when needed.
That said, patches implementing some kind of SRV are welcome. The easiest way might be to introduce an lloadd tier implementation that manages its backend collection accordingly.
Regards,
Ondřej, hello.
On 20 Jan 2023, at 10:47, Ondřej Kuzník wrote:
That said, patches implementing some kind of SRV are welcome. The easiest way might be to introduce an lloadd tier implementation that manages its backend collection accordingly.
It's not an OpenLDAP patch, but I've attached a module which might be of interest here. This exposes a function
char* get_sorted_srv_records(const char* domain);
which does a SRV lookup, and orders the records that come back according to the specification of RFC 2782 (though in a single pass, rather than the clumsy multiple pass algorithm that the RFC suggests).
Best wishes,
Norman
--On Friday, January 20, 2023 3:33 PM +0000 Norman Gray gray@nxg.name wrote:
Ondřej, hello.
On 20 Jan 2023, at 10:47, Ondřej Kuzník wrote:
That said, patches implementing some kind of SRV are welcome. The easiest way might be to introduce an lloadd tier implementation that manages its backend collection accordingly.
It's not an OpenLDAP patch, but I've attached a module which might be of interest here. This exposes a function
char* get_sorted_srv_records(const char* domain);which does a SRV lookup, and orders the records that come back according to the specification of RFC 2782 (though in a single pass, rather than the clumsy multiple pass algorithm that the RFC suggests).
There is no license noted in this file, which would be a requirement to even look at it.
--Quanah
Quanah, hello.
On 20 Jan 2023, at 15:43, Quanah Gibson-Mount wrote:
There is no license noted in this file, which would be a requirement to even look at it.
No problem -- I've reattached a version with the BSD 2-clause licence included. Is that an OK one for you?
Best wishes,
Norman
And...
On 20 Jan 2023, at 15:33, Norman Gray wrote:
This exposes a function
char* get_sorted_srv_records(const char* domain);
which does a SRV lookup, and orders the records that come back according to the specification of RFC 2782 (though in a single pass, rather than the clumsy multiple pass algorithm that the RFC suggests).
I should mention that the way I use this is
ldap_server = "ldap://foo"; // or ldap_server = "DNS:ldap.example.com";
char* server_list; if (strncmp(ldap_server, "DNS:", 4) == 0) { server_list = get_sorted_srv_records(&ldap_server[4]); } else { server_list = ldap_server; }
ldap_initialise(&directory, server_list);
Best wishes,
Norman
Norman Gray wrote:
Ondřej, hello.
On 20 Jan 2023, at 10:47, Ondřej Kuzník wrote:
That said, patches implementing some kind of SRV are welcome. The easiest way might be to introduce an lloadd tier implementation that manages its backend collection accordingly.
It's not an OpenLDAP patch, but I've attached a module which might be of interest here. This exposes a function
char* get_sorted_srv_records(const char* domain);which does a SRV lookup, and orders the records that come back according to the specification of RFC 2782 (though in a single pass, rather than the clumsy multiple pass algorithm that the RFC suggests).
libldap already provides this functionality in ldap_domain2hostlist().
On Fri, Jan 20, 2023 at 03:33:21PM +0000, Norman Gray wrote:
Ondřej, hello.
On 20 Jan 2023, at 10:47, Ondřej Kuzník wrote:
That said, patches implementing some kind of SRV are welcome. The easiest way might be to introduce an lloadd tier implementation that manages its backend collection accordingly.
It's not an OpenLDAP patch, but I've attached a module which might be of interest here. This exposes a function
char* get_sorted_srv_records(const char* domain);which does a SRV lookup, and orders the records that come back according to the specification of RFC 2782 (though in a single pass, rather than the clumsy multiple pass algorithm that the RFC suggests).
Hi Norman, DNS retrieval and parsing of SRV records is the easy part, and libevent has code to help us with that already. It's the management of the backends on the tier (shutting down those that aren't listed anymore, adding those that are new, rearranging the lists) all while the server is still running unless we request a full server pause for it. And any of the backends being removed might still have pending operations, what do we do with those?
And then the code that uses those levels and weights for each request that comes in, in an efficient way. But a lot of that is in lloadd (tier "weighted") and/or libldap already.
All of this is managed if you have another process that manipulates cn=config for you and it can make some of those decisions as appropriate for your own deployment.
Regards,
Ondřej, hello.
On 20 Jan 2023, at 16:19, Ondřej Kuzník wrote:
DNS retrieval and parsing of SRV records is the easy part, and libevent has code to help us with that already. It's the management of the backends on the tier (shutting down those that aren't listed anymore, adding those that are new, rearranging the lists) all while the server is still running unless we request a full server pause for it. And any of the backends being removed might still have pending operations, what do we do with those?
And then the code that uses those levels and weights for each request that comes in, in an efficient way.
Ah, right -- fair enough. I'd misunderstood where the gap was.
Looking at ldap_domain2hostlist, I still think my sort algorithm is more elegant, here, than use of the Fisher-Yates one (*cough*), but it's not like this is going to be a hotspot in the code!
Changing the subject slightly, since there is SRV support in libldap, is there any chance that it'll become possible to specify a SRV domain in ldap.conf, in place of the list of LDAP URIs in option 'URI'. That is, it'd be useful to be able to say 'URI DNS:ldap.example.com' in there. Supporting something similar to that is why I wrote this code.
Best wishes,
Norman
openldap-technical@openldap.org
-
Howard Chu -
Norman Gray -
Ondřej Kuzník -
Quanah Gibson-Mount -
Stefan Kania