Transitioning from slapd.conf to slapd.d, best practices for maintaining configuration comments?
This is more of a practical question than a technical one, but it's prompted by a technical change: I'm *very* **very** belatedly transitioning from flat file slapd.conf config to slapd.d/OLC.
With flat file configuration, it was straightforward to include text comments (e.g. "# blah blah"), but as far as I know there isn't any sort of analog for comments, when using slapd.d. Looking for any tips about how best to annotate slapd configuration, in a slapd.d/olc world. Does anyone have a practice that they find works well for them? Do people just maintain separate documents/wiki pages/etc that describe their servers' configs?
Ben
On 11/15/23 18:58, Ben Poliakoff wrote:
This is more of a practical question than a technical one, but it's prompted by a technical change: I'm *very* **very** belatedly transitioning from flat file slapd.conf config to slapd.d/OLC.
With flat file configuration, it was straightforward to include text comments (e.g. "# blah blah"), but as far as I know there isn't any sort of analog for comments, when using slapd.d. Looking for any tips about how best to annotate slapd configuration, in a slapd.d/olc world. Does anyone have a practice that they find works well for them? Do people just maintain separate documents/wiki pages/etc that describe their servers' configs?
Ben
Hey Ben,
The most straightforward way would be to document the commands you ran in chronological order in a text file/README or shell script.
At some point, I also tried to document all my steps for a setup in a single README.md and update this file as the configuration changed. However, I don't think this works too well. After some time, you may end up with a incomplete documentation of the current setup, which might not even work and is hard to debug.
In conclusion, I would recommend writing all the commands in a text file or bash script, commenting on the changes with a date/time stamp and what the purpose for that change was. With that, you would have a nice chronology, and changes are easier to understand, even for others who are not familiar with the setup.
Regards, Souji
Ben Poliakoff wrote:
This is more of a practical question than a technical one, but it's prompted by a technical change: I'm *very* **very** belatedly transitioning from flat file slapd.conf config to slapd.d/OLC.
With flat file configuration, it was straightforward to include text comments (e.g. "# blah blah"), but as far as I know there isn't any sort of analog for comments, when using slapd.d. Looking for any tips about how best to annotate slapd configuration, in a slapd.d/olc world. Does anyone have a practice that they find works well for them? Do people just maintain separate documents/wiki pages/etc that describe their servers' configs?
There's no real mechanism for commenting in the config itself. We could add optional "description" attributes to all of the config objectclasses, which would at least let you keep some information near the relevant locations, but it still wouldn't be the equivalent of line-by-line commenting.
Il 15/11/23 19:58, Ben Poliakoff ha scritto:
With flat file configuration, it was straightforward to include text comments (e.g. "# blah blah"), but as far as I know there isn't any sort of analog for comments, when using slapd.d. Looking for any tips about how best to annotate slapd configuration, in a slapd.d/olc world. Does anyone have a practice that they find works well for them? Do people just maintain separate documents/wiki pages/etc that describe their servers' configs?
That's the main reason because I don't use slapd.d, and I revert to slapd.conf after installation, if it's used by default.
I don't need the slapd.d features, using slapd.conf configuration is more readable, and I can revert any modification with a stop and a file replacement, automation and deploy is simpler.
Simone
Ben, hello.
On 15 Nov 2023, at 18:58, Ben Poliakoff wrote:
Looking for any tips about how best to annotate slapd configuration, in a slapd.d/olc world. Does anyone have a practice that they find works well for them?
What works for me (in a primary+replicas setup) is to maintain a slapd.ldif file with structured comments in it (ie, #@PRIMARY@ and #@REPLICA@ marking different variants), and when changes need to be made to the configuration, I stop the primary server (leaving things to replicas), slapcat the data, rebuild the slapd.d from scratch with the appropriate version of the configuration file, reload the data, and restart; then do the same with the replicas.
This isn't ideal, but it works for me because the window when no-one can write, because the primary is off, is acceptably small.
The advantages are
* I can version-control (and of course densely comment) the configuration, with all the attendant advantages * I know exactly what the configuration of the server is, without querying the server * because they're both generated from the same source, I know for sure that the primary and replicas have compatible configurations * that means I can have a test server (including scratch regression-test servers) with a duplicate configuration
I can see how I could achieve most of these things using slapd.d as intended. But this route feels more direct, and thus more intelligible to me. The first advantage seems the key one, to me.
Best wishes,
Norman
We use automatio,n that abstracts the setuo complexity a lot. Any changes needed is made to the automation code, with the history tracked into git side.
Regards Jerome
________________________________ De : Norman Gray gray@nxg.name Envoyé : jeudi 16 novembre 2023 11:32 À : Ben Poliakoff benp@reed.edu Cc : openldap-technical@openldap.org openldap-technical@openldap.org Objet : Re: Transitioning from slapd.conf to slapd.d, best practices for maintaining configuration comments?
ATTENTION : Cet e-mail provient de l'extérieur de l'organisation. Ne cliquez pas sur les liens et n'ouvrez pas les pièces jointes à moins que vous ne reconnaissiez l'expéditeur et que vous sachiez que le contenu est sûr.
Ben, hello.
On 15 Nov 2023, at 18:58, Ben Poliakoff wrote:
Looking for any tips about how best to annotate slapd configuration, in a slapd.d/olc world. Does anyone have a practice that they find works well for them?
What works for me (in a primary+replicas setup) is to maintain a slapd.ldif file with structured comments in it (ie, #@PRIMARY@ and #@REPLICA@ marking different variants), and when changes need to be made to the configuration, I stop the primary server (leaving things to replicas), slapcat the data, rebuild the slapd.d from scratch with the appropriate version of the configuration file, reload the data, and restart; then do the same with the replicas.
This isn't ideal, but it works for me because the window when no-one can write, because the primary is off, is acceptably small.
The advantages are
* I can version-control (and of course densely comment) the configuration, with all the attendant advantages * I know exactly what the configuration of the server is, without querying the server * because they're both generated from the same source, I know for sure that the primary and replicas have compatible configurations * that means I can have a test server (including scratch regression-test servers) with a duplicate configuration
I can see how I could achieve most of these things using slapd.d as intended. But this route feels more direct, and thus more intelligible to me. The first advantage seems the key one, to me.
Best wishes,
Norman
-- Norman Gray : https://nxg.me.uk
On 11/15/23 1:58 PM, Ben Poliakoff wrote:
This is more of a practical question than a technical one, but it's prompted by a technical change: I'm *very* **very** belatedly transitioning from flat file slapd.conf config to slapd.d/OLC.
With flat file configuration, it was straightforward to include text comments (e.g. "# blah blah"), but as far as I know there isn't any sort of analog for comments, when using slapd.d. Looking for any tips about how best to annotate slapd configuration, in a slapd.d/olc world. Does anyone have a practice that they find works well for them? Do people just maintain separate documents/wiki pages/etc that describe their servers' configs?
Ben
having jumped into LDAP after the switch to OLC started, i dove into the OLC configs and really tried to learn how the new mechanism worked. i like the benefit of the online updates where changes to the instance occur without having to stop/start. i use fedora, and it includes the file /usr/share/openldap-servers/slapd.ldif which serves as a base config example, with comments in it. i have modified that and then added its contents via slapadd or ldapadd.
as a jumping off point, you might look at "slaptest -f <flat slapd.conf file> -F <slapd.d dir>". this will actually convert from old to new format and may offer an opportunity to see what you flat config would look like in OLC format.
when i wanted to add new object, entries, etc, i often create a purposefully named ldif file, and add the syntax to it that is needed for the specific purpose. i have a myriad of files named according to what the action is or the function of the ldif file is, like CacheModule.ldif and CacheOverlay.ldif. this helps me compartmentalize the configs and not be overwhelmed with everything else going on.
i have found the phpLDAPAdmin project, which provides a browser based admin UI, to be invaluable. getting past the syntax of LDAP allows me to be focused on what i need to do, and not how i need to do it, though you do still need to have proper structure in the syntax you're working with.
HTH
brendan
Hi,
We decided to continue using the (old) slapd.conf file, and rely on our openldap MMR-cluster to keep the LDAP service up, while individual servers are restarted.
Since the our 2.5 MMR setup is behind a load balancer, clients don't notice individual backend servers going down and coming up again.
We love the simplicity of the config file (and various includes that we use) and also like the inline comments.
We configure slapd.conf (and it's includes) through an ansible script.
Happy so far.
MJ
Hi,
We decided to continue using the (old) slapd.conf file, and rely on our openldap MMR-cluster to keep the LDAP service up, while individual servers are restarted.
Since the our 2.5 MMR setup is behind a load balancer, clients don't notice individual backend servers going down and coming up again.
We just love the simplicity of the config file(and various includes that we do) and also like the inline comments.
We configure ldap through ansible scripts.
On 29Nov23 09:05+0100, cYuSeDfZfb cYuSeDfZfb wrote:
We decided to continue using the (old) slapd.conf file, and rely on our openldap MMR-cluster to keep the LDAP service up, while individual servers are restarted.
Since the our 2.5 MMR setup is behind a load balancer, clients don't notice individual backend servers going down and coming up again.
We just love the simplicity of the config file(and various includes that we do) and also like the inline comments.
We configure ldap through ansible scripts.
Same here! We experimented with the olc config, but the benefits were not convincing for us.
Plase also note [1]: ``` The older style slapd.conf(5) file is still supported, but its use is deprecated and support for it will be withdrawn in a future OpenLDAP release. ```
Is this already on the roadmap when this will happen?
1: https://www.openldap.org/doc/admin26/slapdconf2.html
Cheers,
Hi,
We wonder about that deprication as well!
When will that happen..? How long can we continue to drive the more comfortable road? :-)
Have a nice weekend to all!
On Thu, 30 Nov 2023 at 16:06, Bastian Tweddell b.tweddell@fz-juelich.de wrote:
Please also note [1]:
The older style slapd.conf(5) file is still supported, but its use is deprecated and support for it will be withdrawn in a future OpenLDAP release.Is this already on the roadmap when this will happen?
On Thu, 30 Nov 2023 at 16:06, Bastian Tweddell b.tweddell@fz-juelich.de wrote:
Please also note [1]: ``` The older style slapd.conf(5) file is still supported, but its use is deprecated and support for it will be withdrawn in a future OpenLDAP release. ``` Is this already on the roadmap when this will happen?
I really hope this never happens.
The one and only advantage I see to OLC is that you can make some changes on the fly, without restarting the server. But is this ever necessary, or even advisable in a production environment?
In production, people want LDAP servers to be perfectly stable and reliable software-as-an-appliances. They will run 10 (even 20) years this way.
Production configuration should be immutable. The configuration should not need to change from day to day within production. And even when it does, if clients are configured correctly, there is the ability to restart individual servers without impacting the entire service.
As for sync'ing cn=config, I've tried it. I don't see the advantage of it over having one configuration file (or maybe one each for providers and another for consumers) and then deploying each from source control, and controlled with file signature monitoring, for extra security.
You can have the best of both worlds by enabling the config database, but not converting to it. This "converts" your slapd.conf into the memory-based OLC which can be updated on the fly, but not persisted. To me this is the ideal, but then even still, within many of theses setups, I have never needed to use the OLC for on-the-fly-changes, so in retrospect, do not see the necessity of this.
In summary, I see great value to continuing to support the slapd.conf file-based config, especially for production, and I see a lot of risk induced by deprecating it and forcing people to use OLC. OpenLDAP project, would you please consider to not deprecate slapd.conf?
Chris Paul | Rex Consulting | https://www.rexconsulting.net
On Fri, Dec 01, 2023 at 13:02:03 -0800, Christopher Paul wrote:
The one and only advantage I see to OLC is that you can make some changes on the fly, without restarting the server. But is this ever necessary, or even advisable in a production environment?
I prefer the traditional slapd.conf as well, but one very good use case of dynamic configuration is being able to add indexes on a (very) large directory on-line. It can take many hours on a 100 GB directory, and if you have to do that while slapd is down, it also needs to catch up with many hours of replication afterwards, before you can use it again. So being able to do this on-line is really convenient.
We use a hybrid approach to achieve this: we use a traditional slapd.conf (under version control), but when we need to add an index, we temporarily enable a config database in slapd.conf *without a persistent backend*:
database config rootdn cn=config rootpw .........
This exposes our slapd.conf configuration as LDIF, and lets us modify it, but in a non-persisent manner. So we can dynamically add an olcDbIndex entry to create a new index on-line, and then add that same index in our slapd.conf for persistent use after slapd restart.
Best of both worlds?
Geert
--On Friday, December 1, 2023 11:04 PM +0100 Geert Hendrickx geert@hendrickx.be wrote:
database config rootdn cn=config
This is the default value as noted in slapd.conf(5). No need to specify it explicitly.
rootpw .........
May be better to set up cert auth and do a SASL mapping if you need to make config changes from an external host. If you don't need an external host, use LDAPI + EXTERNAL with a mapping for the root user or similar.
--Quanah
--On Friday, December 1, 2023 1:02 PM -0800 Christopher Paul chris.paul@rexconsulting.net wrote:
In summary, I see great value to continuing to support the slapd.conf file-based config, especially for production, and I see a lot of risk induced by deprecating it and forcing people to use OLC. OpenLDAP project, would you please consider to not deprecate slapd.conf?
As has been noted numerous times, slapd.conf is unordered and a constant source of configuration errors and unexpected behavior since people routinely throw statements in the wrong place. I would also note that you are literally running a cn=config system with slapd.conf, even if it doesn't appear that way to you, since slapd just automatically turns slapd.conf into a cn=config db (although it may not function as desired due to preceding note).
For myself, being able to update the servers on the fly has allowed me to:
a) Push ACL changes w/o restart b) Push indexing changes w/o restart c) Push schema changes w/o restart d) Push log level changes w/o restart (Particularly useful when debugging problems in a live environment)
I keep my cn-config db in git & use a test environment confirm changes prior to pushing them live in production.
--Quanah
On Wed, Nov 15, 2023 at 10:58 AM Ben Poliakoff benp@reed.edu wrote:
This is more of a practical question than a technical one, but it's prompted by a technical change: I'm *very* **very** belatedly transitioning from flat file slapd.conf config to slapd.d/OLC.
With flat file configuration, it was straightforward to include text comments (e.g. "# blah blah"), but as far as I know there isn't any sort of analog for comments, when using slapd.d. Looking for any tips about how best to annotate slapd configuration, in a slapd.d/olc world. Does anyone have a practice that they find works well for them? Do people just maintain separate documents/wiki pages/etc that describe their servers' configs?
Ben
Thank you all for your thoughts about this! I've resisted the olc config route for many of the reasons mentioned in your responses. But after having finally spent more time with it, I am particularly taken with one of the features that it makes possible, namely the replication of cn=config. The documents have certainly mentioned for years that the slapd.conf option will go away at some point, and while it's not clear when that might happen, it's enough for me in my own environment to try to move forward with olc.
We use puppet in my environment, and I've looked at the now community maintained puppet forge module ( https://forge.puppetlabs.com/modules/puppet/openldap/readme) but I really don't love the extra layer of abstraction that it employs to manage cn=config. I'm a bit reluctant to move away from a declarative configuration management style for cn=config (I'm still using puppet for all the *other* bits of the configuration of an openldap server), but not so reluctant as to want to go all in with the puppet openldap module.
My current plan is to write a script that pulls the entire cn=config, and writes it out to ldif files, much like what is rendered in slapd.d, but with the attributes of each entry sorted. I can then insert comments/documentation into these ldif files and track them all in a git repo, so that my current config is documented, much as it is in slapd.conf. I've written a separate script (that will be run via cron) that pulls the *active* cn=config and compares it to the commented/documented ldif files that I generated previously (stripping out the comments). At this point a simple diff will show if changes have been made, and if the changes are ones that I want to keep, I can updated the commented ldif files. The sorting of the attributes in the ldif files is key, since otherwise there's no guaranty that this diff-based method would work. This approach feels a little bit backwards, coming from a file based configuration management point of view, but for me it's a reasonable compromise to move forward with cn=config.
Thanks again for all the discussion and comments!
Ben
openldap-technical@openldap.org
-
Bastian Tweddell -
BECOT Jérôme -
Ben Poliakoff -
Brendan Kearney -
Christopher Paul -
cYuSeDfZfb cYuSeDfZfb -
Geert Hendrickx -
Howard Chu -
Norman Gray -
Quanah Gibson-Mount -
sacawulu
-
Simone Piccardi -
Souji Thenria