Thank you for answering my question, comments below:

On Thu, Oct 26, 2023 at 10:52 PM Uwe Sauter <uwe.sauter.de@gmail.com> wrote:

when comparing the LDIF you used to initialize with the slapcat output, what I can see is that you have no distict
definition of olcDatabase={0}config,cn=config. I suspect that OpenLDAP then used default vaules, including the "to * by
* none" ACL.


None of the docs or any examples show to setup a specific section for olcDatabase={0}config,cn=config not even the default ldif file that comes with the distribution. 

This might help others in the future if they encounter Insufficient access (50) from hell:

slapadd and slapmodify did work for me as root directly on the file system
So for example: /usr/sbin/slapmodify -n 0 -F /etc/openldap/slapd.d -l /etc/openldap/secure.ldif 2>&1 
Where secure.ldif would contain something like this:
dn: olcDatabase={1}mdb,cn=config
changetype: modify
delete: olcRootPW

That worked like a charm. 

 
How mission critical is this server? Can you backup/restore? Is this a VM that you can clone?


We are preparing for our first production release so I want to tidy up the bootstrapping before we take it live (next week).
 
First thing I would do is not to use "-Y EXTERNAL -H ldapi:///" because with that you don't connect as RootDN but as
"gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth" as you can see from the ldapmodify output.


Yeah I realized that, and I guess would need to add a specific ACL to allow manager for dn exact match of that. 
I tried all those things but to no avail, I am guessing because of that default rule of to by none 
 
Try the following (and replace with the correct URL):

$ ldifmodify -x -H ldap://localhost/ -D cn=config -W << EOF
 > dn: olcDatabase={0}config,cn=config
 > changetype: modify
 > add: olcRootPW
 > olcRootPW: {SSHA}cZbRoOhRew8MBiWGSEOiFX0XqbAQwXUr
 > EOF


What is ldifmodify ?
 
You will be asked for the old RootPW.

If that fails I would take the slapcat output, clean the operational attributes from it, change the problematic ACL to
something more sensible (and the olcRootPW) and use that to re-create the LDAPs configuration.


I am lucky we are still able to recreate it until we get it right ;-)
 
Following steps are from the top of my head, so don't follow blindly:

- Stop slapd
- Make a backup of your slapd.d directory and your data directory
- Remove the content of the slapd.d directory
- Use slapadd with the prepared LDIF to re-create the slapd.d directory
- Change the ownership of the slapd.d directory
- Start slapd


Thank you for this, I have saved it to my cheat sheet !
 
Again for future people reading this, if you encounter ACL issues and you want to modify the LDIF database in /etc/openldap/slapd.d don't do it manually.
Use slapadd and slapmodify and be sure to select the right database with the -n switch.
Generally the config DB (contained in slapd.d) is -n 0 and your DIT DB is -n 1

Best, and thanks again.

-- 
Alex