Hi,
I just upgraded our servers from Debian 11 to 12. I'm not sure if this is an upstream change, but slapd 2.5 on Debian 12 doesn't support the HDB and BDB backends, so the database needs to be migrated to LMDB. Unfortunately I neglected to check the backend in use for all instances. Our main slapd instance already used LMDB, but another instance, that's just getting a copy of that database through sync replication, was still using HDB. At first I only noticed an error during upgrade. I found a guide (https://sources.debian.org/src/openldap/2.5.13%2Bdfsg-5/debian/slapd.README.... line 255 following) to do the upgrade to 2.5.x if it fails, which showed me the error.
lt_dlopenext failed: (back_hdb) file not foundslapadd: could not add entry dn="cn=module{0},cn=config" (line=16): <olcModuleLoad> handler exited with 1 Closing DB...
So I followed the setps under "BDB/HDB backends removed: migrating to LMDB backend". But upon trying to restore the backup again, it just told me
slapadd: could not add entry dn="cn=config" (line=1): Closing DB...
The first set of lines in cn=config.ldif reads
dn: cn=config objectClass: olcGlobal cn: config olcArgsFile: /var/run/slapd/slapd.args olcLogLevel: none olcPidFile: /var/run/slapd/slapd.pid olcToolThreads: 1 structuralObjectClass: olcGlobal entryUUID: 71b384b4-aca9-1032-883a-d9850217023f creatorsName: cn=config createTimestamp: 20130908080726Z entryCSN: 20130908080726.757296Z#000000#000#000000 modifiersName: cn=config modifyTimestamp: 20130908080726Z
So I'm not sure what it wants to tell me now. I already checked against the config of the main instance, made a few modifications, but the error message is the same. Here the modifications:
dn: cn=config objectClass: olcGlobal cn: config olcArgsFile: /var/run/slapd/slapd.args olcLogLevel: none olcPidFile: /var/run/slapd/slapd.pid olcToolThreads: 1 structuralObjectClass: olcGlobal entryUUID: 71b384b4-aca9-1032-883a-d9850217023f creatorsName: cn=config createTimestamp: 20130908080726Z olcTLSCACertificateFile: /etc/ssl/certs/xyz-chain.pem olcTLSCertificateFile: /etc/ssl/certs/mail.domain.de.cert.pem olcTLSCertificateKeyFile: /etc/ssl/private/mail.domain.de.private.pem entryCSN: 20130908080726.757296Z#000000#000#000000 modifiersName: gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=mail modifyTimestamp: 20130908080726Z
Could anybody tell me what exactly the problem is?
Richard
--On Monday, July 24, 2023 6:16 PM +0200 Richard Rosner rrosner@fsmuw.rwth-aachen.de wrote:
Hi,
I just upgraded our servers from Debian 11 to 12. I'm not sure if this is an upstream change, but slapd 2.5 on Debian 12 doesn't support the HDB and BDB backends
Deprecation of BDB based backends was announced years ago during the OpenLDAP 2.4 series lifecycle.
so the database needs to be migrated to LMDB.
slapadd: could not add entry dn="cn=config" (line=1): Closing DB...
The first set of lines in cn=config.ldif reads
dn: cn=config objectClass: olcGlobal cn: config olcArgsFile: /var/run/slapd/slapd.args olcLogLevel: none olcPidFile: /var/run/slapd/slapd.pid olcToolThreads: 1 structuralObjectClass: olcGlobal entryUUID: 71b384b4-aca9-1032-883a-d9850217023f creatorsName: cn=config createTimestamp: 20130908080726Z entryCSN: 20130908080726.757296Z#000000#000#000000 modifiersName: cn=config modifyTimestamp: 20130908080726Z
Is it actually indented this way? Because that would not be valid LDIF.
--Quanah
It seems to look like it's intended to look like this. I know this isn't the typical ldif syntax, but on the main instance that same file starts with this:
dn: cn=config objectClass: olcGlobal cn: config olcArgsFile: /var/run/slapd/slapd.args olcLogLevel: none olcPidFile: /var/run/slapd/slapd.pid olcToolThreads: 1 structuralObjectClass: olcGlobal entryUUID: 9a5b5f82-56d4-1039-8be0-4705b1c5590c creatorsName: cn=config createTimestamp: 20190819135427Z olcTLSCACertificateFile: /etc/ssl/certs/geant-intermediates.pem olcTLSCertificateFile: /etc/ssl/certs/auth.domain.de.cert.pem olcTLSCertificateKeyFile: /etc/ssl/private/auth.domain.de.private.pem entryCSN: 20230520140839.289572Z#000000#000#000000 modifiersName: gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth modifyTimestamp: 20230520140839Z contextCSN: 20230520140839.289572Z#000000#000#000000
so I guess the rest of the otherwise needed information comes from the command that's used to restore it, or maybe it's not meant to be an ldif file for this to work:
slapadd -F/etc/ldap/slapd.d -n0 -l /var/backups/slapd-2.4.57+dfsg-3+deb11u1/cn=config.ldif
Richard
Am 24.07.2023 um 17:26 schrieb Quanah Gibson-Mount:
--On Monday, July 24, 2023 6:16 PM +0200 Richard Rosner rrosner@fsmuw.rwth-aachen.de wrote:
Hi,
I just upgraded our servers from Debian 11 to 12. I'm not sure if this is an upstream change, but slapd 2.5 on Debian 12 doesn't support the HDB and BDB backends
Deprecation of BDB based backends was announced years ago during the OpenLDAP 2.4 series lifecycle.
so the database needs to be migrated to LMDB.
slapadd: could not add entry dn="cn=config" (line=1): Closing DB...
The first set of lines in cn=config.ldif reads
dn: cn=config objectClass: olcGlobal cn: config olcArgsFile: /var/run/slapd/slapd.args olcLogLevel: none olcPidFile: /var/run/slapd/slapd.pid olcToolThreads: 1 structuralObjectClass: olcGlobal entryUUID: 71b384b4-aca9-1032-883a-d9850217023f creatorsName: cn=config createTimestamp: 20130908080726Z entryCSN: 20130908080726.757296Z#000000#000#000000 modifiersName: cn=config modifyTimestamp: 20130908080726Z
Is it actually indented this way? Because that would not be valid LDIF.
--Quanah
--On Monday, July 24, 2023 6:56 PM +0200 Richard Rosner rrosner@fsmuw.rwth-aachen.de wrote:
dn: cn=config objectClass: olcGlobal cn: config olcArgsFile: /var/run/slapd/slapd.args olcLogLevel: none olcPidFile: /var/run/slapd/slapd.pid olcToolThreads: 1 structuralObjectClass: olcGlobal entryUUID: 9a5b5f82-56d4-1039-8be0-4705b1c5590c creatorsName: cn=config createTimestamp: 20190819135427Z olcTLSCACertificateFile: /etc/ssl/certs/geant-intermediates.pem olcTLSCertificateFile: /etc/ssl/certs/auth.domain.de.cert.pem olcTLSCertificateKeyFile: /etc/ssl/private/auth.domain.de.private.pem entryCSN: 20230520140839.289572Z#000000#000#000000 modifiersName: gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth modifyTimestamp: 20230520140839Z contextCSN: 20230520140839.289572Z#000000#000#000000
slapadd -F/etc/ldap/slapd.d -n0 -l /var/backups/slapd-2.4.57+dfsg-3+deb11u1/cn=config.ldif
If the above is the contents of cn=config.ldif then it's not a backup that was done into LDIF format, but a copy of your cn=config database that needs to be exported to cn=config. And the formatting is invalid if it has those indents. Saying it isn't "typical" LDIF format does not make it valid. It's either complies with the RFC or it doesn't. :)
--Quanah
--On Monday, July 24, 2023 10:05 AM -0700 Quanah Gibson-Mount quanah@fast-mail.org wrote:
If the above is the contents of cn=config.ldif then it's not a backup that was done into LDIF format, but a copy of your cn=config database that needs to be exported to cn=config.
* exported to LDIF. ;)
--Quanah
That's very interesting. I'm only doing what the guide tells me to do. If it tells me this is supposed to work, I expect it to work. So it seems I'll have to take this up to the maintainer of the package as their backup script is broken.
Richard
PS: I guess those indentations are coming from faulty copy & paste.
Am 24.07.2023 um 18:05 schrieb Quanah Gibson-Mount:
--On Monday, July 24, 2023 6:56 PM +0200 Richard Rosner rrosner@fsmuw.rwth-aachen.de wrote:
dn: cn=config objectClass: olcGlobal cn: config olcArgsFile: /var/run/slapd/slapd.args olcLogLevel: none olcPidFile: /var/run/slapd/slapd.pid olcToolThreads: 1 structuralObjectClass: olcGlobal entryUUID: 9a5b5f82-56d4-1039-8be0-4705b1c5590c creatorsName: cn=config createTimestamp: 20190819135427Z olcTLSCACertificateFile: /etc/ssl/certs/geant-intermediates.pem olcTLSCertificateFile: /etc/ssl/certs/auth.domain.de.cert.pem olcTLSCertificateKeyFile: /etc/ssl/private/auth.domain.de.private.pem entryCSN: 20230520140839.289572Z#000000#000#000000 modifiersName: gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth modifyTimestamp: 20230520140839Z contextCSN: 20230520140839.289572Z#000000#000#000000
slapadd -F/etc/ldap/slapd.d -n0 -l /var/backups/slapd-2.4.57+dfsg-3+deb11u1/cn=config.ldif
If the above is the contents of cn=config.ldif then it's not a backup that was done into LDIF format, but a copy of your cn=config database that needs to be exported to cn=config. And the formatting is invalid if it has those indents. Saying it isn't "typical" LDIF format does not make it valid. It's either complies with the RFC or it doesn't. :)
--Quanah
--On Monday, July 24, 2023 8:13 PM +0200 Richard Rosner rrosner@fsmuw.rwth-aachen.de wrote:
That's very interesting. I'm only doing what the guide tells me to do. If it tells me this is supposed to work, I expect it to work. So it seems I'll have to take this up to the maintainer of the package as their backup script is broken.
It's more of a literal backup of the exact contents of cn=config (since cn=config is text) than a backup meant to be used by slapadd. Since it's from OpenLDAP 2.4, you would have to have a 2.4 slapat to export it to LDIF. It's unfortunate, because an actual backup made by slapcat prior to the upgrade would be easier to work with for the purposes of upgrading to LMDB.
The general advice for the last 7-8 years was to update your system to use LMDB while on OpenLDAP 2.4.
--Quanah
True, but for that you'd have to check for that and not expect everything to be fine when the main ldap is already set up right. Or at least the maintainer would have to have written an actual backup script or at least written a description that actually works. I would also have taken a path of just having the upgrade fail and roll back when such incompatibilities are found.
What would be the easiest way to get slapd back up and running? Would a simple "changetype: modify" and "replace: " for any changes that actually needed to be done do the trick? I mean the main part in the description was to just replace all mentions of hdb with lmdb, that wouldn't be that much work.
Richard
Am 24.07.2023 um 19:16 schrieb Quanah Gibson-Mount:
--On Monday, July 24, 2023 8:13 PM +0200 Richard Rosner rrosner@fsmuw.rwth-aachen.de wrote:
That's very interesting. I'm only doing what the guide tells me to do. If it tells me this is supposed to work, I expect it to work. So it seems I'll have to take this up to the maintainer of the package as their backup script is broken.
It's more of a literal backup of the exact contents of cn=config (since cn=config is text) than a backup meant to be used by slapadd. Since it's from OpenLDAP 2.4, you would have to have a 2.4 slapat to export it to LDIF. It's unfortunate, because an actual backup made by slapcat prior to the upgrade would be easier to work with for the purposes of upgrading to LMDB.
The general advice for the last 7-8 years was to update your system to use LMDB while on OpenLDAP 2.4.
--Quanah
--On Monday, July 24, 2023 8:29 PM +0200 Richard Rosner rrosner@fsmuw.rwth-aachen.de wrote:
True, but for that you'd have to check for that and not expect everything to be fine when the main ldap is already set up right. Or at least the maintainer would have to have written an actual backup script or at least written a description that actually works. I would also have taken a path of just having the upgrade fail and roll back when such incompatibilities are found.
Honestly the upgrade process should have detected that hdb/bdb was in use prior to upgrading and aborted.
What would be the easiest way to get slapd back up and running? Would a simple "changetype: modify" and "replace: " for any changes that actually needed to be done do the trick? I mean the main part in the description was to just replace all mentions of hdb with lmdb, that wouldn't be that much work.
That, and remove any BDB specific options in the config database. Basically, I'd recusrively copy your 'upgraded' slapd.d/cn=config somewhere so you have it before making changes in case you hit errors, and then modify the contents of slapd.d/cn=config while slapd is stopped. I.e., soemthing like:
stop slapd
cp -pr /etc/openldap/slapd.d /some/backup/location/
cd /etc/openldap/slapd.d
(start making edits where necessary)
restart slapd to ensure it comes up. Note that you will have lost all ability to access the contents of the existing binary BDB-based database, so hopefully that was actually backed up as part of the upgrade process.
stop slapd again
back up the new fixed config database:
slapcat -n 0 -F /etc/openldap/slapd.d -l config.ldif
move your hacked one elsewhere:
mv /etc/openldap/slapd.d /some/other/backup/location
mkdir -p /etc/openldap/slapd.d
slapadd -n 0 -F /etc/openldap/slapd.d -l config.ldif
chown -R <whatever> /etc/openldap/slapd.d if it runs as a non-root user
start slapd
This will generate all the checksums in the config db so you don't get warnings about checksum mismatches.
--Quanah
--On Monday, July 24, 2023 9:27 PM +0200 Richard Rosner rrosner@fsmuw.rwth-aachen.de wrote:
Am 24.07.2023 um 19:55 schrieb Quanah Gibson-Mount:
That, and remove any BDB specific options in the config database.
With that do you mean this part from the guide?
For each configured BDB or HDB database: - Delete any olcDbConfig attributes
Correct :)
--Quanah
Thanks, that did the trick. Slapd is now running again as supposed.
Richard
Am 24.07.2023 um 19:55 schrieb Quanah Gibson-Mount:
--On Monday, July 24, 2023 8:29 PM +0200 Richard Rosner rrosner@fsmuw.rwth-aachen.de wrote:
True, but for that you'd have to check for that and not expect everything to be fine when the main ldap is already set up right. Or at least the maintainer would have to have written an actual backup script or at least written a description that actually works. I would also have taken a path of just having the upgrade fail and roll back when such incompatibilities are found.
Honestly the upgrade process should have detected that hdb/bdb was in use prior to upgrading and aborted.
What would be the easiest way to get slapd back up and running? Would a simple "changetype: modify" and "replace: " for any changes that actually needed to be done do the trick? I mean the main part in the description was to just replace all mentions of hdb with lmdb, that wouldn't be that much work.
That, and remove any BDB specific options in the config database. Basically, I'd recusrively copy your 'upgraded' slapd.d/cn=config somewhere so you have it before making changes in case you hit errors, and then modify the contents of slapd.d/cn=config while slapd is stopped. I.e., soemthing like:
stop slapd
cp -pr /etc/openldap/slapd.d /some/backup/location/
cd /etc/openldap/slapd.d
(start making edits where necessary)
restart slapd to ensure it comes up. Note that you will have lost all ability to access the contents of the existing binary BDB-based database, so hopefully that was actually backed up as part of the upgrade process.
stop slapd again
back up the new fixed config database:
slapcat -n 0 -F /etc/openldap/slapd.d -l config.ldif
move your hacked one elsewhere:
mv /etc/openldap/slapd.d /some/other/backup/location
mkdir -p /etc/openldap/slapd.d
slapadd -n 0 -F /etc/openldap/slapd.d -l config.ldif
chown -R <whatever> /etc/openldap/slapd.d if it runs as a non-root user
start slapd
This will generate all the checksums in the config db so you don't get warnings about checksum mismatches.
--Quanah
--On Monday, July 24, 2023 10:24 PM +0200 Richard Rosner rrosner@fsmuw.rwth-aachen.de wrote:
Thanks, that did the trick. Slapd is now running again as supposed.
Glad to hear it!
Regards, Quanah
openldap-technical@openldap.org
-
Quanah Gibson-Mount -
Richard Rosner