Hello,
we have a cluster of LDAP servers consisting of one provider and 4 consumer. We're upgrading the os of the systems by proving an replacement system for each of the five systems. Then we stop the slapd on the system that should be replaced, dump the database with slapcat, copy it to new system, switch hostname and ip of both systems. We shut down the old and reboot the new system. Then we slapadd the database on the new system. So far so good...
We started with the provider. After importing the database and starting the slapd on the new provider, we get errors for the syncrepl state on all consumer systems: "Can't get Context CSN with SID <x> from ldap+tls://localhost. Please set SID with -I option."
We're using check_ldap_syncrepl_status from ltb-project with icinga2 to monitor the replication.
I don't know how to fix this. When I modify an entry in the provider, it is synced to the consumer. But the status error stays. On all four consumers.
I've then 'upgraded' a consumer by starting the slapd on the replacment system without any database. The system started the sync and after completion the error was gone. But syncing takes more time than importing a dump from a local file. And in a case where we have to rebuild the provider from scratch after a crash, it might be not an option to resync one consumer after the other to rebuild the complete cluster.
Is there a method to fix the problem?
Syncrepl configuration on the provider
overlay syncprov syncprov-checkpoint 100 10 syncprov-sessionlog 100
Syncrepl configuration on the consumer
syncrepl rid=<x> provider=ldap://yxz:389 binddn="cn=xxxxxxx" credentials="xxxxxxx" tls_cacertdir=/etc/pki/tls/certs tls_reqcert=demand bindmethod=simple starttls=critical searchbase="xxxxxx" type=refreshAndPersist retry="10 10 300 +" filter="(objectClass=*)" scope=sub attrs="*,+" sizelimit=unlimited timelimit=unlimited schemachecking=off
Regards
Berthold Cogel
On Mon, 21 Sep 2020, Berthold Cogel wrote:
we have a cluster of LDAP servers consisting of one provider and 4 consumer. We're upgrading the os of the systems by proving an replacement system for each of the five systems. Then we stop the slapd on the system that should be replaced, dump the database with slapcat, copy it to new system, switch hostname and ip of both systems. We shut down the old and reboot the new system. Then we slapadd the database on the new system. So far so good...
The options passed to slapadd (and maybe slapcat) matter. In particular, use (or not) of -w and -S are relevant. Please provide more information about what you did.
Philip Guenther
Am 21.09.20 um 21:37 schrieb Philip Guenther:
On Mon, 21 Sep 2020, Berthold Cogel wrote:
we have a cluster of LDAP servers consisting of one provider and 4 consumer. We're upgrading the os of the systems by proving an replacement system for each of the five systems. Then we stop the slapd on the system that should be replaced, dump the database with slapcat, copy it to new system, switch hostname and ip of both systems. We shut down the old and reboot the new system. Then we slapadd the database on the new system. So far so good...
The options passed to slapadd (and maybe slapcat) matter. In particular, use (or not) of -w and -S are relevant. Please provide more information about what you did.
Philip Guenther
Hello Philip,
dump is done with a script that acts as a wrapper around slapcat. It zips the dumps and discards older dumps after a configurable time.
We simply call slapcat like this:
SLAPCAT=/usr/sbin/slapcat ... $SLAPCAT -f $SLAPD_CONFIG -b $BASE -l $BACKUP_PATH/$DUMP_FILE
For slapadd we use this one:
slapadd -f /etc/openldap/slapd.conf -b <Base> -l <Path to dump>
Regards Berthold Cogel
--On Monday, September 21, 2020 6:32 PM +0200 Berthold Cogel cogel@uni-koeln.de wrote:
We started with the provider. After importing the database and starting the slapd on the new provider, we get errors for the syncrepl state on all consumer systems: "Can't get Context CSN with SID <x> from ldap+tls://localhost. Please set SID with -I option."
You left out most of the configuration bits so it's impossible to say how you've done this. Certainly not an error I've ever seen. Please provide the full provider config MINUS passwords.
Regards, Quanah
--
Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
Am 22.09.20 um 17:21 schrieb Quanah Gibson-Mount:
--On Monday, September 21, 2020 6:32 PM +0200 Berthold Cogel cogel@uni-koeln.de wrote:
We started with the provider. After importing the database and starting the slapd on the new provider, we get errors for the syncrepl state on all consumer systems: "Can't get Context CSN with SID <x> from ldap+tls://localhost. Please set SID with -I option."
You left out most of the configuration bits so it's impossible to say how you've done this. Certainly not an error I've ever seen. Please provide the full provider config MINUS passwords.
You asked for it... ;-)
We use a static configuration generated from templates by cfengine. And we've split the configuration in a number of includes. Some are generated and some are simlpy copied to all LDAP servers. It's easier to handle for a small team with only few people with a kind of LDAP knowledge...
slapd.conf (the framework) ----------
######################################### # # # THIS FILE IS CONTROLLED BY CFENGINE # # # # *** DO NOT EDIT LOCALLY *** # # # #########################################
# # See slapd.conf(5) for details on configuration options. # This file should NOT be world readable. #
# Load schema files include /etc/openldap/schema.inc
# Define global ACLs to disable default read access.
# allgemeine ACLs ausgelagert fuer Verteilung z.B. durch cfengine include /etc/openldap/acl.inc
# Allow LDAPv2 client connections. This is NOT the default. allow bind_v2
# Do not enable referrals until AFTER you have a working directory # service AND an understanding of referrals. #referral ldap://root.openldap.org
pidfile /var/run/openldap/slapd.pid argsfile /var/run/openldap/slapd.args
# Load dynamic backend modules: modulepath /usr/lib64/openldap # moduleload back_bdb.la # moduleload back_hdb.la # moduleload back_ldap.la # auditlog static # moduleload auditlog.la
# fuer Provider moduleload syncprov.la moduleload accesslog.la
# TLS definitions TLSCACertificatePath /etc/pki/tls/certs TLSCipherSuite HIGH:TLSv1.2:!SSLv3:!SSLv2
TLSCertificateFile /var/security/certs/sphinx.cert/sphinx_cert.pem TLSCertificateKeyFile /var/security/certs/sphinx.cert/sphinx.key TLSDHParamFile /var/security/certs/sphinx.cert/dhparams.pem
# Sample security restrictions # Require integrity protection (prevent hijacking) # Require 112-bit (3DES or better) encryption for updates # Require 63-bit encryption for simple bind security ssf=64 update_ssf=112 simple_bind=64
localSSF 256
sizelimit unlimited idletimeout 0 loglevel none
####################################################################### # DIT database definitions #######################################################################
database mdb suffix "dc=uni-koeln,dc=de" rootdn "cn=ldapmgr,dc=uni-koeln,dc=de"
directory /var/lib/ldap maxsize 3221225472 checkpoint 4096 10
# Define Limits
include /etc/openldap/limits.inc
# Define Index and Replication
include /etc/openldap/slapd.inc
####################################################################### # Monitoring database definitions #######################################################################
include /etc/openldap/monitor.inc
####################################################################### # Config database definitions #######################################################################
include /etc/openldap/config.inc
slapd.inc (the host specific part) ----------
######################################### # # # THIS FILE IS CONTROLLED BY CFENGINE # # # # *** DO NOT EDIT LOCALLY *** # # # #########################################
index mail eq index cn eq,sub index sn eq,sub index givenname eq,sub index uid eq index uidNumber eq index gidNumber eq index memberUid eq index member eq index objectclass eq index entryCSN eq index entryUUID eq index UniCologneMailPolicy eq index description sub index UniCologneDescription4Klips sub index UniColognePersonStatus eq index UniCologneKIMIdentID eq index UniColognePrimaryMailAlias eq index UniCologneMailAlias eq,sub index UniColognePersk eq,sub index UniCologneKST eq,sub index UniColognePKZKST eq,sub index UniCologneCourseOfStudy eq index UniCologneQualification eq index UniCologneFaculty eq index UniCologneAccountStatus eq index UniCologneHPCFlag eq
overlay syncprov syncprov-checkpoint 100 10 syncprov-sessionlog 100
limits.inc ----------
######################################### # # # THIS FILE IS CONTROLLED BY CFENGINE # # # # *** DO NOT EDIT LOCALLY *** # # # #########################################
# Limits for user/groups
# Unlimited limits group/organizationalRole/roleOccupant="cn=admins,ou=OrgRoles,dc=uni-koeln,dc=de" size=unlimited time=unlimited limits group/organizationalRole/roleOccupant="cn=consumer,ou=OrgRoles,dc=uni-koeln,dc=de" size=unlimited time=unlimited limits group/organizationalRole/roleOccupant="cn=nolimits,ou=OrgRoles,dc=uni-koeln,dc=de" size=unlimited time=unlimited
acl.inc ----------
######################################### # # # THIS FILE IS CONTROLLED BY CFENGINE # # # # *** DO NOT EDIT LOCALLY *** # # # #########################################
# Basis ACLs #
# Lesender Vollzugriff für Monitoring auf cn=monitor access to dn.sub="cn=monitor" by dn.exact="cn=ldapmgr,cn=monitor" read by group/organizationalRole/roleOccupant="cn=admins,ou=OrgRoles,dc=uni-koeln,dc=de" read by group/organizationalRole/roleOccupant="cn=monitoring,ou=OrgRoles,dc=uni-koeln,dc=de" read by * break
# Lesender Vollzugriff für Monitoring auf cn=config access to dn.sub="cn=config" by group/organizationalRole/roleOccupant="cn=admins,ou=OrgRoles,dc=uni-koeln,dc=de" read by * break
# Schreibender Vollzugriff für Admin # Lesender Vollzugriff für Replikation # Replikations-User müssen alles lesen dürfen access to * by group/organizationalRole/roleOccupant="cn=admins,ou=OrgRoles,dc=uni-koeln,dc=de" write by group/organizationalRole/roleOccupant="cn=consumer,ou=OrgRoles,dc=uni-koeln,dc=de" read by * break
# Subtree Group # Schreibender Vollzugriff für IDM auf Nodes access to dn.subtree="ou=Group,dc=uni-koeln,dc=de" by group/organizationalRole/roleOccupant="cn=provider,ou=OrgRoles,dc=uni-koeln,dc=de" write by anonymous read by * read by * break
...... several other trees for special purposes
### START: ACLs für authconsumer # # per <ldap-group>.acl.json generiert #
# ou=People, uid # access to dn.subtree="ou=People,dc=uni-koeln,dc=de" attrs="entry,objectclass,uid" by group/organizationalRole/roleOccupant="cn=r-uid,ou=OrgRoles,dc=uni-koeln,dc=de" read by * break
...... gives acces to special attributes
# # # ### END: Acls für authconsumer
# Subtree AuthConsumer # Lesender Zugriff und Auth.. access to dn.subtree="ou=AuthConsumer,dc=uni-koeln,dc=de" by self read by anonymous auth by * break
# Dito für Subtree DSA # Lesender Zugriff und Auth.. access to dn.subtree="ou=DSA,dc=uni-koeln,dc=de" by self read by anonymous auth by * break
# Subtree OrgRoles # Lesender Zugriff und Auth.. access to dn.subtree="ou=OrgRoles,dc=uni-koeln,dc=de" by self read by anonymous auth by * break
# Authentisierten Usern das Auslesen der Root_DSE, der BaseDN und der Schemata erlauben
# Alle dürfen Root-DSE lesen access to dn.exact="" by * read
# Alle dürfen Subschemata lesen access to dn.base="cn=Subschema" by * read
# Alle dürfen den Suffix lesen access to dn.base="dc=uni-koeln,dc=de" by * read
# Erste Ebene unterhalb von dc=uni-koeln,dc=de für alle lesbar, # nicht weiter, da sonst alles lesbar # Regel am Ende, sonst werden erweiterte Schreibrechte tiefer im Baum negiert access to dn.one="dc=uni-koeln,dc=de" by * read
And at the end monitor.inc and config.inc: Access to monitoring and online configation (to change loglevel on the fly)
Importing the dump of the database into a clean system leads to a broken synchonisation with all consumers. But when I delete the database on a consumer, the synchronisation starts at once.
Regards Berthold Cogel
--On Wednesday, September 23, 2020 11:06 AM +0200 Berthold Cogel cogel@uni-koeln.de wrote:
Am 22.09.20 um 17:21 schrieb Quanah Gibson-Mount:
--On Monday, September 21, 2020 6:32 PM +0200 Berthold Cogel cogel@uni-koeln.de wrote:
We started with the provider. After importing the database and starting the slapd on the new provider, we get errors for the syncrepl state on all consumer systems: "Can't get Context CSN with SID <x> from ldap+tls://localhost. Please set SID with -I option."
You left out most of the configuration bits so it's impossible to say how you've done this. Certainly not an error I've ever seen. Please provide the full provider config MINUS passwords.
This does not look like the config for the provider, as requested.
Regards, Quanah
--
Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
--On Wednesday, September 23, 2020 9:23 AM -0700 Quanah Gibson-Mount quanah@symas.com wrote:
We started with the provider. After importing the database and starting the slapd on the new provider, we get errors for the syncrepl state on all consumer systems: "Can't get Context CSN with SID <x> from ldap+tls://localhost. Please set SID with -I option."
You left out most of the configuration bits so it's impossible to say how you've done this. Certainly not an error I've ever seen. Please provide the full provider config MINUS passwords.
This does not look like the config for the provider, as requested.
Or to be more precise, you talk about multiple providers, but the config you sent could at most be the config for a single provider, in which case I think I'd expect the error you have hit, since multiprovider replication requires the providers to have non-zero serverID's.
--Quanah
--
Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
Am 23.09.20 um 21:55 schrieb Quanah Gibson-Mount:
--On Wednesday, September 23, 2020 9:23 AM -0700 Quanah Gibson-Mount quanah@symas.com wrote:
We started with the provider. After importing the database and starting the slapd on the new provider, we get errors for the syncrepl state on all consumer systems: "Can't get Context CSN with SID <x> from ldap+tls://localhost. Please set SID with -I option."
You left out most of the configuration bits so it's impossible to say how you've done this. Certainly not an error I've ever seen. Please provide the full provider config MINUS passwords.
This does not look like the config for the provider, as requested.
Or to be more precise, you talk about multiple providers, but the config you sent could at most be the config for a single provider, in which case I think I'd expect the error you have hit, since multiprovider replication requires the providers to have non-zero serverID's.
--Quanah
--
Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
We use cfengine to generate the configuration for all our LDAP servers. Provider and Consumer. To make it easier to handle we split the configuration in different parts. Some are simply copied and some are generated from templates or JSON files. This process is controled by machine classes and hostnames. Each part starts with this header:
######################################### # # # THIS FILE IS CONTROLLED BY CFENGINE # # # # *** DO NOT EDIT LOCALLY *** # # # #########################################
All those different fragments are includes for the slapd.conf. What I've send are the configuration parts for one specific provider.
The consumers that are associated with this provider, do get a different slapd.inc file, with the same index definitions and the consumer part of the configuration. And of course they do get other certificate names in the slapd.conf.
Regards Berthold Cogel
--On Thursday, September 24, 2020 12:06 PM +0200 Berthold Cogel cogel@uni-koeln.de wrote:
We use cfengine to generate the configuration for all our LDAP servers. Provider and Consumer. To make it easier to handle we split the configuration in different parts. Some are simply copied and some are generated from templates or JSON files. This process is controled by machine classes and hostnames. Each part starts with this header:
Actually, I just looked at the OpenLDAP source, and the error you are reporting appears to come from the LTB project. It could be that their tool is only designed to work with multi provider instances, and your setup as far as OpenLDAP is concerned may be ok. I can't really say because it's difficult to determine what your end goal is.
--Quanah
--
Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
openldap-technical@openldap.org
-
Berthold Cogel -
Philip Guenther -
Quanah Gibson-Mount