Hello.
I'm struggling a bit with setting up syncrepl replication between two OpenLDAP servers (using version 2.4.39 compiled by the LTB project, on top of RHEL6, if that matters in this case).
Does anyone here have some suggestions on what I should look deeper into here? Is it a known newbie-error I'm making? I can post configuration files, describe how I attempt to set up the replication etc.
The two servers have a replicated cn=config, in addition to two suffixes with their own HDB backend. The first of those suffixes are meant for administrative data, replication user account, etc., and the second suffix is for some end-user accounts/settings.
I seem to have managed to get the first HDB backend to replicate, but I can't get the 2nd to work for some reason (most likely because I'm doing something wrong).
When I start OpenLDAP with some debug logging, I see several log line, but the first ones that catches my interest looks like:
53ac30ff slapd starting 53ac30ff slap_client_connect: URI=ldap://ldap01-testing.aminor.no DN="cn=replicator,ou=admins,ou=internal,o=aminor" ldap_sasl_bind_s failed (49) 53ac30ff do_syncrepl: rid=005 rc 49 retrying (4 retries left)
(this was seen on the node ldap02-testing.aminor.no. The hostnames exist in DNS internally, the two nodes can see each other on the IP level etc.)
Both the working and non-working suffix are configured to use the same replication user (which lives in the 1st suffix). In my case, I have 2 hdb backends, one seems to replicate just fine, the other doesn't. I can use ldapsearch on the suffix for that non-replicating hdb from both nodes to both nodes, and get replies back (running ldapsearch -x, with -D and -w giving the cn=replicator,ou=admins...etc. and password).
I went to the #openldap IRC channel and asked about this issue earlier today, and I saw another person ask about the same "ldap_sasl_bind_s failed (49)" error message as well. He was using a somewhat older OpenLDAP though (2.4.23) on Debian though.
Regards Eivind Olsen
Eivind Olsen wrote:
53ac30ff slapd starting 53ac30ff slap_client_connect: URI=ldap://ldap01-testing.aminor.no DN="cn=replicator,ou=admins,ou=internal,o=aminor" ldap_sasl_bind_s failed (49) 53ac30ff do_syncrepl: rid=005 rc 49 retrying (4 retries left)
49 is "invalidCredentials".
Likely either one of the following reasons are causing this: - entry cn=replicator,ou=admins,ou=internal,o=aminor does not exist - the password is wrong - some ACLs reject authentication
(this was seen on the node ldap02-testing.aminor.no.
Try from ldap02-testing.aminor.no:
ldapwhoami -H ldap://ldap01-testing.aminor.no -x \ -D "cn=replicator,ou=admins,ou=internal,o=aminor" -w <password>
For further questions you should post your config.
Ciao, Michael.
Michael Ströder wrote:
49 is "invalidCredentials". Likely either one of the following reasons are causing this:
- entry cn=replicator,ou=admins,ou=internal,o=aminor does not exist
- the password is wrong
- some ACLs reject authentication
That's what puzzles me. I can from both nodes do ldapsearch as the replication user to both nodes, and that part behaves as I'd expect it to (I get a connection with answers, and if I try to connect with the wrong password I get "ldap_bind: Invalid credentials (49)").
Try from ldap02-testing.aminor.no: ldapwhoami -H ldap://ldap01-testing.aminor.no -x \ -D "cn=replicator,ou=admins,ou=internal,o=aminor" -w <password>
[root@ldap02-testing ~]# ldapwhoami -H ldap://ldap01-testing.aminor.no -x -D "cn=replicator,ou=admins,ou=internal,o=aminor" -w <a_password> dn:cn=replicator,ou=admins,ou=internal,o=aminor
For further questions you should post your config.
I will. At the risk of someone going "OMG, why the h... are you doing it like this?" :)
Here's the contents of cn=config (minus all the stuff under cn=schema,cn=config - since that would have added over 2000 lines). I have hidden the passwords. I have done a diff on the output from cn=config on both servers and it's identical.
[root@ldap01-testing ~]# ldapsearch -x -b "cn=config" -D "cn=admin,cn=config" -w <CONFIG-password> -h ldap01-testing.aminor.no -LLL dn: cn=config objectClass: olcGlobal cn: config olcConfigFile: /usr/local/openldap/etc/openldap/slapd.conf olcConfigDir: /usr/local/openldap/etc/openldap/slapd.d olcArgsFile: /usr/local/openldap/var/run/slapd.args olcAttributeOptions: lang- olcAuthzPolicy: none olcConcurrency: 0 olcConnMaxPending: 100 olcConnMaxPendingAuth: 1000 olcGentleHUP: FALSE olcIdleTimeout: 0 olcIndexSubstrIfMaxLen: 4 olcIndexSubstrIfMinLen: 2 olcIndexSubstrAnyLen: 4 olcIndexSubstrAnyStep: 2 olcIndexIntLen: 4 olcListenerThreads: 1 olcLocalSSF: 71 olcLogLevel: 0 olcPidFile: /usr/local/openldap/var/run/slapd.pid olcReadOnly: FALSE olcSaslSecProps: noplain,noanonymous olcServerID: 101 ldap://ldap01-testing.aminor.no olcServerID: 201 ldap://ldap02-testing.aminor.no olcSockbufMaxIncoming: 262143 olcSockbufMaxIncomingAuth: 16777215 olcThreads: 16 olcTLSCRLCheck: none olcTLSVerifyClient: never olcTLSProtocolMin: 0.0 olcToolThreads: 1 olcWriteTimeout: 0
dn: cn=module{0},cn=config objectClass: olcModuleList cn: module{0} olcModuleLoad: {0}syncprov.la
dn: olcDatabase={-1}frontend,cn=config objectClass: olcDatabaseConfig objectClass: olcFrontendConfig olcDatabase: {-1}frontend olcAccess: {0}to dn.base="" by * read olcAccess: {1}to dn.base="cn=subschema" by * read olcAccess: {2}to * by self write by users read by anonymous auth olcAddContentAcl: FALSE olcLastMod: TRUE olcMaxDerefDepth: 0 olcReadOnly: FALSE olcSchemaDN: cn=Subschema olcSyncUseSubentry: FALSE olcMonitoring: FALSE
dn: olcDatabase={0}config,cn=config objectClass: olcDatabaseConfig olcDatabase: {0}config olcAccess: {0}to * by dn.base="gidNumber=0+uidNumber=0,cn=peercred,cn=externa l,cn=auth" manage by * +0 break olcAddContentAcl: TRUE olcLastMod: TRUE olcMaxDerefDepth: 15 olcReadOnly: FALSE olcRootDN: cn=admin,cn=config olcRootPW: <CONFIG-password> olcSyncUseSubentry: FALSE olcSyncrepl: {0}rid=001 provider=ldap://ldap01-testing.aminor.no binddn ="cn=admin,cn=config" bindmethod=simple credentials=<CONFIG-password> searchbase="cn=co nfig" type=refreshAndPersist retry="5 5 300 5" timeout=1 olcSyncrepl: {1}rid=002 provider=ldap://ldap02-testing.aminor.no binddn ="cn=admin,cn=config" bindmethod=simple credentials=<CONFIG-password> searchbase="cn=co nfig" type=refreshAndPersist retry="5 5 300 5" timeout=1 olcMirrorMode: TRUE olcMonitoring: FALSE
dn: olcOverlay={0}syncprov,olcDatabase={0}config,cn=config objectClass: olcOverlayConfig objectClass: olcSyncProvConfig olcOverlay: {0}syncprov
dn: olcDatabase={1}monitor,cn=config objectClass: olcDatabaseConfig olcDatabase: {1}monitor olcAccess: {0}to * by dn.base="cn=admin,cn=config" read by * none olcAddContentAcl: FALSE olcLastMod: TRUE olcMaxDerefDepth: 15 olcReadOnly: FALSE olcSyncUseSubentry: FALSE olcMonitoring: FALSE
dn: olcDatabase={2}hdb,cn=config objectClass: olcDatabaseConfig objectClass: olcHdbConfig olcDatabase: {2}hdb olcDbDirectory: /usr/local/openldap/var/openldap-data/internal olcSuffix: ou=internal,o=aminor olcAccess: {0}to attrs=userPassword by self write by anonymous auth by dn.chil dren="ou=admins,ou=internal,o=aminor" write by * none olcAccess: {1}to * by self write by dn.children="ou=admins,ou=internal,o=ami nor" write by * read olcLimits: {0}dn.exact="cn=Manager,ou=internal,o=aminor" time.soft=unlimit ed time.hard=unlimited size.soft=unlimited size.hard=unlimited olcRootDN: cn=Manager,ou=internal,o=aminor olcRootPW: <MANAGER-password> olcSyncrepl: {0}rid=003 provider=ldap://ldap01-testing.aminor.no binddn ="cn=replicator,ou=admins,ou=internal,o=aminor" bindmethod=simple creden tials=<REPLICATOR-password> searchbase="ou=internal,o=aminor" type=refreshAndPersist retry="5 5 5 +" timeout=3 olcSyncrepl: {1}rid=004 provider=ldap://ldap02-testing.aminor.no binddn ="cn=replicator,ou=admins,ou=internal,o=aminor" bindmethod=simple creden tials=<REPLICATOR-password> searchbase="ou=internal,o=aminor" type=refreshAndPersist retry="5 5 5 +" timeout=3 olcMirrorMode: TRUE olcDbCacheSize: 1000 olcDbCheckpoint: 1024 10 olcDbConfig: {0}set_cachesize 0 10485760 0 olcDbConfig: {1}set_lg_bsize 2097152 olcDbConfig: {2}set_lg_dir /usr/local/berkeleydb/openldap-logs/internal olcDbConfig: {3}set_flags DB_LOG_AUTOREMOVE olcDbIDLcacheSize: 3000 olcDbIndex: uid pres,eq,sub olcDbIndex: cn,sn,displayName pres,eq,approx,sub olcDbIndex: uidNumber,gidNumber eq olcDbIndex: memberUid eq olcDbIndex: objectClass eq olcDbIndex: entryUUID pres,eq olcDbIndex: entryCSN pres,eq
dn: olcOverlay={0}syncprov,olcDatabase={2}hdb,cn=config objectClass: olcOverlayConfig objectClass: olcSyncProvConfig olcOverlay: {0}syncprov
dn: olcDatabase={3}hdb,cn=config objectClass: olcDatabaseConfig objectClass: olcHdbConfig olcDatabase: {3}hdb olcDbDirectory: /usr/local/openldap/var/openldap-data/radius olcSuffix: ou=radius,ou=no,o=aminor olcAccess: {0}to attrs=userPassword by self write by anonymous auth by dn.chil dren="ou=admins,ou=radius,ou=no,o=aminor" write by * none olcAccess: {1}to * by self write by dn.children="ou=admins,ou=internal,o=ami nor" write by group.exact="cn=radius write,ou=groups,ou=internal,o=amin or" write by group.exact="cn=radius read,ou=groups,ou=internal,o=aminor" read by * read olcLimits: {0}dn.exact="cn=Manager,ou=radius,ou=no,o=aminor" time.soft=unl imited time.hard=unlimited size.soft=unlimited size.hard=unlimited olcRootDN: cn=Manager,ou=radius,ou=no,o=aminor olcRootPW: <MANAGER-password> olcSyncrepl: {0}rid=005 provider=ldap://ldap01-testing.aminor.no binddn ="cn=replicator,ou=admins,ou=internal,o=aminor" bindmethod=simple credent ials=<REPLICATOR-password> searchbase="ou=radius,ou=no,o=aminor" type=refreshAndPersis t retry="5 5 5 +" timeout=3 olcSyncrepl: {1}rid=006 provider=ldap://ldap02-testing.aminor.no binddn ="cn=replicator,ou=admins,ou=internal,o=aminor" bindmethod=simple credent ials=<REPLICATOR-password> searchbase="ou=radius,ou=no,o=aminor" type=refreshAndPersi st retry="5 5 5 +" timeout=3 olcMirrorMode: TRUE olcDbCacheSize: 1000 olcDbCheckpoint: 1024 10 olcDbConfig: {0}set_cachesize 0 10485760 0 olcDbConfig: {1}set_lg_bsize 2097152 olcDbConfig: {2}set_lg_dir /usr/local/berkeleydb/openldap-logs/radius olcDbConfig: {3}set_flags DB_LOG_AUTOREMOVE olcDbIDLcacheSize: 3000 olcDbIndex: uid pres,eq,sub olcDbIndex: cn,sn,displayName pres,eq,approx,sub olcDbIndex: uidNumber,gidNumber eq olcDbIndex: memberUid eq olcDbIndex: objectClass eq olcDbIndex: entryUUID pres,eq olcDbIndex: entryCSN pres,eq olcDbIndex: amiCustomerId pres,eq,sub olcDbIndex: amipppProfileType pres,eq olcDbIndex: amiLineId pres,eq,sub
dn: olcOverlay={0}syncprov,olcDatabase={3}hdb,cn=config objectClass: olcOverlayConfig objectClass: olcSyncProvConfig olcOverlay: {0}syncprov
Here is the contents of ou=internal,o=aminor (I also verified that this looks the same on both servers):
dn: ou=internal,o=aminor objectClass: top objectClass: organizationalUnit ou: internal
dn: ou=groups,ou=internal,o=aminor objectClass: organizationalUnit ou: groups description: generic groups branch
dn: cn=radius read,ou=groups,ou=internal,o=aminor objectClass: groupOfNames cn: radius read description: read permission to radius tree member: cn=radius app user,ou=applications,ou=internal,o=aminor
dn: cn=radius write,ou=groups,ou=internal,o=aminor objectClass: groupOfNames cn: radius write description: write permission to radius tree member: cn=radius app user,ou=applications,ou=internal,o=aminor
dn: ou=people,ou=internal,o=aminor objectClass: organizationalUnit ou: people description: generic people branch
dn: ou=admins,ou=internal,o=aminor objectClass: organizationalUnit ou: admins description: administrative accounts
dn: cn=replicator,ou=admins,ou=internal,o=aminor cn: replicator sn: user objectClass: person userPassword: <REPLICATOR-password>
dn: ou=applications,ou=internal,o=aminor objectClass: organizationalUnit ou: applications description: application users
dn: cn=radius app user,ou=applications,ou=internal,o=aminor objectClass: inetOrgPerson objectClass: organizationalPerson objectClass: person objectClass: top cn: radius app user sn: radius app user userPassword: <RADIUS-password>
Regards Eivind Olsen
Eivind Olsen wrote:
Michael Ströder wrote:
49 is "invalidCredentials". Likely either one of the following reasons are causing this:
- entry cn=replicator,ou=admins,ou=internal,o=aminor does not exist
- the password is wrong
- some ACLs reject authentication
That's what puzzles me. I can from both nodes do ldapsearch as the replication user to both nodes, and that part behaves as I'd expect it to (I get a connection with answers, and if I try to connect with the wrong password I get "ldap_bind: Invalid credentials (49)").
dn: olcDatabase={3}hdb,cn=config objectClass: olcDatabaseConfig objectClass: olcHdbConfig olcDatabase: {3}hdb olcDbDirectory: /usr/local/openldap/var/openldap-data/radius olcSuffix: ou=radius,ou=no,o=aminor
olcSyncrepl: {0}rid=005 provider=ldap://ldap01-testing.aminor.no binddn ="cn=replicator,ou=admins,ou=internal,o=aminor" bindmethod=simple credent ials=<REPLICATOR-password> searchbase="ou=radius,ou=no,o=aminor" type=refreshAndPersis t retry="5 5 5 +" timeout=3 olcSyncrepl: {1}rid=006 provider=ldap://ldap02-testing.aminor.no binddn ="cn=replicator,ou=admins,ou=internal,o=aminor" bindmethod=simple credent ials=<REPLICATOR-password> searchbase="ou=radius,ou=no,o=aminor" type=refreshAndPersi st retry="5 5 5 +" timeout=3
Clearly you have a mistake in the password of one of these two lines, because if they were identical they would be identical in length, but they wrap the "refreshAndPersist" in two different positions.
Howard Chu wrote:
Eivind Olsen wrote:
Michael Ströder wrote:
49 is "invalidCredentials". Likely either one of the following reasons are causing this:
- entry cn=replicator,ou=admins,ou=internal,o=aminor does not exist
- the password is wrong
- some ACLs reject authentication
That's what puzzles me. I can from both nodes do ldapsearch as the replication user to both nodes, and that part behaves as I'd expect it to (I get a connection with answers, and if I try to connect with the wrong password I get "ldap_bind: Invalid credentials (49)").
dn: olcDatabase={3}hdb,cn=config objectClass: olcDatabaseConfig objectClass: olcHdbConfig olcDatabase: {3}hdb olcDbDirectory: /usr/local/openldap/var/openldap-data/radius olcSuffix: ou=radius,ou=no,o=aminor
olcSyncrepl: {0}rid=005 provider=ldap://ldap01-testing.aminor.no binddn ="cn=replicator,ou=admins,ou=internal,o=aminor" bindmethod=simple credent ials=<REPLICATOR-password> searchbase="ou=radius,ou=no,o=aminor" type=refreshAndPersis t retry="5 5 5 +" timeout=3 olcSyncrepl: {1}rid=006 provider=ldap://ldap02-testing.aminor.no binddn ="cn=replicator,ou=admins,ou=internal,o=aminor" bindmethod=simple credent ials=<REPLICATOR-password> searchbase="ou=radius,ou=no,o=aminor" type=refreshAndPersi st retry="5 5 5 +" timeout=3
Clearly you have a mistake in the password of one of these two lines, because if they were identical they would be identical in length, but they wrap the "refreshAndPersist" in two different positions.
PS: Most mistakes are obvious if you actually pay attention to details. But LDIF config format makes mistakes like these even more obvious. Good luck emailing a slapd.conf with this type of mistake in it and having the problem still be apparent after being mangled and rewrapped by multiple mail agents.
Howard Chu wrote:
Howard Chu wrote:
Clearly you have a mistake in the password of one of these two lines, because if they were identical they would be identical in length, but they wrap the "refreshAndPersist" in two different positions.
PS: Most mistakes are obvious if you actually pay attention to details.
Hmm, maybe it's obvious for you but I don't see the fault since there are no real passwords in there. The line wrapping is messed up my mail formatting anyway.
But LDIF config format makes mistakes like these even more obvious. Good luck emailing a slapd.conf with this type of mistake in it and having the problem still be apparent after being mangled and rewrapped by multiple mail agents.
LDIF can be mangled too because RFC 2849 mandates 76 chars per line and most non-HTML MUAs wrap lines at 72 chars per line.
I prefer examining attribute 'olcSyncrepl' with web2ldap. Recent versions display a clickable LDAP URL parsed from the attribute value including bind-DN and password which you can directly chase with web2ldap with one click. This gives you a really quick and correct check whether the value actually works. :-)
Ciao, Michael.
Howard Chu wrote:
Clearly you have a mistake in the password of one of these two lines, because if they were identical they would be identical in length, but
they wrap the
"refreshAndPersist" in two different positions.
PS: Most mistakes are obvious if you actually pay attention to details.
Excellent! Well spotted! You are absolutely correct. I'm now kicking myself for not seeing that myself already.
Cheers! Eivind Olsen
openldap-technical@openldap.org
-
Eivind Olsen -
Howard Chu -
Michael Ströder