Hello
my syncrepl consumer does get initial synchronisation, but then is never updated
I struggled a lot , to finally discover that contextCNS is not created on that replica, that might explain why it is not updated after initial sync .
after different "try and catch" (ACLs, checpoint values etc ...) I finally discovered that setting schemachecking to *Off* (cf full syncrepl config below) on the consumer syncrepl config corrected the pb, contextCNS is then finally created after inital sync (after few dozen of seconds ...) and subsequent updated on master does get replicated on consumer.
Shemas a stricly the same on provider (master) end consumer (replica)
what would explain that schemachecking directive to break syncrepl contextCNS to be created ?
is it wrong to run replicas with schemacheck Off ?
maybe related , but no real solution to me :
https://stackoverflow.com/questions/6525984/syncrepl-syncing-but-not-updatin...
regards
syncrepl rid=081 provider=ldaps://master.int.fr type=refreshAndPersist searchbase="dc=int,dc=fr" filter="(objectClass=*)" attrs="*,+" scope=sub schemachecking=*off* bindmethod=simple retry="60 10 300 +" binddn="cn=rep,ou=dsa,dc=int,dc=fr" credentials="secret" updateref ldaps://master.int.fr:636
--On Wednesday, February 12, 2020 9:16 PM +0100 jehan.procaccia@imtbs-tsp.eu wrote:
What version of OpenLDAP are you using?
--Quanah
--
Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
I am running on a centos 8 system that does'nt provide openldap-server packages as you might know cf https://access.redhat.com/solutions/2440481
So I run openldap server from ltb-project packages: https://ltb-project.org/documentation/openldap-rpm which is actually: *openldap-ltb-2.4.48-2.el8.x86_64*
thanks .
Le 12/02/2020 à 23:54, Quanah Gibson-Mount a écrit :
--On Wednesday, February 12, 2020 9:16 PM +0100 jehan.procaccia@imtbs-tsp.eu wrote:
What version of OpenLDAP are you using?
--Quanah
--
Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
--On Thursday, February 13, 2020 8:53 AM +0100 jehan.procaccia@imtbs-tsp.eu wrote:
I am running on a centos 8 system that does'nt provide openldap-server packages as you might know
Ok, thanks. I wanted to be sure it was something recent prior to going to far into the depths of syncrepl. :)
As a general note, I do see that you're using standard Syncrepl. I would highly advise using delta-syncrepl instead.
For your specific issue, it would be useful to have a copy of the cn=config database (slapcat -n 0 -l /tmp/config.ldif) from both the replica and the master, with all passwords scrubbed. The syncrepl statement on its own doesn't really provide any useful information as far potentially being able to diagnose issues.
Regards, Quanah
--
Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
indeed , https://www.openldap.org/doc/admin24/replication.html#Delta-syncrepl seems to be far more efficient in most cases, I'll give it a try , thanks .
back to my contextCNS missing when schemachecking if On , here are the full cn=config requested: master : https://filez.imtbs-tsp.eu/kj557s7ps replica: https://filez.imtbs-tsp.eu/lgcznk11jp
be aware that I load lots of schema [1], that's why those config are a bit large I have also relatively complex ACLs (13 lignes) , I tried to simplified them while debuging but with no success in regard of my pb .
thanks a lot for your help and analyse .
[1] include /usr/local/openldap/etc/openldap/schema/core.schema include /usr/local/openldap/etc/openldap/schema/corba.schema include /usr/local/openldap/etc/openldap/schema/cosine.schema include /usr/local/openldap/etc/openldap/schema/duaconf.schema include /usr/local/openldap/etc/openldap/schema/dyngroup.schema include /usr/local/openldap/etc/openldap/schema/inetorgperson.schema include /usr/local/openldap/etc/openldap/schema/java.schema include /usr/local/openldap/etc/openldap/schema/misc.schema include /usr/local/openldap/etc/openldap/schema/nis.schema include /usr/local/openldap/etc/openldap/schema/openldap.schema include /usr/local/openldap/etc/openldap/schema/ppolicy.schema include /usr/local/openldap/etc/openldap/schema/collective.schema include /usr/local/openldap/etc/openldap/schema/supann-2019-02-05.schema include /usr/local/openldap/etc/openldap/schema/eduperson-200412.schema include /usr/local/openldap/etc/openldap/schema/schac-20090326-1.4.0.schema include /usr/local/openldap/etc/openldap/schema/samba.schema include /usr/local/openldap/etc/openldap/schema/autofs.schema include /usr/local/openldap/etc/openldap/schema/int.schema
Jehan PROCACCIA Ingénieur systèmes et réseaux Membre du comité de pilotage REVE : Réseau d’Évry Val d'Essonne et THD +33160764436 9 rue Charles Fourier - 91011 Evry Cedex [ https://www.imt-bs.eu/ | www.imt-bs.eu ] - [ https://www.telecom-sudparis.eu/ | www.telecom-sudparis.eu ]
----- Mail original ----- De: "Quanah Gibson-Mount" quanah@symas.com À: "jehan procaccia" jehan.procaccia@imtbs-tsp.eu, openldap-technical@openldap.org Envoyé: Jeudi 13 Février 2020 22:50:16 Objet: Re: syncrepl syncing but not updating, contextCNS missing and schemacheck off
--On Thursday, February 13, 2020 8:53 AM +0100 jehan.procaccia@imtbs-tsp.eu wrote:
I am running on a centos 8 system that does'nt provide openldap-server packages as you might know
Ok, thanks. I wanted to be sure it was something recent prior to going to far into the depths of syncrepl. :)
As a general note, I do see that you're using standard Syncrepl. I would highly advise using delta-syncrepl instead.
For your specific issue, it would be useful to have a copy of the cn=config database (slapcat -n 0 -l /tmp/config.ldif) from both the replica and the master, with all passwords scrubbed. The syncrepl statement on its own doesn't really provide any useful information as far potentially being able to diagnose issues.
Regards, Quanah
--
Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
--On Friday, February 14, 2020 8:51 AM +0100 Jehan PROCACCIA jehan.procaccia@imtbs-tsp.eu wrote:
indeed , https://www.openldap.org/doc/admin24/replication.html#Delta-syncrepl seems to be far more efficient in most cases, I'll give it a try , thanks
Hi,
I have the following notes:
This ACL on the replica (and similar one on the master) is non-sensical:
{0}to attrs=userPassword,sambaNTPassword,sambaPwdLastSet by self auth by anonymous auth by dn.base="cn=repint,ou=dsa,dc=int,dc=fr" r ead by * none
"by self auth" doesn't do anything, since you have to already be authenticated to be "self". This should be fixed on both the master and the replica.
You also switch format on the replica in ACLs {1} and {2}, although they are covering the same namespace. You could make them read cleaner by being consistent (Either use * or dn.subtree="dc=int,dc=fr" consistently).
On the replica, your syncrepl stanza is a bit odd. You specify you want to use a simple bind, and then you provide a TLS key and TLS cert, which would be used for client side cert auth. If you're doing a simple bind, you should only need to set the tls_cacert and tls_reqcert parameters here. I would also strongly advise setting a keepalive (I usually use 240:10:30)
You also seem to have three different replication accounts in use, as the master has ACLs for these identities:
cn=repint,ou=dsa,dc=int,dc=fr cn=replicator,ou=system,dc=int,dc=fr cn=repz,ou=system,dc=int,dc=fr
On the master, ACL {1} is a bit odd, in that you list a bunch of entities as having read access, and then end with "by * read". You could drastically simplify this ACL by making it just list your write access for the ldapadmin group followed with by * read
On the master, ACL {3} is problematic as it grants write access to internal attributes that only the slapd process should be modifying, such as: modifyTimestamp,createTimestamp,structuralObjectClass,creatorsName,entryCSN,modifiersName,subschemaSubentry,hasSubordinates,ObjectClass
ACL {6} has a similar bit I noted previously about by * read.
ACL {7} should just be "by * read"
on the master, the olcLimits could be simplified to just be:
olcLimits: {0}dn.base="cn=repint,ou=dsa,dc=int,dc=fr" size=unlimited time=unlimited
I would note that there are missing limit exceptions for the other 2 replicator accounts if that is indeed what they are.
Generally I don't see anything obviously incorrect as far as the replication portion goes, so it's not clear to me why you're seeing the behavior you got. I've never set schemachecking to be on when configuring sync replication of either type, so I wonder if there's a bug in there. The OpenLDAP test suite also does not set schema checking to be on, so I'll see if tweaking it there causes any incorrect failures.
Regards, Quanah
--
Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
I corrected all my meaningless ACLs , thank for the advices ! some of them last from more than 15 years ago when I was discovering openldap and was messing with ACLs .. I also added the keepalive parameter to syncrepl regarding the different replication account, it's for differents replicas externalise to other entities which has their own credential, but you're right that last two ones miss the size=unlimited time=unlimited
regarding my initial pb, for now, I keep schemachecking to Off on the replicas in odrder to have contextCNS created after inital sync (after few dozen of seconds ...) . Then subsequent updated on master also does get replicated on consumer, so everything works fine. After all if schemacheck is On on the master, as it is the only one that do the Writes, if there is a write attempts that does'nt respect schema, it will be refused on the master and never be presented on the replicas .
If ever you get to test this schemacking directive to On for syncrepl, let us know .
Thanks .
Jehan PROCACCIA Ingénieur systèmes et réseaux Membre du comité de pilotage REVE : Réseau d’Évry Val d'Essonne et THD +33160764436 9 rue Charles Fourier - 91011 Evry Cedex [ https://www.imt-bs.eu/ | www.imt-bs.eu ] - [ https://www.telecom-sudparis.eu/ | www.telecom-sudparis.eu ]
----- Mail original ----- De: "Quanah Gibson-Mount" quanah@symas.com À: "jehan procaccia" jehan.procaccia@imtbs-tsp.eu Cc: "openldap-technical" openldap-technical@openldap.org Envoyé: Vendredi 14 Février 2020 20:05:32 Objet: Re: syncrepl syncing but not updating, contextCNS missing and schemacheck off
--On Friday, February 14, 2020 8:51 AM +0100 Jehan PROCACCIA jehan.procaccia@imtbs-tsp.eu wrote:
indeed , https://www.openldap.org/doc/admin24/replication.html#Delta-syncrepl seems to be far more efficient in most cases, I'll give it a try , thanks
Hi,
I have the following notes:
This ACL on the replica (and similar one on the master) is non-sensical:
{0}to attrs=userPassword,sambaNTPassword,sambaPwdLastSet by self auth by anonymous auth by dn.base="cn=repint,ou=dsa,dc=int,dc=fr" r ead by * none
"by self auth" doesn't do anything, since you have to already be authenticated to be "self". This should be fixed on both the master and the replica.
You also switch format on the replica in ACLs {1} and {2}, although they are covering the same namespace. You could make them read cleaner by being consistent (Either use * or dn.subtree="dc=int,dc=fr" consistently).
On the replica, your syncrepl stanza is a bit odd. You specify you want to use a simple bind, and then you provide a TLS key and TLS cert, which would be used for client side cert auth. If you're doing a simple bind, you should only need to set the tls_cacert and tls_reqcert parameters here. I would also strongly advise setting a keepalive (I usually use 240:10:30)
You also seem to have three different replication accounts in use, as the master has ACLs for these identities:
cn=repint,ou=dsa,dc=int,dc=fr cn=replicator,ou=system,dc=int,dc=fr cn=repz,ou=system,dc=int,dc=fr
On the master, ACL {1} is a bit odd, in that you list a bunch of entities as having read access, and then end with "by * read". You could drastically simplify this ACL by making it just list your write access for the ldapadmin group followed with by * read
On the master, ACL {3} is problematic as it grants write access to internal attributes that only the slapd process should be modifying, such as: modifyTimestamp,createTimestamp,structuralObjectClass,creatorsName,entryCSN,modifiersName,subschemaSubentry,hasSubordinates,ObjectClass
ACL {6} has a similar bit I noted previously about by * read.
ACL {7} should just be "by * read"
on the master, the olcLimits could be simplified to just be:
olcLimits: {0}dn.base="cn=repint,ou=dsa,dc=int,dc=fr" size=unlimited time=unlimited
I would note that there are missing limit exceptions for the other 2 replicator accounts if that is indeed what they are.
Generally I don't see anything obviously incorrect as far as the replication portion goes, so it's not clear to me why you're seeing the behavior you got. I've never set schemachecking to be on when configuring sync replication of either type, so I wonder if there's a bug in there. The OpenLDAP test suite also does not set schema checking to be on, so I'll see if tweaking it there causes any incorrect failures.
Regards, Quanah
--
Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
openldap-technical@openldap.org
-
Jehan PROCACCIA -
jehan.procaccia＠imtbs-tsp.eu
-
Quanah Gibson-Mount