Trouble with delta-syncrepl MMR: delta-sync lost sync on X, switching to REFRESH

Hi,

I'm trying to create a 2-node delta-syncrepl Multi-Master setup with the help of the Admin Guide, man pages and tests/scripts/test063-delta-multimaster. I see the following problem repeat on the "slave" master aka ldap02 which initially syncs with ldap01 aka the "primary" master:

Oct 28 04:12:14 ldap02 slapd[9998]: do_syncrep2: rid=001 delta-sync lost sync on (reqStart=20131028012214.000002Z,cn=accesslog), switching to REFRESH

I found ITS#7274 which mentions that some order should be changed (syncprov before olcServerID) but I have no idea how that applies to my setup. Being new to all this magic I came up empty. So here's my config. Hopefully it isn't too messed up :) I would appreciate it if someone could share a clue or 2 how to make this work. Comments on the config not related to the problem are also most welcome.

OS: CentOS 6.4 x86_64 - clean install, all OpenLDAP dirs are empty OpenLDAP version: RE24 git rev f9e417a from around 10/23/2013.

On the initial/"primary" master (note that the config is the same for the "slave" up to the comment):

$ sudo /usr/local/sbin/slapadd -v -F $LDAP_ETC/slapd.d \ -l ./delta-syncrepl-MMR.ldif -n 0

$ cat ./delta-syncrepl-MMR.ldif

# global configuration settings dn: cn=config objectClass: olcGlobal cn: config olcArgsFile: /var/run/openldap-2.4/slapd-2.4.args olcPidFile: /var/run/openldap-2.4/slapd-2.4.pid olcLogFile: /var/log/openldap-2.4/slapd-2.4.log olcLogLevel: conns stats stats2 sync olcTLSCACertificateFile: /etc/pki/tls/certs/DS-CA.crt olcTLSCertificateFile: /etc/pki/tls/certs/slapd.crt olcTLSCertificateKeyFile: /etc/pki/tls/private/slapd.key.crt olcTLSCipherSuite: TLSv1+HIGH:!SSLv2:!aNULL:!eNULL:!3DES:!RC4:@STRENGTH olcTLSVerifyClient: demand olcLocalSSF: 256 olcSecurity: ssf=256 olcPasswordCryptSaltFormat: $6$%s olcPasswordHash: {CRYPT} olcServerID: 1 ldap://ldap01 olcServerID: 2 ldap://ldap02

# load modules dn: cn=module,cn=config objectClass: olcModuleList cn: module olcModulePath: /usr/local/lib64/openldap-2.4 olcModuleload: {0}syncprov.la olcModuleload: {1}accesslog.la olcModuleLoad: {2}back_mdb.la olcModuleLoad: {3}back_monitor.la olcModuleLoad: {4}memberof.la olcModuleLoad: {5}refint.la olcModuleLoad: {6}ppolicy.la

# schema definitions dn: cn=schema,cn=config objectClass: olcSchemaConfig cn: schema

# include the schemas include: file:///etc/openldap-2.4/schema/core.ldif include: file:///etc/openldap-2.4/schema/corba.ldif include: file:///etc/openldap-2.4/schema/cosine.ldif include: file:///etc/openldap-2.4/schema/duaconf.ldif include: file:///etc/openldap-2.4/schema/dyngroup.ldif include: file:///etc/openldap-2.4/schema/inetorgperson.ldif include: file:///etc/openldap-2.4/schema/java.ldif include: file:///etc/openldap-2.4/schema/misc.ldif include: file:///etc/openldap-2.4/schema/nis.ldif include: file:///etc/openldap-2.4/schema/openldap.ldif include: file:///etc/openldap-2.4/schema/ppolicy.ldif include: file:///etc/openldap-2.4/schema/collective.ldif

# global database parameters dn: olcDatabase=frontend,cn=config objectClass: olcDatabaseConfig objectClass: olcFrontendConfig olcDatabase: frontend

# setup cn=config (password = 1234) dn: olcDatabase={0}config,cn=config objectClass: olcDatabaseConfig olcDatabase: {0}config olcRootPW: {CRYPT}$6$... olcSyncrepl: {0}rid=001 provider=ldap://ldap01 binddn="cn=Manager,dc=test" bindmethod=sasl saslmech=external searchbase="cn=config" type=refreshAndPersist retry="5 5 300 5" timeout=1 schemachecking=off interval=00:00:00:5 retry="5 +" starttls=critical tls_cert=/etc/pki/tls/certs/Manager.crt tls_key=/etc/pki/tls/private/Manager.key.crt tls_cacert=/etc/pki/tls/certs/DS-CA.crt tls_reqcert=demand logbase="cn=accesslog" logfilter="(&(objectClass=auditWriteObject)(reqResult=0))" syncdata=accesslog olcSyncrepl: {1}rid=002 provider=ldap://ldap02 binddn="cn=Manager,dc=test" bindmethod=sasl saslmech=external searchbase="cn=config" type=refreshAndPersist retry="5 5 300 5" timeout=1 schemachecking=off interval=00:00:00:5 retry="5 +" starttls=critical tls_cert=/etc/pki/tls/certs/Manager.crt tls_key=/etc/pki/tls/private/Manager.key.crt tls_cacert=/etc/pki/tls/certs/DS-CA.crt tls_reqcert=demand logbase="cn=accesslog" logfilter="(&(objectClass=auditWriteObject)(reqResult=0))" syncdata=accesslog olcMirrorMode: TRUE olcAccess: to * by dn.exact="cn=Manager,dc=test" write by * none olcLimits: dn.exact="cn=Manager,dc=test" time.soft=unlimited time.hard=unlimited size.soft=unlimited size.hard=unlimited

# add the syncprov overlay to the cn=config database dn: olcOverlay=syncprov,olcDatabase={0}config,cn=config objectClass: olcOverlayConfig objectClass: olcSyncProvConfig olcOverlay: syncprov olcSpNoPresent: TRUE olcSpReloadHint: TRUE

# setup monitoring dn: olcDatabase=monitor,cn=config objectClass: olcDatabaseConfig objectClass: olcMonitorConfig olcDatabase: monitor olcAccess: to dn.subtree=cn=Monitor by dn.exact="cn=config" write by dn.exact="cn=Manager,dc=test" write by * none

# setup Accesslog database definitions dn: olcDatabase={2}mdb,cn=config objectClass: olcDatabaseConfig objectClass: olcMdbConfig olcDatabase: {2}mdb olcDbDirectory: /var/lib/ldap-2.4/accesslog olcSuffix: cn=accesslog olcAccess: {0}to dn.subtree="cn=accesslog" by dn.exact="cn=Manager,dc=test" read olcRootDN: cn=Manager,dc=test olcDbIndex: objectClass,entryCSN,reqStart,reqEnd,reqResult,reqDN eq olcDbMode: 0600 # max size in bytes - 1GB = 1073741824 bytes olcDbMaxsize: 1073741824

# add the syncprov overlay to the cn=accesslog database dn: olcOverlay=syncprov,olcDatabase={2}mdb,cn=config objectClass: olcOverlayConfig objectClass: olcSyncProvConfig olcOverlay: syncprov olcSpNoPresent: TRUE olcSpReloadHint: TRUE

# up to here ^^^ is also the config for the "slave"

# main mdb database definition dn: olcDatabase={3}mdb,cn=config objectClass: olcDatabaseConfig objectClass: olcMdbConfig olcDatabase: {3}mdb olcSuffix: dc=test olcDbDirectory: /var/lib/ldap-2.4/test olcRootDN: cn=Manager,dc=test olcRootPW: {CRYPT}$6$... olcSyncrepl: {0}rid=003 provider=ldap://ldap01 binddn="cn=Manager,dc=test" bindmethod=sasl saslmech=external searchbase="dc=test" type=refreshAndPersist retry="5 5 300 5" timeout=1 schemachecking=off interval=00:00:00:5 retry="5 +" starttls=critical tls_cert=/etc/pki/tls/certs/Manager.crt tls_key=/etc/pki/tls/private/Manager_nopass.key.crt tls_cacert=/etc/pki/tls/certs/DS-CA.crt tls_reqcert=demand logbase="cn=accesslog" logfilter="(&(objectClass=auditWriteObject)(reqResult=0))" syncdata=accesslog olcSyncrepl: {1}rid=004 provider=ldap://ldap02 binddn="cn=Manager,test" bindmethod=sasl saslmech=external searchbase="dc=test" type=refreshAndPersist retry="5 5 300 5" timeout=1 schemachecking=off interval=00:00:00:5 retry="5 +" starttls=critical tls_cert=/etc/pki/tls/certs/Manager.crt tls_key=/etc/pki/tls/private/Manager.key.crt tls_cacert=/etc/pki/tls/certs/DS-CA.crt tls_reqcert=demand logbase="cn=accesslog" logfilter="(&(objectClass=auditWriteObject)(reqResult=0))" syncdata=accesslog olcMirrorMode: TRUE olcDbIndex: cn pres,eq,sub olcDbIndex: gidNumber pres,eq olcDbIndex: mail pres,eq,sub olcDbIndex: memberUid pres,eq olcDbIndex: objectClass pres,eq olcDbIndex: ou pres,eq,sub olcDbIndex: sn pres,eq,sub olcDbIndex: uid pres,eq olcDbIndex: uidNumber pres,eq olcDbIndex: entryCSN,entryUUID eq olcDbMode: 0600 # max size in bytes - 1GB = 1073741824 bytes olcDbMaxSize: 5368709120 olcAccess: to attrs=userPassword by dn.exact="cn=Manager,dc=test" write by self write by anonymous auth by * none olcAccess: to * by dn.exact="cn=Manager,dc=test" write by self read by * read olcLimits: dn.exact="cn=Manager,dc=test" time.soft=unlimited time.hard=unlimited size.soft=unlimited size.hard=unlimited

# add the syncprov overlay to the main mdb database dn: olcOverlay={0}syncprov,olcDatabase={3}mdb,cn=config objectClass: olcOverlayConfig objectClass: olcSyncProvConfig olcOverlay: syncprov olcSpCheckPoint: 20 10

# add the accesslog overlay to the main mdb database dn: olcOverlay={1}accesslog,olcDatabase={3}mdb,cn=config objectClass: olcOverlayConfig objectClass: olcAccessLogConfig olcOverlay: accesslog olcAccessLogDB: cn=accesslog olcAccessLogOps: writes olcAccessLogSuccess: TRUE olcAccessLogPurge: 01+00:00 04+00:00

# add memberof overlay to mdb database dn: olcOverlay={2}memberof,olcDatabase={3}mdb,cn=config objectClass: olcConfig objectClass: olcMemberOf objectClass: olcOverlayConfig objectClass: top olcOverlay: memberof olcMemberOfDangling: ignore olcMemberOfRefInt: TRUE olcMemberOfGroupOC: groupOfNames olcMemberOfMemberAD: member olcMemberOfMemberOfAD: memberOf

# add refint overlay to mdb database dn: olcOverlay={3}refint,olcDatabase={3}mdb,cn=config objectClass: olcConfig objectClass: olcOverlayConfig objectClass: olcRefintConfig objectClass: top olcOverlay: refint olcRefintAttribute: member memberOf olcRefintNothing: cn=Manager,dc=test

# add the ppolicy overlay to the main mdb database dn: olcOverlay={4}ppolicy,olcDatabase={3}mdb,cn=config objectClass: olcOverlayConfig objectClass: olcPPolicyConfig olcOverlay: ppolicy olcPPolicyDefault: cn=default,ou=policies,dc=test olcPPolicyHashCleartext: TRUE

$ sudo /usr/local/bin/ldapadd -v -f ./test-data.ldif x -D "cn=Manager,dc=test" -w $(cat ./ldap.secret) -d0 -ZZ

# Organization dn: dc=test objectClass: dcObject objectClass: organization dc: test o: Test description: Test LDAP Root

# add ppolicy ou dn: ou=policies,dc=test ou: policies objectClass: top objectClass: organizationalUnit

# add password policy dn: cn=default,ou=policies,dc=test cn: default objectClass: pwdPolicy objectClass: pwdPolicyChecker objectClass: person objectClass: top pwdAllowUserChange: TRUE pwdAttribute: userPassword pwdCheckQuality: 2 pwdExpireWarning: 600 pwdFailureCountInterval: 30 pwdGraceAuthNLimit: 5 pwdInHistory: 5 pwdLockout: TRUE pwdLockoutDuration: 0 pwdMaxAge: 0 pwdMaxFailure: 5 pwdMinAge: 0 pwdMinLength: 5 pwdMustChange: FALSE pwdSafeModify: FALSE pwdCheckModule: check_password.so sn: dummy value ...

Thanks!

Regards, Patrick