Hi,
I'm trying to create a 2-node delta-syncrepl Multi-Master setup with the help of the Admin Guide, man pages and tests/scripts/test063-delta-multimaster. I see the following problem repeat on the "slave" master aka ldap02 which initially syncs with ldap01 aka the "primary" master:
Oct 28 04:12:14 ldap02 slapd[9998]: do_syncrep2: rid=001 delta-sync lost sync on (reqStart=20131028012214.000002Z,cn=accesslog), switching to REFRESH
I found ITS#7274 which mentions that some order should be changed (syncprov before olcServerID) but I have no idea how that applies to my setup. Being new to all this magic I came up empty. So here's my config. Hopefully it isn't too messed up :) I would appreciate it if someone could share a clue or 2 how to make this work. Comments on the config not related to the problem are also most welcome.
OS: CentOS 6.4 x86_64 - clean install, all OpenLDAP dirs are empty OpenLDAP version: RE24 git rev f9e417a from around 10/23/2013.
On the initial/"primary" master (note that the config is the same for the "slave" up to the comment):
$ sudo /usr/local/sbin/slapadd -v -F $LDAP_ETC/slapd.d \ -l ./delta-syncrepl-MMR.ldif -n 0
$ cat ./delta-syncrepl-MMR.ldif
# global configuration settings dn: cn=config objectClass: olcGlobal cn: config olcArgsFile: /var/run/openldap-2.4/slapd-2.4.args olcPidFile: /var/run/openldap-2.4/slapd-2.4.pid olcLogFile: /var/log/openldap-2.4/slapd-2.4.log olcLogLevel: conns stats stats2 sync olcTLSCACertificateFile: /etc/pki/tls/certs/DS-CA.crt olcTLSCertificateFile: /etc/pki/tls/certs/slapd.crt olcTLSCertificateKeyFile: /etc/pki/tls/private/slapd.key.crt olcTLSCipherSuite: TLSv1+HIGH:!SSLv2:!aNULL:!eNULL:!3DES:!RC4:@STRENGTH olcTLSVerifyClient: demand olcLocalSSF: 256 olcSecurity: ssf=256 olcPasswordCryptSaltFormat: $6$%s olcPasswordHash: {CRYPT} olcServerID: 1 ldap://ldap01 olcServerID: 2 ldap://ldap02
# load modules dn: cn=module,cn=config objectClass: olcModuleList cn: module olcModulePath: /usr/local/lib64/openldap-2.4 olcModuleload: {0}syncprov.la olcModuleload: {1}accesslog.la olcModuleLoad: {2}back_mdb.la olcModuleLoad: {3}back_monitor.la olcModuleLoad: {4}memberof.la olcModuleLoad: {5}refint.la olcModuleLoad: {6}ppolicy.la
# schema definitions dn: cn=schema,cn=config objectClass: olcSchemaConfig cn: schema
# include the schemas include: file:///etc/openldap-2.4/schema/core.ldif include: file:///etc/openldap-2.4/schema/corba.ldif include: file:///etc/openldap-2.4/schema/cosine.ldif include: file:///etc/openldap-2.4/schema/duaconf.ldif include: file:///etc/openldap-2.4/schema/dyngroup.ldif include: file:///etc/openldap-2.4/schema/inetorgperson.ldif include: file:///etc/openldap-2.4/schema/java.ldif include: file:///etc/openldap-2.4/schema/misc.ldif include: file:///etc/openldap-2.4/schema/nis.ldif include: file:///etc/openldap-2.4/schema/openldap.ldif include: file:///etc/openldap-2.4/schema/ppolicy.ldif include: file:///etc/openldap-2.4/schema/collective.ldif
# global database parameters dn: olcDatabase=frontend,cn=config objectClass: olcDatabaseConfig objectClass: olcFrontendConfig olcDatabase: frontend
# setup cn=config (password = 1234) dn: olcDatabase={0}config,cn=config objectClass: olcDatabaseConfig olcDatabase: {0}config olcRootPW: {CRYPT}$6$... olcSyncrepl: {0}rid=001 provider=ldap://ldap01 binddn="cn=Manager,dc=test" bindmethod=sasl saslmech=external searchbase="cn=config" type=refreshAndPersist retry="5 5 300 5" timeout=1 schemachecking=off interval=00:00:00:5 retry="5 +" starttls=critical tls_cert=/etc/pki/tls/certs/Manager.crt tls_key=/etc/pki/tls/private/Manager.key.crt tls_cacert=/etc/pki/tls/certs/DS-CA.crt tls_reqcert=demand logbase="cn=accesslog" logfilter="(&(objectClass=auditWriteObject)(reqResult=0))" syncdata=accesslog olcSyncrepl: {1}rid=002 provider=ldap://ldap02 binddn="cn=Manager,dc=test" bindmethod=sasl saslmech=external searchbase="cn=config" type=refreshAndPersist retry="5 5 300 5" timeout=1 schemachecking=off interval=00:00:00:5 retry="5 +" starttls=critical tls_cert=/etc/pki/tls/certs/Manager.crt tls_key=/etc/pki/tls/private/Manager.key.crt tls_cacert=/etc/pki/tls/certs/DS-CA.crt tls_reqcert=demand logbase="cn=accesslog" logfilter="(&(objectClass=auditWriteObject)(reqResult=0))" syncdata=accesslog olcMirrorMode: TRUE olcAccess: to * by dn.exact="cn=Manager,dc=test" write by * none olcLimits: dn.exact="cn=Manager,dc=test" time.soft=unlimited time.hard=unlimited size.soft=unlimited size.hard=unlimited
# add the syncprov overlay to the cn=config database dn: olcOverlay=syncprov,olcDatabase={0}config,cn=config objectClass: olcOverlayConfig objectClass: olcSyncProvConfig olcOverlay: syncprov olcSpNoPresent: TRUE olcSpReloadHint: TRUE
# setup monitoring dn: olcDatabase=monitor,cn=config objectClass: olcDatabaseConfig objectClass: olcMonitorConfig olcDatabase: monitor olcAccess: to dn.subtree=cn=Monitor by dn.exact="cn=config" write by dn.exact="cn=Manager,dc=test" write by * none
# setup Accesslog database definitions dn: olcDatabase={2}mdb,cn=config objectClass: olcDatabaseConfig objectClass: olcMdbConfig olcDatabase: {2}mdb olcDbDirectory: /var/lib/ldap-2.4/accesslog olcSuffix: cn=accesslog olcAccess: {0}to dn.subtree="cn=accesslog" by dn.exact="cn=Manager,dc=test" read olcRootDN: cn=Manager,dc=test olcDbIndex: objectClass,entryCSN,reqStart,reqEnd,reqResult,reqDN eq olcDbMode: 0600 # max size in bytes - 1GB = 1073741824 bytes olcDbMaxsize: 1073741824
# add the syncprov overlay to the cn=accesslog database dn: olcOverlay=syncprov,olcDatabase={2}mdb,cn=config objectClass: olcOverlayConfig objectClass: olcSyncProvConfig olcOverlay: syncprov olcSpNoPresent: TRUE olcSpReloadHint: TRUE
# up to here ^^^ is also the config for the "slave"
# main mdb database definition dn: olcDatabase={3}mdb,cn=config objectClass: olcDatabaseConfig objectClass: olcMdbConfig olcDatabase: {3}mdb olcSuffix: dc=test olcDbDirectory: /var/lib/ldap-2.4/test olcRootDN: cn=Manager,dc=test olcRootPW: {CRYPT}$6$... olcSyncrepl: {0}rid=003 provider=ldap://ldap01 binddn="cn=Manager,dc=test" bindmethod=sasl saslmech=external searchbase="dc=test" type=refreshAndPersist retry="5 5 300 5" timeout=1 schemachecking=off interval=00:00:00:5 retry="5 +" starttls=critical tls_cert=/etc/pki/tls/certs/Manager.crt tls_key=/etc/pki/tls/private/Manager_nopass.key.crt tls_cacert=/etc/pki/tls/certs/DS-CA.crt tls_reqcert=demand logbase="cn=accesslog" logfilter="(&(objectClass=auditWriteObject)(reqResult=0))" syncdata=accesslog olcSyncrepl: {1}rid=004 provider=ldap://ldap02 binddn="cn=Manager,test" bindmethod=sasl saslmech=external searchbase="dc=test" type=refreshAndPersist retry="5 5 300 5" timeout=1 schemachecking=off interval=00:00:00:5 retry="5 +" starttls=critical tls_cert=/etc/pki/tls/certs/Manager.crt tls_key=/etc/pki/tls/private/Manager.key.crt tls_cacert=/etc/pki/tls/certs/DS-CA.crt tls_reqcert=demand logbase="cn=accesslog" logfilter="(&(objectClass=auditWriteObject)(reqResult=0))" syncdata=accesslog olcMirrorMode: TRUE olcDbIndex: cn pres,eq,sub olcDbIndex: gidNumber pres,eq olcDbIndex: mail pres,eq,sub olcDbIndex: memberUid pres,eq olcDbIndex: objectClass pres,eq olcDbIndex: ou pres,eq,sub olcDbIndex: sn pres,eq,sub olcDbIndex: uid pres,eq olcDbIndex: uidNumber pres,eq olcDbIndex: entryCSN,entryUUID eq olcDbMode: 0600 # max size in bytes - 1GB = 1073741824 bytes olcDbMaxSize: 5368709120 olcAccess: to attrs=userPassword by dn.exact="cn=Manager,dc=test" write by self write by anonymous auth by * none olcAccess: to * by dn.exact="cn=Manager,dc=test" write by self read by * read olcLimits: dn.exact="cn=Manager,dc=test" time.soft=unlimited time.hard=unlimited size.soft=unlimited size.hard=unlimited
# add the syncprov overlay to the main mdb database dn: olcOverlay={0}syncprov,olcDatabase={3}mdb,cn=config objectClass: olcOverlayConfig objectClass: olcSyncProvConfig olcOverlay: syncprov olcSpCheckPoint: 20 10
# add the accesslog overlay to the main mdb database dn: olcOverlay={1}accesslog,olcDatabase={3}mdb,cn=config objectClass: olcOverlayConfig objectClass: olcAccessLogConfig olcOverlay: accesslog olcAccessLogDB: cn=accesslog olcAccessLogOps: writes olcAccessLogSuccess: TRUE olcAccessLogPurge: 01+00:00 04+00:00
# add memberof overlay to mdb database dn: olcOverlay={2}memberof,olcDatabase={3}mdb,cn=config objectClass: olcConfig objectClass: olcMemberOf objectClass: olcOverlayConfig objectClass: top olcOverlay: memberof olcMemberOfDangling: ignore olcMemberOfRefInt: TRUE olcMemberOfGroupOC: groupOfNames olcMemberOfMemberAD: member olcMemberOfMemberOfAD: memberOf
# add refint overlay to mdb database dn: olcOverlay={3}refint,olcDatabase={3}mdb,cn=config objectClass: olcConfig objectClass: olcOverlayConfig objectClass: olcRefintConfig objectClass: top olcOverlay: refint olcRefintAttribute: member memberOf olcRefintNothing: cn=Manager,dc=test
# add the ppolicy overlay to the main mdb database dn: olcOverlay={4}ppolicy,olcDatabase={3}mdb,cn=config objectClass: olcOverlayConfig objectClass: olcPPolicyConfig olcOverlay: ppolicy olcPPolicyDefault: cn=default,ou=policies,dc=test olcPPolicyHashCleartext: TRUE
$ sudo /usr/local/bin/ldapadd -v -f ./test-data.ldif x -D "cn=Manager,dc=test" -w $(cat ./ldap.secret) -d0 -ZZ
# Organization dn: dc=test objectClass: dcObject objectClass: organization dc: test o: Test description: Test LDAP Root
# add ppolicy ou dn: ou=policies,dc=test ou: policies objectClass: top objectClass: organizationalUnit
# add password policy dn: cn=default,ou=policies,dc=test cn: default objectClass: pwdPolicy objectClass: pwdPolicyChecker objectClass: person objectClass: top pwdAllowUserChange: TRUE pwdAttribute: userPassword pwdCheckQuality: 2 pwdExpireWarning: 600 pwdFailureCountInterval: 30 pwdGraceAuthNLimit: 5 pwdInHistory: 5 pwdLockout: TRUE pwdLockoutDuration: 0 pwdMaxAge: 0 pwdMaxFailure: 5 pwdMinAge: 0 pwdMinLength: 5 pwdMustChange: FALSE pwdSafeModify: FALSE pwdCheckModule: check_password.so sn: dummy value ...
Thanks!
Regards, Patrick
--On Monday, October 28, 2013 5:14 AM +0100 Patrick Lists openldap-list@puzzled.xs4all.nl wrote:
Hi,
I'm trying to create a 2-node delta-syncrepl Multi-Master setup with the help of the Admin Guide, man pages and tests/scripts/test063-delta-multimaster. I see the following problem repeat on the "slave" master aka ldap02 which initially syncs with ldap01 aka the "primary" master:
Why do you have your cn=config db reading from the same accesslog for replication as your primary DB?
If you are going to set up cn=config AND your primary db both as delta-syncrepl, you're going to need 2 different accesslog DBs.
I would also note your cn=config DB is completely missing the accesslog overlay.
Personally, I've never done replicated cn=config, with or without delta-syncrepl. I would focus on just getting replication to work for your primary DB first.
--Quanah
--
Quanah Gibson-Mount Architect - Server Zimbra, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration
Quanah Gibson-Mount quanah@zimbra.com schrieb am 28.10.2013 um 18:14 in
Nachricht <97CEE97E42D7FD3A860611C8@[192.168.1.93]>:
--On Monday, October 28, 2013 5:14 AM +0100 Patrick Lists openldap-list@puzzled.xs4all.nl wrote:
Hi,
I'm trying to create a 2-node delta-syncrepl Multi-Master setup with the help of the Admin Guide, man pages and tests/scripts/test063-delta-multimaster. I see the following problem repeat on the "slave" master aka ldap02 which initially syncs with ldap01 aka the "primary" master:
Why do you have your cn=config db reading from the same accesslog for replication as your primary DB?
If you are going to set up cn=config AND your primary db both as delta-syncrepl, you're going to need 2 different accesslog DBs.
Hi!
Why? That's completely unobvious: accesslog is write-only, right? So why can't two sources write to one accesslog?
I would also note your cn=config DB is completely missing the accesslog overlay.
Personally, I've never done replicated cn=config, with or without delta-syncrepl. I would focus on just getting replication to work for your primary DB first.
--Quanah
--
Quanah Gibson-Mount Architect - Server Zimbra, Inc.
Zimbra :: the leader in open source messaging and collaboration
Ulrich Windl wrote:
Quanah Gibson-Mount quanah@zimbra.com schrieb am 28.10.2013 um 18:14 in
Nachricht <97CEE97E42D7FD3A860611C8@[192.168.1.93]>:
--On Monday, October 28, 2013 5:14 AM +0100 Patrick Lists openldap-list@puzzled.xs4all.nl wrote:
Hi,
I'm trying to create a 2-node delta-syncrepl Multi-Master setup with the help of the Admin Guide, man pages and tests/scripts/test063-delta-multimaster. I see the following problem repeat on the "slave" master aka ldap02 which initially syncs with ldap01 aka the "primary" master:
Why do you have your cn=config db reading from the same accesslog for replication as your primary DB?
If you are going to set up cn=config AND your primary db both as delta-syncrepl, you're going to need 2 different accesslog DBs.
Hi!
Why? That's completely unobvious: accesslog is write-only, right? So why can't two sources write to one accesslog?
They can, but it means your syncrepl consumers must use a more specific filter to extract the modifications that are relevant. Also it's useful to use separate logs because different databases will have different rates of modifications, and you can thus configure a log purge interval more suited to each.
Hi Quanah,
Thank you for your feedback. It was most helpful.
On 10/28/2013 06:14 PM, Quanah Gibson-Mount wrote: [snip]
Why do you have your cn=config db reading from the same accesslog for replication as your primary DB?
Doh.
If you are going to set up cn=config AND your primary db both as delta-syncrepl, you're going to need 2 different accesslog DBs.
Got it.
I would also note your cn=config DB is completely missing the accesslog overlay.
Double doh. The result of too much changing things around I guess.
Personally, I've never done replicated cn=config, with or without delta-syncrepl. I would focus on just getting replication to work for your primary DB first.
The reason that I went down this road is because it's described that way in the Admin Guide: http://www.openldap.org/doc/admin24/guide.html#N-Way%20Multi-Master
After I fixed the issues you pointed out I've now got delta-syncrepl replication working of both cn=config and the main database. Or at least it seems so :-)
Now if I understand that multi-master config example in the Admin Guide correctly (see link above) it should also be possible to use this config:
ldap01: config with cn=config + syncprov & accesslog overlays and the main mdb + syncprov & accesslog overlays. Both with olcMirrorMode: TRUE
ldap02: config with cn=config + syncprov & accesslog overlays with olcMirrorMode: TRUE
And when ldap02 is started it should sync the entire config from ldap01 (including the main database) and also replicate the data in the main database. At least that is how I read the sentence "We still have to replicate the actual data, not just the config, so add to the master (all active and configured consumers/masters will pull down this config, as they are all syncing)". Is my interpretation correct?
Regards, Patrick
--On Tuesday, October 29, 2013 5:56 PM +0100 Patrick Lists openldap-list@puzzled.xs4all.nl wrote:
Now if I understand that multi-master config example in the Admin Guide correctly (see link above) it should also be possible to use this config:
ldap01: config with cn=config + syncprov & accesslog overlays and the main mdb + syncprov & accesslog overlays. Both with olcMirrorMode: TRUE
ldap02: config with cn=config + syncprov & accesslog overlays with olcMirrorMode: TRUE
And when ldap02 is started it should sync the entire config from ldap01 (including the main database) and also replicate the data in the main database. At least that is how I read the sentence "We still have to replicate the actual data, not just the config, so add to the master (all active and configured consumers/masters will pull down this config, as they are all syncing)". Is my interpretation correct?
No idea... as I said, I've never set up cn=config replication, and I've never run with mismatched configurations for MMR. Clearly ldap02's config for the primary db doesn't match, so I would hope it would work? I'd say test and report back. ;)
--Quanah
--
Quanah Gibson-Mount Architect - Server Zimbra, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration
On 10/29/2013 07:39 PM, Quanah Gibson-Mount wrote:
--On Tuesday, October 29, 2013 5:56 PM +0100 Patrick Lists openldap-list@puzzled.xs4all.nl wrote:
Now if I understand that multi-master config example in the Admin Guide correctly (see link above) it should also be possible to use this config:
ldap01: config with cn=config + syncprov & accesslog overlays and the main mdb + syncprov & accesslog overlays. Both with olcMirrorMode: TRUE
ldap02: config with cn=config + syncprov & accesslog overlays with olcMirrorMode: TRUE
And when ldap02 is started it should sync the entire config from ldap01 (including the main database) and also replicate the data in the main database. At least that is how I read the sentence "We still have to replicate the actual data, not just the config, so add to the master (all active and configured consumers/masters will pull down this config, as they are all syncing)". Is my interpretation correct?
No idea... as I said, I've never set up cn=config replication, and I've never run with mismatched configurations for MMR. Clearly ldap02's config for the primary db doesn't match, so I would hope it would work? I'd say test and report back. ;)
I guess I should have mentioned that I tested it with mismatched delta-syncrepl multi-master configs and it did not work. All I saw was ldap01's cn=config replicate to ldap02 but not the main database. And ldap01 was complaining that it could not reach the rid of the main database on ldap02 which makes sense. Would be cool if this worked though.
Regards, Patrick
openldap-technical@openldap.org
-
Howard Chu -
Patrick Lists -
Quanah Gibson-Mount -
Ulrich Windl