Peter Mogensen wrote:

Hallvard B Furuseth wrote:

...

Peter Mogensen writes:

...

I'm trying to understand why changes made to SID 1 in my mirror set while SID 2 is down does not get propagated to SID 2 when it comes up.

Maybe your mirror is configured with refreshAndPersist mode and you have not specified a retry interval? Then thed default is 1 hour, according to the slapd.conf manpage.

No. retry is "60 +" (1 minute from what I read).

Since no one has complained against the 8 step procedure I posted, I will assume that it is the correct way to load an huge LDIF into an empty mirrormode setup.

There is no need for your step #2.

Given a valid slapcat from OpenLDAP 2.3 you should be able to slapadd it directly in 2.4 without using -S or -w in your step #3. Therefore you don't need step #4.

So since, it's not the procedure, it must be either my configuration or a bug. I'll assume it's my configuration though I suspect this message is about the same problem: http://www.openldap.org/lists/openldap-software/200911/msg00058.html

No, that message refers to a bug that is definitely fixed in 2.4.20. (ITS#6367)

So here's my configuration in a step-by-step sequence. I do:

• First install openldap 2.4.20 / db 4.8.24 on two debian Lenny systems.

• Set /etc/ldap/slapd.conf to this:

=================================================================== gentlehup on pidfile /var/run/slapd/slapd.pid argsfile /var/run/slapd/slapd.args loglevel none

tool-threads 8

You have 8 CPUs?

# Modules modulepath /usr/lib/ldap moduleload back_hdb moduleload syncprov

# Schemas include /etc/ldap/schema/core.schema include /etc/ldap/schema/cosine.schema include /etc/ldap/schema/inetorgperson.schema

# Limits disallow bind_anon idletimeout 120 sizelimit 2000

# TLS/Auth TLSCACertificateFile /etc/ldap/ssl/ca.crt TLSCertificateFile /etc/ldap/ssl/server.crt TLSCertificateKeyFile /etc/ldap/ssl/server.nopass.key TLSCipherSuite "NULL-SHA"

# Allow root to configure slapd via ldapi:/// TLSVerifyClient demand authz-regexp "gidNumber=0\+uidNumber=0,cn=peercred,cn=external,cn=auth" "cn=config"

Neatness nit: your TLSVerifyClient is obviously under the wrong comment.

authz-regexp "email=root@example.com,cn=config,ou=dev,o=example.com,st=Denmark,c=DK" "cn=config"

##### Mirror mode #### serverID 1

database config

limits dn.exact="cn=config" time.soft=unlimited time.hard=unlimited size.soft=unlimited size.hard=unlimited

The rootdn is always unlimited, this clause is superfluous.

syncrepl rid=1 provider=ldaps://server1.example.com:636/ searchbase="cn=config" type=refreshAndPersist retry="60 +" scope=sub schemachecking=on bindmethod=sasl binddn="cn=config" saslmech="EXTERNAL" tls_cert=/etc/ldap/ssl/config.crt tls_key=/etc/ldap/ssl/config.nopass.key tls_cacert=/etc/ldap/ssl/ca.crt tls_cipher_suite="NULL-SHA"

syncrepl rid=2 provider=ldaps://server2.example.com:636/ searchbase="cn=config" type=refreshAndPersist retry="60 +" scope=sub schemachecking=on bindmethod=sasl binddn="cn=config" saslmech="EXTERNAL" tls_cert=/etc/ldap/ssl/config.crt tls_key=/etc/ldap/ssl/config.nopass.key tls_cacert=/etc/ldap/ssl/ca.crt tls_cipher_suite="NULL-SHA"

overlay syncprov syncprov-checkpoint 100 10 syncprov-sessionlog 100 syncprov-reloadhint TRUE

mirrormode on

• Then, I run slaptest -f /etc/ldap/slapd.conf -F /etc/ldap/slapd.d to convert the above to a cn=config based setup.

• Then I start slapd on both servers. $ /usr/sbin/slapd -h ldapi:/// ldaps://server1.example.com:636/ \ ldap://server1.example.com/ -g openldap -u openldap \ -F /etc/ldap/slapd.d -4

That won't work in typical Unix shells without quotes.

... all of the above of course different wrt. server1/server2, SID 1/2

• The I load the following LDIF files on server 1 with $ ldapadd -YEXTERNAL -H ldapi:/// -f<LDIFFILE> In sequence:

============================== dn: cn=module{0},cn=config changetype: modify add: olcModuleLoad olcModuleLoad: refint ============================= dn: cn=module{0},cn=config changetype: modify add: olcModuleLoad olcModuleLoad: back_bdb ==============================

Could have just used one mod request for both of those. Why are you loading back-bdb when you're just using back-hdb and it's already loaded?

... a bunch of schemas, like: dn: cn=evolutionperson,cn=schema,cn=config ============================== dn: olcDatabase={1}hdb,cn=config objectClass: olcHdbConfig objectClass: olcDatabaseConfig olcDatabase: hdb olcSuffix: cn=data,dc=example,dc=com olcRootDN: cn=config olcDbDirectory: /var/lib/ldap/cn=data,dc=example,dc=com olcDbMode: 0660 olcDbConfig: set_cachesize 2 0 0 olcDbConfig: set_lg_bsize 2097512 olcDbConfig: set_lg_dir /var/lib/ldap/cn=data,dc=example,dc=com-log olcDbConfig: set_flags DB_LOG_AUTOREMOVE olcDbConfig: set_lk_max_objects 5000 olcDbConfig: set_lk_max_locks 5000 olcDbConfig: set_lk_max_lockers 5000 olcDbCheckpoint: 1024 10 olcDbCachefree: 16 olcDbCachesize: 100000 olcDbIDLcacheSize: 300000 olcDbLinearIndex: FALSE olcDbIndex: objectClass eq olcDbIndex: entryUUID eq olcDbIndex: entryCSN eq olcDbIndex: cn eq,sub olcDbIndex: uid eq olcDbIndex: ou eq olcDbIndex: o eq olcDbIndex: givenName eq,sub olcDbIndex: sn eq,sub olcDbIndex: mail eq,sub olcDbIndex: member eq olcDbIndex: reader eq olcDbIndex: writer eq olcDbIndex: admin eq olcAccess: to dn.base="cn=data,dc=example,dc=com" attrs=userPassword by * auth olcAccess: to dn.base="cn=data,dc=example,dc=com" by dn.base="cn=data,dc=example,dc=com" search olcAccess: to dn.children="cn=data,dc=example,dc=com" by dn.base="cn=data,dc=example,dc=com" write olcSyncRepl: rid=3 provider=ldaps://server1.example.com:636/ searchbase="cn=data,dc=example,dc=com" type=refreshAndPersist retry="60 +" scope=sub schemachecking=on bindmethod=sasl binddn="cn=config" saslmech="EXTERNAL" tls_cert=/etc/ldap/ssl/config.crt tls_key=/etc/ldap/ssl/config.nopass.key tls_cacert=/etc/ldap/ssl/ca.crt tls_cipher_suite="NULL-SHA" olcSyncRepl: rid=4 provider=ldaps://server2.example.com:636/ searchbase="cn=data,dc=example,dc=com" type=refreshAndPersist retry="60 +" scope=sub schemachecking=on bindmethod=sasl binddn="cn=config" saslmech="EXTERNAL" tls_cert=/etc/ldap/ssl/config.crt tls_key=/etc/ldap/ssl/config.nopass.key tls_cacert=/etc/ldap/ssl/ca.crt tls_cipher_suite="NULL-SHA" olcMirrorMode: TRUE olcLimits: dn.base="cn=config" size.soft=unlimited size.hard=unlimited time.soft=unlimited time.hard=unlimited

dn: olcOverlay=syncprov,olcDatabase={1}hdb,cn=config objectClass: olcOverlayConfig objectClass: olcSyncProvConfig olcOverlay: syncprov olcSpCheckpoint: 100 600 olcSpSessionlog: 100 olcSpReloadHint: TRUE

dn: olcOverlay=refint,olcDatabase={1}hdb,cn=config objectClass: olcOverlayConfig objectClass: olcRefintConfig olcOverlay: refint olcRefintAttribute: member ===========================================

• All of the above gets properly replicated to server2.

• Then I take an LDIF from slapcat on slapd 2.3.30 and run: $ cat dump.ldif | grep -v -E '^(entryCSN:|contextCSN:)'> load_noCSN.ldif

  The data (dump.ldif) looks like this (the root object):

=================================== dn: cn=data,dc=example,dc=com objectClass: top objectClass: NamedObject objectClass: dcObject objectClass: simpleSecurityObject cn: data userPassword:: BASE64 structuralObjectClass: NamedObject entryUUID: ab7d5590-3e90-102c-8c03-91e70ecd3b46 creatorsName: cn=data,dc=example,dc=com modifiersName: cn=data,dc=example,dc=com createTimestamp: 20071214130312Z modifyTimestamp: 20071214130312Z entryCSN: 20071214130312Z#000000#00#000000 contextCSN: 20091118105948Z#000001#00#000000 =====================================

• Then I STOP slapd on both servers.

• Then I load the output on server1: $ slapadd -S 1 -q -w -l ~/load_noCSN.ldif

• Then I immediately slapcat this and move it to server2: $ slapcat> ~/toserver2.ldif

• And load it on server2: $ slapadd -q -l ~/toserver2.ldif

• I start server1, but BEFORE I start server2 I make ONE SINGLE CHANGE:

================= dn: cn=data,dc=example,dc=com changetype: modify replace: userPassword userPassword: NEWBASE64 =================

• THEN I start server2 and monitor it's data.

What I find is that the contextCSN from server1 gets replicated, but the change doesn't. Also I see a contextCSN on server2 with SID 002 without I've done any operations on server2.

No idea what that is. Your debug logs should tell what it was doing.