(ITS#5866) syncrepl : all entries not replicated (glue entryCSN problem) - openldap-bugs

19 Dec 2008


      Full_Name: Julien Combes
Version: 2.4.13
OS: debian 4.0 etch
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (212.23.162.38)
Hello,
I am studying the migration of my architecture from 2.3.39 to 2.4.13.
The provider is in 2.4.13. I have a problem with syncrepl (and delta-syncrepl)
with two consumers in openldap 2.4.13 when I start them with an empty database :
some entries are not replicated.  A consumer with openldap 2.3.39 is not
affected by the problem : all entries are replicated correctly.
All consumers are using the same replication account.
As I am in a testing environment, I have retried several times the test: every
time, the same entries are not replicated on both consumers.
All entries not replicated are "organizationalUnit" on the provider and have
entries below them. All entries below them are correctly replicated. With a
slapcat on the consumers I can see that the entries not replicated are "glue".
Servers :
-> provider : openldap 2.4.13, BDB 4.7.25 - number of entries : 191961
-> consumer-1 : openldap 2.4.13, BDB 4.7.25 - number of entries after
replication : 191937
-> consumer-2 : openldap 2.4.13, BDB 4.7.25 - number of entries after
replication : 191937
-> consumer-3 : openldap 2.3.39, BDB 4.2.52 - number of entries after
replication : 191961
The logfile (sync, stats) indicate that :
-> one entry below the organizationalUnit (not replicated) is replicated before
the organizationalUnit itself
Dec 18 18:18:17 consumer-2 slapd[13897]: syncrepl_entry: rid=003
mineqRDN=_uid_john.doe,ou=foo6,ou=foo5,ou=foo4,ou=foo3,ou=foo2,ou=foo,ou=bar,dc=my,dc=domain
-> when it's the turn of this organizationalUnit to be replicated, the log
indicate lignes like that :
Dec 18 18:18:17 consumer-2 slapd[13897]: syncrepl_entry: rid=003
ou=foo6,ou=foo5,ou=foo4,ou=foo3,ou=foo2,ou=foo,ou=bar,dc=my,dc=domain
Dec 18 18:18:17 consumer-2 slapd[13897]: syncrepl_entry: rid=003 be_add (68)
Dec 18 18:18:17 consumer-2 slapd[13897]: dn_callback : new entry is older than
ours ou=foo6,ou=foo5,ou=foo4,ou=foo3,ou=foo2,ou=foo,ou=bar,dc=my,dc=domain ours
20081204104512.468400Z#000000#000#000000, new
20081020233519.000000Z#000018#000#000000
Dec 18 18:18:17 consumer-2 slapd[13897]: syncrepl_entry: rid=003 entry
unchanged, ignored (ou=foo6,ou=foo5,ou=foo4,ou=foo3,ou=foo2,ou=foo,ou=bar,dc=my,dc=domain)
About the entryCSN of this log :
-> "ours 20081204104512.468400Z#000000#000#000000" seems to be the entryCSN of
the entry below replicated befor the organizationalUnit :
On consumer-[12] :
dn: mineqRDN=_uid_john.doe,ou=foo6,ou=foo5,ou=foo4,ou=foo3,ou=foo2,ou=foo,ou=bar,dc=my,dc=domain
entryCSN: 20081204104512.468400Z#000000#000#000000
On Provider :
dn: mineqRDN=_uid_john.doe,ou=foo6,ou=foo5,ou=foo4,ou=foo3,ou=foo2,ou=foo,ou=bar,dc=my,dc=domain
entryCSN: 20081204104512.468400Z#000000#000#000000
-> "new 20081020233519.000000Z#000018#000#000000" is the entryCSN of the
organisazionalUnit on the provider :
dn: ou=foo6,ou=foo5,ou=foo4,ou=foo3,ou=foo2,ou=foo,ou=bar,dc=my,dc=domain
entryCSN: 20081020233519.000000Z#000018#000#000000
All servers are a the same time.
Configuration files :
=====================================================================
provider : openldap 2.4.13, BDB 4.7.25
=====================================================================
include         /etc/ldap/schema/core.schema
include         /etc/ldap/schema/cosine.schema
include         /etc/ldap/schema/nis.schema
include         /etc/ldap/schema/inetorgperson.schema
include         /etc/ldap/schema/pamela.schema
include         /etc/ldap/schema/rfc2739.schema
include         /etc/ldap/schema/samba.schema
sizelimit       5000
timelimit       10
pidfile         /var/run/slapd/slapd.pid
argsfile        /var/run/slapd/slapd.args
loglevel        stats sync
modulepath      /usr/lib/ldap
moduleload      back_hdb
moduleload      syncprov
moduleload      accesslog
moduleload      back_monitor
defaultsearchbase       "dc=my,dc=domain"
TLSCACertificateFile    /etc/certs/CA_cert.pem
TLSCertificateFile      /etc/certs/provider.my.domain.pem
TLSCertificateKeyFile   /etc/certs/provider.my.domain.key
access to dn.subtree="cn=monitor"
        by peername.ip=127.0.0.1 read
        by * none
access to dn.subtree="cn=accesslog"
        by dn.base="cn=admin,ou=admin,ou=ressources,dc=my,dc=domain" read
        by dn.base="cn=syncuser.csac,ou=admin,ou=ressources,dc=my,dc=domain"
read
        by * none
access to attrs=userPassword
        by dn.base="cn=admin,ou=admin,ou=ressources,dc=my,dc=domain" write
        by dn.base="cn=syncuser.csac,ou=admin,ou=ressources,dc=my,dc=domain"
read
        by anonymous auth
        by self write
        by * none
access to *
        by dn.base="cn=admin,ou=admin,ou=ressources,dc=my,dc=domain" write
        by dn.base="cn=syncuser.csac,ou=admin,ou=ressources,dc=my,dc=domain"
read
        by * read
database        hdb
suffix "cn=accesslog"
rootdn "cn=accesslog"
directory       "/var/lib/ldap/accesslog"
index entryCSN,objectClass,reqEnd,reqResult,reqStart eq
overlay syncprov
syncprov-nopresent      TRUE
syncprov-reloadhint     TRUE
limits dn.base="cn=syncuser.csac,ou=admin,ou=ressources,dc=my,dc=domain"
        size.soft=unlimited size.hard=unlimited
        time.soft=unlimited time.hard=unlimited
limits dn.base="cn=admin,ou=admin,ou=ressources,dc=my,dc=domain"
        size.soft=unlimited size.hard=unlimited
        time.soft=unlimited time.hard=unlimited
database        hdb
suffix          "dc=my,dc=domain"
directory       "/var/lib/ldap/database"
cachesize    75000
cachefree      1000
idlcachesize 225000
heckpoint 512 10
index           objectClass eq
index mineqMelPartages,mineqLiensImport,mineqMelmailEmission,cn pres,eq,sub
index mail pres,eq
index uid,mineqMelMembres,employeeNumber,ou,gidnumber,uidNumber,mineqTypeEntree,sn,drink,aliasedObjectName,memberUid
pres,eq
index entryCSN,entryUUID eq
lastmod         on
overlay syncprov
syncprov-checkpoint 100 10
overlay accesslog
logdb "cn=accesslog"
logops writes
logsuccess TRUE
logpurge 07+00:00 01+00:00
limits dn.base="cn=syncuser.csac,ou=admin,ou=ressources,dc=my,dc=domain"
        size.soft=unlimited size.hard=unlimited
        time.soft=unlimited time.hard=unlimited
limits dn.base="cn=admin,ou=admin,ou=ressources,dc=my,dc=domain"
        size.soft=unlimited size.hard=unlimited
        time.soft=unlimited time.hard=unlimited
database        monitor
=====================================================================
consumer-1 : openldap 2.4.13, BDB 4.7.25
=====================================================================
allow bind_v2
include         /etc/ldap/schema/core.schema
include         /etc/ldap/schema/cosine.schema
include         /etc/ldap/schema/nis.schema
include         /etc/ldap/schema/inetorgperson.schema
include         /etc/ldap/schema/pamela.schema
include         /etc/ldap/schema/rfc2739.schema
include         /etc/ldap/schema/samba.schema
sizelimit       20000
pidfile         /var/run/slapd/slapd.pid
argsfile        /var/run/slapd/slapd.args
loglevel        stats sync
modulepath      /usr/lib/ldap
moduleload      back_hdb
moduleload      back_monitor
defaultsearchbase       "dc=my,dc=domain"
TLSCACertificateFile    /etc/certs/CA_cert.pem
TLSCertificateFile      /etc/certs/consumer-01.my.domain.pem
TLSCertificateKeyFile   /etc/certs/consumer-01.my.domain.key
access to dn.subtree="cn=monitor"
        by peername.ip=127.0.0.1 read
        by * none
access to attrs=userPassword
        by dn="cn=admin,ou=admin,ou=ressources,dc=my,dc=domain" write
        by anonymous auth
        by self write
        by * none
access to *
        by dn="cn=admin,ou=admin,ou=ressources,dc=my,dc=domain" write
        by * read
database        hdb
suffix          "dc=my,dc=domain"
rootdn          "dc=my,dc=domain"
limits dn.exact="cn=admin,ou=admin,ou=ressources,dc=my,dc=domain"
        size.soft=unlimited size.hard=unlimited
        time.soft=unlimited time.hard=unlimited
syncrepl rid=2
        provider=ldaps://provider.my.domain
        type=refreshAndPersist
        retry="10 3 30 3 60 +"
        searchbase="dc=my,dc=domain"
        filter="(objectClass=*)"
        scope=sub
        schemachecking=off
        bindmethod=simple
        binddn="cn=syncuser.csac,ou=admin,ou=ressources,dc=my,dc=domain"
        credentials=<secret>
        logbase="cn=accesslog"
        syncdata=accesslog
updateref       ldaps://provider.my.domain
directory       "/var/lib/ldap/database"
cachesize    75000
cachefree      1000
idlcachesize 225000
checkpoint 0 10
index           objectClass eq
index           uid,mail pres,eq
index           cn pres,eq,sub
index           entryCSN,entryUUID eq
lastmod         on
database        monitor
=====================================================================
consumer-2 : openldap 2.4.13, BDB 4.7.25
=====================================================================
include         /etc/ldap/schema/core.schema
include         /etc/ldap/schema/cosine.schema
include         /etc/ldap/schema/nis.schema
include         /etc/ldap/schema/inetorgperson.schema
include         /etc/ldap/schema/pamela.schema
include         /etc/ldap/schema/rfc2739.schema
include         /etc/ldap/schema/samba.schema
sizelimit       5000
timelimit       10
pidfile         /var/run/slapd/slapd.pid
argsfile        /var/run/slapd/slapd.args
loglevel        stats sync
modulepath      /usr/lib/ldap
moduleload      back_hdb
moduleload      back_monitor
defaultsearchbase       "dc=my,dc=domain"
TLSCACertificateFile    /etc/certs/CA_cert.pem
TLSCertificateFile      /etc/certs/consumer-02.my.domain.pem
TLSCertificateKeyFile   /etc/certs/consumer-02.my.domain.key
access to dn.subtree="cn=monitor"
        by peername.ip=127.0.0.1 read
        by * none
access to attrs=userPassword
        by dn.base="cn=admin,ou=admin,ou=ressources,dc=my,dc=domain" write
        by anonymous auth
        by self write
        by * none
access to *
        by dn.base="cn=admin,ou=admin,ou=ressources,dc=my,dc=domain" write
        by * read
database        hdb
suffix          "dc=my,dc=domain"
rootdn          "dc=my,dc=domain"
directory       "/var/lib/ldap/database"
cachesize    75000
cachefree      1000
idlcachesize 225000
checkpoint 512 10
index           objectClass eq
index mineqMelPartages,mineqLiensImport,mineqMelmailEmission,cn eq,sub
index mail eq
index uid,mineqMelMembres,employeeNumber,ou,gidnumber,uidNumber,mineqTypeEntree,sn,drink,aliasedObjectName,memberUid
eq
index entryCSN,entryUUID eq
lastmod         on
syncrepl rid=3
        provider=ldaps://provider.my.domain
        type=refreshAndPersist
        retry="10 3 30 3 60 +"
        searchbase="dc=my,dc=domain"
        filter="(objectClass=*)"
        scope=sub
        schemachecking=off
        bindmethod=simple
        binddn="cn=syncuser.csac,ou=admin,ou=ressources,dc=my,dc=domain"
        credentials=<secret>
updateref       ldaps://provider.my.domain
limits dn.base="cn=admin,ou=admin,ou=ressources,dc=my,dc=domain"
        size.soft=unlimited size.hard=unlimited
        time.soft=unlimited time.hard=unlimited
database        monitor
=====================================================================
consumer-3 : openldap 2.3.39, BDB 4.2.52
=====================================================================
allow bind_v2
include         /etc/ldap/schema/core.schema
include         /etc/ldap/schema/cosine.schema
include         /etc/ldap/schema/nis.schema
include         /etc/ldap/schema/inetorgperson.schema
include         /etc/ldap/schema/pamela.schema
include         /etc/ldap/schema/rfc2739.schema
include         /etc/ldap/schema/samba.schema
schemacheck     on
sizelimit       20000
pidfile         /var/run/slapd/slapd.pid
argsfile        /var/run/slapd/slapd.args
loglevel        stats sync
modulepath      /usr/lib/ldap
moduleload      back_hdb
moduleload      back_monitor
defaultsearchbase       "dc=my,dc=domain"
TLSCACertificateFile    /etc/certs/CA_cert.pem
TLSCertificateFile      /etc/certs/consumer-03.my.domain.pem
TLSCertificateKeyFile   /etc/certs/consumer-03.my.domain.key
access to dn.subtree="cn=monitor"
        by peername.ip=127.0.0.1 read
        by * none
access to attrs=userPassword
        by dn="cn=admin,ou=admin,ou=ressources,dc=my,dc=domain" write
        by anonymous auth
        by self write
        by * none
access to *
        by dn="cn=admin,ou=admin,ou=ressources,dc=my,dc=domain" write
        by * read
database        hdb
suffix          "dc=my,dc=domain"
rootdn          "dc=my,dc=domain"
limits dn.exact="cn=admin,ou=admin,ou=ressources,dc=my,dc=domain"
        size.soft=unlimited size.hard=unlimited
        time.soft=unlimited time.hard=unlimited
syncrepl rid=1
        provider=ldaps://provider.my.domain
        type=refreshAndPersist
        retry="10 3 30 3 60 +"
        searchbase="dc=my,dc=domain"
        filter="(objectClass=*)"
        scope=sub
        schemachecking=off
        bindmethod=simple
        binddn="cn=syncuser.csac,ou=admin,ou=ressources,dc=my,dc=domain"
        credentials=<secret>
        logbase="cn=accesslog"
        syncdata=accesslog
updateref       ldaps://provider.my.domain
directory       "/var/lib/ldap/database"
cachesize    75000
cachefree      1000
idlcachesize 225000
checkpoint 0 10
index           objectClass eq
index           uid,mail eq
index           cn eq,sub
index           entryCSN,entryUUID eq
lastmod         on
database        monitor
Regards,
Julien