Re: Replication restores deleted user - openldap-technical

31 Oct 2014


      Quanah,
Thank you for the suggestion to move to delta-syncrepl MMR. Unfortunately,
I am having problems setting this up properly. After reading through some
documentation, I thought it would be simple but when I bring up slapd on my
two servers, they both start using around 100% CPU and in the debug output
the two servers are constantly looping through all of the objects in my DIT
and saying that the objects have not changed:
5453d6b5 @(#) $OpenLDAP: slapd 2.4.39 (Jun 18 2014 05:19:18) $
        mockbuild@x86-028.build.eng.bos.redhat.com:
/builddir/build/BUILD/openldap-2.4.39/openldap-2.4.39/build-servers/servers/slapd
5453d6b5 hdb_monitor_db_open: monitoring disabled; configure monitor
database to enable
5453d6b5 slapd starting
...
5453d6b5 syncrepl_message_to_entry: rid=001 DN: dc=example,dc=com, UUID:
1cfcd560-f564-1033-9f47-b521eabdb6ad
5453d6b5 syncrepl_entry: rid=001 LDAP_RES_SEARCH_ENTRY(LDAP_SYNC_ADD)
5453d6b5 syncrepl_entry: rid=001 inserted UUID
1cfcd560-f564-1033-9f47-b521eabdb6ad
5453d6b5 dn_callback : entries have identical CSN dc=example,dc=com
20141031161001.910968Z#000000#000#000000
5453d6b5 syncrepl_entry: rid=001 be_search (0)
5453d6b5 syncrepl_entry: rid=001 dc=example,dc=com
5453d6b5 syncrepl_entry: rid=001 entry unchanged, ignored
(dc=example,dc=com)
5453d6b5 syncrepl_message_to_entry: rid=001 DN: ou=users,dc=example,dc=com,
UUID: 1cfe8cf2-f564-1033-9f48-b521eabdb6ad
5453d6b5 syncrepl_entry: rid=001 LDAP_RES_SEARCH_ENTRY(LDAP_SYNC_ADD)
5453d6b5 syncrepl_entry: rid=001 inserted UUID
1cfe8cf2-f564-1033-9f48-b521eabdb6ad
5453d6b5 dn_callback : entries have identical CSN
ou=users,dc=example,dc=com 20141031161001.922234Z#000000#000#000000
5453d6b5 syncrepl_entry: rid=001 be_search (0)
5453d6b5 syncrepl_entry: rid=001 ou=users,dc=example,dc=com
5453d6b5 syncrepl_entry: rid=001 entry unchanged, ignored
(ou=users,dc=example,dc=com)
.....
5453d6b5 do_syncrep2: rid=001 LDAP_RES_SEARCH_RESULT
5453d6b5 do_syncrep2: rid=001 cookie=
... Repeated forever ...
Am I configuring something incorrectly?
To refresh your memory, I am running 2.4.39-8. I have two servers (server1
and server2) that I want to setup in delta-syncrepl MMR MirrorMode.
slapd.conf for server1:
----------------------------------------------
modulepath /usr/lib64/openldap
moduleload accesslog.la
moduleload memberof.la
moduleload ppolicy.la
moduleload refint.la
moduleload syncprov.la
access to *
    by dn.exact="cn=syncuser,dc=example,dc=com" read
    by * break
access to dn.base=""
    by * read
access to dn.base="cn=subschema"
    by * read
access to attrs=userPassword,pwdHistory
    by self write
    by anonymous auth
    by * none
access to dn.subtree="dc=example,dc=com"
    by self write
    by users read
    by anonymous read
database config
access to *
    by dn.exact="gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth"
manage
    by * none
access to * by * none
database    hdb
suffix        "cn=accesslog"
directory       /var/lib/ldap/accesslog
rootdn            "cn=accesslog"
index            default eq
index            entryCSN,objectClass,reqEnd,reqResult,reqStart
overlay syncprov
syncprov-nopresent TRUE
syncprov-reloadhint TRUE
limits dn.exact="cn=syncuser,dc=example,dc=com" time.soft=unlimited
time.hard=unlimited size.soft=unlimited size.hard=unlimited
database    hdb
suffix        "dc=example,dc=com"
directory    /var/lib/ldap
rootdn        "cn=admin,dc=example,dc=com"
index objectClass                       eq,pres
index ou,cn,mail,surname,givenname      eq,pres,sub
index uidNumber,gidNumber,loginShell    eq,pres
index uid,memberUid                     eq,pres,sub
index nisMapName,nisMapEntry            eq,pres,sub
index entryCSN                          eq
index entryUUID                         eq
limits dn.exact="cn=syncuser,dc=example,dc=com" time.soft=unlimited
time.hard=unlimited size.soft=unlimited size.hard=unlimited
overlay syncprov
syncprov-checkpoint 1000 60
serverID 1
syncrepl rid=001
    provider=ldap://server2/
    type=refreshAndPersist
    retry="10 +"
    searchbase="dc=example,dc=com"
    bindmethod=simple
    binddn="cn=syncuser,dc=example,dc=com"
    credentials=secret
    syncdata=accesslog
mirrormode on
overlay accesslog
logdb cn=accesslog
logops writes
logsuccess TRUE
logpurge 07+00:00 01+00:00
---------------------------------------
slapd.conf for server2:
-------------------------------------------
modulepath /usr/lib64/openldap
moduleload accesslog.la
moduleload memberof.la
moduleload ppolicy.la
moduleload refint.la
moduleload syncprov.la
access to *
    by dn.exact="cn=syncuser,dc=example,dc=com" read
    by * break
access to dn.base=""
    by * read
access to dn.base="cn=subschema"
    by * read
access to attrs=userPassword,pwdHistory
    by self write
    by anonymous auth
    by * none
access to dn.subtree="dc=example,dc=com"
    by self write
    by users read
    by anonymous read
database config
access to *
    by dn.exact="gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth"
manage
    by * none
access to * by * none
database    hdb
suffix        "cn=accesslog"
directory       /var/lib/ldap/accesslog
rootdn            "cn=accesslog"
index            default eq
index            entryCSN,objectClass,reqEnd,reqResult,reqStart
overlay syncprov
syncprov-nopresent TRUE
syncprov-reloadhint TRUE
limits dn.exact="cn=syncuser,dc=example,dc=com" time.soft=unlimited
time.hard=unlimited size.soft=unlimited size.hard=unlimited
database    hdb
suffix        "dc=example,dc=com"
directory    /var/lib/ldap
rootdn        "cn=admin,dc=example,dc=com"
index objectClass                       eq,pres
index ou,cn,mail,surname,givenname      eq,pres,sub
index uidNumber,gidNumber,loginShell    eq,pres
index uid,memberUid                     eq,pres,sub
index nisMapName,nisMapEntry            eq,pres,sub
index entryCSN                          eq
index entryUUID                         eq
limits dn.exact="cn=syncuser,dc=example,dc=com" time.soft=unlimited
time.hard=unlimited size.soft=unlimited size.hard=unlimited
overlay syncprov
syncprov-checkpoint 1000 60
serverID 2
syncrepl rid=001
    provider=ldap://server1/
    type=refreshAndPersist
    retry="10 +"
    searchbase="dc=example,dc=com"
    bindmethod=simple
    binddn="cn=syncuser,dc=example,dc=com"
    credentials=secret
    syncdata=accesslog
mirrormode on
overlay accesslog
logdb cn=accesslog
logops writes
logsuccess TRUE
logpurge 07+00:00 01+00:00
-------------------------------------------
Are there any obvious errors that I am making?
Thanks,
Kevin
On Thu, Oct 30, 2014 at 4:24 PM, Quanah Gibson-Mount quanah@zimbra.com
wrote:
...
--On Thursday, October 30, 2014 4:55 PM -0400 kevin sullivan <
kevin4sullivan@gmail.com> wrote:
I am also looking for any resources that explain exactly how replication
...
works and how conflicts are resolved in a multi-master configuration. For
example, I know there are two phases of replication: the present phase
and the delete phase. Could server1 be replicating the data for
'deletedUser' to server2 in the present phase before server2 has
communicated to server1 that 'deletedUser' no longer exists in the delete
phase? I don't believe this problem happens when I just delete
'deletedUser' (instead of deleting AND modifying). I know I am butchering
the technical details a little bit, which is why I am curious how this
situation is expected to be resolved.
Personaly I use delta-syncrepl MMR to avoid the general issues with
syncrepl.  I've found it to be much more reliable.
--Quanah
--
Quanah Gibson-Mount
Server Architect
Zimbra, Inc.

Zimbra ::  the leader in open source messaging and collaboration