Re: Replication restores deleted user
by kevin sullivan
Quanah,
Thank you for the suggestion to move to delta-syncrepl MMR. Unfortunately,
I am having problems setting this up properly. After reading through some
documentation, I thought it would be simple but when I bring up slapd on my
two servers, they both start using around 100% CPU and in the debug output
the two servers are constantly looping through all of the objects in my DIT
and saying that the objects have not changed:
5453d6b5 @(#) $OpenLDAP: slapd 2.4.39 (Jun 18 2014 05:19:18) $
mockbuild(a)x86-028.build.eng.bos.redhat.com:
/builddir/build/BUILD/openldap-2.4.39/openldap-2.4.39/build-servers/servers/slapd
5453d6b5 hdb_monitor_db_open: monitoring disabled; configure monitor
database to enable
5453d6b5 slapd starting
...
5453d6b5 syncrepl_message_to_entry: rid=001 DN: dc=example,dc=com, UUID:
1cfcd560-f564-1033-9f47-b521eabdb6ad
5453d6b5 syncrepl_entry: rid=001 LDAP_RES_SEARCH_ENTRY(LDAP_SYNC_ADD)
5453d6b5 syncrepl_entry: rid=001 inserted UUID
1cfcd560-f564-1033-9f47-b521eabdb6ad
5453d6b5 dn_callback : entries have identical CSN dc=example,dc=com
20141031161001.910968Z#000000#000#000000
5453d6b5 syncrepl_entry: rid=001 be_search (0)
5453d6b5 syncrepl_entry: rid=001 dc=example,dc=com
5453d6b5 syncrepl_entry: rid=001 entry unchanged, ignored
(dc=example,dc=com)
5453d6b5 syncrepl_message_to_entry: rid=001 DN: ou=users,dc=example,dc=com,
UUID: 1cfe8cf2-f564-1033-9f48-b521eabdb6ad
5453d6b5 syncrepl_entry: rid=001 LDAP_RES_SEARCH_ENTRY(LDAP_SYNC_ADD)
5453d6b5 syncrepl_entry: rid=001 inserted UUID
1cfe8cf2-f564-1033-9f48-b521eabdb6ad
5453d6b5 dn_callback : entries have identical CSN
ou=users,dc=example,dc=com 20141031161001.922234Z#000000#000#000000
5453d6b5 syncrepl_entry: rid=001 be_search (0)
5453d6b5 syncrepl_entry: rid=001 ou=users,dc=example,dc=com
5453d6b5 syncrepl_entry: rid=001 entry unchanged, ignored
(ou=users,dc=example,dc=com)
.....
5453d6b5 do_syncrep2: rid=001 LDAP_RES_SEARCH_RESULT
5453d6b5 do_syncrep2: rid=001 cookie=
... Repeated forever ...
Am I configuring something incorrectly?
To refresh your memory, I am running 2.4.39-8. I have two servers (server1
and server2) that I want to setup in delta-syncrepl MMR MirrorMode.
slapd.conf for server1:
----------------------------------------------
modulepath /usr/lib64/openldap
moduleload accesslog.la
moduleload memberof.la
moduleload ppolicy.la
moduleload refint.la
moduleload syncprov.la
access to *
by dn.exact="cn=syncuser,dc=example,dc=com" read
by * break
access to dn.base=""
by * read
access to dn.base="cn=subschema"
by * read
access to attrs=userPassword,pwdHistory
by self write
by anonymous auth
by * none
access to dn.subtree="dc=example,dc=com"
by self write
by users read
by anonymous read
database config
access to *
by dn.exact="gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth"
manage
by * none
access to * by * none
database hdb
suffix "cn=accesslog"
directory /var/lib/ldap/accesslog
rootdn "cn=accesslog"
index default eq
index entryCSN,objectClass,reqEnd,reqResult,reqStart
overlay syncprov
syncprov-nopresent TRUE
syncprov-reloadhint TRUE
limits dn.exact="cn=syncuser,dc=example,dc=com" time.soft=unlimited
time.hard=unlimited size.soft=unlimited size.hard=unlimited
database hdb
suffix "dc=example,dc=com"
directory /var/lib/ldap
rootdn "cn=admin,dc=example,dc=com"
index objectClass eq,pres
index ou,cn,mail,surname,givenname eq,pres,sub
index uidNumber,gidNumber,loginShell eq,pres
index uid,memberUid eq,pres,sub
index nisMapName,nisMapEntry eq,pres,sub
index entryCSN eq
index entryUUID eq
limits dn.exact="cn=syncuser,dc=example,dc=com" time.soft=unlimited
time.hard=unlimited size.soft=unlimited size.hard=unlimited
overlay syncprov
syncprov-checkpoint 1000 60
serverID 1
syncrepl rid=001
provider=ldap://server2/
type=refreshAndPersist
retry="10 +"
searchbase="dc=example,dc=com"
bindmethod=simple
binddn="cn=syncuser,dc=example,dc=com"
credentials=secret
syncdata=accesslog
mirrormode on
overlay accesslog
logdb cn=accesslog
logops writes
logsuccess TRUE
logpurge 07+00:00 01+00:00
---------------------------------------
slapd.conf for server2:
-------------------------------------------
modulepath /usr/lib64/openldap
moduleload accesslog.la
moduleload memberof.la
moduleload ppolicy.la
moduleload refint.la
moduleload syncprov.la
access to *
by dn.exact="cn=syncuser,dc=example,dc=com" read
by * break
access to dn.base=""
by * read
access to dn.base="cn=subschema"
by * read
access to attrs=userPassword,pwdHistory
by self write
by anonymous auth
by * none
access to dn.subtree="dc=example,dc=com"
by self write
by users read
by anonymous read
database config
access to *
by dn.exact="gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth"
manage
by * none
access to * by * none
database hdb
suffix "cn=accesslog"
directory /var/lib/ldap/accesslog
rootdn "cn=accesslog"
index default eq
index entryCSN,objectClass,reqEnd,reqResult,reqStart
overlay syncprov
syncprov-nopresent TRUE
syncprov-reloadhint TRUE
limits dn.exact="cn=syncuser,dc=example,dc=com" time.soft=unlimited
time.hard=unlimited size.soft=unlimited size.hard=unlimited
database hdb
suffix "dc=example,dc=com"
directory /var/lib/ldap
rootdn "cn=admin,dc=example,dc=com"
index objectClass eq,pres
index ou,cn,mail,surname,givenname eq,pres,sub
index uidNumber,gidNumber,loginShell eq,pres
index uid,memberUid eq,pres,sub
index nisMapName,nisMapEntry eq,pres,sub
index entryCSN eq
index entryUUID eq
limits dn.exact="cn=syncuser,dc=example,dc=com" time.soft=unlimited
time.hard=unlimited size.soft=unlimited size.hard=unlimited
overlay syncprov
syncprov-checkpoint 1000 60
serverID 2
syncrepl rid=001
provider=ldap://server1/
type=refreshAndPersist
retry="10 +"
searchbase="dc=example,dc=com"
bindmethod=simple
binddn="cn=syncuser,dc=example,dc=com"
credentials=secret
syncdata=accesslog
mirrormode on
overlay accesslog
logdb cn=accesslog
logops writes
logsuccess TRUE
logpurge 07+00:00 01+00:00
-------------------------------------------
Are there any obvious errors that I am making?
Thanks,
Kevin
On Thu, Oct 30, 2014 at 4:24 PM, Quanah Gibson-Mount <quanah(a)zimbra.com>
wrote:
> --On Thursday, October 30, 2014 4:55 PM -0400 kevin sullivan <
> kevin4sullivan(a)gmail.com> wrote:
>
> I am also looking for any resources that explain exactly how replication
>> works and how conflicts are resolved in a multi-master configuration. For
>> example, I know there are two phases of replication: the present phase
>> and the delete phase. Could server1 be replicating the data for
>> 'deletedUser' to server2 in the present phase before server2 has
>> communicated to server1 that 'deletedUser' no longer exists in the delete
>> phase? I don't believe this problem happens when I just delete
>> 'deletedUser' (instead of deleting AND modifying). I know I am butchering
>> the technical details a little bit, which is why I am curious how this
>> situation is expected to be resolved.
>>
>
> Personaly I use delta-syncrepl MMR to avoid the general issues with
> syncrepl. I've found it to be much more reliable.
>
>
> --Quanah
>
>
>
> --
>
> Quanah Gibson-Mount
> Server Architect
> Zimbra, Inc.
> --------------------
> Zimbra :: the leader in open source messaging and collaboration
>
7 years, 1 month
LDAP Crafted Search Request Access Allowed
by Net Warrior
Hi there guys.
Recently we had an internal audit and it seems that our opelndap server is
not configured properly, it seems that null bind are allowed and Crafted
request as well are permited and it would be nice if anyone of you could
lend me a hand to fix this:
There are the access list:
olcAccess: {0}to attrs=userPassword by dn="cn=Manager,dc=domain,dc=com" wr
ite by anonymous auth by self write by * none
olcAccess: {1}to
attrs=cn,sn,memberUid,uidNumber,pwdHistory,pwdPolicySubentry,
gidNumber,homeDirectory,givenName,description,loginShell by self write by
ano
nymous read by * none
olcAccess: {2}to dn.base="" by * read
olcAccess: {3}to * by dn="cn=Manager,dc=domain,dc=com" write by * read
And from a client I got the following:
ldapsearch -x -s base -b '' -H ldap://ldapserver "(objectClass=*)" "*" +
# extended LDIF
#
# LDAPv3
# base <> with scope baseObject
# filter: (objectClass=*)
# requesting: * +
#
dn:
objectClass: top
objectClass: OpenLDAProotDSE
structuralObjectClass: OpenLDAProotDSE
configContext: cn=config
monitorContext: cn=Monitor
namingContexts: dc=domain,dc=com
supportedControl: 2.16.840.1.113730.3.4.18
supportedControl: 2.16.840.1.113730.3.4.2
supportedControl: 1.3.6.1.4.1.4203.1.10.1
supportedControl: 1.2.840.113556.1.4.319
supportedControl: 1.2.826.0.1.3344810.2.3
supportedControl: 1.3.6.1.1.13.2
supportedControl: 1.3.6.1.1.13.1
supportedControl: 1.3.6.1.1.12
supportedExtension: 1.3.6.1.4.1.1466.20037
supportedExtension: 1.3.6.1.4.1.4203.1.11.1
supportedExtension: 1.3.6.1.4.1.4203.1.11.3
supportedExtension: 1.3.6.1.1.8
supportedFeatures: 1.3.6.1.1.14
supportedFeatures: 1.3.6.1.4.1.4203.1.5.1
supportedFeatures: 1.3.6.1.4.1.4203.1.5.2
supportedFeatures: 1.3.6.1.4.1.4203.1.5.3
supportedFeatures: 1.3.6.1.4.1.4203.1.5.4
supportedFeatures: 1.3.6.1.4.1.4203.1.5.5
supportedLDAPVersion: 3
entryDN:
subschemaSubentry: cn=Subschema
It seems it's no good at all, any help appreciated
Best regards
7 years, 1 month
#Overview in openldap2/guide/admin/guide.html
by Ulrich Windl
Hi!
Can someone check whether there are still multiple "#Overview" HREFs in guide.html? In 2.4.26 the hyperlink (just one example) for "11.9.1. Overview" (Relay) jumps to "11.1.1. Overview" (BDB)...
# grep '#Overview' /usr/share/doc/packages/openldap2/guide/admin/guide.html
<A HREF="#Overview">11.1.1. Overview</A>
<A HREF="#Overview">11.2.1. Overview</A>
<A HREF="#Overview">11.3.1. Overview</A>
<A HREF="#Overview">11.4.1. Overview</A>
<A HREF="#Overview">11.5.1. Overview</A>
<A HREF="#Overview">11.6.1. Overview</A>
<A HREF="#Overview">11.7.1. Overview</A>
<A HREF="#Overview">11.8.1. Overview</A>
<A HREF="#Overview">11.9.1. Overview</A>
<A HREF="#Overview">11.10.1. Overview</A>
<A HREF="#Overview">12.1.1. Overview</A>
<A HREF="#Overview">12.2.1. Overview</A>
<A HREF="#Overview">12.3.1. Overview</A>
<A HREF="#Overview">12.4.1. Overview</A>
<A HREF="#Overview">12.5.1. Overview</A>
<A HREF="#Overview">12.6.1. Overview</A>
<A HREF="#Overview">12.7.1. Overview</A>
<A HREF="#Overview">12.8.1. Overview</A>
<A HREF="#Overview">12.9.1. Overview</A>
<A HREF="#Overview">12.10.1. Overview</A>
<A HREF="#Overview">12.11.1. Overview</A>
<A HREF="#Overview">12.12.1. Overview</A>
<A HREF="#Overview">12.13.1. Overview</A>
<A HREF="#Overview">12.14.1. Overview</A>
<A HREF="#Overview">12.15.1. Overview</A>
<A HREF="#Overview">12.16.1. Overview</A>
<A HREF="#Overview">12.17.1. Overview</A>
<A HREF="#Overview">12.18.1. Overview</A>
Regards,
Ulrich
7 years, 1 month
Re: Need information on alock file in data directory of OpenLDAP 2.4.39
by pramod kulkarni
Thanks for the information.
I tried few more tests,slapcat doesn't always hang and it hangs every
second restart of slapd.
I don't know really whether it is sharing of locks issue or something else.
Currently lock detect is set to DB_LOCK_DEFAULT even if I increase the lock
detect to Maximum locks no change in the state of slapcat.
If I upgrade BDB will this solve the issue of slapcat hanging ? , Have your
faced such situation of hanging in slapcat
I get this below message when I run the slapcat and it never goes forward
5452a973 The first database does not allow slapcat; using the first
available on e (2)
Regards,
Pramod
On Thu, Oct 30, 2014 at 7:41 PM, Quanah Gibson-Mount <quanah(a)zimbra.com>
wrote:
> --On Thursday, October 30, 2014 6:04 PM +0100 pramod kulkarni <
> pammu.kulkarni(a)gmail.com> wrote:
>
>
>>
>> Thanks for the response.
>> What if I disable the alock file generation in the back-bdb
>> initialization? will it affect other functionalities like replication etc.
>>
>
> You should understand what the alock file is for before you think about
> deleting it. It tracks the state of your database. If your slapcat is
> hanging, it sounds like you have DB corruption at the BDB level.
>
> --Quanah
>
>
> --
>
> Quanah Gibson-Mount
> Server Architect
> Zimbra, Inc.
> --------------------
> Zimbra :: the leader in open source messaging and collaboration
>
7 years, 1 month
Re: Replication restores deleted user
by kevin sullivan
Sorry about the confusion, I specified MirrorMode since that is a specific
type of Multi-Master Replication and I wanted to be exact.
To answer Michael's questions:
1. Check whether cn=syncuser,dc=example,dc=com can really read *everything*
and no ACLs are preventing access to relevant CSN attributes etc.
My replication user can read everything. In fact, these deletes work when
both instances of slapd are online when the delete happens. My problem only
occurs when one of the instances of slapd is offline when the delete occurs.
2. Try to reproduce your issue with 2.4.40.
I will try this out and let you know what happens.
I am also looking for any resources that explain exactly how replication
works and how conflicts are resolved in a multi-master configuration. For
example, I know there are two phases of replication: the present phase and
the delete phase. Could server1 be replicating the data for 'deletedUser'
to server2 in the present phase before server2 has communicated to server1
that 'deletedUser' no longer exists in the delete phase? I don't believe
this problem happens when I just delete 'deletedUser' (instead of deleting
AND modifying). I know I am butchering the technical details a little bit,
which is why I am curious how this situation is expected to be resolved.
Thanks,
Kevin
On Thu, Oct 30, 2014 at 2:42 PM, Quanah Gibson-Mount <quanah(a)zimbra.com>
wrote:
> --On Thursday, October 30, 2014 7:33 PM +0100 Michael Ströder <
> michael(a)stroeder.com> wrote:
>
> kevin sullivan wrote:
>>
>>> I have two servers (server1 and server2) running openldap 2.4.39-8 and
>>> they are configured to replicate via MirrorMode replication.
>>>
>>
>> Really mirror mode? Not MMR? You're config looks like MMR.
>>
>
> Mirror mode is a way of configuring MMR. It is *not* something "separate"
> from MMR. Mirror mode will *always* be MMR.
>
> --Quanah
>
> --
>
> Quanah Gibson-Mount
> Server Architect
> Zimbra, Inc.
> --------------------
> Zimbra :: the leader in open source messaging and collaboration
>
7 years, 1 month
Replication restores deleted user
by kevin sullivan
I have two servers (server1 and server2) running openldap 2.4.39-8 and they
are configured to replicate via MirrorMode replication.
Here is what I am seeing in order:
1) On server1, I create a two users: deleteUser and modifyUser.
2) I can see that these users are then properly synced to server2.
3) On server1, slapd is stopped.
4) On server2, I now delete 'deleteUser' and I modify 'modifyUser'.
5) On server1, slapd is started.
6) The two slapd instances replicate.
Outcome:
Both servers now have the deleted user 'deleteUser' in their databases like
the user was never deleted. However, the user 'modifyUser' was properly
updated in both places.
Expected outcome:
I would expect that 'deleteUser' wouldn't exist in either database. I would
expect that 'modifyUser' would be properly modified on both servers.
Why would this happen? Do I need to configure something specifically so
deletes are handled properly? Is this just a quirk with how replication
works?
Below are the relevant parts of each server's slapd.conf.
server1's configuration:
serverID 1
...
overlay syncprov
syncrepl rid=001
provider=ldap://server2/
type=refreshAndPersist
retry="10 +"
searchbase="dc=example,dc=com"
bindmethod=simple
binddn="cn=syncuser,dc=example,dc=com
credentials=secret
mirrormode on
...
server2's configuration:
serverID 2
...
overlay syncprov
syncrepl rid=001
provider=ldap://server1/
type=refreshAndPersist
retry="10 +"
searchbase="dc=example,dc=com"
bindmethod=simple
binddn="cn=syncuser,dc=example,dc=com
credentials=secret
mirrormode on
...
Thanks,
Kevin
7 years, 1 month
Need information on alock file in data directory of OpenLDAP 2.4.39
by pramod kulkarni
Hi,
I have taken the OpenLDAP 2.4.39 and able to work properly but when I do
slapcat then it hangs and always complains on alock.
But if I delete alock it works fine.
I have 4.6.21 version of bdb
I don't know much about alock file in bdb .
Can you please suggest is it ok to delete alock file for running slapd and
slapcat ?
Thanks,
Pramod
7 years, 1 month
2.4.41?
by Michael Ströder
HI!
It seems to me some import fixes were committed to git-master after cutting
2.4.40.
Will there be a 2.4.41 release?
Ciao, Michael.
7 years, 1 month
Overlays in Consumer and provider configuration
by Uli Tehrani
Hello all,
I have some questions regarding overlay configuration for consumer and
provider server.
It is necessary to load the same overlay modules in a consumer
configuration?
Is it also necessary to configure the overlay like in the provider
configuration ?
There are overlays that are needed only for the provider, e.g. syncprov.
Others are only needed for the consumer like the chaining overlay.
But for other overlays it is not that clear:
Dynlist needs to be configured for both servers.
But what's about dds, ppolicy and refinit?
Here maybe a provider configuration is enough.
Is there a simple best practise ?
Thanks in advance.
Regards
Uli
--
===================================
Ulrich Tehrani
Am Ulrichshof 19
79189 Bad Krozingen
+497633806246
u_tehrani(a)yahoo.de
===================================
7 years, 1 month
syncrepl: turn consumer into a stand-alone ldap server
by Yannick Barbeaux
Hi everyone,
I have configured an ldap replication based on the producer-consumer
mechanism using the syncrepl module.
It worked fine but at first, the ldap tree was only partially imported on
the consumer because the autofs.schema was missing.
It took me a few hours (days?) to find out that I had to import the
autofs.ldif manually on the consumer to make it work properly:
ldapadd -Y EXTERNAL -H ldapi:/// -f /etc/ldap/schema/autofs.ldif
(that might not be the recommended way to achieve the sync but that worked).
Now that I have my producer and consumer perfectly in sync, I would like to
get rid of the producer server and turn my consumer into the master ldap
server (that might be used later as a producer). I wonder if it is
possible...
I almost achieved "un-configuring" the consumer mechanism but the
"ex-consumer" has now an odd behaviour : it allows me to modify the ldap
tree with ldapmodify (normally impossible on consumer) but the tree is
effectively modified on the ex-producer and not on the consumer itself. Yet
when I perform an ldapsearch, it searches in the consumer tree, not on the
producer side.
To initially configure the consumer, I had injected the following ldif file:
### consumer.ldif ###
#Load the syncprov module.
dn: cn=module{0},cn=config
changetype: modify
add: olcModuleLoad
olcModuleLoad: syncprov
#syncrepl specific indices
dn: olcDatabase={1}bdb,cn=config
changetype: modify
add: olcDbIndex
olcDbIndex: entryUUID eq
-
add: olcSyncRepl
olcSyncRepl: rid=001 provider=ldap://10.50.1.11 bindmethod=simple
binddn="cn=synchronisator,dc=office,dc=myorg,dc=be" credentials=mysecret
searchbase="dc=office,dc=myorg,dc=be" logbase="cn=accesslog"
logfilter="(&(objectClass=auditWriteObject)(reqResult=0))"
schemachecking=on type=refreshAndPersist retry="60 +" syncdata=accesslog
-
add: olcUpdateRef
olcUpdateRef: ldap://10.50.1.11
So I thought that after sync, disabling the consumer property would be as
simple as unloading the syncprov module and removing the olcSyncRepl
directive... this way:
### removeSyncprovModule.ldif ###
dn: cn=module{0},cn=config
changetype: modify
delete: olcModuleLoad
olcModuleLoad: {1}syncprov
### disableConsumer.ldif ###
dn: olcDatabase={1}bdb,cn=config
changetype: modify
delete: olcSyncrepl
-
delete: olcUpdateRef
But the ldapmodify returned an error when trying to inject that ldif file.
Such operations are not allowed on the consumer.
So I had no other choice than to edit the ldif manually (though it is not
recommended!) to delete the corresponding directives and restart the ldap
server ( /etc/ldap/slapd.d/cn=config/cn=module{0}.ldif )
Of course the server complained about wrong checksums. So I applied the
method suggested on this page to fix it:
http://serverfault.com/questions/499856/is-there-any-bad-thing-happens-if...
(basically removing and re-adding the schemas+data using *slapcat* and
*slapadd)*
Since that, the consumer is not sync-ed with the producer anymore (good)
but as I mentionned above, any attempt to modify the tree on the
ex-consumer sides results in a modification on the ex-producer side and not
on the consumer.
Is there any easier and working way to turn a consumer into a stand-alone
master ldap server?
Thank you.
Yannick
7 years, 1 month
