openldap-technical

openldap-technical@openldap.org

9 participants
9902 discussions

Re: Replication restores deleted user
by Quanah Gibson-Mount 31 Oct '14

31 Oct '14

--On Friday, October 31, 2014 4:13 PM -0400 kevin sullivan <kevin4sullivan(a)gmail.com> wrote: > > > > > > Quanah, > > Thank you for the suggestion to move to delta-syncrepl MMR. > Unfortunately, I am having problems setting this up properly. After > reading through some documentation, I thought it would be simple but when > I bring up slapd on my two servers, they both start using around 100% CPU > and in the debug output the two servers are constantly looping through > all of the objects in my DIT and saying that the objects have not changed: > > 5453d6b5 @(#) $OpenLDAP: slapd 2.4.39 (Jun 18 2014 05:19:18) $ > > mockbuild@x86-028.build.eng.bos.redhat.com:/builddir/build/BUILD/openldap > -2.4.39/openldap-2.4.39/build-servers/servers/slapd > 5453d6b5 hdb_monitor_db_open: monitoring disabled; configure monitor > database to enable > 5453d6b5 slapd starting > ... > 5453d6b5 syncrepl_message_to_entry: rid=001 DN: dc=example,dc=com, UUID: > 1cfcd560-f564-1033-9f47-b521eabdb6ad > 5453d6b5 syncrepl_entry: rid=001 LDAP_RES_SEARCH_ENTRY(LDAP_SYNC_ADD) > 5453d6b5 syncrepl_entry: rid=001 inserted UUID > 1cfcd560-f564-1033-9f47-b521eabdb6ad > 5453d6b5 dn_callback : entries have identical CSN dc=example,dc=com > 20141031161001.910968Z#000000#000#000000 > 5453d6b5 syncrepl_entry: rid=001 be_search (0) > 5453d6b5 syncrepl_entry: rid=001 dc=example,dc=com > 5453d6b5 syncrepl_entry: rid=001 entry unchanged, ignored > (dc=example,dc=com) > 5453d6b5 syncrepl_message_to_entry: rid=001 DN: > ou=users,dc=example,dc=com, UUID: 1cfe8cf2-f564-1033-9f48-b521eabdb6ad > 5453d6b5 syncrepl_entry: rid=001 LDAP_RES_SEARCH_ENTRY(LDAP_SYNC_ADD) > 5453d6b5 syncrepl_entry: rid=001 inserted UUID > 1cfe8cf2-f564-1033-9f48-b521eabdb6ad > 5453d6b5 dn_callback : entries have identical CSN > ou=users,dc=example,dc=com 20141031161001.922234Z#000000#000#000000 > 5453d6b5 syncrepl_entry: rid=001 be_search (0) > 5453d6b5 syncrepl_entry: rid=001 ou=users,dc=example,dc=com > 5453d6b5 syncrepl_entry: rid=001 entry unchanged, ignored > (ou=users,dc=example,dc=com) > ..... > 5453d6b5 do_syncrep2: rid=001 LDAP_RES_SEARCH_RESULT > 5453d6b5 do_syncrep2: rid=001 cookie= > ... Repeated forever ... > > Am I configuring something incorrectly? > > To refresh your memory, I am running 2.4.39-8. I have two servers > (server1 and server2) that I want to setup in delta-syncrepl MMR > MirrorMode. Hi Kevin, I stopped using the deprecated slapd.conf format years ago, as it's prone to misuse and incorrect setup. Your supplied config is an example of why it's a bad idea to use slapd.conf (For example, you have serverID under the database section, when it's a global option, etc). I'd strongly advise you to switch to using cn=config so that you can have an actual validated configuration. It also makes troubleshooting a lot easier to do. Aside from that, your log simply shows the server comparing each entry between the two servers... Sounds like they didn't start out believing they were in sync and are trying to get there. Here's an example of my config, using cn=config (slapcat export) minus the schema: <http://pastebin.com/nv1WNX2y> --Quanah -- Quanah Gibson-Mount Server Architect Zimbra, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

Re: Replication restores deleted user
by kevin sullivan 31 Oct '14

31 Oct '14

Quanah, Thank you for the suggestion to move to delta-syncrepl MMR. Unfortunately, I am having problems setting this up properly. After reading through some documentation, I thought it would be simple but when I bring up slapd on my two servers, they both start using around 100% CPU and in the debug output the two servers are constantly looping through all of the objects in my DIT and saying that the objects have not changed: 5453d6b5 @(#) $OpenLDAP: slapd 2.4.39 (Jun 18 2014 05:19:18) $ mockbuild(a)x86-028.build.eng.bos.redhat.com: /builddir/build/BUILD/openldap-2.4.39/openldap-2.4.39/build-servers/servers/slapd 5453d6b5 hdb_monitor_db_open: monitoring disabled; configure monitor database to enable 5453d6b5 slapd starting ... 5453d6b5 syncrepl_message_to_entry: rid=001 DN: dc=example,dc=com, UUID: 1cfcd560-f564-1033-9f47-b521eabdb6ad 5453d6b5 syncrepl_entry: rid=001 LDAP_RES_SEARCH_ENTRY(LDAP_SYNC_ADD) 5453d6b5 syncrepl_entry: rid=001 inserted UUID 1cfcd560-f564-1033-9f47-b521eabdb6ad 5453d6b5 dn_callback : entries have identical CSN dc=example,dc=com 20141031161001.910968Z#000000#000#000000 5453d6b5 syncrepl_entry: rid=001 be_search (0) 5453d6b5 syncrepl_entry: rid=001 dc=example,dc=com 5453d6b5 syncrepl_entry: rid=001 entry unchanged, ignored (dc=example,dc=com) 5453d6b5 syncrepl_message_to_entry: rid=001 DN: ou=users,dc=example,dc=com, UUID: 1cfe8cf2-f564-1033-9f48-b521eabdb6ad 5453d6b5 syncrepl_entry: rid=001 LDAP_RES_SEARCH_ENTRY(LDAP_SYNC_ADD) 5453d6b5 syncrepl_entry: rid=001 inserted UUID 1cfe8cf2-f564-1033-9f48-b521eabdb6ad 5453d6b5 dn_callback : entries have identical CSN ou=users,dc=example,dc=com 20141031161001.922234Z#000000#000#000000 5453d6b5 syncrepl_entry: rid=001 be_search (0) 5453d6b5 syncrepl_entry: rid=001 ou=users,dc=example,dc=com 5453d6b5 syncrepl_entry: rid=001 entry unchanged, ignored (ou=users,dc=example,dc=com) ..... 5453d6b5 do_syncrep2: rid=001 LDAP_RES_SEARCH_RESULT 5453d6b5 do_syncrep2: rid=001 cookie= ... Repeated forever ... Am I configuring something incorrectly? To refresh your memory, I am running 2.4.39-8. I have two servers (server1 and server2) that I want to setup in delta-syncrepl MMR MirrorMode. slapd.conf for server1: ---------------------------------------------- modulepath /usr/lib64/openldap moduleload accesslog.la moduleload memberof.la moduleload ppolicy.la moduleload refint.la moduleload syncprov.la access to * by dn.exact="cn=syncuser,dc=example,dc=com" read by * break access to dn.base="" by * read access to dn.base="cn=subschema" by * read access to attrs=userPassword,pwdHistory by self write by anonymous auth by * none access to dn.subtree="dc=example,dc=com" by self write by users read by anonymous read database config access to * by dn.exact="gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth" manage by * none access to * by * none database hdb suffix "cn=accesslog" directory /var/lib/ldap/accesslog rootdn "cn=accesslog" index default eq index entryCSN,objectClass,reqEnd,reqResult,reqStart overlay syncprov syncprov-nopresent TRUE syncprov-reloadhint TRUE limits dn.exact="cn=syncuser,dc=example,dc=com" time.soft=unlimited time.hard=unlimited size.soft=unlimited size.hard=unlimited database hdb suffix "dc=example,dc=com" directory /var/lib/ldap rootdn "cn=admin,dc=example,dc=com" index objectClass eq,pres index ou,cn,mail,surname,givenname eq,pres,sub index uidNumber,gidNumber,loginShell eq,pres index uid,memberUid eq,pres,sub index nisMapName,nisMapEntry eq,pres,sub index entryCSN eq index entryUUID eq limits dn.exact="cn=syncuser,dc=example,dc=com" time.soft=unlimited time.hard=unlimited size.soft=unlimited size.hard=unlimited overlay syncprov syncprov-checkpoint 1000 60 serverID 1 syncrepl rid=001 provider=ldap://server2/ type=refreshAndPersist retry="10 +" searchbase="dc=example,dc=com" bindmethod=simple binddn="cn=syncuser,dc=example,dc=com" credentials=secret syncdata=accesslog mirrormode on overlay accesslog logdb cn=accesslog logops writes logsuccess TRUE logpurge 07+00:00 01+00:00 --------------------------------------- slapd.conf for server2: ------------------------------------------- modulepath /usr/lib64/openldap moduleload accesslog.la moduleload memberof.la moduleload ppolicy.la moduleload refint.la moduleload syncprov.la access to * by dn.exact="cn=syncuser,dc=example,dc=com" read by * break access to dn.base="" by * read access to dn.base="cn=subschema" by * read access to attrs=userPassword,pwdHistory by self write by anonymous auth by * none access to dn.subtree="dc=example,dc=com" by self write by users read by anonymous read database config access to * by dn.exact="gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth" manage by * none access to * by * none database hdb suffix "cn=accesslog" directory /var/lib/ldap/accesslog rootdn "cn=accesslog" index default eq index entryCSN,objectClass,reqEnd,reqResult,reqStart overlay syncprov syncprov-nopresent TRUE syncprov-reloadhint TRUE limits dn.exact="cn=syncuser,dc=example,dc=com" time.soft=unlimited time.hard=unlimited size.soft=unlimited size.hard=unlimited database hdb suffix "dc=example,dc=com" directory /var/lib/ldap rootdn "cn=admin,dc=example,dc=com" index objectClass eq,pres index ou,cn,mail,surname,givenname eq,pres,sub index uidNumber,gidNumber,loginShell eq,pres index uid,memberUid eq,pres,sub index nisMapName,nisMapEntry eq,pres,sub index entryCSN eq index entryUUID eq limits dn.exact="cn=syncuser,dc=example,dc=com" time.soft=unlimited time.hard=unlimited size.soft=unlimited size.hard=unlimited overlay syncprov syncprov-checkpoint 1000 60 serverID 2 syncrepl rid=001 provider=ldap://server1/ type=refreshAndPersist retry="10 +" searchbase="dc=example,dc=com" bindmethod=simple binddn="cn=syncuser,dc=example,dc=com" credentials=secret syncdata=accesslog mirrormode on overlay accesslog logdb cn=accesslog logops writes logsuccess TRUE logpurge 07+00:00 01+00:00 ------------------------------------------- Are there any obvious errors that I am making? Thanks, Kevin On Thu, Oct 30, 2014 at 4:24 PM, Quanah Gibson-Mount <quanah(a)zimbra.com> wrote: > --On Thursday, October 30, 2014 4:55 PM -0400 kevin sullivan < > kevin4sullivan(a)gmail.com> wrote: > > I am also looking for any resources that explain exactly how replication >> works and how conflicts are resolved in a multi-master configuration. For >> example, I know there are two phases of replication: the present phase >> and the delete phase. Could server1 be replicating the data for >> 'deletedUser' to server2 in the present phase before server2 has >> communicated to server1 that 'deletedUser' no longer exists in the delete >> phase? I don't believe this problem happens when I just delete >> 'deletedUser' (instead of deleting AND modifying). I know I am butchering >> the technical details a little bit, which is why I am curious how this >> situation is expected to be resolved. >> > > Personaly I use delta-syncrepl MMR to avoid the general issues with > syncrepl. I've found it to be much more reliable. > > > --Quanah > > > > -- > > Quanah Gibson-Mount > Server Architect > Zimbra, Inc. > -------------------- > Zimbra :: the leader in open source messaging and collaboration >

1 0

LDAP Crafted Search Request Access Allowed
by Net Warrior 31 Oct '14

31 Oct '14

Hi there guys. Recently we had an internal audit and it seems that our opelndap server is not configured properly, it seems that null bind are allowed and Crafted request as well are permited and it would be nice if anyone of you could lend me a hand to fix this: There are the access list: olcAccess: {0}to attrs=userPassword by dn="cn=Manager,dc=domain,dc=com" wr ite by anonymous auth by self write by * none olcAccess: {1}to attrs=cn,sn,memberUid,uidNumber,pwdHistory,pwdPolicySubentry, gidNumber,homeDirectory,givenName,description,loginShell by self write by ano nymous read by * none olcAccess: {2}to dn.base="" by * read olcAccess: {3}to * by dn="cn=Manager,dc=domain,dc=com" write by * read And from a client I got the following: ldapsearch -x -s base -b '' -H ldap://ldapserver "(objectClass=*)" "*" + # extended LDIF # # LDAPv3 # base <> with scope baseObject # filter: (objectClass=*) # requesting: * + # dn: objectClass: top objectClass: OpenLDAProotDSE structuralObjectClass: OpenLDAProotDSE configContext: cn=config monitorContext: cn=Monitor namingContexts: dc=domain,dc=com supportedControl: 2.16.840.1.113730.3.4.18 supportedControl: 2.16.840.1.113730.3.4.2 supportedControl: 1.3.6.1.4.1.4203.1.10.1 supportedControl: 1.2.840.113556.1.4.319 supportedControl: 1.2.826.0.1.3344810.2.3 supportedControl: 1.3.6.1.1.13.2 supportedControl: 1.3.6.1.1.13.1 supportedControl: 1.3.6.1.1.12 supportedExtension: 1.3.6.1.4.1.1466.20037 supportedExtension: 1.3.6.1.4.1.4203.1.11.1 supportedExtension: 1.3.6.1.4.1.4203.1.11.3 supportedExtension: 1.3.6.1.1.8 supportedFeatures: 1.3.6.1.1.14 supportedFeatures: 1.3.6.1.4.1.4203.1.5.1 supportedFeatures: 1.3.6.1.4.1.4203.1.5.2 supportedFeatures: 1.3.6.1.4.1.4203.1.5.3 supportedFeatures: 1.3.6.1.4.1.4203.1.5.4 supportedFeatures: 1.3.6.1.4.1.4203.1.5.5 supportedLDAPVersion: 3 entryDN: subschemaSubentry: cn=Subschema It seems it's no good at all, any help appreciated Best regards

4 18

#Overview in openldap2/guide/admin/guide.html
by Ulrich Windl 31 Oct '14

31 Oct '14

Hi! Can someone check whether there are still multiple "#Overview" HREFs in guide.html? In 2.4.26 the hyperlink (just one example) for "11.9.1. Overview" (Relay) jumps to "11.1.1. Overview" (BDB)... # grep '#Overview' /usr/share/doc/packages/openldap2/guide/admin/guide.html <A HREF="#Overview">11.1.1. Overview</A> <A HREF="#Overview">11.2.1. Overview</A> <A HREF="#Overview">11.3.1. Overview</A> <A HREF="#Overview">11.4.1. Overview</A> <A HREF="#Overview">11.5.1. Overview</A> <A HREF="#Overview">11.6.1. Overview</A> <A HREF="#Overview">11.7.1. Overview</A> <A HREF="#Overview">11.8.1. Overview</A> <A HREF="#Overview">11.9.1. Overview</A> <A HREF="#Overview">11.10.1. Overview</A> <A HREF="#Overview">12.1.1. Overview</A> <A HREF="#Overview">12.2.1. Overview</A> <A HREF="#Overview">12.3.1. Overview</A> <A HREF="#Overview">12.4.1. Overview</A> <A HREF="#Overview">12.5.1. Overview</A> <A HREF="#Overview">12.6.1. Overview</A> <A HREF="#Overview">12.7.1. Overview</A> <A HREF="#Overview">12.8.1. Overview</A> <A HREF="#Overview">12.9.1. Overview</A> <A HREF="#Overview">12.10.1. Overview</A> <A HREF="#Overview">12.11.1. Overview</A> <A HREF="#Overview">12.12.1. Overview</A> <A HREF="#Overview">12.13.1. Overview</A> <A HREF="#Overview">12.14.1. Overview</A> <A HREF="#Overview">12.15.1. Overview</A> <A HREF="#Overview">12.16.1. Overview</A> <A HREF="#Overview">12.17.1. Overview</A> <A HREF="#Overview">12.18.1. Overview</A> Regards, Ulrich

2 1

Re: Need information on alock file in data directory of OpenLDAP 2.4.39
by pramod kulkarni 30 Oct '14

30 Oct '14

Thanks for the information. I tried few more tests,slapcat doesn't always hang and it hangs every second restart of slapd. I don't know really whether it is sharing of locks issue or something else. Currently lock detect is set to DB_LOCK_DEFAULT even if I increase the lock detect to Maximum locks no change in the state of slapcat. If I upgrade BDB will this solve the issue of slapcat hanging ? , Have your faced such situation of hanging in slapcat I get this below message when I run the slapcat and it never goes forward 5452a973 The first database does not allow slapcat; using the first available on e (2) Regards, Pramod On Thu, Oct 30, 2014 at 7:41 PM, Quanah Gibson-Mount <quanah(a)zimbra.com> wrote: > --On Thursday, October 30, 2014 6:04 PM +0100 pramod kulkarni < > pammu.kulkarni(a)gmail.com> wrote: > > >> >> Thanks for the response. >> What if I disable the alock file generation in the back-bdb >> initialization? will it affect other functionalities like replication etc. >> > > You should understand what the alock file is for before you think about > deleting it. It tracks the state of your database. If your slapcat is > hanging, it sounds like you have DB corruption at the BDB level. > > --Quanah > > > -- > > Quanah Gibson-Mount > Server Architect > Zimbra, Inc. > -------------------- > Zimbra :: the leader in open source messaging and collaboration >

2 1

Re: Replication restores deleted user
by kevin sullivan 30 Oct '14

30 Oct '14

Sorry about the confusion, I specified MirrorMode since that is a specific type of Multi-Master Replication and I wanted to be exact. To answer Michael's questions: 1. Check whether cn=syncuser,dc=example,dc=com can really read *everything* and no ACLs are preventing access to relevant CSN attributes etc. My replication user can read everything. In fact, these deletes work when both instances of slapd are online when the delete happens. My problem only occurs when one of the instances of slapd is offline when the delete occurs. 2. Try to reproduce your issue with 2.4.40. I will try this out and let you know what happens. I am also looking for any resources that explain exactly how replication works and how conflicts are resolved in a multi-master configuration. For example, I know there are two phases of replication: the present phase and the delete phase. Could server1 be replicating the data for 'deletedUser' to server2 in the present phase before server2 has communicated to server1 that 'deletedUser' no longer exists in the delete phase? I don't believe this problem happens when I just delete 'deletedUser' (instead of deleting AND modifying). I know I am butchering the technical details a little bit, which is why I am curious how this situation is expected to be resolved. Thanks, Kevin On Thu, Oct 30, 2014 at 2:42 PM, Quanah Gibson-Mount <quanah(a)zimbra.com> wrote: > --On Thursday, October 30, 2014 7:33 PM +0100 Michael Ströder < > michael(a)stroeder.com> wrote: > > kevin sullivan wrote: >> >>> I have two servers (server1 and server2) running openldap 2.4.39-8 and >>> they are configured to replicate via MirrorMode replication. >>> >> >> Really mirror mode? Not MMR? You're config looks like MMR. >> > > Mirror mode is a way of configuring MMR. It is *not* something "separate" > from MMR. Mirror mode will *always* be MMR. > > --Quanah > > -- > > Quanah Gibson-Mount > Server Architect > Zimbra, Inc. > -------------------- > Zimbra :: the leader in open source messaging and collaboration >

2 1

Replication restores deleted user
by kevin sullivan 30 Oct '14

30 Oct '14

I have two servers (server1 and server2) running openldap 2.4.39-8 and they are configured to replicate via MirrorMode replication. Here is what I am seeing in order: 1) On server1, I create a two users: deleteUser and modifyUser. 2) I can see that these users are then properly synced to server2. 3) On server1, slapd is stopped. 4) On server2, I now delete 'deleteUser' and I modify 'modifyUser'. 5) On server1, slapd is started. 6) The two slapd instances replicate. Outcome: Both servers now have the deleted user 'deleteUser' in their databases like the user was never deleted. However, the user 'modifyUser' was properly updated in both places. Expected outcome: I would expect that 'deleteUser' wouldn't exist in either database. I would expect that 'modifyUser' would be properly modified on both servers. Why would this happen? Do I need to configure something specifically so deletes are handled properly? Is this just a quirk with how replication works? Below are the relevant parts of each server's slapd.conf. server1's configuration: serverID 1 ... overlay syncprov syncrepl rid=001 provider=ldap://server2/ type=refreshAndPersist retry="10 +" searchbase="dc=example,dc=com" bindmethod=simple binddn="cn=syncuser,dc=example,dc=com credentials=secret mirrormode on ... server2's configuration: serverID 2 ... overlay syncprov syncrepl rid=001 provider=ldap://server1/ type=refreshAndPersist retry="10 +" searchbase="dc=example,dc=com" bindmethod=simple binddn="cn=syncuser,dc=example,dc=com credentials=secret mirrormode on ... Thanks, Kevin

3 2

Need information on alock file in data directory of OpenLDAP 2.4.39
by pramod kulkarni 30 Oct '14

30 Oct '14

Hi, I have taken the OpenLDAP 2.4.39 and able to work properly but when I do slapcat then it hangs and always complains on alock. But if I delete alock it works fine. I have 4.6.21 version of bdb I don't know much about alock file in bdb . Can you please suggest is it ok to delete alock file for running slapd and slapcat ? Thanks, Pramod

3 3

2.4.41?
by Michael Ströder 30 Oct '14

30 Oct '14

HI! It seems to me some import fixes were committed to git-master after cutting 2.4.40. Will there be a 2.4.41 release? Ciao, Michael.

2 1

Overlays in Consumer and provider configuration
by Uli Tehrani 30 Oct '14

30 Oct '14

Hello all, I have some questions regarding overlay configuration for consumer and provider server. It is necessary to load the same overlay modules in a consumer configuration? Is it also necessary to configure the overlay like in the provider configuration ? There are overlays that are needed only for the provider, e.g. syncprov. Others are only needed for the consumer like the chaining overlay. But for other overlays it is not that clear: Dynlist needs to be configured for both servers. But what's about dds, ppolicy and refinit? Here maybe a provider configuration is enough. Is there a simple best practise ? Thanks in advance. Regards Uli -- =================================== Ulrich Tehrani Am Ulrichshof 19 79189 Bad Krozingen +497633806246 u_tehrani(a)yahoo.de ===================================

2 2

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

openldap-technical