openldap-technical

openldap-technical@openldap.org

14 participants
9796 discussions

LMDB, doing the impossible
by Abilio Marques 02 Apr '19

02 Apr '19

Hello everyone, About a week ago, I wrote about moving a piece of software out of writing multiple files and into LMDB. Currently, the software will run on UBIFS (which is unsafe for NOSYNC option). Is multi-threaded and has no concept of transactions. Write operations are spread between threads. It only uses a mutex to avoid multiple writers, and tend to happen in a burst. Storage is implemented by a layer that has a similar API (read, write, delete) to LMDB. My main goal is to replace the guts of this layer with LMDB (while keeping the API), to get an idea on how the program would work. Because of the lack of transaction concept in the current API, my initial approach was to accumulate transactions (after the first sync) for a couple of seconds, and then call env_sync. This solution is not power failure robust because of UBIFS characteristics. I was trying to figure out other solutions that can be put this deferred concept in place, again, with the goal of proving worthiness of a full refactoring of the code in order to properly use LMDB. Any suggestions? Best, Abilio Marques

1 0

Question about ppolicy usage
by Mikael Bak 02 Apr '19

02 Apr '19

Hi list, I realize I'm trying to use the ppolicy overlay a little differently from how it was designed to be used. The problem is that the ppolicy overlay is the closest thing I have found. My use case: 1) I want to be able to disable users. I can do this by setting: pwdAccountLockedTime: 000001010000Z That works. Great! 2) I want to be able to set a date in the future when a user account will expire / deactivate. I was hoping to be able to set "pwdAccountLockedTime" to a date in the future and after that date the user account would be locked. Unfortunately this isn't the case. ppolicy seems to lock out every account that has the "pwdAccountLockedTime" attribute set to a valid value. Reading the source code for ppolicy I find an interesting block in the function "account_locked()" at line 356: /* Still in the future? not yet in effect */ if (now < then) return 0; This leads me to believe that the author's intension may have been to allow what I want to do. Perhaps is there another attribute I need to set in order to tweek ppolicy to do wat I want. Here's how the default policy looks like: dn: cn=passwordDefault,ou=Policies,ou=local objectClass: pwdPolicy objectClass: person objectClass: top cn: passwordDefault sn: passwordDefault pwdAttribute: userPassword pwdCheckQuality: 0 pwdMinAge: 0 pwdMaxAge: 0 pwdMinLength: 0 pwdInHistory: 0 pwdMaxFailure: 0 pwdFailureCountInterval: 0 pwdLockout: TRUE pwdLockoutDuration: 0 pwdAllowUserChange: FALSE pwdExpireWarning: 0 pwdGraceAuthNLimit: 0 pwdMustChange: FALSE pwdSafeModify: FALSE All help greatly appreciated! TIA, Mikael

3 4

RE: mdb_equality_candidates: (entryUUID) not indexed, with olcDbIndex=entryUUID pres, eq
by Marc Roos 02 Apr '19

02 Apr '19

Yes, I have a default hdb slap.d tree setup from the el7 rpm install. I also noticed this olcHdbConfig. I have decided to stick with the hdb for now. -----Original Message----- From: Quanah Gibson-Mount Sent: 02 April 2019 15:37 To: Marc Roos; openldap-technical Subject: RE: mdb_equality_candidates: (entryUUID) not indexed, with olcDbIndex=entryUUID pres, eq --On Tuesday, April 02, 2019 2:48 PM +0200 Marc Roos> wrote: > > Hi Quanah, > > Missed your response, I am not sure if I understand your info request, > so I am sending you these. Thanks for looking. > > dn: olcDatabase={2}mdb > objectClass: olcDatabaseConfig > objectClass: olcHdbConfig The above is clearly an invalid configuration, it looks like you tried to convert from HDB to MDB manually? I doubt slapd is really able to function correctly at all. --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

RE: mdb_equality_candidates: (entryUUID) not indexed, with olcDbIndex=entryUUID pres, eq
by Marc Roos 02 Apr '19

02 Apr '19

Hi Quanah, Missed your response, I am not sure if I understand your info request, so I am sending you these. Thanks for looking. [@cn=config]# ls -arlt total 20 -rw------- 1 ldap ldap 378 Mar 30 13:21 cn=schema.ldif -rw------- 1 ldap ldap 562 Mar 30 13:21 olcDatabase={1}monitor.ldif -rw------- 1 ldap ldap 678 Mar 30 14:58 olcDatabase={0}config.ldif drwxr-x--- 2 ldap ldap 284 Mar 30 15:03 cn=schema -rw------- 1 ldap ldap 905 Mar 30 15:04 olcDatabase={-1}frontend.ldif -rw------- 1 ldap ldap 2333 Mar 31 23:42 olcDatabase={2}mdb.ldif [@ cn=config]# cat 'olcDatabase={1}monitor.ldif' # AUTO-GENERATED FILE - DO NOT EDIT!! Use ldapmodify. # CRC32 ff86e0ea dn: olcDatabase={1}monitor objectClass: olcDatabaseConfig olcDatabase: {1}monitor olcAccess: {0}to * by dn.base="gidNumber=0+uidNumber=0,cn=peercred,cn=extern al,cn=auth" read by dn.base="cn=Manager,dc=my-domain,dc=com" read by * none structuralObjectClass: olcDatabaseConfig entryUUID: 06b54f3e-e732-1038-8480-f782455a1e70 creatorsName: cn=config createTimestamp: 20190330122101Z entryCSN: 20190330122101.772335Z#000000#000#000000 modifiersName: cn=config modifyTimestamp: 20190330122101Z [@ cn=config]# head 'olcDatabase={2}mdb.ldif' # AUTO-GENERATED FILE - DO NOT EDIT!! Use ldapmodify. # CRC32 29266174 dn: olcDatabase={2}mdb objectClass: olcDatabaseConfig objectClass: olcHdbConfig olcDatabase: {2}mdb olcDbDirectory: /var/lib/ldap olcDbIndex: objectClass eq,pres olcDbIndex: sendmailMTACluster pres,eq olcDbIndex: sendmailMTAHost pres,eq -----Original Message----- From: Quanah Gibson-Mount Sent: 01 April 2019 16:30 To: Marc Roos; openldap-technical Subject: Re: mdb_equality_candidates: (entryUUID) not indexed, with olcDbIndex=entryUUID pres, eq --On Monday, April 01, 2019 12:53 AM +0200 Marc wrote: > > First time I am installing the slapd with a mdb backend. Could this be > because the syntax has changed? > > I am getting these messages: > > mdb_equality_candidates: (entryUUID) not indexed > > While I have in {2}mdb > > olcDbIndex entryUUID pres,eq What database is in index {1}? --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

2 1

mdb_equality_candidates: (entryUUID) not indexed
by Marc Roos 02 Apr '19

02 Apr '19

I am getting mdb_equality_candidates: (entryUUID) not indexed While I have [@ slapd.d]# grep entryUU cn\=config/olcDatabase\=\{2\}mdb.ldif olcDbIndex: entryUUID pres,eq entryUUID: 06b55588-e732-1038-8481-f782455a1e70 openldap-servers-2.4.44-21.el7_6.x86_64

1 0

mdb_equality_candidates: (entryUUID) not indexed, with olcDbIndex=entryUUID pres, eq
by Marc Roos 01 Apr '19

01 Apr '19

First time I am installing the slapd with a mdb backend. Could this be because the syntax has changed? I am getting these messages: mdb_equality_candidates: (entryUUID) not indexed While I have in {2}mdb olcDbIndex entryUUID pres,eq CentOS Linux release 7.6.1810 (Core) openldap-clients-2.4.44-21.el7_6.x86_64 openldap-2.4.44-21.el7_6.x86_64 openldap-servers-2.4.44-21.el7_6.x86_64

2 1

back_mdb: does rtxnsize affect slapcat live backups?
by Maxime Besson 01 Apr '19

01 Apr '19

Hi! I am running OpenLDAP 2.4.47 with significant write activity (in the hundreds of modify/add/del ops per second), and a large volume of data compared to available RAM (about 20G) For backups, I am trying to do a live slapcat dump to maintain availability, and encountered the growth in MDB database size described in those ITS: http://www.openldap.org/its/index.cgi/Software%20Bugs?id=7904 http://www.openldap.org/its/index.cgi/Software%20Bugs?id=8226 I am however a little puzzled over the rtxnsize option. My understanding is that this option splits large read transactions (such as backups) into smaller transactions, allowing freed pages to be reused before the end of the full operation. This is critical for me because backups take more than 10 minutes and a significant portion of my MDB file (100G) gets filled up during that time. However, setting rtxnsize to a smaller value from the default does not seem to have any effect on the growth of my MDB file. I have made several tries with different rtxnsizes from 1M to 100, starting from a freshly rebuilt MDB database every time. I saw that the rtxnsize option was added to handle an issue with replication, so perhaps it was never supposed to help in my particular case. For the whole duration of a slapcat backup, I see the same transaction ID from the slapcat process in mdb_stat -r. I noticed that using ldapsearch for backups behaves much better (MDB only grows by a few thousand pages in my workload), but takes a while longer, regardless of the rtxnsize setting Is it normal that rtxnsize does not affect slapcat dumps while slapd is running, or am I missing something? How would you handle live backups of a database with lots of write activity? -- Maxime Besson Systems Engineer - Worteks maxime.besson(a)worteks.com

5 6

LMDB mdb_txn_begin parent in other thread
by Abilio Marques 29 Mar '19

29 Mar '19

Hello, I read the documentation for the mdb_txn_begin function, and the note reads "a transaction and its cursors must only be used by a single thread". Does this rule also apply to being a parent for another thread transaction? I gave it a quick shot, and it seems to work for me, but I'm not sure if I'm breaking the rules. Can anybody confirm this? Best, Abilio

2 1

Tree versus group
by Olivier - 29 Mar '19

29 Mar '19

3 3

Add module crashes slapd
by Gao 29 Mar '19

29 Mar '19

Hi all, I am relatively new to openldap and I am having an issue when I try to setup Master-Slave replication. I don't know how to solve this issue and need your help please. On the master I made a ldif file: [root@test-ldap ldif]# cat syncprov_mod.ldif dn: cn=module,cn=config objectClass: olcModuleList cn: module olcModulePath: /usr/lib64/openldap olcModuleLoad: syncprov.la Then I run command: [root@test-ldap ldif]# ldapadd -Y EXTERNAL -H ldapi:/// -f syncprov_mod.ldif SASL/EXTERNAL authentication started SASL username: gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth SASL SSF: 0 adding new entry "cn=module,cn=config" The output seems normal. But actually the slapd got messed up. So if I do slapcat I got: [root@test-ldap ldif]# slapcat -n 0 5c9d1323 config error processing cn={0}module,cn=config: slapcat: bad configuration file! And restart slapd failed. journalctl shows: Mar 28 11:40:39 test-ldap systemd[1]: Starting OpenLDAP Server Daemon... -- Subject: Unit slapd.service has begun start-up -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit slapd.service has begun starting up. Mar 28 11:40:39 test-ldap runuser[3427]: pam_unix(runuser:session): session opened for user ldap by (uid=0) Mar 28 11:40:39 test-ldap runuser[3427]: pam_unix(runuser:session): session closed for user ldap Mar 28 11:40:39 test-ldap check-config.sh[3424]: Checking configuration file failed: Mar 28 11:40:39 test-ldap check-config.sh[3424]: 5c9d1527 config error processing cn={0}module,cn=config: Mar 28 11:40:39 test-ldap check-config.sh[3424]: slaptest: bad configuration file! Mar 28 11:40:39 test-ldap slapd[3438]: @(#) $OpenLDAP: slapd 2.4.40 (Nov 6 2016 01:21:28) $ mockbuild@worker1.bsys.centos.org:/builddir/build/BUILD/openldap-2.4.40/openldap-2.4.40 Mar 28 11:40:39 test-ldap slapd[3438]: config error processing cn={0}module,cn=config: Mar 28 11:40:39 test-ldap slapd[3438]: slapd stopped. Mar 28 11:40:39 test-ldap slapd[3438]: connections_destroy: nothing to destroy. Mar 28 11:40:39 test-ldap systemd[1]: slapd.service: control process exited, code=exited status=1 Mar 28 11:40:39 test-ldap systemd[1]: Failed to start OpenLDAP Server Daemon. -- Subject: Unit slapd.service has failed -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit slapd.service has failed. -- -- The result is failed. Mar 28 11:40:39 test-ldap systemd[1]: Unit slapd.service entered failed state. Mar 28 11:40:39 test-ldap systemd[1]: slapd.service failed. Thank you for your help. Gao

3 2

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

openldap-technical