openldap-technical August 2024

openldap-technical@openldap.org

19 participants
17 discussions

certutil - nsldap + nss trustargs
by venugopal chinnakotla 04 Sep '24

04 Sep '24

Hi Team, We are migrating our source code to use openldap client libraries instead of nsldap libraries. We have the below query for the openldap team. Please share your thoughts on this feature availability in the openldap library. nsldap has the below feature while validating TLS Connections. please refer to the nss tool, and refer to the section "Options and Arguments" https://www-archive.mozilla.org/projects/security/pki/nss/tools/certutil -t trustargs Specify the trust attributes to modify in an existing certificate or to apply to a certificate when creating it or adding it to a database. There are three available trust categories for each certificate, expressed in this order: "SSL, email, object signing". In each category position use zero or more of the following attribute codes: p Valid peer P Trusted peer (implies p) c Valid CA T Trusted CA to issue client certificates (implies c) C Trusted CA to issue server certificates (SSL only) (implies c) u Certificate can be used for authentication or signing w Send warning (use with other attributes to include a warning when the certificate is used in that context) The attribute codes for the categories are separated by commas, and the entire set of attributes enclosed by quotation marks. For example: -t "TCu,Cu,Tuw" Use the -L option to see a list of the current certificates and trust attributes in a certificate database. our query: Does openldap provide trustargs support to verify TLS connection? Please share details on this. Thanks In advance. -- Thanks, *c.venugopal*

4 3

Syncrepl getting stuck on partial sync
by jose Pereira 04 Sep '24

04 Sep '24

Hello All! I have multiple setups with a Provider(2.6.4) and consumer(2.6.2) with RefreshAndPersist. From time to time I see data missing in the consumer e.g only groups are present and all users are missing, even though users were never removed from the Provider, and were present in the Consumer beforehand. On the Provider all data is present and correct. I can still see the incoming connection, from the Consumers Sync, in the Provider without errors. No errors on the Comsumer logs from syncrepl (with -256 logs). Attempted to look in debug (-1 logs), but nothing caught my eye. When this happened I attempted restart services on both the Provider and the Consumer, disabling and re-enabling the sync on the Consumer. Only clearing the Consumers database seemed to fix the issue. There maybe be sporatic failures in the network connectivity between both systems. This is the config on the Consumer olcSyncrepl: {0}rid=000 provider=ldaps://Provider:636 type=refreshAndPersist retry="30 5 300 +" searchbase="ou=subtree,dc=test,dc=net" scope=sub schemachecking=on binddn="cn=Manager,dc=test,dc=net" credentials=credentials keepalive=240:10:30 Any advice on how to troubleshoot this or fix it?

4 18

long restart time of openldap
by Norbert 29 Aug '24

29 Aug '24

Hi, we have ldap servers with many ldap entries (3 million and more) these are replicating with other instances (Master-Master + Read slaves). When we need to restart and ldap due to maintenance or other problems those are taking a lot of time before they serving a requests again. It looks like that in most cases it does a full refresh of the whole tree (printed to the log) but there are is also a timespan where nothing is printed to the log but the slapd process is really busy and consuming more than one core. We are currently using 2.5.16.1 of openldap but have experienced this with older version too. Any hint about how to improve this? Thanks, Norbert

2 5

LMDB write commit time hockey-sticks [corrected]
by Alec Matusis 22 Aug '24

22 Aug '24

Problem: Our application first writes data into LMDB. After the write cursor is closed, the LMDB is used with a read-only cursor. Our problem is that writing commit time hockey-sticks. Here is the graph on the machine with 128GB of RAM: https://i.imgur.com/py83dRV.jpeg (happens at 3200) We used a virtual machine with 16 GB, to wait less for this hockey-stick: https://i.imgur.com/CYohM3O.jpeg (red curve is the commit time, happens at 3080; the lmdb env stats are on the bottom graph. Size on the top graph is total program size from /proc/[pid]/statm) . We tried various combinations of environment flags: MDB_WRITEMAP, MDB_MAPASYNC, MDB_NOSYNC and MDB_NOMETASYNC. We also tried increasing mapsize. Nothing seems to matter (e.g. https://i.imgur.com/MkZJkxp.jpeg with 64G mapsize and MDB_NOSYNC), the best combination seems to be without any flags at all. Smaller mapsizes are worse, but increasing mapsize above 4GB in a 16GB RAM machine doesn't seem to gain anything. Can you recommend a way to prevent or delay this hockey-stick when we are in the write regime and the reads are very minimal? We just want to write the data to disk fast, we don’t even need mmap() for read performance, we will need it in the read-only mode later, after the write cursor finishes and a read-only cursor is used. Details: The environment contains two databases, one with no flags (keys are 32 bytes, values are 8 byte integers), second one with MDB_INTEGERKEY (keys are 8 byte integers, values are 32 bytes, uses MDB_APPEND for writing). All commits are similar sized, 6000-8000 pages (page size is 4096). Each commit therefore is ~400k keys total for boths dbs, (7000*4096/((32+8)*2). The VM has no disk swap, from perf output here https://i.imgur.com/zuTGawz.jpeg we can see that the machine is under page swap load. On a VM with 16GB RAM, hockey-stick happens when the LMDB size is only 1.5GB, 12% of the available RAM. Detailed stats: Below is the excerpt from the application log on each commit, the same data is plotted in the linked graphs above. Memory stats are from. /proc/[pid]/statm , lmdb stats are from the environment.“Difference” the time the commit takes, it degrades from 45 seconds to 3+ minutes. Final LMDB size in this log is pgs_in_use 448993*4096 = 1.84GB. Start of log before write degradation: 2024-08-09 02:26:19 [INFO][src/importer.rs:198] File 3054: 132932384 bytes. Memory stats: Size: 27540910 pages, Resident: 286072 pages, Shared: 99484 pages, Text: 981 pages, Data: 191849 pages, lmdb: dict_env: branch_pgs 0 leaf_pgs 1 of_pgs 0 last_pgno 108959 last_txnid 18 max_readers 126 num_readers 1 free_pgs 56171 pgs_in_use 52788 Difference: 0 min 44 sec 2024-08-09 02:27:03 [INFO][src/importer.rs:198] File 3055: 133951354 bytes. Memory stats: Size: 27541730 pages, Resident: 298463 pages, Shared: 111020 pages, Text: 981 pages, Data: 192669 pages, lmdb: dict_env: branch_pgs 0 leaf_pgs 1 of_pg s 0 last_pgno 121928 last_txnid 19 max_readers 126 num_readers 1 free_pgs 62881 pgs_in_use 59047 Difference: 0 min 48 sec 2024-08-09 02:27:51 [INFO][src/importer.rs:198] File 3056: 133218473 bytes. Memory stats: Size: 27543735 pages, Resident: 313441 pages, Shared: 123980 pages, Text: 981 pages, Data: 194674 pages, lmdb: dict_env: branch_pgs 0 leaf_pgs 1 of_pg s 0 last_pgno 133243 last_txnid 20 max_readers 126 num_readers 1 free_pgs 69672 pgs_in_use 63571 Difference: 0 min 40 sec …… End of log after write degradation: 2024-08-09 04:39:33 [INFO][src/importer.rs:198] File 3127: 132694310 bytes. Memory stats: Size: 27654397 pages, Resident: 1028693 pages, Shared: 727992 pages, Text: 981 pages, Data: 305336 pages, lmdb: dict_env: branch_pgs 0 leaf_pgs 1 of_pgs 0 last_pgno 745440 last_txnid 91 max_readers 126 num_readers 1 free_pgs 308699 pgs_in_use 436741 Difference: 2 min 36 sec 2024-08-09 04:42:09 [INFO][src/importer.rs:198] File 3128: 133418687 bytes. Memory stats: Size: 27654397 pages, Resident: 1033516 pages, Shared: 732815 pages, Text: 981 pages, Data: 305336 pages, lmdb: dict_env: branch_pgs 0 leaf_pgs 1 of_pgs 0 last_pgno 745440 last_txnid 92 max_readers 126 num_readers 1 free_pgs 304043 pgs_in_use 441397 Difference: 3 min 18 sec 2024-08-09 04:45:27 [INFO][src/importer.rs:198] File 3129: 133118782 bytes. Memory stats: Size: 27654397 pages, Resident: 1043876 pages, Shared: 743175 pages, Text: 981 pages, Data: 305336 pages, lmdb: dict_env: branch_pgs 0 leaf_pgs 1 of_pgs 0 last_pgno 762922 last_txnid 93 max_readers 126 num_readers 1 free_pgs 313929 pgs_in_use 448993

1 1

start slapd with mdb on readonly filesystem
by Zach Hanson-Hart 19 Aug '24

19 Aug '24

Hello list, When starting slapd when an lmdb backend in readonly mode is on a readonly filesystem, slapd fails to start with: 66c347f4.334226ce 0x7fc9c14ffb08 @(#) $OpenLDAP: slapd 2.6.6 (Oct 22 2023 00:57:15) $ 2024-08-19T09:26:12.865756736-04:00 openldap 2024-08-19T09:26:12.875884302-04:00 66c347f4.342f703c 0x7fc9c14ffb08 olcDbDirectory: value #0: invalid path: Read-only file system 2024-08-19T09:26:12.875912369-04:00 66c347f4.3430d2ab 0x7fc9c14ffb08 config error processing olcDatabase={1}mdb,cn=config: olcDbDirectory: value #0: invalid path: Read-only file system 2024-08-19T09:26:12.875942620-04:00 66c347f4.3434526e 0x7fc9c14ffb08 slapd stopped. 2024-08-19T09:26:12.875950918-04:00 66c347f4.343518c7 0x7fc9c14ffb08 connections_destroy: nothing to destroy. Is there something I can set to make it start even though the filesystem is readonly? For reference, the slapd configuration is below: dn: cn=config objectClass: olcGlobal cn: config olcPidFile: /etc/openldap/scratch/run/slapd.pid olcArgsFile: /etc/openldap/scratch/run/slapd.args olcLogLevel: stats olcAuthzPolicy: none olcSecurity: ssf=0 tls=0 simple_bind=0 transport=0 olcReadOnly: FALSE dn: olcDatabase={-1}frontend,cn=config objectClass: olcDatabaseConfig objectClass: olcFrontendConfig olcDatabase: {-1}frontend olcAccess: to dn.base="" by * read olcAccess: to dn.base="cn=Subschema" by * read dn: cn=module,cn=config objectClass: olcModuleList cn: module olcModulepath: /usr/lib/openldap olcModuleload: back_mdb.so olcModuleload: back_ldap.so olcModuleLoad: ppolicy dn: cn=schema,cn=config objectClass: olcSchemaConfig cn: schema include: file:///etc/openldap/schema/core.ldif include: file:///etc/openldap/schema/cosine.ldif include: file:///etc/openldap/schema/inetorgperson.ldif dn: olcDatabase={0}config,cn=config objectClass: olcDatabaseConfig olcDatabase: {0}config olcAccess: to dn.base="cn=config" by dn.exact=gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth manage by * read olcAccess: to dn.subtree="cn=config" by dn.exact="gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth" manage by * read olcAccess: to * by * read dn: olcDatabase={1}mdb,cn=config objectClass: olcDatabaseConfig objectClass: olcMdbConfig olcDatabase: {1}mdb olcSuffix: ou=data,dc=example,dc=com olcRootDN: cn=root,ou=data,dc=example,dc=com olcRootPW: <REDACTED> olcReadOnly: TRUE olcDbDirectory: /var/lib/openldap/data olcAccess: to dn.base="ou=data,dc=example,dc=com" by * read # Allow full read access for now for testing. olcAccess: to dn.subtree="ou=data,dc=example,dc=com" by * read olcLimits: * size=unlimited time=unlimited olcDbMaxsize: 171798691840 olcDbIndex: objectClass eq,pres olcDbIndex: entryUUID eq olcDbIndex: entryCSN eq olcDbIndex: cn eq Thanks in advance. Zach

1 0

Need support to configure ldap for duo MFA for auth proxy.
by Ajay Kumar 16 Aug '24

16 Aug '24

Hi Team, we get stuck at configuring Openldap for enabling Multifactor authentication for ldap users. As per duo support team, for doing the same we need to configure schema includes the memberOf overlay for groups and that the following requirements to satisfied: Synced groups must have the groupOfNames object class. Synced groups must list their members by DN (directoryName) via the member attribute. Synced groups must have a cn attribute, used as the Duo group name after import. Synced groups must also have the attributes entrydn (used as the distinguished name) and entryuuid (the group unique identifier). Synced users must list group memberships by DN using the memberOf attribute. Synced users must have the organizationalPerson object class. We are trying to enable Multifactor authentication using duo auth proxy & duo admin panel configuration for ldap users. Ldap server is getting synced successfully with Duo admin portal but groups and users details not getting fetched at duo admin portal. Duo support team mentioned to change ldap configuration as mention in below mentioned article. Can you pls share some reference document to make required changes. https://duo.my.site.com/s/article/4529?language=en_US Regards, Ajay Kumar Engineering Cloud Ops | Bharti Airtel Ltd. Mob.: +91 8510020994 [photo_6075787602121832799_m] *********************************************************************************************************************************************************************** This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited . The information contained in this mail is propriety and strictly confidential. *********************************************************************************************************************************************************************** “CONFIDENTIALITY NOTICE This e-mail message and any attachments are only for the use of the intended recipient and may contain information that is privileged, confidential or exempt from disclosure under applicable law. If you are not the intended recipient, any disclosure, distribution or other use of this e-mail message or attachments is prohibited. If you have received this e-mail message in error, please delete and notify the sender immediately. Thank you.”

2 1

Push-based synchronization & dynamic attributes
by Benjamin Renard 13 Aug '24

13 Aug '24

Hello, Since updating our LDAP infrastructure to OpenLDAP 2.5 (2.5.13 from the Debian repository), we have encountered desynchronization issues with certain objects. The synchronization in place is in push mode as documented in the official documentation (https://www.openldap.org/doc/admin25/replication.html#Syncrepl%20Proxy). The problem seems to stem from the fact that we are using dynamic attributes via the dynlist overlay, and the master server attempts to push these dynamic attributes to the slave servers when modifications are made to these objects, which the slaves refuse. Log extract from one of our slave servers: Aug 13 10:12:28 volubilis slapd[2005636]: conn=20983 op=54 MOD dn="uid=ele-ladon-test,ou=contracts,o=easter-eggs" Aug 13 10:12:28 volubilis slapd[2005636]: conn=20983 op=54 MOD attr=eeTechnicalContact eeStartDate eeStorageDisks entryCSN modifiersName modifyTimestamp eeContractId eeContractIncludedService Aug 13 10:12:28 volubilis slapd[2005636]: conn=20983 op=54 RESULT tag=103 err=16 qtime=0.000022 etime=0.000536 text=modify/delete: eeContractId: no such attribute Note: The attribute eeContractId is a dynamic attribute of the object in question. Previously, with OpenLDAP 2.4, we did not have this issue, and the attributes were dynamic (and therefore managed by dynlist) on both our master server and slave servers. To immediately resolve this problem, we have removed the dynlist configuration on our slave servers. These attributes are therefore "static" on our slave servers, and desynchronizations are potentially possible. How can we correctly manage this situation? Please note that some of the affected attributes are both static on certain objects and dynamic on others, so we cannot simply exclude them completely from synchronization. Thank you in advance. -- Benjamin Renard - Easter-eggs 44-46 rue de l'Ouest - 75014 Paris - France - Métro Gaité Phone: +33 (0) 1 43 35 00 37 - mailto:brenard@easter-eggs.com

1 0

Help using the memberOf overlay
by awoestmann＠intevation.de 12 Aug '24

12 Aug '24

Hi everyone, i am trying to set up a OpenLDAP (2.6.7 in a Ubuntu noble container) with the memberOf Overlay activated. Adding the users and groups seems to work fine but i was not yet able to activate the overlay and query the memberships of a given user. I am using the following ldif to load the overlay: dn: cn=module,cn=config cn: module objectClass: olcModuleList olcModuleLoad: memberof olcModulePath: /usr/lib/ldap dn: olcOverlay={0}memberof,olcDatabase={1}mdb,cn=config objectClass: olcConfig objectClass: olcMemberOf objectClass: olcOverlayConfig objectClass: top olcOverlay: memberof olcMemberOfDangling: ignore olcMemberOfRefInt: TRUE olcMemberOfGroupOC: groupOfNames olcMemberOfMemberAD: member olcMemberOfMemberOfAD: memberOf The config is added using "slapadd -n0 -l /path/to/ldif", after that i add the users using slapadd. To check if the overlay works i am trying to query the memberOf attribute using ldapvi: "ldapvi --discover "(uid=exampleUid)" memberOf" Does anyone has a hint for me what i am doing wrong? Best regards, Alexander

5 5

LMDB write commit time hockey-sticks
by Alec Matusis 12 Aug '24

12 Aug '24

Problem: Our application first writes data into LMDB. After the data is written, the LMDB is used in the read-only mode. Our problem is that writing commit time hockey-sticks. Here is the graph on the machine with 128GB of RAM: https://i.imgur.com/py83dRV.jpeg (happens at 3200) We used a virtual machine with 16 GB, to wait less for this hockey-stick: https://i.imgur.com/CYohM3O.jpeg (red curve is the commit time, happens at 3080; the lmdb env stats are on the bottom graph. Size on the top graph is total program size from /proc/[pid]/statm) . We tried various combinations of environment flags: MDB_WRITEMAP, MDB_MAPASYNC, MDB_NOSYNC and MDB_NOMETASYNC. We also tried increasing mapsize. Nothing seems to matter (e.g. https://i.imgur.com/MkZJkxp.jpeg with 64G mapsize and MDB_NOSYNC), the best combination seems to be without any flags at all. Smaller mapsizes are worse, but increasing mapsize above 4GB in a 16GB RAM machine doesn't seem to gain anything. Can you recommend a way to prevent or delay this hockey-stick when we are in the write-only, append-only regime? We just want to write the data to disk fast, we don’t even need mmap() here, we will need it in the read-only mode later. Details: The environment contains two databases, one with no flags (keys are 32 bytes, values are 8 byte integers), second one with MDB_INTEGERKEY (keys are 8 byte integers, values are 32 bytes, uses MDB_APPEND for writing). All commits are similar sized, 6000-8000 pages (page size is 4096). Each commit therefore is ~400k keys total for boths dbs, (7000*4096/((32+8)*2). The VM has no disk swap, from perf output here https://i.imgur.com/zuTGawz.jpeg we can see that the machine is under page swap load. On a VM with 16GB RAM, hockey-stick happens when the LMDB size is only 1.5GB, 12% of the available RAM. Detailed stats: Below is the excerpt from the application log on each commit, the same data is plotted in the linked graphs above. Memory stats are from. /proc/[pid]/statm , lmdb stats are from the environment.“Difference” the time the commit takes, it degrades from 45 seconds to 3+ minutes. Final LMDB size in this log is pgs_in_use 448993*4096 = 1.84GB. Start of log before write degradation: 2024-08-09 02:26:19 [INFO][src/importer.rs:198] File 3054: 132932384 bytes. Memory stats: Size: 27540910 pages, Resident: 286072 pages, Shared: 99484 pages, Text: 981 pages, Data: 191849 pages, lmdb: dict_env: branch_pgs 0 leaf_pgs 1 of_pgs 0 last_pgno 108959 last_txnid 18 max_readers 126 num_readers 1 free_pgs 56171 pgs_in_use 52788 Difference: 0 min 44 sec 2024-08-09 02:27:03 [INFO][src/importer.rs:198] File 3055: 133951354 bytes. Memory stats: Size: 27541730 pages, Resident: 298463 pages, Shared: 111020 pages, Text: 981 pages, Data: 192669 pages, lmdb: dict_env: branch_pgs 0 leaf_pgs 1 of_pg s 0 last_pgno 121928 last_txnid 19 max_readers 126 num_readers 1 free_pgs 62881 pgs_in_use 59047 Difference: 0 min 48 sec 2024-08-09 02:27:51 [INFO][src/importer.rs:198] File 3056: 133218473 bytes. Memory stats: Size: 27543735 pages, Resident: 313441 pages, Shared: 123980 pages, Text: 981 pages, Data: 194674 pages, lmdb: dict_env: branch_pgs 0 leaf_pgs 1 of_pg s 0 last_pgno 133243 last_txnid 20 max_readers 126 num_readers 1 free_pgs 69672 pgs_in_use 63571 Difference: 0 min 40 sec …… End of log after write degradation: 2024-08-09 04:39:33 [INFO][src/importer.rs:198] File 3127: 132694310 bytes. Memory stats: Size: 27654397 pages, Resident: 1028693 pages, Shared: 727992 pages, Text: 981 pages, Data: 305336 pages, lmdb: dict_env: branch_pgs 0 leaf_pgs 1 of_pgs 0 last_pgno 745440 last_txnid 91 max_readers 126 num_readers 1 free_pgs 308699 pgs_in_use 436741 Difference: 2 min 36 sec 2024-08-09 04:42:09 [INFO][src/importer.rs:198] File 3128: 133418687 bytes. Memory stats: Size: 27654397 pages, Resident: 1033516 pages, Shared: 732815 pages, Text: 981 pages, Data: 305336 pages, lmdb: dict_env: branch_pgs 0 leaf_pgs 1 of_pgs 0 last_pgno 745440 last_txnid 92 max_readers 126 num_readers 1 free_pgs 304043 pgs_in_use 441397 Difference: 3 min 18 sec 2024-08-09 04:45:27 [INFO][src/importer.rs:198] File 3129: 133118782 bytes. Memory stats: Size: 27654397 pages, Resident: 1043876 pages, Shared: 743175 pages, Text: 981 pages, Data: 305336 pages, lmdb: dict_env: branch_pgs 0 leaf_pgs 1 of_pgs 0 last_pgno 762922 last_txnid 93 max_readers 126 num_readers 1 free_pgs 313929 pgs_in_use 448993

1 0

ppolicy: passing operational attribute changes from replica to provider
by Nick Urbanik 08 Aug '24

08 Aug '24

Dear Team, I love your work. in man(5) slapo-ppolicy we see: "In general, authentication attempts on a replica server only affect the copy of the operational attributes on that replica and will not affect any attributes for a user's entry on the provider." I would prefer to have the replica send the changes to the provider. Is the chain overlay the right approach for this?

2 1

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

openldap-technical August 2024