openldap-technical January 2023

openldap-technical@openldap.org

24 participants
31 discussions

RE25 testing call (2.5.14) #2
by Quanah Gibson-Mount 26 Jan '23

26 Jan '23

This is the second testing call for OpenLDAP 2.5.14. Depending on the results, this may be the final testing call. Generally, get the code for RE25: <https://git.openldap.org/openldap/openldap/-/archive/OPENLDAP_REL_ENG_2_5/o…> Extract, configure, and build. Execute the test suite (via make test) after it is built. Optionally, cd tests && make its to run through the regression suite. Thanks! OpenLDAP 2.5.14 Engineering Fixed client tools to remove 'h' and 'p' options (ITS#9917,ITS#8618) Fixed ldapsearch memory leak with paged results (ITS#9860) Fixed libldap ldif_open_urlto check for failure (ITS#9904) Fixed libldap ldap_url_parsehosts check for failure (ITS#9904) Fixed liblunicode UTF8bvnormalize buffer size (ITS#9955) Fixed lloadd race in epoch.c (ITS#9947) Fixed lloadd to keep listener base around when not active (ITS#9984) Fixed lloadd object reclamation sequencing (ITS#9983) Fixed slapd memory leak with olcAuthIDRewrite (ITS#6035) Fixed slapd transactions extended operations cleanup after write (ITS#9892) Fixed slapd deadlock with replicated cn=config (ITS#9930) Fixed slapd-mdb max number of index databases to 256 (ITS#9895) Fixed slapd-mdb to always release entries from ADD operations (ITS#9942) Fixed slapd-mdb to fully init empty DN in tool_entry_get (ITS#9940) Fixed slapd-monitor to free remembered cookies (ITS#9339) Fixed slapo-accesslog reqStart ordering matching rule (ITS#9880) Fixed slapo-deref memory leak (ITS#9924) Fixed slapo-dynlist to ignore irrelevant objectClasses (ITS#9897) Fixed slapo-dynlist to avoid unnecessary searches (ITS#9929) Fixed slapo-dynlist to mark internal searches as such (ITS#9960) Fixed slapo-pcache crash in consistency_check (ITS#9966) Fixed slapo-remoteauth memory leaks (ITS#9438) Build Environment Fixed ancient DOS related ifdef checks (ITS#9925) Fixed build process to not use gmake specific features (ITS#9894) Fixed source tree to remove symlinks (ITS#9926) Fixed slapo-otp testdir creation (ITS#9437) Fixed slapd-tester memory leak (ITS#9908) Fixed usage of non-standard C syntax (ITS#9898, ITS#9899, ITS#9901) Fixed usage of bashism (ITS#9900) Fixed test suite portability (ITS#9931) Documentation Fixed ldap_bind(3) to document ber_bvfree in ldap_sasl_bind (ITS#9976) Fixed slapo-asyncmeta(5) to clarify scheduling for target connections (ITS#9941) Fixed slapo-dynlist(5) to clarify configuration settings (ITS#9957) Fixed slapo-unique(5) to clarify when quoting should be used (ITS#9915) Minor cleanup ITS#9935 ITS#9336 ITS#9337 Regards, Quanah

2 1

"container" structural class
by Timothy Stonis 26 Jan '23

26 Jan '23

Hi All, I’ve searched the internet, but can’t find any info, so sorry in advance if this is a basic question… I’m trying to setup a “standard” DIT in an OpenLDAP 2.6.3 deployment. I checked out my existing Active Directory deployment and also an old macOS Server implementation, and they both make heavy use of the “container” structural class. For example, users are in cn=users,dc=…,dc=… which is objectClass container. I see this class is defined in the msuser schema, but in 2.6.3 it’s definition is commented out in the msuser.schema file. Can anyone help shed some light on why this is the case and maybe a pointer to what a modern best practices DIT might look like? Thanks in advance Tim

4 3

LDAP Proxy (meta type)
by bourguijl＠gmail.com 25 Jan '23

25 Jan '23

Dears, I tried to configure a proxy ldap type (meta) but without success as I get following error message when I try to start it : 63d13d3c.03f49ea0 0x7f2e4ff3b1c0 backend_startup_one: starting "o=mobistar.be" 63d13d3c.03f4ab38 0x7f2e4ff3b1c0 meta_back_db_open: no targets defined 63d13d3c.03f53e41 0x7f2e4ff3b1c0 backend_startup_one (type=meta, suffix="o=mobistar.be"): bi_db_open failed! (1) Here is my configuration : dn: olcDatabase={2}meta objectClass: olcDatabaseConfig objectClass: olcMdbConfig olcDatabase: {2}meta olcSuffix: o=mobistar.be olcDbURI: "ldap://acccorpldapproxym1.nonprod.priv.orange.be:389/ou=staff,o=mobistar.be" olcLastMod: FALSE olcReadOnly: TRUE olcRootDN: cn=directory manager,o=mobistar.be olcRootPW:: e1NTSEF9aFVwQ2t0RHdOeHJDZHJ3UHA2bkc2dEVHdFJRQzFwWk8= structuralObjectClass: olcMdbConfig To be sure it's not a Firewall issue, I started the target on the same server but no luck, it doesn't start. What can be my misconfiguration ? Thx for help, Jean-Luc.

1 0

RE26 testing call (2.6.4) #2
by Quanah Gibson-Mount 23 Jan '23

23 Jan '23

This is the second testing call for OpenLDAP 2.6.4. Depending on the results, this may be the final testing call. Generally, get the code for RE26: <https://git.openldap.org/openldap/openldap/-/archive/OPENLDAP_REL_ENG_2_6/o…> Extract, configure, and build. Execute the test suite (via make test) after it is built. Optionally, cd tests && make its to run through the regression suite. Thanks! OpenLDAP 2.6.4 Engineering Fixed client tools to remove 'h' and 'p' options (ITS#9917,ITS#8618) Fixed ldapsearch memory leak with paged results (ITS#9860) Fixed libldap ldif_open_urlto check for failure (ITS#9904) Fixed libldap ldap_url_parsehosts check for failure (ITS#9904) Fixed liblunicode UTF8bvnormalize buffer size (ITS#9955) Fixed lloadd memory leaks (ITS#9907) Fixed lloadd shutdown code to protect memory correctly (ITS#9913) Fixed lloadd race in epoch.c (ITS#9947) Fixed lloadd potential deadlock with cn=monitor (ITS#9951) Fixed lloadd to keep listener base around when not active (ITS#9984) Fixed lloadd object reclamation sequencing (ITS#9983) Fixed slapd memory leak with olcAuthIDRewrite (ITS#6035) Fixed slapd free of redundant cmdline option (ITS#9912) Fixed slapd transactions extended operations cleanup after write (ITS#9892) Fixed slapd deadlock with replicated cn=config (ITS#9930) Fixed slapd bconfig locking of cn=config entries (ITS#9045) Fixed slapd-mdb max number of index databases to 256 (ITS#9895) Fixed slapd-mdb to always release entries from ADD operations (ITS#9942) Fixed slapd-mdb to fully init empty DN in tool_entry_get (ITS#9940) Fixed slapd-monitor memory leaks with lloadd (ITS#9906) Fixed slapd-monitor to free remembered cookies (ITS#9339) Fixed slapo-accesslog reqStart ordering matching rule (ITS#9880) Fixed slapo-deref memory leak (ITS#9924) Fixed slapo-dynlist to ignore irrelevant objectClasses (ITS#9897) Fixed slapo-dynlist to avoid unnecessary searches (ITS#9929) Fixed slapo-dynlist to mark internal searches as such (ITS#9960) Fixed slapo-pcache crash in consistency_check (ITS#9966) Fixed slapo-remoteauth memory leaks (ITS#9438) Fixed slapo-rwm memory leaks (ITS#9817) Build Environment Fixed ancient DOS related ifdef checks (ITS#9925) Fixed build process to not use gmake specific features (ITS#9894) Fixed source tree to remove symlinks (ITS#9926) Fixed slapo-otp testdir creation (ITS#9437) Fixed slapd-tester memory leak (ITS#9908) Fixed usage of non-standard C syntax (ITS#9898, ITS#9899, ITS#9901) Fixed usage of bashism (ITS#9900) Fixed test suite portability (ITS#9931) Documentation Fixed ldap_bind(3) to document ber_bvfree in ldap_sasl_bind (ITS#9976) Fixed slapo-asyncmeta(5) to clarify scheduling for target connections (ITS#9941) Fixed slapo-dynlist(5) to clarify configuration settings (ITS#9957) Fixed slapo-unique(5) to clarify when quoting should be used (ITS#9915) Minor cleanup ITS#9935 ITS#9336 ITS#9337 Regards, Quanah

2 1

Re: RE26 testing call (2.6.4) #2
by Jean-Luc Bourguignon 23 Jan '23

23 Jan '23

Hello Quanah, Extract, configure, and build. make test make its were OK on Red Hat Enterprise Linux release 8.6 (Ootpa) Brgds, Jean-Luc. On Thu, Jan 19, 2023 at 9:29 PM Quanah Gibson-Mount <quanah(a)fast-mail.org> wrote: > This is the second testing call for OpenLDAP 2.6.4. Depending on the > results, this may be the final testing call. > > Generally, get the code for RE26: > > < > https://git.openldap.org/openldap/openldap/-/archive/OPENLDAP_REL_ENG_2_6/o… > > > > Extract, configure, and build. > > Execute the test suite (via make test) after it is built. Optionally, cd > tests && make its to run through the regression suite. > > Thanks! > > OpenLDAP 2.6.4 Engineering > Fixed client tools to remove 'h' and 'p' options (ITS#9917,ITS#8618) > Fixed ldapsearch memory leak with paged results (ITS#9860) > Fixed libldap ldif_open_urlto check for failure (ITS#9904) > Fixed libldap ldap_url_parsehosts check for failure (ITS#9904) > Fixed liblunicode UTF8bvnormalize buffer size (ITS#9955) > Fixed lloadd memory leaks (ITS#9907) > Fixed lloadd shutdown code to protect memory correctly (ITS#9913) > Fixed lloadd race in epoch.c (ITS#9947) > Fixed lloadd potential deadlock with cn=monitor (ITS#9951) > Fixed lloadd to keep listener base around when not active (ITS#9984) > Fixed lloadd object reclamation sequencing (ITS#9983) > Fixed slapd memory leak with olcAuthIDRewrite (ITS#6035) > Fixed slapd free of redundant cmdline option (ITS#9912) > Fixed slapd transactions extended operations cleanup after write > (ITS#9892) > Fixed slapd deadlock with replicated cn=config (ITS#9930) > Fixed slapd bconfig locking of cn=config entries (ITS#9045) > Fixed slapd-mdb max number of index databases to 256 (ITS#9895) > Fixed slapd-mdb to always release entries from ADD operations > (ITS#9942) > Fixed slapd-mdb to fully init empty DN in tool_entry_get (ITS#9940) > Fixed slapd-monitor memory leaks with lloadd (ITS#9906) > Fixed slapd-monitor to free remembered cookies (ITS#9339) > Fixed slapo-accesslog reqStart ordering matching rule (ITS#9880) > Fixed slapo-deref memory leak (ITS#9924) > Fixed slapo-dynlist to ignore irrelevant objectClasses (ITS#9897) > Fixed slapo-dynlist to avoid unnecessary searches (ITS#9929) > Fixed slapo-dynlist to mark internal searches as such (ITS#9960) > Fixed slapo-pcache crash in consistency_check (ITS#9966) > Fixed slapo-remoteauth memory leaks (ITS#9438) > Fixed slapo-rwm memory leaks (ITS#9817) > Build Environment > Fixed ancient DOS related ifdef checks (ITS#9925) > Fixed build process to not use gmake specific features (ITS#9894) > Fixed source tree to remove symlinks (ITS#9926) > Fixed slapo-otp testdir creation (ITS#9437) > Fixed slapd-tester memory leak (ITS#9908) > Fixed usage of non-standard C syntax (ITS#9898, ITS#9899, ITS#9901) > Fixed usage of bashism (ITS#9900) > Fixed test suite portability (ITS#9931) > Documentation > Fixed ldap_bind(3) to document ber_bvfree in ldap_sasl_bind > (ITS#9976) > Fixed slapo-asyncmeta(5) to clarify scheduling for target > connections (ITS#9941) > Fixed slapo-dynlist(5) to clarify configuration settings (ITS#9957) > Fixed slapo-unique(5) to clarify when quoting should be used > (ITS#9915) > Minor cleanup > ITS#9935 > ITS#9336 > ITS#9337 > > Regards, > Quanah >

1 0

Help moving from old server with bdb, etc.
by Chandler Sobel-Sorenson 22 Jan '23

22 Jan '23

Hi all, I'm trying to move our LDAP from an old CentOS 6 server to new Debian 11 server, but I'm running into the problem importing the config on the new server with slapadd: 63cc2422 <= str2entry: str2ad(olcDbCacheSize): attribute type undefined slapadd: could not parse entry (line=2076) I read that it's because of BDB being removed and no longer supported. What should I do, then? I admit LDAP is an area I practically know nothing about, so I'll appreciate your patience. This instance has been in place before I started my position over 6.5 years ago! It's been running great, but the server it's running on is also getting old and used for intense computation still at times, so I'm trying to be proactive and separate the LDAP service off to another server that is newer, that I can keep updated. The current, old server is running: @(#) $OpenLDAP: slapd 2.4.40 (Mar 22 2017 06:29:21) $ mockbuild@c1bm.rdu2.centos.org:/builddir/build/BUILD/openldap-2.4.40/openldap-2.4.40/build-servers/servers/slapd Included static backends: config ldif monitor bdb hdb ldap mdb meta null passwd relay shell sock and new server is running: @(#) $OpenLDAP: slapd 2.4.57+dfsg-3+deb11u1 (May 14 2022 18:32:57) $ Debian OpenLDAP Maintainers <pkg-openldap-devel(a)lists.alioth.debian.org> Included static backends: config ldif On the old server, I just ran: # cd /tmp # slapcat -n 0 -l config.ldif 63cc1841 PROXIED attributeDescription "DC" inserted. # slapcat -n 1 -l user.ldif 63cc1943 PROXIED attributeDescription "DC" inserted. # Then transferred the *.ldif to the new server and ran: # slapadd -n 0 -l /tmp/config.ldif This didn't work right off, so I adjusted the values of some parameters inside, such as: olcConfigFile, olcConfigDir, olcArgsFile, olcPidFile, olcTLSCertificateFile, olcTLSCertificateKeyFile. These were all in the `dn: cn=config`. After some more tries, figured I needed the -F option so am now using this command: # slapadd -n 0 -l /tmp/config.ldif -F /etc/ldap/slapd.d 63cc1f0f PROXIED attributeDescription "DC" inserted. 63cc1f0f <= str2entry: str2ad(olcDbCacheSize): attribute type undefined slapadd: could not parse entry (line=2076) _################### 99.05% eta none elapsed none spd 13.5 M/s Closing DB... # which brings us to the present situation. So, let me know if you have any ideas of what I should do/try next! Best, Chandler -- The University of Arizona block 'A' logo. *Chandler Sobel-Sorenson* Systems Administrator, Senior Arizona Genomics Institute School of Plant Sciences, Research THE UNIVERSITY OF ARIZONA Thomas W. Keating Bioresearch Bldg. | Rm. 200A24 1657 E. Helen St. | Tucson, AZ 85721 Office: 520-626-9589 | Cell: 520-907-4352 chandler(a)genome.arizona.edu <mailto: chandler(a)genome.arizona.edu> Pronouns: he/him/his *www.genome.arizona.edu* <https://www.genome.arizona.edu/> Integrity, Compassion, Exploration, Adaptation, Inclusion, Determination <https://brand.arizona.edu/signature>

4 7

using SRV-records in syncrepl
by Stefan Kania 20 Jan '23

20 Jan '23

hi to all, is it somehow possible to finde the provider in "syncrepl" of a consumer via DNS SRV-records. If I have several providers with lloadd in front of it and the consumers are only contacting the loadbalancer, the it would be nice to use the SRV-Records of the DNS. I could then set up two loadbalancer with a different priority, so if one failed the consumer would switch to the second loadbalancer. Just an idea :-). Because using just one loadbalancer it will be a single point of failure. Thanks Stefan

5 10

Re: Slow Mod operations on LDAP
by Bhanush Mehta 19 Jan '23

19 Jan '23

Hi Quanah The current mdb data file is GB on disk. We are using XFS, we tried with ext2 and ext4, we saw the same behavior for slow mods. 2376455:/var/lib/ldap$ sudo du -sh data.mdb 11G data.mdb 2376455:/var/lib/ldap$ sudo du -s data.mdb 10519776 data.mdb The dump from slapcat is 200 MB approx. Whenever we add it to a new instance, it balloons up to 10 GB and then stays around the same size. local-backups$ du -sh ldapdump202301180000.ldif 204M ldapdump202301180000.ldif Can DB size be a reason for slow mods? And, how do we investigate the slow mod further? Regards Bhanush On Tue, Jan 17, 2023 at 10:53 PM Quanah Gibson-Mount <quanah(a)fast-mail.org> wrote: > > > --On Monday, January 16, 2023 10:47 AM +0530 Bhanush Mehta > <bhanush.mehta(a)flipkart.com> wrote: > > > > > Hi Quanah, > > > > > > We see the same issue with 2.4.58 (compiled from source). > > > > > > I am able to debug that mod operations are fast on a fresh mdb, but after > > a certain number of operations the mdb size is going from 300 MB to 10 > > GB. > > Hello, > > Are you sure that the database is actually 10GB in size? Keep in mind that > back-mdb uses a sparse file, so the filesystem will often show the > database > as being the "size" of the MDB maxsize value, regardless of what the > actual > DB size is. > > You can use du -h "file.mdb" to see the actual filesize. > > For example, in a test system, I have: > > maxsize 1073741824 > > which is 1GB. > > > If we look at the output of ls: > > total 2704 > -rw------- 1 root root 1073741824 May 17 2022 data.mdb > > but the actual size on disk is: > > du -h data.mdb > 2.7M data.mdb > > Regards, > Quanah > > > > -- *-----------------------------------------------------------------------------------------* *This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error, please notify the system manager. This message contains confidential information and is intended only for the individual named. If you are not the named addressee, you should not disseminate, distribute or copy this email. Please notify the sender immediately by email if you have received this email by mistake and delete this email from your system. If you are not the intended recipient, you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.***** **** *Any views or opinions presented in this email are solely those of the author and do not necessarily represent those of the organization. Any information on shares, debentures or similar instruments, recommended product pricing, valuations and the like are for information purposes only. It is not meant to be an instruction or recommendation, as the case may be, to buy or to sell securities, products, services nor an offer to buy or sell securities, products or services unless specifically stated to be so on behalf of the Flipkart group. Employees of the Flipkart group of companies are expressly required not to make defamatory statements and not to infringe or authorise any infringement of copyright or any other legal right by email communications. Any such communication is contrary to organizational policy and outside the scope of the employment of the individual concerned. The organization will not accept any liability in respect of such communication, and the employee responsible will be personally liable for any damages or other liability arising.***** **** *Our organization accepts no liability for the content of this email, or for the consequences of any actions taken on the basis of the information *provided,* unless that information is subsequently confirmed in writing. If you are not the intended recipient, you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.* _-----------------------------------------------------------------------------------------_

4 5

Re: Slow Mod operations on LDAP
by Bhanush Mehta 18 Jan '23

18 Jan '23

Hi Quanah, We have 80 GB RAM on the system and 300 GB SSD disk allocated for the directory. Regards Bhanush On Wed, Jan 18, 2023 at 7:20 PM Quanah Gibson-Mount <quanah(a)fast-mail.org> wrote: > > > --On Wednesday, January 18, 2023 6:31 PM +0530 Bhanush Mehta > <bhanush.mehta(a)flipkart.com> wrote: > > > > > > > Hi Quanah > > > > > > The current mdb data file is GB on disk. We are using XFS, we tried with > > ext2 and ext4, we saw the same behavior for slow mods. > > > > 2376455:/var/lib/ldap$ sudo du -sh data.mdb > > 11G data.mdb > > 2376455:/var/lib/ldap$ sudo du -s data.mdb > > 10519776 data.mdb > > > > The dump from slapcat is 200 MB approx. Whenever we add it to a new > > instance, it balloons up to 10 GB and then stays around the same size. > > local-backups$ du -sh ldapdump202301180000.ldif204M > > ldapdump202301180000.ldif > > > > > > Can DB size be a reason for slow mods? And, how do we investigate the > > slow mod further? > > One is a flat text file export of the database (.ldif) > > The other is a set of binary data based off of the flat text file export, > but includes things such as indices. You cannot correlate the two size > wise. > > If your binary database after an import with slapadd -q is ~11GB, than > that's the "real" size of your database. > > How much memory do you have on the system you're using? > > I would note that it is not advised to use XFS with back-mdb. > > --Quanah > > > > -- *-----------------------------------------------------------------------------------------* *This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error, please notify the system manager. This message contains confidential information and is intended only for the individual named. If you are not the named addressee, you should not disseminate, distribute or copy this email. Please notify the sender immediately by email if you have received this email by mistake and delete this email from your system. If you are not the intended recipient, you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.***** **** *Any views or opinions presented in this email are solely those of the author and do not necessarily represent those of the organization. Any information on shares, debentures or similar instruments, recommended product pricing, valuations and the like are for information purposes only. It is not meant to be an instruction or recommendation, as the case may be, to buy or to sell securities, products, services nor an offer to buy or sell securities, products or services unless specifically stated to be so on behalf of the Flipkart group. Employees of the Flipkart group of companies are expressly required not to make defamatory statements and not to infringe or authorise any infringement of copyright or any other legal right by email communications. Any such communication is contrary to organizational policy and outside the scope of the employment of the individual concerned. The organization will not accept any liability in respect of such communication, and the employee responsible will be personally liable for any damages or other liability arising.***** **** *Our organization accepts no liability for the content of this email, or for the consequences of any actions taken on the basis of the information *provided,* unless that information is subsequently confirmed in writing. If you are not the intended recipient, you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.* _-----------------------------------------------------------------------------------------_

2 1

questions on index_hash64
by Geert Hendrickx 18 Jan '23

18 Jan '23

Hi I have a few questions on the index_hash64 / olcIndexHash64 feature; > Use a 64 bit hash for indexing. The default is to use 32 bit hashes. > These hashes are used for equality and substring indexing. The 64 bit > version may be needed to avoid index collisions when the number of > indexed values exceeds ~64 million. (Note that substring indexing > generates multiple index values per actual attribute value.) Indices > generated with 32 bit hashes are incompatible with the 64 bit version, > and vice versa. Any existing databases must be fully reloaded when > changing this setting. This directive is only supported on 64 bit CPUs. What is the effect if hash collisions happen? Will slapd actually return incorrect results, or does it just need to perform extra work to discard false positives from an index search result? Since this feature only impacts indexes and not data, isn't "slapindex" enough to to regenerate indexes, instead of a full slapcat/slapadd ? Finally, is there some way to detect if a given data.mdb file uses 32- or 64-bit index hashes? I can load a "legacy" data.mdb in a server with index_hash64 enabled, and slapd does not complain, indexes are just broken. Thanks Geert

2 1

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

openldap-technical January 2023