openldap-technical May 2024

openldap-technical@openldap.org

19 participants
17 discussions

Re: Facing issues with Symas LDAP upgrade from 2.4 to 2.5
by anil kumar pathuri 24 Jul '24

24 Jul '24

hi Quanah, Hope you are doing well. Thank you for your email. PFA This is what I did: Installed openldap 2.4.44 --> Upgraded to Symas OpenLDAP 2.4.57 --> Upgrading from Symas OpenLDAP 2.4.57 to Symas OpenLDAP 2.6.7 ***PS: I am not using slapd.conf, what should I do in this case. Regards, Anil P On Fri, 17 May 2024 at 01:57, Quanah Gibson-Mount <quanah(a)fast-mail.org> wrote: > > > --On Thursday, May 16, 2024 7:55 PM +0000 anilkumar.pathuri7(a)gmail.com > wrote: > > > Hi Team, GM > > Successful while trying to upgrade from openLDAP 2.4.44 to > SymasLDAP2.4.57 > > > > Facing issues while trying to upgrade to 2.5.x and 2.6.x from > > SymasLDAP2.4.57 https://repo.symas.com/soldap2.5/upgrading/ > > > > Issues while trying to upgrade from SymasLDAP2.4.57 to SymasLDAP 2.5.17. > > 1. config error processing olcDatabase={2}mdb,cn=config > > UNKNOWN attributeDescription "OLCDBINDEX" inserted > > Providing a config snippet is not particularly useful. > > You also say issue going from 2.5.x to 2.6.x but then you say the error is > in going from 2.4.57 to 2.5.17. > > Your error generally indicates that you're missing a module load on the > MDB > backend module. > > --Quanah > >

7 13

[OpenLDAP 2.5] Accesslog size and performance issues
by BECOT Jérôme 27 May '24

27 May '24

Hello, Here is our setup: 2 master in mirrormode with accesslog replication and several slaves with lastbind and chain overlay. Everything works, but the accesslog db is quite large and filled with authtimestamp updates from client hosts running sssd. We also have a disk performance issue when the cleanup process is launched every 4h. We plan to reduce the accesslog history by setting 12h retention and increase cleanup . It should reduce the number of entries and therefore lighten the cleanup process but: I wonder if increasing the frequency will also have a positive effects (as the though here is that there will be less entries to delete). I think that allocated pages in the mdb database won't change. Is there a safe way to reduce the db size ? Finally, we considered not logging the sssd auth timestamp updates, but lastbind doesn't have any option to filter which branch is allowed to update, the only option is to set the precision to a higher value. Is there a way to achieve this ? Any suggestion will be warmly appreciated. Thank you Jerome

4 7

its8752 failure
by patrick+openldap.org＠laimbock.com 26 May '24

26 May '24

Hi all, Almalinux 9.4 x86_64. When building 2.6.8 with gcc 11.4.1 20231218 (Red Hat 11.4.1-3) or gcc 13.2.1 20231205 (Red Hat 13.2.1-6) I get a failure in 'make -j1 its' in its8752: >>>>> Starting its8752 ... running defines.sh This test tracks a case where slapd deadlocks during a significant write load See https://bugs.openldap.org/show_bug.cgi?id=8752 for more information. Starting slapd on TCP/IP port 9011... Using ldapsearch to check that slapd is running... Populating database on first provider... Stopping slapd and reworking configuration for MPR... Starting provider slapd on TCP/IP URI ldap://localhost:9011/ Using ldapsearch to check that provider slapd is running... Starting provider slapd on TCP/IP URI ldap://localhost:9012/ Using ldapsearch to check that provider slapd is running... Starting provider slapd on TCP/IP URI ldap://localhost:9013/ Using ldapsearch to check that provider slapd is running... Starting provider slapd on TCP/IP URI ldap://localhost:9014/ Using ldapsearch to check that provider slapd is running... Setting up accesslog on each provider... Modifying dn: cn=Elmer_Fudd,ou=People,dc=example,dc=com on provider 1 Modifying dn: cn=Elmer_Fudd,ou=People,dc=example,dc=com on provider 2 Modifying dn: cn=Elmer_Fudd,ou=People,dc=example,dc=com on provider 3 Modifying dn: cn=Elmer_Fudd,ou=People,dc=example,dc=com on provider 4 Waiting 7 seconds for servers to catch up... Waiting 7 seconds for servers to catch up... Waiting 7 seconds for servers to catch up... Waiting 7 seconds for servers to catch up... Waiting 7 seconds for servers to catch up... Waiting 7 seconds for servers to catch up... Servers did not replicate in time >>>>> ./data/regressions/its8752/its8752 failed (exit 1) make[1]: Leaving directory '/builddir/build/BUILD/openldap-2.6.8/tests' make[1]: *** [Makefile:364: mdb-its-yes] Error 1 make: *** [Makefile:355: regressions] Error 2 error: Bad exit status from /var/tmp/rpm-tmp.cKP3aW (%check) Looking at its8752 in bugzilla there is a target of 2.7.0 and there are a number of other ITSs mentioned: https://bugs.openldap.org/show_bug.cgi?id=9782 (confirmed) https://bugs.openldap.org/show_bug.cgi?id=9580 (in progress) https://bugs.openldap.org/show_bug.cgi?id=9341 (unconfirmed) Is this test supposed to pass successfully on 2.6.8? Thanks, Patrick

1 0

el9 bind ip address
by Marc 23 May '24

23 May '24

Anyone know if this file is still working in el9? Looks like if I put SLAPD_URLS it is not read. /etc/sysconfig/slapd

4 7

Unable to enable auditlog in openldap 2.4
by kalybox2020＠gmail.com 20 May '24

20 May '24

We are using cn=config file for configuration. Using ldapmodify dn: olcOverlay={8}auditlog,olcDatabase={1}hdb,cnc=config objectclass: olcOverlayConfig objectClass: olcAuditlogConfig olcAuditLogFile: /var/log/ldapauditlog.ldif restarted slapd, Gave permissions to this file, and restarted slapd, still not able to generate logs. Thanks -Kaly

1 0

How to setup replication in openldap 2.6.7
by kalybox2020＠gmail.com 20 May '24

20 May '24

How to setup replication in openldap 2.6.7 Please let me know Thanks -Kaly

3 2

Facing issues with Symas LDAP upgrade from 2.4 to 2.5
by anilkumar.pathuri7＠gmail.com 16 May '24

16 May '24

Hi Team, GM Successful while trying to upgrade from openLDAP 2.4.44 to SymasLDAP2.4.57 Facing issues while trying to upgrade to 2.5.x and 2.6.x from SymasLDAP2.4.57 https://repo.symas.com/soldap2.5/upgrading/ Issues while trying to upgrade from SymasLDAP2.4.57 to SymasLDAP 2.5.17. 1. config error processing olcDatabase={2}mdb,cn=config UNKNOWN attributeDescription "OLCDBINDEX" inserted if I comment "OLCDBINDEX" 2. ldif_read_file: checksum error on "/opt/symas/etc/openldap/slapd.d/cn=config/olcDatabase={2}mdb.ldif config error processing olcDatabase={2}mdb,cn=config: Can you please guide by looking into this issue. Regards, Anil P ************************************************************* cat olcDatabase={2}mdb.ldif # AUTO-GENERATED FILE - DO NOT EDIT!! Use ldapmodify. # CRC32 f21f2837 dn: olcDatabase={2}mdb objectClass: olcDatabaseConfig objectClass: olcMdbConfig olcDatabase: {2}mdb olcDbDirectory: /var/lib/ldap olcSuffix: dc=hadoop,dc=com olcRootDN: cn=ldapadm,dc=hadoop,dc=com olcRootPW:: ************ olcDbIndex: objectClass eq,pres structuralObjectClass: olcMdbConfig entryUUID: ba4f359a-a605-103e-9085-7fcb0f4ffaf5 creatorsName: cn=config createTimestamp: 20240514061956Z entryCSN: 20240514062200.341950Z#000000#000#000000 modifiersName: gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth modifyTimestamp: 20240514062200Z

2 1

[openldap 2.4]olcdbcheckpoint parameter is not working
by norihisa.kimura＠ctc-g.co.jp 15 May '24

15 May '24

To whom it may concern, Could you confirm if the following understanding is correct? Regarding the olcDbCheckpoint parameter in openldap-2.4.46-18, it is mentioned in the 2.4 manual but it does not seem to work even when the setting is enabled and tested. Upon investigating the source code, it appears that the parameter is deactivated. Based on these results, it is speculated that it will not work as expected because it is an unimplemented item, Is this hypothesis right? * 2.4 manual https://www.openldap.org/doc/admin24/slapdconf2.html 5.2.6.3. olcDbCheckpoint: <kbyte> <min> I am looking forward to your response. regards

2 1

ldapauditlog is too big
by kalybox2020＠gmail.com 14 May '24

14 May '24

Hi Team, We are running openldap 2.4. and I see ldapauditlog is too big about 14GB, never rotated for more than a year. What is the best way to rotate it? I am thinking. 1. Stop the ldap instance mv ldapauditlog file to ldapauditlog.bak touch ldapauditlog start the isntance Is there any better way? Please confirm Thanks -Kaly

2 1

Storage overhead of LMDB vs SQLite3 for file hierarchy indexing scenario
by Jérôme Carretero 13 May '24

13 May '24

Hi, I wanted to share some thoughts and findings and perhaps collect some comments about a small test I did: I used LMDB vs. SQLite3 to perform indexing of a file tree. In this scenario, we're capturing file paths, along with some metadata (size, mtime, and hashes, let's use only sha1 for this example), and we want to be able to efficiently walk the tree, as well as resolve a file's info by id or by path. I'm showing the SQL schema first, as it helps formalizing the "NoSQL" case: CREATE TABLE IF NOT EXISTS files ( id INTEGER PRIMARY KEY AUTOINCREMENT, sha1 BLOB, size INTEGER, mtime INTEGER ); CREATE TABLE IF NOT EXISTS pathsegments ( id INTEGER PRIMARY KEY AUTOINCREMENT, parent_id INTEGER, file_id INTEGER, segment_name TEXT, FOREIGN KEY(parent_id) REFERENCES pathsegments(id), FOREIGN KEY(file_id) REFERENCES files(id) ); CREATE INDEX IF NOT EXISTS idx_sha1 ON files (sha1); CREATE INDEX IF NOT EXISTS idx_fileid2segment ON pathsegments (file_id); CREATE INDEX IF NOT EXISTS idx_parent_id2segment ON pathsegments (parent_id); Capturing the full path of each entry as a string was not something I wanted, and was also wasteful in the few datasets I tested it on. So we're using path segments, and a full path can be reconstructed; in the walking situation, from root to leaves, or if we have a file ID and want its path, from the leaves to the root, using the fileid2segment index and the parent information. And if files get moved around, we can alter the database without rewriting many paths. With lmdb, as I was conscious that each database would "intrinsically" build an index, I packed the file_id->{mtime,size,hash} as fid2attrs and segment_id->{parent_id,name} as sid2kids databases, and we still have segment_id->kids DUPSORT sid2kids for doing asciibetical readdir like the SQL idx_parent_id2segment. So we have our 2 tables with 2+3 indexes in SQL, and 4 DBs in lmdb (fid2attrs, sid2kids, sid2dirname, hash2fid). I implemented this (in Python) with sqlite and lmdb (using COBS for serialization, for simplicity and because it has the nice property of keeping serialized varint unsigned integers with preserved ordering), and went on to build dbs from a dataset of 474782 small files and 45014 directories (which I've coincidentally uploaded at https://archive.org/download/ftp.modland.com_pub_modules). Well, I was surprised that the SQLite3 file was weighing 67,149,824 bytes, and the lmdb one was larger at 82,718,720 bytes. In particular, looking at database statistics, it seems that the hash to id index/DB had unexpected (for me) overhead of ~ 112% to payload size with lmdb (and it was using less payload), with a size of 24932352 bytes for lmdb's DB vs. 15298560 for SQLite's index; SQLite statistics: Percentage of total database...................... 22.8% Number of entries................................. 474782 Bytes of storage consumed......................... 15298560 Bytes of payload.................................. 12311437 80.5% Bytes of metadata................................. 1469162 9.6% B-tree depth...................................... 3 Average payload per entry......................... 25.93 Average unused bytes per entry.................... 3.20 Average metadata per entry........................ 3.09 Average fanout.................................... 109.00 Non-sequential pages.............................. 2986 80.0% Maximum payload per entry......................... 26 Entries that use overflow......................... 0 0.0% Index pages used.................................. 34 Primary pages used................................ 3701 Overflow pages used............................... 0 Total pages used.................................. 3735 Unused bytes on index pages....................... 16998 12.2% Unused bytes on primary pages..................... 1500963 9.9% Unused bytes on overflow pages.................... 0 Unused bytes on all pages......................... 1517961 9.9% LMDB: depth=3 branch_pages=66 leaf_pages=6021 overflow_pages=0 Keys size: 9495640 Values size: 2261081 Payload size: 11756721 Pages size: 24932352 Average key size: 20.000 (20-20) Average value size: 4.762 (1-5) Average entry size: 24.762 Entries number: 474782 Overhead : 13175631 Overhead/payload ratio: 1.121 Overhead/entry avg: 27.751 I checked the overhead for in-order insertions and it's still around 50% for this key/value sizes, much more than SQLite does (~21%) for random insertions. But even if we were to remove this index, SQLite's file would still be tighter... That's it! Best regards, -- Jérôme

2 1

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

openldap-technical May 2024