openldap-technical June 2023

openldap-technical@openldap.org

30 participants
37 discussions

Re: migration 2.4 -> 2.5 (bdb -> mdb) | monitoring & health checks
by cYuSeDfZfb cYuSeDfZfb 22 Jun '23

22 Jun '23

Hi Quanah, Thanks for your answer and kind suggestions! We will implement them. And anyone here using zabbix, and has some scripting for monitoring laying around..? Thanks! On Fri, 16 Jun 2023 at 16:42, Quanah Gibson-Mount <quanah(a)fast-mail.org> wrote: > > > --On Thursday, June 15, 2023 10:51 PM +0200 sacawulu <cyusedfzfb(a)gmail.com> > > wrote: > > > > Most of these things can be done similarly in 2.4/bdb and 2.5/mbd but not > > the db_verify, which is bdb specific. > > > > I don't find a lot of info on the maintenance / integrity checks that can > > be done on mdb databases. In fact: Mostly what I can find is: "MDB uses > > no caching and requires no tuning to deliver maximum search performance" > > There's very little necessary to be done when using back-mdb as you've > already discovered > > > That is very good. :-) We have already discovered that we need to > > increase maxsize, as our database is (much) larger than 10MB. > > The rule of thumb for maxsize is to set it to something larger than you > ever expect to hit. For example, with a database I'm using that's ~3.4GB > in size, I have a 20GB maxsize set. You may want to monitor the size of > your database vs the configured maxsize. This can be done with data > available in the back-monitor database (at least with 2.6, not sure with > 2.5) or via the mdb_stat utility. > > python-ldap3 snippet using back-monitor: > > try: > tls = Tls(validate=ssl.CERT_NONE, version=ssl.PROTOCOL_TLSv1_2) > conn = Connection(Server(ldap_url, tls=tls, use_ssl=True), > auto_bind=True) > conn.search( > "cn=monitor", > "(&(objectClass=olmMDBDatabase))", > attributes=["olmMDBPagesMax", "olmMDBPagesUsed", > "namingContexts"], > ) > > for db in conn.entries: > suffix = db.namingContexts.value > # 4096 is the page size in use > # can be found with mdb_stat -e /path/to/database > max_size = int(db.olmMDBPagesMax.value) * 4096 > current_size = int(db.olmMDBPagesUsed.value) * 4096 > pct_used = float(current_size / max_size * 100) > > > > > Anyone else with tips and tricks on daily maintenance or monitoring? > > Scripts to share..? Perhaps zabbix templates..? > > The only other bits I would recommend is if you have object in the > directory that have very large multi-valued attribute data. For example, > if you use groups extensively, and the 'member' attribute has hundreds of > entries, you would probably want to do something like: > > a) Add member as an attribute to be handled by olcSortvals > > b) Add a multival configuration to back-mdb if the attribute is indexed. > For example, if you indexed member "eq". I usually do 100,10. This will > put the index databse into its own subdb, which helps keep fragmentation > from being an issue. > > For example, I had a ~5.5GB DB that was swelling to over 11GB in size due > to fragmentation before implementing those two configuration options. > > > Regards, > Quanah > > > >

2 2

Data migration
by Jean-Luc Chandezon 16 Jun '23

16 Jun '23

Hello dear community, I have to migrate our ldap database from OpenLDAP (from 2019 version hosted on Debian, to Symas OpenLDAP 2.4.59 on ). I need to change our DN from "dc=lexp,dc=fr" to "dc=directory,dc=lan-explore,dc=fr". I installed OpenLDAP and configure directory with new DN. For data migration, I exported data: slapcat -b dc=lexp,dc=fr > /tmp/20230616_save_data.ldif .. and for recovery, I changed all dn mentions from old to new dn. 1. Is the correct way? When import, you can see error message: slapadd: slap_init invalid suffix ("dc=directory,dc=lan-explore,dc=fr ") 2. When searching, I discovered that each line can contains 77 columns. Next, line is truncated, and it's difficult to replace all dn. Thanks, Jean-Luc

4 4

migration 2.4 -> 2.5 (bdb -> mdb) | monitoring & health checks
by sacawulu 16 Jun '23

16 Jun '23

Hi, We are in the middle of a move from 2.4/bdb to 2.5/mdb, and I am working on implementing monitoring for the new systems. In the 2.4/bdb environment, we run several scripts to check sanity of the servers: * checks of the contextCSN value across the masters/slaves * regular db_verify on the actual bdb files * compare all DN & entryCSN in the DB across all the masters/slaves * do a write on a master, and verify that it is replicated everywhere Most of these things can be done similarly in 2.4/bdb and 2.5/mbd but not the db_verify, which is bdb specific. I don't find a lot of info on the maintenance / integrity checks that can be done on mdb databases. In fact: Mostly what I can find is: "MDB uses no caching and requires no tuning to deliver maximum search performance" That is very good. :-) We have already discovered that we need to increase maxsize, as our database is (much) larger than 10MB. Two questions: - Is there really nothing else to tune or adjust? - Is there really no way to verify the internal structures within the MDB file, to make sure everything is valid and healthy? Perhaps things I have missed with our four checks above? Anyone else with tips and tricks on daily maintenance or monitoring? Scripts to share..? Perhaps zabbix templates..? If anyone would be interested in (for example) my script to verify the complete directory contents based on DN&entryCSN across a (multi)master/(multi)slave setup, I'd be happy to share too, of course. Thanks!

2 1

Vulnerability CVE-2023-2953 found in latest version
by Sahil Sharma D 15 Jun '23

15 Jun '23

Hi Team, We are currently using OpenLDAP version 2.4.57 in which vulnerability CVE-2023-2953 was not there. We are planning to upgrade to OpenLDAP-2.6.4 however in this version we found CVE-2023-2953 in our scanning, can you please help in understanding why this vulnerability is opened in latest release. In which release we can expect this vulnerability to get resolved? Appreciate your earliest response. Regards, Sahil

2 2

Re: no contextCSN attribute on consumers
by cYuSeDfZfb cYuSeDfZfb 15 Jun '23

15 Jun '23

Hi! I am using the cn=admin,o=infra,c=com with correct password to connect. I will further check ACLs! Thank you for that suggestion. Just to make things more concrete, below are two samples. One on the MASTER with contextCSN, and one from the SLAVE without contextCSN. EXAMPLE SLAVE: ldapsearch -x -H ldaps://$SERVER -D $LDAPBINDDN -w $ADMINPW -b "o=infra,c=com" -s base contextCSN # extended LDIF # # LDAPv3 # base <o=infra,c=com> with scope baseObject # filter: (objectclass=*) # requesting: contextCSN # # search result search: 2 result: 0 Success # numResponses: 1 EXAMPLE MASTER: ldapsearch -x -H ldaps://$SERVER -D $LDAPBINDDN -w $ADMINPW -b "o=infra,c=com" -s base contextCSN # extended LDIF # # LDAPv3 # base <o=infra,c=com> with scope baseObject # filter: (objectclass=*) # requesting: contextCSN # # infra, com dn: o=infra,c=com contextCSN: 20180917142109.765066Z#000000#000#000000 contextCSN: 20230612144901.796553Z#000000#001#000000 contextCSN: 20230612144904.899482Z#000000#002#000000 # search result search: 2 result: 0 Success # numResponses: 2 # numEntries: 1 On Mon, 12 Jun 2023 at 16:42, Quanah Gibson-Mount <quanah(a)fast-mail.org> wrote: > > > --On Monday, June 12, 2023 5:36 PM +0200 cYuSeDfZfb cYuSeDfZfb > <cyusedfzfb(a)gmail.com> wrote: > > > Hi Quanah, > > > > Thanks for the swift reponse! I think I do, yes, see, from consumer one: > > > > olcSyncrepl: {0}rid=202 > > provider=ldap://master-acc-02.ldap.infra.com:389 bindmethod=simple > > filter="(objectClass=*)" scope=sub binddn="cn=mirrormode,ou=Directory > > Access,o=infra,c=com" credentials=XYZ searchbase="o=infra,c=com" > > schemachecking=on type=refreshAndPersist retry="60 +" attrs="*,+" > > starttls=critical tls_reqcert=demand > > olcSyncrepl: {1}rid=201 > > provider=ldap://master-acc-01.ldap.infra.com:389 bindmethod=simple > > filter="(objectClass=*)" scope=sub binddn="cn=mirrormode,ou=Directory > > Access,o=infra,c=com" credentials=XYZ searchbase="o=infra,c=com" > > schemachecking=on type=refreshAndPersist retry="60 +" attrs="*,+" > > starttls=critical tls_reqcert=demand > > olcUpdateRef: ldap://provider-acc-02.ldap.infra.com:389 > > olcUpdateRef: ldap://provider-acc-01.ldap.infra.com:389 > > That's syncrepl, not syncprov. > > However, the issue could be ACLs. If you use the rootdn for your database > to run the query, can you see the contextCSN value stored in your database > root? > > --Quanah > > > >

3 4

Problem with restoring using ldif, OpenLDAP 2.6.3
by awsome jang 14 Jun '23

14 Jun '23

Hello, I am running openldap 2.6.3 configured as provider-consumer and I would like to do backup/restore of the database on the provider. I create a backup on working sladp using command(ends with success, file created): slapd.exe –T cat –f slapd.conf –l backup.ldif And then trying to load it using slapd command into a new database(restore): slapd.exe -T add -f slapd.conf -l backup.ldif -d -1 Command ends with: . creatorsName: uid=root,ou=invalid,dc=vmbox,dc=int createTimestamp: 20230410111050Z entryCSN: 20230410111051.368528Z#000000#000#000000 modifiersName: uid=root,ou=invalid,dc=vmbox,dc=int modifyTimestamp: 20230410111050Z contextCSN: 20230410125515.-401856Z#000000#000#000000 " >>> dnPrettyNormal: <dc=vmbox,dc=int> <<< dnPrettyNormal: <dc=vmbox,dc=int>, <dc=vmbox,dc=int> >>> dnNormalize: <uid=root,ou=invalid,dc=vmbox,dc=int> <<< dnNormalize: <uid=root,ou=invalid,dc=vmbox,dc=int> >>> dnNormalize: <uid=root,ou=invalid,dc=vmbox,dc=int> <<< dnNormalize: <uid=root,ou=invalid,dc=vmbox,dc=int> <= str2entry NULL (smr_normalize contextCSN 21) slapadd: could not parse entry (line=1) slapadd shutdown: initiated slapadd destroy: freeing system resources. When I look into backup.ldif files I see that there is an '-' in contextCSN timestamp which cause this behaviour: entryCSN: 20230410111051.368528Z#000000#000#000000 modifiersName: uid=root,ou=invalid,dc=vmbox,dc=int modifyTimestamp: 20230410111050Z contextCSN: 20230410125515.-401856Z#000000#000#000000 ^^^^^^ contextCSN: 20230412130738.035155Z#000000#002#000000 I also see that entryCSN also can be negative which also causes similar problems when loading files. I would like to ask if there is any way to restore backup in this scenario in a proper way? for instance removing the "-" sign? Thank in advance for your response, Best regards, Peter

2 1

Re: troubleshooting replication issues
by Abby Guenther 14 Jun '23

14 Jun '23

2.4.59 Postgres backend Slapd doesn't always crash. Sometimes it just stops replicating for seemingly no reason. Thanks On Tue, Jun 13, 2023, 1:38 PM Quanah Gibson-Mount <quanah(a)fast-mail.org> wrote: > > > --On Wednesday, May 31, 2023 3:54 AM +0000 abby.knows.it(a)gmail.com wrote: > > > We have three nodes and want to replicate between them. This setup > > worked fine for over a year. Recently replication fails and the slapd > > process crashes. > > OpenLDAP version? What database backend? Does the process actually crash > or does it shut down? > > --Quanah > > >

2 1

troubleshooting replication issues
by abby.knows.it＠gmail.com 13 Jun '23

13 Jun '23

We have three nodes and want to replicate between them. This setup worked fine for over a year. Recently replication fails and the slapd process crashes. In a general sense I'm looking for a FAQ on troubleshooting replication, the common issues and solutions. Also how to turn on logging for replication specifically would help. Thanks. Abby

4 3

Re: RE26 testing call (2.6.5) #1
by Simon Pichugin 12 Jun '23

12 Jun '23

Hi folks, OpenLDAP 2.6.5 was built and tested cleanly on Fedora 38 (Rawhide). Regards, Simon On Thu, May 25, 2023 at 1:39 PM Quanah Gibson-Mount <quanah(a)fast-mail.org> wrote: > This is the first testing call for OpenLDAP 2.6.5. Depending on the > results, this may be the only testing call. > > Generally, get the code for RE26: > > < > https://git.openldap.org/openldap/openldap/-/archive/OPENLDAP_REL_ENG_2_6/o… > > > > Extract, configure, and build. > > Execute the test suite (via make test) after it is built. Optionally, cd > tests && make its to run through the regression suite. > > Thanks! > > OpenLDAP 2.6.5 Engineering > Fixed libldap handling of TCP KEEPALIVE options (ITS#10015) > Fixed libldap with async connections (ITS#10023) > Fixed libldap openssl TLSv1.3 cipher suite handling (ITS#10035) > Fixed slapd callback handling with overlays that do extended > operations > (ITS#9990) > Fixed slapd conversion of pcache configurations (ITS#10031) > Fixed slapd cn=config modification handling with abandon (ITS#10045) > Fixed slapd-mdb online indexer termination and cleanup (ITS#9993) > Fixed slapd-mdb online indexer when interrupted (ITS#10047) > Fixed slapd-monitor connection cleanup (ITS#10042) > Fixed slapo-constraint handling of push replication (ITS#9953) > Fixed slapo-dynlist filter evaluation efficiency (ITS#10041) > Fixed slapo-pcache handling of invalid schema (ITS#10032) > Fixed slapo-ppolicy handling of push replication (ITS#9953) > Fixed slapo-ppolicy handling of pwdMinDelay (ITS#10028) > Fixed slapo-syncprov abandon handling (ITS#10016) > Fixed slapo-translucent handling of invalid schema (ITS#10032) > Fixed slapo-unique handling of push replication (ITS#9953) > Fixed slapo-variant to improve regex handling (ITS#10048) > Build Environment > Fixed compatibility with stricter C99 compilers (ITS#10011) > Keep .pc files during make clean (ITS#9989) > Contrib > Fixed slapo-variant handling of push replication (ITS#9953) > Minor Cleanup > ITS#9855 > ITS#9995 > ITS#9996 > ITS#9997 > ITS#9998 > ITS#9999 > ITS#10000 > ITS#10003 > ITS#10004 > ITS#10033 > ITS#10037 > ITS#10039 > ITS#10046 > > Regards, > Quanah > >

1 0

olcAccess rule with groupOfURLs
by Souji Thenria 12 Jun '23

12 Jun '23

Hello, I am trying to use a group of the type "groupOfURLs" within an olcAccess rule. However, I am having trouble getting this to work. For testing, I use a simple rule like: by set="[cn=test,ou=System,dc=example,dc=local]/member & user" write where "test" consists of the objectClass "grouOfURLs", with the following query in the "memberURL" attribute: ldap:///ou=Users,dc=example,dc=local??one?(uid=*) In my understanding, the query is executed when the group's content is requested. And it works; if I use "ldapsearch", I get the expected result. However, if I use this group in the olcAccess rule, it won't work. I tried to use a static group with the objectClass "groupOfMembers", which works. All I found suggested that I could use a dynamic list within a rule. Am I missing something, or does it simply not work with dynamic lists? Regards, Souji Thenria

4 9

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

openldap-technical June 2023