dynlist vs memberof performance issues
by Mark Cairney
Hi,
I've been out the LDAP loop for a bit but the recent discussion of the
memberof overlay on 2.5 piqued my curiosity. Having upgraded a Dev box,
removed the memberof elements from the database and replaced the
memberof overlay with dynlist the queries appear to work as expected but
are both a) slow and b) heavily CPU-intensive on the LDAP server.
2021-09-01T12:47:17.603513+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 fd=12 ACCEPT from IP=192.168.152.33:58738
(IP=129.215.17.9:636)
2021-09-01T12:47:17.687488+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 fd=12 TLS established tls_ssf=256 ssf=256
tls_proto=TLSv1.3 tls_cipher=TLS_AES_256_GCM_SHA384
2021-09-01T12:47:17.688032+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=0 SRCH base="" scope=0 deref=0
filter="(objectClass=*)"
2021-09-01T12:47:17.688470+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=0 SRCH attr=supportedSASLMechanisms
2021-09-01T12:47:17.688878+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=0 SEARCH RESULT tag=101 err=0 qtime=0.000014
etime=0.000214 nentries=1 text=
2021-09-01T12:47:17.811279+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=1 BIND dn="" method=163
2021-09-01T12:47:17.819249+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=1 RESULT tag=97 err=14 qtime=0.000030
etime=0.009084 text=SASL(0): successful result:
2021-09-01T12:47:17.908889+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=2 BIND dn="" method=163
2021-09-01T12:47:17.909836+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=2 RESULT tag=97 err=14 qtime=0.000031
etime=0.000181 text=SASL(0): successful result:
2021-09-01T12:47:17.938839+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=3 BIND dn="" method=163
2021-09-01T12:47:17.939621+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=3 BIND authcid="mcairney(a)EASE.ED.AC.UK"
authzid="mcairney(a)EASE.ED.AC.UK"
2021-09-01T12:47:17.940213+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=3 BIND
dn="uid=mcairney,ou=people,ou=central,dc=authorise-dev,dc=ed,dc=ac,dc=uk"
mech=GSSAPI bind_ssf=256 ssf=256
2021-09-01T12:47:17.940616+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=3 RESULT tag=97 err=0 qtime=0.000024
etime=0.000409 text=
2021-09-01T12:47:18.227342+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=4 SRCH
base="dc=authorise-dev,dc=ed,dc=ac,dc=uk" scope=2 deref=0
filter="(uid=mcairney)"
2021-09-01T12:47:18.227703+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=4 SRCH attr=* +
2021-09-01T12:47:31.392255+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=5 UNBIND
2021-09-01T12:47:31.460705+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=4 SEARCH RESULT tag=101 err=0 qtime=0.000031
etime=13.233679 nentries=1 text=
2021-09-01T12:47:31.461098+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 fd=12 closed
I'm guessing that as the values are computed that this will be heavier
on the CPU but it seems a bit excessive? Has anyone else noticed any
similar performance issues?
This is a relatively low-specced DEV server (2 vCPUs, 4GB RAM) so I
guess this could be a factor but there's no io waiting on the server and
no swapping?
The database is on a par in size with our Production service ( about
400K user objects with 1 group object per user and then about 80K actual
groups of users)
The config for the primary DB (ACLs and rootPW redacted) is:
dn: olcDatabase={1}mdb,cn=config
objectClass: olcDatabaseConfig
objectClass: olcMdbConfig
olcDatabase: {1}mdb
olcDbDirectory: /opt/openldap/var/openldap-data/authorise
olcSuffix: dc=authorise-dev,dc=ed,dc=ac,dc=uk
olcAddContentAcl: FALSE
olcLastMod: TRUE
olcMaxDerefDepth: 2
olcReadOnly: FALSE
olcSecurity: ssf=1
olcSecurity: update_ssf=112
olcSecurity: simple_bind=64
olcSizeLimit: unlimited
olcSyncUseSubentry: FALSE
olcTimeLimit: unlimited
olcMonitoring: TRUE
olcDbEnvFlags: writemap
olcDbEnvFlags: nometasync
olcDbNoSync: FALSE
olcDbIndex: objectClass eq
olcDbIndex: entryUUID eq
olcDbIndex: entryCSN eq
olcDbIndex: cn pres,eq,sub
olcDbIndex: uid pres,eq,sub
olcDbIndex: uidNumber pres,eq
olcDbIndex: gidNumber pres,eq
olcDbIndex: eduniType eq
olcDbIndex: gecos pres,eq,sub
olcDbIndex: eduniCategory eq
olcDbIndex: mail pres,eq,sub
olcDbIndex: eduniSchoolCode eq
olcDbIndex: eduniIDStatus eq
olcDbIndex: eduniCollegeCode eq
olcDbIndex: eduniOrgCode eq
olcDbIndex: memberOf pres,eq
olcDbIndex: eduniLibraryBarcode pres,eq
olcDbIndex: eduniOrganisation pres,eq,sub
olcDbIndex: eduniServiceCode pres,eq
olcDbIndex: krbName pres,eq
olcDbIndex: eduPersonAffiliation pres,eq
olcDbIndex: eduPersonEntitlement pres,eq
olcDbIndex: sn pres,eq,sub
olcDbIndex: eduniIdmsId pres,eq
olcDbIndex: member pres,eq
olcDbIndex: memberUid pres,eq
olcDbIndex: eduniRefNo pres,eq
olcDbIndex: eduniTitle pres,eq
olcDbIndex: title pres,eq,sub
olcDbIndex: eduniCardNumber pres,eq
olcDbIndex: eduniYearOfStudy eq
olcDbIndex: description pres,eq,sub
olcDbIndex: givenName pres,eq,sub
olcDbIndex: aliasedObjectName eq
olcDbIndex: yubiKeyId pres,eq
olcDbIndex: isMemberOf pres,eq
olcDbIndex: hasMember pres,eq
olcDbIndex: proxyAddresses pres,eq,sub
olcDbMaxReaders: 96
olcDbMaxSize: 32212254720
olcDbMode: 0600
olcDbSearchStack: 16
structuralObjectClass: olcMdbConfig
dn: olcOverlay={0}syncprov,olcDatabase={1}mdb,cn=config
objectClass: olcOverlayConfig
objectClass: olcConfig
objectClass: top
objectClass: olcSyncProvConfig
olcOverlay: {0}syncprov
structuralObjectClass: olcSyncProvConfig
dn: olcOverlay={1}accesslog,olcDatabase={1}mdb,cn=config
objectClass: olcOverlayConfig
objectClass: olcAccessLogConfig
olcOverlay: {1}accesslog
olcAccessLogDB: cn=accesslog
olcAccessLogOps: writes
olcAccessLogPurge: 02+00:00 00+04:00
olcAccessLogSuccess: TRUE
structuralObjectClass: olcAccessLogConfig
dn: olcOverlay={2}dynlist,olcDatabase={1}mdb,cn=config
objectClass: olcOverlayConfig
objectClass: olcDynListConfig
olcOverlay: {2}dynlist
olcDynListAttrSet: {0}groupOfURLs memberURL member+memberOf@groupOfNames
structuralObjectClass: olcDynListConfig
--
/****************************
Mark Cairney
ITI Enterprise Services
Information Services
University of Edinburgh
Tel: 0131 650 6565
Email: Mark.Cairney(a)ed.ac.uk
*******************************/
The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. Is e buidheann carthannais a th’ ann an Oilthigh Dhùn Èideann, clàraichte an Alba, àireamh clàraidh SC005336.
2 months, 2 weeks
Q: "Error: Invalid DN syntax (34), additional info: invalid new RDN"
by Ulrich Windl
Hi!
I'm programming some automated changes to our LDAP database, and I have an
issue:
# Error: Invalid DN syntax (34), additional info: invalid new RDN
dn: cn=subntbcst_tftp@247/tcp,dc=services,dc=net,dc=...,dc=de
changetype: modrdn
newrdn: subntbcst-tftp@247/tcp
deleteoldrdn: 1
So is the new RDN "subntbcst-tftp@247/tcp" really invalid? If so it seems an
older version of OpenLDAP accepted that as we have such an entry:
dn: cn=subntbcst_tftp@247/tcp,dc=services,dc=net,dc=...,dc=de
objectClass: ipService
cn: subntbcst_tftp
cn: subntbcst_tftp@247/tcp
createTimestamp: 20130719093351Z
...
I saw this exaple in RFC 2849 (so I thought my LDIF shuld be OK):
# Modify an entry’s relative distinguished name
dn: cn=Paul Jensen, ou=Product Development, dc=airius, dc=com
changetype: modrdn
newrdn: cn=Paula Jensen
deleteoldrdn: 1
Regards,
Ulrich
7 months
measurable slower result time with large subtree
by Norbert
Hi,
with OpenLDAP 2.4.47 (running on Debian 10) but also with 2.5.13 from ltb-project.org (running on same Debian
10) I can observe the following:
given following rough idea of a tree
o=base
|- bn=subtree1
| - handful of entries
|- bn=subtree2
| - millions of entries
|- bn=subtree3
| |- bn=sub-subtree1
| |- bn=sub-subtree2
| |- some entries
|- bn=subtree4
ou is an indexed attribute with pres,eq,sub. When now searching with the filter bn=<value> then there is a
significant difference when searching for either subtree1 and subtree2 as values in the range of seconds. This
means on command line ldapsearch with subtree1 it is around 0.007 seconds and with subtree2 4.5 seconds before
the fully finishes.
When I change the search filter to "(&(objectClass=bnClass)(bn=<value>))" then this has no real impact to the
time needed searching for subtree1 but improves searching subtree2, still it more than 2 times slower than
searching for subtree1. Only entries with objectClass=bnClass have the bn attribute but no other entries.
Changing the scope to one, yields similar times when searching for subtree1 and subtree2 but the search itself
also to cover searching for sub-subtree1 or sub-subtree2 with the same search task as it is not known where
such sub-subtree could be found.
The only pattern I've found so far is that if there are millions of entries in a subtree then finishing the
search takes much longer.
The LDAP is using MDB, and has been completely rebuilt (means slapcat/slapadd) several times with always
showing the same results. There was no difference between 2.4 and 2.5.13.
Is there something which can be improved by changing the configuration?
Thanks,
Norbert
7 months
pcache makes slapd die after some seconds
by Norbert
Hi,
when trying to use the pcache overlay for an MDB database instance, it seems to work, at least some first test
search seem to be significantly improved but then the slapd process without anything in the logs (no segfault
at least). Unfortunately it is not possible to restart the slapd process either, as it also soon dies after
starting.
The config used looks something like the following:
dn: olcOverlay={0}pcache,olcDatabase={1}mdb,cn=config
objectClass: olcOverlayConfig
objectClass: olcPcacheConfig
olcOverlay: {0}pcache
olcPcache: mdb 1000 1 1000 100
olcPcacheAttrset: 0 bn attribute1 attribute2 objectClass
olcPcacheTemplate: "(&(|(objectClass=)(objectClass=))(bn=))" 0 120
olcPcacheTemplate: "(bn=)" 0 120
dn: olcDatabase={0}mdb,olcOverlay={0}pcache,olcDatabase={1}mdb,cn=config
objectClass: olcMdbConfig
objectClass: olcPcacheDatabase
olcDatabase: {0}mdb
olcDbDirectory: /data/pcache
olcDbIndex: objectClass eq
olcDbIndex: bn pres,eq,sub
Is pcache supposed to work as an overlay for mdb database? Or does it only work with an ldap backend db?
Thanks,
Norbert
7 months
ipService anyone?
by Ulrich Windl
Hi!
Several years ago I added ipService to our LDAP Database, then I thought it's time to update it.
Now I have a conceptual problem:
Some services have multiple protocols and port numbers. For example "compressnet".
While it's possible to assign unique names like
cn=compressnet@2/tcp,...
cn=compressnet@2/tcp,...
cn=compressnet@3/tcp,...
cn=compressnet@3/udp,...
I wonder how a standard query for "compressnet" should look like.
Basically I'd like to leave out the port (@n), but the IANA registry is not unique.
I identified these:
compressnet
mit-ml-dev
sql-net
rap
meter
pip
csdmbase
csdm
nmsp
uma
raid-cd
...
optohost004
It seems ipService can have multiple values for ipServiceProtocol, cn, and ipServiceProtocol, but wouldn't that silently assume that all port numbers are valid for all listed protocols? For the example it would be any combination of (UDP,TCP) and (2,3).
To me the only solution seems to force IANA to clean up the mess.
RFC-6335/BCP-165 claims: "Service names are the unique key in the Service Name and Transport Protocol Port Number registry." (page 8)
Opinions?
Regards,
Ulrich
7 months, 1 week
Empty error when configuring OpenLDAP
by Cezary Drożak
Hello,
I am trying to set up OpenLDAP on Arch Linux on my server, following
instruction on Arch Wiki[1]. I prepared the config.ldif file, replacing
every $BASEDN and $PASSWD in the example configuration:
# The root config entry
dn: cn=config
objectClass: olcGlobal
cn: config
olcArgsFile: /run/openldap/slapd.args
olcPidFile: /run/openldap/slapd.pid
# Schemas
dn: cn=schema,cn=config
objectClass: olcSchemaConfig
cn: schema
# TODO: Include further schemas as necessary
include: file:///etc/openldap/schema/core.ldif
# The config database
dn: olcDatabase=config,cn=config
objectClass: olcDatabaseConfig
olcDatabase: config
olcRootDN: cn=Manager,dc=example,dc=com
# The database for our entries
dn: olcDatabase=mdb,cn=config
objectClass: olcDatabaseConfig
objectClass: olcMdbConfig
olcDatabase: mdb
olcSuffix: dc=example,dc=com
olcRootDN: cn=Manager,dc=example,dc=com
olcRootPW: {SSHA}xZqSQN4wG4+C5I57dB/Qm02vJ+kQcwd7
olcDbDirectory: /var/lib/openldap/openldap-data
# TODO: Create further indexes
olcDbIndex: objectClass eq
Then I executed the following command:
sudo -u ldap slapadd -n 0 -F /etc/openldap/slapd.d/ -l ./config.ldif
This gave me the following error:
invalid config directory /etc/openldap/slapd.d/, error 2
slapadd: bad configuration directory!
I checked that the directory did not exist, so I created it and changed
owner to `ldap`. The wiki page did not mention that the directory should
be created earlier, so maybe it should have been created by a post
installation script. If that's the case, I will report it to package
maintainers.
After I created the directory, I ran the command again, this time having
a different error message:
slapadd: could not add entry dn="cn=config" (line=1):
Closing DB...
I have no idea what is wrong now and I cannot find anything useful on
the internet. Does anyone have an idea what I may be doing wrong here?
[1]: https://wiki.archlinux.org/title/OpenLDAP
7 months, 1 week
slapo-variant and searching virtual attrs
by Michael Ströder
HI!
I have the need to search a whole sub-tree for something like collective
attributes which AFAIK slapo-collect does not support.
Now I'm wondering whether it's possible to search for the virtual
attributes generated by slapo-variant. And probably I'd like to use the
regex variant.
I've read the section LIMITATIONS in slapo-variant(5) which says that
regex variants only support reading the virtual attributes with scope
base. It also implies that searching for these virtual attributes is not
possible at all.
Is that correct?
Ciao, Michael.
7 months, 1 week
using memberof to authenticate Linux with PAM
by Alceu Rodrigues de Freitas Junior
Greetings,
For a matter of studying OpenLDAP, I decided to create a CLI in Golang
that is based on the migrationtools
(https://gitlab.com/future-ad-laboratory/migrationtools), which is
written in Bash and (very old) Perl code.
All the Golang module is available here:
https://github.com/glasswalk3r/aprendendo-openldap/tree/main/migration.
After learning about the memberof overlay, I've being wondering if it is
possible to use it to maintain the UNIX groups at /etc/group instead of
just replicating the same information over an over.
I've tried to find references in the documentation of using PAM and NSCD
in the Linux clients for authenticating from a OpenLDAP server, but
found nothing regarding those requirements, neither a detailed
explanation (without resorting looking into the source code) of how
those requests from a Linux client would be sent to OpenLDAP in order to
check that.
If any has any pointers on the subject, I would be glad to receive them.
Thanks in advance,
Alceu
7 months, 1 week
Proxying an insecure LDAP and authenticate using another
by Kevin Olbrich
Hi!
I have a case where I got access to a read-only LDAP that has no
authentication (everything is public). It is located in a secure and
trusted environment.
Now I want to expose the directory to another network and like to add
authentication but can not add it to the source (proprietary
software).
I need to proxy this server and authenticate the users using active directory.
I know how to proxy AD including passthrough authentication but can I
proxy another directory and authenticate using another? If yes, how?
Thank you very much.
Kind regards
Kevin Olbrich
7 months, 1 week
Re: Pass-through
by Stéphane Veyret
Could it be that the SASL global configuration (also given in first
message) is wrong? I only set those 2 options:
olcSaslHost: localhost
olcSaslSecProps: none
7 months, 1 week
