dynlist vs memberof performance issues
by Mark Cairney
Hi,
I've been out the LDAP loop for a bit but the recent discussion of the
memberof overlay on 2.5 piqued my curiosity. Having upgraded a Dev box,
removed the memberof elements from the database and replaced the
memberof overlay with dynlist the queries appear to work as expected but
are both a) slow and b) heavily CPU-intensive on the LDAP server.
2021-09-01T12:47:17.603513+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 fd=12 ACCEPT from IP=192.168.152.33:58738
(IP=129.215.17.9:636)
2021-09-01T12:47:17.687488+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 fd=12 TLS established tls_ssf=256 ssf=256
tls_proto=TLSv1.3 tls_cipher=TLS_AES_256_GCM_SHA384
2021-09-01T12:47:17.688032+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=0 SRCH base="" scope=0 deref=0
filter="(objectClass=*)"
2021-09-01T12:47:17.688470+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=0 SRCH attr=supportedSASLMechanisms
2021-09-01T12:47:17.688878+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=0 SEARCH RESULT tag=101 err=0 qtime=0.000014
etime=0.000214 nentries=1 text=
2021-09-01T12:47:17.811279+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=1 BIND dn="" method=163
2021-09-01T12:47:17.819249+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=1 RESULT tag=97 err=14 qtime=0.000030
etime=0.009084 text=SASL(0): successful result:
2021-09-01T12:47:17.908889+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=2 BIND dn="" method=163
2021-09-01T12:47:17.909836+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=2 RESULT tag=97 err=14 qtime=0.000031
etime=0.000181 text=SASL(0): successful result:
2021-09-01T12:47:17.938839+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=3 BIND dn="" method=163
2021-09-01T12:47:17.939621+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=3 BIND authcid="mcairney(a)EASE.ED.AC.UK"
authzid="mcairney(a)EASE.ED.AC.UK"
2021-09-01T12:47:17.940213+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=3 BIND
dn="uid=mcairney,ou=people,ou=central,dc=authorise-dev,dc=ed,dc=ac,dc=uk"
mech=GSSAPI bind_ssf=256 ssf=256
2021-09-01T12:47:17.940616+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=3 RESULT tag=97 err=0 qtime=0.000024
etime=0.000409 text=
2021-09-01T12:47:18.227342+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=4 SRCH
base="dc=authorise-dev,dc=ed,dc=ac,dc=uk" scope=2 deref=0
filter="(uid=mcairney)"
2021-09-01T12:47:18.227703+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=4 SRCH attr=* +
2021-09-01T12:47:31.392255+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=5 UNBIND
2021-09-01T12:47:31.460705+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=4 SEARCH RESULT tag=101 err=0 qtime=0.000031
etime=13.233679 nentries=1 text=
2021-09-01T12:47:31.461098+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 fd=12 closed
I'm guessing that as the values are computed that this will be heavier
on the CPU but it seems a bit excessive? Has anyone else noticed any
similar performance issues?
This is a relatively low-specced DEV server (2 vCPUs, 4GB RAM) so I
guess this could be a factor but there's no io waiting on the server and
no swapping?
The database is on a par in size with our Production service ( about
400K user objects with 1 group object per user and then about 80K actual
groups of users)
The config for the primary DB (ACLs and rootPW redacted) is:
dn: olcDatabase={1}mdb,cn=config
objectClass: olcDatabaseConfig
objectClass: olcMdbConfig
olcDatabase: {1}mdb
olcDbDirectory: /opt/openldap/var/openldap-data/authorise
olcSuffix: dc=authorise-dev,dc=ed,dc=ac,dc=uk
olcAddContentAcl: FALSE
olcLastMod: TRUE
olcMaxDerefDepth: 2
olcReadOnly: FALSE
olcSecurity: ssf=1
olcSecurity: update_ssf=112
olcSecurity: simple_bind=64
olcSizeLimit: unlimited
olcSyncUseSubentry: FALSE
olcTimeLimit: unlimited
olcMonitoring: TRUE
olcDbEnvFlags: writemap
olcDbEnvFlags: nometasync
olcDbNoSync: FALSE
olcDbIndex: objectClass eq
olcDbIndex: entryUUID eq
olcDbIndex: entryCSN eq
olcDbIndex: cn pres,eq,sub
olcDbIndex: uid pres,eq,sub
olcDbIndex: uidNumber pres,eq
olcDbIndex: gidNumber pres,eq
olcDbIndex: eduniType eq
olcDbIndex: gecos pres,eq,sub
olcDbIndex: eduniCategory eq
olcDbIndex: mail pres,eq,sub
olcDbIndex: eduniSchoolCode eq
olcDbIndex: eduniIDStatus eq
olcDbIndex: eduniCollegeCode eq
olcDbIndex: eduniOrgCode eq
olcDbIndex: memberOf pres,eq
olcDbIndex: eduniLibraryBarcode pres,eq
olcDbIndex: eduniOrganisation pres,eq,sub
olcDbIndex: eduniServiceCode pres,eq
olcDbIndex: krbName pres,eq
olcDbIndex: eduPersonAffiliation pres,eq
olcDbIndex: eduPersonEntitlement pres,eq
olcDbIndex: sn pres,eq,sub
olcDbIndex: eduniIdmsId pres,eq
olcDbIndex: member pres,eq
olcDbIndex: memberUid pres,eq
olcDbIndex: eduniRefNo pres,eq
olcDbIndex: eduniTitle pres,eq
olcDbIndex: title pres,eq,sub
olcDbIndex: eduniCardNumber pres,eq
olcDbIndex: eduniYearOfStudy eq
olcDbIndex: description pres,eq,sub
olcDbIndex: givenName pres,eq,sub
olcDbIndex: aliasedObjectName eq
olcDbIndex: yubiKeyId pres,eq
olcDbIndex: isMemberOf pres,eq
olcDbIndex: hasMember pres,eq
olcDbIndex: proxyAddresses pres,eq,sub
olcDbMaxReaders: 96
olcDbMaxSize: 32212254720
olcDbMode: 0600
olcDbSearchStack: 16
structuralObjectClass: olcMdbConfig
dn: olcOverlay={0}syncprov,olcDatabase={1}mdb,cn=config
objectClass: olcOverlayConfig
objectClass: olcConfig
objectClass: top
objectClass: olcSyncProvConfig
olcOverlay: {0}syncprov
structuralObjectClass: olcSyncProvConfig
dn: olcOverlay={1}accesslog,olcDatabase={1}mdb,cn=config
objectClass: olcOverlayConfig
objectClass: olcAccessLogConfig
olcOverlay: {1}accesslog
olcAccessLogDB: cn=accesslog
olcAccessLogOps: writes
olcAccessLogPurge: 02+00:00 00+04:00
olcAccessLogSuccess: TRUE
structuralObjectClass: olcAccessLogConfig
dn: olcOverlay={2}dynlist,olcDatabase={1}mdb,cn=config
objectClass: olcOverlayConfig
objectClass: olcDynListConfig
olcOverlay: {2}dynlist
olcDynListAttrSet: {0}groupOfURLs memberURL member+memberOf@groupOfNames
structuralObjectClass: olcDynListConfig
--
/****************************
Mark Cairney
ITI Enterprise Services
Information Services
University of Edinburgh
Tel: 0131 650 6565
Email: Mark.Cairney(a)ed.ac.uk
*******************************/
The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. Is e buidheann carthannais a th’ ann an Oilthigh Dhùn Èideann, clàraichte an Alba, àireamh clàraidh SC005336.
4 months, 2 weeks
slow memberOf queries in 2.5 with dynlist overlay
by Paul B. Henson
I'm still trying to figure out why my servers sometimes get into a state
where queries requesting the memberOf attribute take an exceedingly long
time to process, for example:
Feb 13 19:46:05 ldap-02 slapd[13564]: conn=297643 fd=104 ACCEPT from PATH=/var/symas/run/ldapi (PATH=/var/symas/run/ldapi)
Feb 13 19:46:05 ldap-02 slapd[13564]: conn=297643 op=0 BIND dn="cn=ldaproot,dc=cpp,dc=edu" method=128
Feb 13 19:46:05 ldap-02 slapd[13564]: conn=297643 op=0 BIND dn="cn=ldaproot,dc=cpp,dc=edu" mech=SIMPLE bind_ssf=0 ssf=71
Feb 13 19:46:05 ldap-02 slapd[13564]: conn=297643 op=0 RESULT tag=97 err=0 qtime=0.000006 etime=0.000050 text=
Feb 13 19:46:05 ldap-02 slapd[13564]: conn=297643 op=1 SRCH base="dc=cpp,dc=edu" scope=2 deref=0 filter="(uid=henson)"
Feb 13 19:46:05 ldap-02 slapd[13564]: conn=297643 op=1 SRCH attr=memberOf
Feb 13 19:46:42 ldap-02 slapd[13564]: conn=297643 op=1 SEARCH RESULT tag=101 err=0 qtime=0.000012 etime=36.955710 nentries=1 text=
How is the qtime calculated? It is nice and short, despite the wall clock
reading over 30 seconds :(.
Usually I have to reboot the server completely to clear this up, but this
time I just had to restart it, and then the queries were lickity split
again:
Feb 13 19:55:01 ldap-02 slapd[89655]: conn=1556 fd=40 ACCEPT from PATH=/var/symas/run/ldapi (PATH=/var/symas/run/ldapi)
Feb 13 19:55:01 ldap-02 slapd[89655]: conn=1556 op=0 BIND dn="cn=ldaproot,dc=cpp,dc=edu" method=128
Feb 13 19:55:01 ldap-02 slapd[89655]: conn=1556 op=0 BIND dn="cn=ldaproot,dc=cpp,dc=edu" mech=SIMPLE bind_ssf=0 ssf=71
Feb 13 19:55:01 ldap-02 slapd[89655]: conn=1556 op=0 RESULT tag=97 err=0 qtime=0.000009 etime=0.000088 text=
Feb 13 19:55:01 ldap-02 slapd[89655]: conn=1556 op=1 SRCH base="dc=cpp,dc=edu" scope=2 deref=0 filter="(uid=henson)"
Feb 13 19:55:01 ldap-02 slapd[89655]: conn=1556 op=1 SRCH attr=memberOf
Feb 13 19:55:01 ldap-02 slapd[89655]: conn=1556 op=1 SEARCH RESULT tag=101 err=0 qtime=0.000013 etime=0.213149 nentries=1 text=
Over 30 seconds elapsed time to .2 seconds? I'd like to see the .2 all
the time :).
When the server gets like this, there's a very high read IO load, 200-300Mb/s,
compared to generally less than 1Mb/s when things are working right.
It almost seems like it's doing a full disk scan searching for members
every time.
Any suggestions on what to dig into?
Thanks...
12 months
STARTTLS vs LDAPS
by thomaswilliampritchard＠gmail.com
At risk of beating a dead horse, I'd like to hear considerations on STARTTLS vs LDAPS. I'm also particularly interested if openldap plans to support LDAPS long term or if there's actually a deprecation effort going on around LDAPS where it would one day no longer be supported by openldap.
This seems to be the most comprehensive post discussing the virtue of the two. https://security.stackexchange.com/questions/257749/is-ldaps-or-starttls-...
I also found a post in this Archive from 2018 that seems to indicate a change of opinion where LDAPS should be preferred, and not deprecated.
https://lists.openldap.org/hyperkitty/list/openldap-technical@openldap.or...
Does openldap agree that LDAPS should now be the preferred implementation and STARTLS should be discouraged?
I do not have a security background and there is certainly a lot of room for me to misunderstand, but it seems like STARTTLS leaves the door open for a "tls downgrade attack" where a man in the middle could essentially reply to a client effectively saying start tls is not supported and then the client falls back to non tls communication (which is obviously unfortunate). Even if the backend server is properly not responding to clients until STARTTLS is initiated, the man in the middle could initiate a connection with STARTTLS to the ldap server and be talking plaintext to the client. Is that legitimately possible or am I missing a nuance? If one were to only support clients over LDAPS it seems this would be mitigated?
Thanks for the considerations, looking forward to hearing the expert opinions on the topic.
1 year, 1 month
delete glue entry
by Michael Ströder
HI!
Had a MDB database with a glue entry in it on all replicas in a
multi-provider setup (release 2.6.1). I could not update this entry anymore.
Is it possible to delete a glue entry via LDAP? All subordinate entries
were already removed before.
Ciao, Michael.
1 year, 1 month
RE26 testing call #1 (OpenLDAP 2.6.2)
by Quanah Gibson-Mount
This is the first testing call for OpenLDAP 2.6.2. Depending on the
results, this may be the only testing call.
Generally, get the code for RE26:
<https://git.openldap.org/openldap/openldap/-/archive/OPENLDAP_REL_ENG_2_6...>
Extract, configure, and build.
Execute the test suite (via make test) after it is built. Optionally, cd
tests && make its to run through the regression suite.
Thanks!
OpenLDAP 2.6.2 Engineering
Added libldap support for OpenSSL 3.0 (ITS#9436)
Added slapd support for OpenSSL 3.0 (ITS#9436)
Fixed ldapdelete to prune LDAP subentries (ITS#9737)
Fixed libldap to drop connection when non-LDAP data is received
(ITS#9803)
Fixed libldap to allow newlines at end of included file (ITS#9811)
Fixed slapd slaptest conversion of olcLastBind (ITS#9808)
Fixed slapd to correctly init global_host earlier (ITS#9787)
Fixed slapd bconfig locking for cn=config replication (ITS#9584)
Fixed slapd usage of thread local counters (ITS#9789)
Fixed slapd to clear runqueue task correctly (ITS#9785)
Fixed slapd syncrepl handling of new sessions (ITS#9584)
Fixed slapd to clear connections on bind (ITS#9799)
Fixed slapd syncrepl ODSEE replication of unknown attr (ITS#9801)
Fixed slapd-asyncmeta memory leak in keepalive setting (ITS#9802)
Fixed slapd-ldap memory leak in keepalive setting (ITS#9802)
Fixed slapd-meta SEGV on config rewrite (ITS#9802)
Fixed slapd-meta ordering on config rewrite (ITS#9802)
Fixed slapd-meta memory leak in keepalive setting (ITS#9802)
Fixed slapd-monitor SEGV on shutdown (ITS#9809)
Added slapo-autoca support for OpenSSL 3.0 (ITS#9436)
Added slapo-otp support for OpenSSL 3.0 (ITS#9436)
Fixed slapo-pcache SEGV on shutdown (ITS#9809)
Fixed slapo-ppolicy operation handling to be consistent (ITS#9794)
Build Enviornment
Add ability to override default compile time paths (ITS#9675)
Fix compiliation with certain versions of gcc (ITS#9790)
Fix compilation with openssl exclusions (ITS#9791)
Fix warnings from make jobserver (ITS#9788)
Documentation
admin26 Document new lloadd features (ITS#9780)
Fixed slapd.conf(5)/slapd-config(5) syncrepl sizelimit/timelimit
documentation (ITS#9804)
Fixed slapd-sock(5) to clarify "sockresps result" behavior
(ITS#8255)
Regards,
Quanah
1 year, 1 month
RE25 Testing call #1 (OpenLDAP 2.5.12)
by Quanah Gibson-Mount
This is the first testing call for OpenLDAP 2.5.12. Depending on the
results, this may be the only testing call.
Generally, get the code for RE25:
<https://git.openldap.org/openldap/openldap/-/archive/OPENLDAP_REL_ENG_2_5...>
Extract, configure, and build.
Execute the test suite (via make test) after it is built. Optionally, cd
tests && make its to run through the regression suite.
Thanks!
OpenLDAP 2.5.12 Engineering
Fixed libldap to drop connection when non-LDAP data is received
(ITS#9803)
Fixed libldap to allow newlines at end of included file (ITS#9811)
Fixed slapd slaptest conversion of olcLastBind (ITS#9808)
Fixed slapd usage of thread local counters (ITS#9789)
Fixed slapd to clear runqueue task correctly (ITS#9785)
Fixed slapd bconfig locking for cn=config replication (ITS#9584)
Fixed slapd syncrepl handling of new sessions (ITS#9584)
Fixed slapd to clear connections on bind (ITS#9799)
Fixed slapd syncrepl ODSEE replication of unknown attr (ITS#9801)
Fixed slapd-asyncmeta memory leak in keepalive setting (ITS#9802)
Fixed slapd-ldap memory leak in keepalive setting (ITS#9802)
Fixed slapd-meta SEGV on config rewrite (ITS#9802)
Fixed slapd-meta ordering on config rewrite (ITS#9802)
Fixed slapd-meta memory leak in keepalive setting (ITS#9802)
Fixed slapd-monitor SEGV on shutdown (ITS#9809)
Fixed slapo-pcache SEGV on shutdown (ITS#9809)
Fixed slapo-ppolicy operation handling to be consistent (ITS#9794)
Build Environment
Fix compilation with openssl exclusions (ITS#9791)
Fix warnings from make jobserver (ITS#9788)
Fix compiliation with certain versions of gcc (ITS#9790)
Documentation
Fixed slapd.conf(5)/slapd-config(5) syncrepl sizelimit/timelimit
documentation (ITS#9804)
Regards,
Quanah
1 year, 1 month
Re: STARTTLS vs LDAPS
by Braiam
On Thu, Mar 31, 2022 at 11:46 AM Quanah Gibson-Mount <quanah(a)fast-mail.org>
wrote:
>
>
> --On Thursday, March 31, 2022 12:16 PM -0400 Braiam <braiamp(a)gmail.com>
> wrote:
>
> > What would be the process to modify content on the openldap.org page?
>
> Depends on the content. The main web pages are in the OpenLDAP Web git
> repository.
>
This repository seems to be the FAQ:
https://git.openldap.org/project/faq/-/tree/master/data/item
There's a website project too. I couldn't figure out what markup language
items
on the faq project uses.
> --Quanah
>
--
Braiam
1 year, 2 months
multiple TOTP token for one user in otp overlay slapo-otp
by Rene Zeipelt
Hello,
is it possible to save multiple TOTP token (with different sha
algorithms and different secrets) to one user with the otp overlay? In
the totp module multi values like {TOTP1}XXXXXXX and {TOTP256}XXXXXX are
working. Thanks for any help.
Best regards
Rene Zeipelt
ps. system is debian bookworm with slapd 2.5.11
--
_________________________________________________________
BERGISCHE UNIVERSITÄT WUPPERTAL
Zentrum fuer Informations- und Medienverarbeitung - ZIM
1 year, 2 months
Deny write access to new accounts
by beren beren
Hi.
Is it possible to make admin Bob unable to edit accounts (delete, create,
change passwords)created this year ? There is an idea to move them to a
group or OU and give Bob the rights to write only there. Is there a more
elegant solution ?
1 year, 2 months
olclimit dn.base <> group/groupOfNames/member
by bourguijl＠gmail.com
Dears,
Openldap version : 2.5.7
env 2 MMR 2 Replicas (test env)
I've set and olclimit for one user (dn.base) on my DB and it works fine but in order to move it on my production env, I decided to modify my olclimit by using (group/groupOfNames/member) and place this user as member of the group. This is also works fine on my test env.
I did the same config on my production env which is 4 MMR 4 Replicas and it didn't work :-(
I did a lot of checks to see if there was any difference but it was exactly the same configuration.
I did some other test on replicas first by adding a new olclimit for the concerned user ( dn.base) which solved the issue.
I decided to remove this newly user olclimit, the olclimit (group/groupOfNames/member) was still there, and was not my surprise, the limitation for my user was still set to unlimited as expected.
I did the same on all replicas, adding concerned user, remove it and limits were OK .... very strange.
As it was working on replicas, I did try the same on master but no luck, my user stay still limited to 500 entries.
Questions :
Is there an order to respect in olclimit type ?
why the config is working on test env and not on production one ?
Thx to advice,
Jean-Luc
1 year, 2 months
