dynlist vs memberof performance issues
by Mark Cairney
Hi,
I've been out the LDAP loop for a bit but the recent discussion of the
memberof overlay on 2.5 piqued my curiosity. Having upgraded a Dev box,
removed the memberof elements from the database and replaced the
memberof overlay with dynlist the queries appear to work as expected but
are both a) slow and b) heavily CPU-intensive on the LDAP server.
2021-09-01T12:47:17.603513+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 fd=12 ACCEPT from IP=192.168.152.33:58738
(IP=129.215.17.9:636)
2021-09-01T12:47:17.687488+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 fd=12 TLS established tls_ssf=256 ssf=256
tls_proto=TLSv1.3 tls_cipher=TLS_AES_256_GCM_SHA384
2021-09-01T12:47:17.688032+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=0 SRCH base="" scope=0 deref=0
filter="(objectClass=*)"
2021-09-01T12:47:17.688470+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=0 SRCH attr=supportedSASLMechanisms
2021-09-01T12:47:17.688878+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=0 SEARCH RESULT tag=101 err=0 qtime=0.000014
etime=0.000214 nentries=1 text=
2021-09-01T12:47:17.811279+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=1 BIND dn="" method=163
2021-09-01T12:47:17.819249+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=1 RESULT tag=97 err=14 qtime=0.000030
etime=0.009084 text=SASL(0): successful result:
2021-09-01T12:47:17.908889+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=2 BIND dn="" method=163
2021-09-01T12:47:17.909836+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=2 RESULT tag=97 err=14 qtime=0.000031
etime=0.000181 text=SASL(0): successful result:
2021-09-01T12:47:17.938839+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=3 BIND dn="" method=163
2021-09-01T12:47:17.939621+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=3 BIND authcid="mcairney(a)EASE.ED.AC.UK"
authzid="mcairney(a)EASE.ED.AC.UK"
2021-09-01T12:47:17.940213+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=3 BIND
dn="uid=mcairney,ou=people,ou=central,dc=authorise-dev,dc=ed,dc=ac,dc=uk"
mech=GSSAPI bind_ssf=256 ssf=256
2021-09-01T12:47:17.940616+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=3 RESULT tag=97 err=0 qtime=0.000024
etime=0.000409 text=
2021-09-01T12:47:18.227342+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=4 SRCH
base="dc=authorise-dev,dc=ed,dc=ac,dc=uk" scope=2 deref=0
filter="(uid=mcairney)"
2021-09-01T12:47:18.227703+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=4 SRCH attr=* +
2021-09-01T12:47:31.392255+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=5 UNBIND
2021-09-01T12:47:31.460705+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=4 SEARCH RESULT tag=101 err=0 qtime=0.000031
etime=13.233679 nentries=1 text=
2021-09-01T12:47:31.461098+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 fd=12 closed
I'm guessing that as the values are computed that this will be heavier
on the CPU but it seems a bit excessive? Has anyone else noticed any
similar performance issues?
This is a relatively low-specced DEV server (2 vCPUs, 4GB RAM) so I
guess this could be a factor but there's no io waiting on the server and
no swapping?
The database is on a par in size with our Production service ( about
400K user objects with 1 group object per user and then about 80K actual
groups of users)
The config for the primary DB (ACLs and rootPW redacted) is:
dn: olcDatabase={1}mdb,cn=config
objectClass: olcDatabaseConfig
objectClass: olcMdbConfig
olcDatabase: {1}mdb
olcDbDirectory: /opt/openldap/var/openldap-data/authorise
olcSuffix: dc=authorise-dev,dc=ed,dc=ac,dc=uk
olcAddContentAcl: FALSE
olcLastMod: TRUE
olcMaxDerefDepth: 2
olcReadOnly: FALSE
olcSecurity: ssf=1
olcSecurity: update_ssf=112
olcSecurity: simple_bind=64
olcSizeLimit: unlimited
olcSyncUseSubentry: FALSE
olcTimeLimit: unlimited
olcMonitoring: TRUE
olcDbEnvFlags: writemap
olcDbEnvFlags: nometasync
olcDbNoSync: FALSE
olcDbIndex: objectClass eq
olcDbIndex: entryUUID eq
olcDbIndex: entryCSN eq
olcDbIndex: cn pres,eq,sub
olcDbIndex: uid pres,eq,sub
olcDbIndex: uidNumber pres,eq
olcDbIndex: gidNumber pres,eq
olcDbIndex: eduniType eq
olcDbIndex: gecos pres,eq,sub
olcDbIndex: eduniCategory eq
olcDbIndex: mail pres,eq,sub
olcDbIndex: eduniSchoolCode eq
olcDbIndex: eduniIDStatus eq
olcDbIndex: eduniCollegeCode eq
olcDbIndex: eduniOrgCode eq
olcDbIndex: memberOf pres,eq
olcDbIndex: eduniLibraryBarcode pres,eq
olcDbIndex: eduniOrganisation pres,eq,sub
olcDbIndex: eduniServiceCode pres,eq
olcDbIndex: krbName pres,eq
olcDbIndex: eduPersonAffiliation pres,eq
olcDbIndex: eduPersonEntitlement pres,eq
olcDbIndex: sn pres,eq,sub
olcDbIndex: eduniIdmsId pres,eq
olcDbIndex: member pres,eq
olcDbIndex: memberUid pres,eq
olcDbIndex: eduniRefNo pres,eq
olcDbIndex: eduniTitle pres,eq
olcDbIndex: title pres,eq,sub
olcDbIndex: eduniCardNumber pres,eq
olcDbIndex: eduniYearOfStudy eq
olcDbIndex: description pres,eq,sub
olcDbIndex: givenName pres,eq,sub
olcDbIndex: aliasedObjectName eq
olcDbIndex: yubiKeyId pres,eq
olcDbIndex: isMemberOf pres,eq
olcDbIndex: hasMember pres,eq
olcDbIndex: proxyAddresses pres,eq,sub
olcDbMaxReaders: 96
olcDbMaxSize: 32212254720
olcDbMode: 0600
olcDbSearchStack: 16
structuralObjectClass: olcMdbConfig
dn: olcOverlay={0}syncprov,olcDatabase={1}mdb,cn=config
objectClass: olcOverlayConfig
objectClass: olcConfig
objectClass: top
objectClass: olcSyncProvConfig
olcOverlay: {0}syncprov
structuralObjectClass: olcSyncProvConfig
dn: olcOverlay={1}accesslog,olcDatabase={1}mdb,cn=config
objectClass: olcOverlayConfig
objectClass: olcAccessLogConfig
olcOverlay: {1}accesslog
olcAccessLogDB: cn=accesslog
olcAccessLogOps: writes
olcAccessLogPurge: 02+00:00 00+04:00
olcAccessLogSuccess: TRUE
structuralObjectClass: olcAccessLogConfig
dn: olcOverlay={2}dynlist,olcDatabase={1}mdb,cn=config
objectClass: olcOverlayConfig
objectClass: olcDynListConfig
olcOverlay: {2}dynlist
olcDynListAttrSet: {0}groupOfURLs memberURL member+memberOf@groupOfNames
structuralObjectClass: olcDynListConfig
--
/****************************
Mark Cairney
ITI Enterprise Services
Information Services
University of Edinburgh
Tel: 0131 650 6565
Email: Mark.Cairney(a)ed.ac.uk
*******************************/
The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. Is e buidheann carthannais a th’ ann an Oilthigh Dhùn Èideann, clàraichte an Alba, àireamh clàraidh SC005336.
3 weeks, 1 day
slow memberOf queries in 2.5 with dynlist overlay
by Paul B. Henson
I'm still trying to figure out why my servers sometimes get into a state
where queries requesting the memberOf attribute take an exceedingly long
time to process, for example:
Feb 13 19:46:05 ldap-02 slapd[13564]: conn=297643 fd=104 ACCEPT from PATH=/var/symas/run/ldapi (PATH=/var/symas/run/ldapi)
Feb 13 19:46:05 ldap-02 slapd[13564]: conn=297643 op=0 BIND dn="cn=ldaproot,dc=cpp,dc=edu" method=128
Feb 13 19:46:05 ldap-02 slapd[13564]: conn=297643 op=0 BIND dn="cn=ldaproot,dc=cpp,dc=edu" mech=SIMPLE bind_ssf=0 ssf=71
Feb 13 19:46:05 ldap-02 slapd[13564]: conn=297643 op=0 RESULT tag=97 err=0 qtime=0.000006 etime=0.000050 text=
Feb 13 19:46:05 ldap-02 slapd[13564]: conn=297643 op=1 SRCH base="dc=cpp,dc=edu" scope=2 deref=0 filter="(uid=henson)"
Feb 13 19:46:05 ldap-02 slapd[13564]: conn=297643 op=1 SRCH attr=memberOf
Feb 13 19:46:42 ldap-02 slapd[13564]: conn=297643 op=1 SEARCH RESULT tag=101 err=0 qtime=0.000012 etime=36.955710 nentries=1 text=
How is the qtime calculated? It is nice and short, despite the wall clock
reading over 30 seconds :(.
Usually I have to reboot the server completely to clear this up, but this
time I just had to restart it, and then the queries were lickity split
again:
Feb 13 19:55:01 ldap-02 slapd[89655]: conn=1556 fd=40 ACCEPT from PATH=/var/symas/run/ldapi (PATH=/var/symas/run/ldapi)
Feb 13 19:55:01 ldap-02 slapd[89655]: conn=1556 op=0 BIND dn="cn=ldaproot,dc=cpp,dc=edu" method=128
Feb 13 19:55:01 ldap-02 slapd[89655]: conn=1556 op=0 BIND dn="cn=ldaproot,dc=cpp,dc=edu" mech=SIMPLE bind_ssf=0 ssf=71
Feb 13 19:55:01 ldap-02 slapd[89655]: conn=1556 op=0 RESULT tag=97 err=0 qtime=0.000009 etime=0.000088 text=
Feb 13 19:55:01 ldap-02 slapd[89655]: conn=1556 op=1 SRCH base="dc=cpp,dc=edu" scope=2 deref=0 filter="(uid=henson)"
Feb 13 19:55:01 ldap-02 slapd[89655]: conn=1556 op=1 SRCH attr=memberOf
Feb 13 19:55:01 ldap-02 slapd[89655]: conn=1556 op=1 SEARCH RESULT tag=101 err=0 qtime=0.000013 etime=0.213149 nentries=1 text=
Over 30 seconds elapsed time to .2 seconds? I'd like to see the .2 all
the time :).
When the server gets like this, there's a very high read IO load, 200-300Mb/s,
compared to generally less than 1Mb/s when things are working right.
It almost seems like it's doing a full disk scan searching for members
every time.
Any suggestions on what to dig into?
Thanks...
8 months, 1 week
rewriting non-DN attribute values with rwm?
by Kartik Subbarao
I'd like to rewrite the following entry:
dn: uid=user(a)olddomain.com,dc=olddomain,dc=com
uid: user(a)olddomain.com
mail: user(a)olddomain.com
to appear and behave like this:
dn: uid=user(a)newdomain.com,dc=newdomain,dc=com
uid: user(a)newdomain.com
mail: user(a)newdomain.com
I can get the DN rewritten with slapd-relay and rwm-suffixmassage, and
can use rwm-rewriterule with the searchFilter and searchEntryDN contexts
to return the entry when querying for mail=user(a)newdomain.com. But I
can't figure out how to rewrite the *values* of the uid and mail
attributes in the returned entry to user(a)newdomain.com. What is the best
way to achieve this?
Thanks,
-Kartik
8 months, 1 week
result not in cache
by Stefan Kania
I'm testing the openldap cache module pcache with OpenLDAP 2.6 on
Debian11 (symas-packages). The proxy has the following config:
(I'm testing caching so no security is set)
--------------
include /opt/symas/etc/openldap/schema/core.schema
include /opt/symas/etc/openldap/schema/cosine.schema
include /opt/symas/etc/openldap/schema/nis.schema
include /opt/symas/etc/openldap/schema/inetorgperson.schema
pidfile /var/symas/run/slapd.pid
argsfile /var/symas/run/slapd.args
loglevel any
modulepath /opt/symas/lib/openldap
moduleload back_mdb.la
moduleload argon2.la
moduleload back_ldap
moduleload pcache
sizelimit 500
tool-threads 1
database ldap
suffix "dc=example,dc=net"
uri "ldap://ldap-server.example.net"
rootdn "cn=admin,dc=example,dc=net"
protocol-version 3
rebind-as-user
overlay pcache
pcachePersist TRUE
pcache mdb 100000 2 1000 100
pcacheAttrset 0 mail postaladdress telephonenumber givenname
pcacheAttrset 1 uid employeetype
pcacheTemplate (sn=) 0 3600
pcacheTemplate (&(sn=)(givenName=)) 0 3600
pcacheTemplate (&(departmentNumber=)(secretary=*)) 0 3600
directory /var/symas/cache
index objectclass eq
index uid,cn,sn,mail,givenname pres,eq,sub
--------------
The following host are involved:
ldap-server<----->ldap-proxy<----->ldap-client
The ldap-client can only access the ldap-proxy. (ldap.conf ist pointing
to the ldap-proxy) Now I do a:
ldapsearch -x '(&(sn=Kania)(givenName=Stefan))' givenname
The first time I can see that the proxy is asking the ldap-server and is
giving the result to the ldap-client.
Each time I repeat the command on the ldap-client, only the log from the
ldap-proxy is showing the access from the ldap-client. The ldap-client
is getting the result from the proxy.
I can even shutdown the ldap-server and the client is still getting the
result from the proxy. Up to this point I understand the log but if I
set "loglevel any" I see:
------------
access_allowed: result not in cache (givenName)
------------
But I think the result IS in the cache otherwise I would not get the
result with the ldap-server turned off.
So why do I get this messages?
8 months, 1 week
Made a change to cn=schema,cn=config, can't revert?
by Sander Smeenk
Hi,
I'm trying to add an 'olcAttributeType' to an already existing LDAP
schema. I wrote this LDIF, which was wrong as it turns out:
| dn: cn=schema,cn=config
| changetype: modify
| add: olcAttributetypes
| olcAttributetypes: ( bitHttpAttribute:26.2008.07.29.1 NAME
| 'testEntry' DESC 'Test' EQUALITY integerMatch SYNTAX
| LDAPInteger SINGLE-VALUE )
This changed my 'dn: cn=schema,cn=config' from:
| dn: cn=schema,cn=config
| objectClass: olcSchemaConfig
| cn: schema
| structuralObjectClass: olcSchemaConfig
| entryUUID: 9faa0e66-41f5-1032-82e1-974d55020d34
| creatorsName: cn=config
| createTimestamp: 20130425131311Z
| entryCSN: 20130425131311.204787Z#000000#000#000000
| modifiersName: cn=config
| modifyTimestamp: 20130425131311Z
To:
| dn: cn=schema,cn=config
| objectClass: olcSchemaConfig
| cn: schema
| structuralObjectClass: olcSchemaConfig
| entryUUID: 9faa0e66-41f5-1032-82e1-974d55020d34
| creatorsName: cn=config
| createTimestamp: 20130425131311Z
| olcAttributeTypes: {0}( bitHttpAttribute:26.2008.07.29.1 NAME
| 'testEntry' DESC 'Test' EQUALITY integerMatch SYNTAX
| LDAPInteger SINGLE-VALUE )
| entryCSN: 20220525132055.795116Z#000000#000#000000
| modifiersName: gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth
| modifyTimestamp: 20220525132055Z
Which is not what i wanted but it happened, so now i want it gone.
But it seems i can't remove that olcAttributeTypes entry?
This LDIF won't work:
| dn: cn=schema,cn=config
| changetype: modify
| delete: olcAttributeTypes
Yielding:
| # ldapmodify -Y EXTERNAL -H ldapi:/// -f kak.ldif
| [..]
| modifying entry "cn=schema,cn=config"
| ldap_modify: Other (e.g., implementation specific) error (80)
When i put in the exact olcAttributeType line i want to remove (instead
of all of 'em) like this:
| dn: cn=schema,cn=config
| changetype: modify
| delete: olcAttributeTypes
| olcAttributeTypes: ( bitHttpAttribute:26.2008.07.29.1 NAME 'testEntry' DESC 'Test' EQUALITY integerMatch SYNTAX LDAPInteger SINGLE-VALUE )
Now the same ldapmodify command yields:
| # ldapmodify -Y EXTERNAL -H ldapi:/// -f kak.ldif
| [..]
| modifying entry "cn=schema,cn=config"
| ldap_modify: Operations error (1)
Can anyone explain what i am doing wrong and how to revert my change
to the 'cn=schema,cn=config' dn?
Thanks a bunch!
Regards,
-Sander.
--
| I did a theatrical performance about puns. It was a play on words.
| 4096R/20CC6CD2 - 6D40 1A20 B9AA 87D4 84C7 FBD6 F3A9 9442 20CC 6CD2
8 months, 1 week
consumer state is newer than provider
by Rallavagu Kon
Hello All,
Running OpenLDAP version 2.4.58 on Ubuntu 20.04 in multicaster mode.
Replication Configuration:
olcSyncrepl: {0}rid=101 provider=ldaps://<host1>:10636 binddn="<binddn>" bindm
ethod=simple credentials=<creds> searchbase="dc=company,dc=com" type
=refreshAndPersist interval=00:00:00:30 retry="5 5 60 +" timeout=1 keepalive=
"240:10:30"
olcSyncrepl: {1}rid=102 provider=ldaps://<host2>:10636 binddn="<binddn>" bindm
ethod=simple credentials=<creds> searchbase="dc=company,dc=com" type
=refreshAndPersist interval=00:00:00:30 retry="5 5 60 +" timeout=1 keepalive=
"240:10:30"
olcSyncrepl: {2}rid=103 provider=ldaps://<host3>:10636 binddn="<binddn>" bindm
ethod=simple credentials=<creds> searchbase="dc=company,dc=com" type
=refreshAndPersist interval=00:00:00:30 retry="5 5 60 +" timeout=1 keepalive=
"240:10:30"
olcSyncrepl: {3}rid=104 provider=ldaps://<host4>:636 binddn="<binddn>" bindmethod=si
mple credentials=<creds> searchbase="dc=company,dc=com" type=refresh
AndPersist interval=00:00:00:30 retry="5 5 60 +" timeout=1 keepalive="240:10:
30"
olcMirrorMode: TRUE
# {1}syncprov, {2}mdb, config
dn: olcOverlay={1}syncprov,olcDatabase={2}mdb,cn=config
objectClass: olcOverlayConfig
objectClass: olcSyncProvConfig
olcOverlay: {1}syncprov
olcSpCheckpoint: 100 10
Under normal conditions, all operations perform normally in terms of replication consistency. However, when performing a stream of “ADD” operations eventually producer/consumer go out of sync where consumer thinks it has latest state than producer. All the write operations including ADD are pointed to a producer node (not load balancing the writes). Introduced 30 second delay to “ADD” operations (for the sake of testing) and experiencing the same behavior. The issue is manifesting after adding around 4k entries (sometimes even sooner).
Here are the log snippets.
May 4 23:06:47 openldap-service-0 slapd[199]: do_syncrep2: rid=102 LDAP_RES_SEARCH_RESULT (53) Server is unwilling to perform
May 4 23:06:47 openldap-service-0 slapd[199]: do_syncrep2: rid=102 (53) Server is unwilling to perform
May 4 23:06:37 openldap-service-1 slapd[199]: conn=1046 op=1 SEARCH RESULT tag=101 err=53 nentries=0 text=consumer state is newer than provider!
Checked for the clock synchronization and verified that all instances have NTP enabled. Wondering where and what I should be looking for.
Thanks in advance.
8 months, 2 weeks
slapo-syncprov on read-only consumers
by Michael Ströder
HI!
Is it still highly recommended to configure slapo-syncprov on read-only
consumers?
Background:
I have a tier of read-only consumers with "lastbind on" and
slapo-ppolicy configured, but no chaining to the writeable providers
(e.g. no ppolicy_forward_updates). All providers and consumers are 2.6.2.
In case of a simple bind to this read-only consumer it makes local
modification to the user's entry (pwdFailureTime and pwdLastSuccess) and
it's ok for me that this is local to this consumer.
But with slapo-syncprov configured on the consumer slapd generates a new
entryCSN with server ID 0 which results in a contextCSN value with SID
0. This is what I'd like to avoid. Hence my above question.
Ciao, Michael.
8 months, 2 weeks
Solution for syncrepl & memberof replacement with non-dynamic group and users
by René Gallati
Hello list,
I have an openldap 2.4.49 (ubuntu 20.04 LTS) server pair running with
syncrepl. I also have memberof overlay activated and during a debug
session found out that this is a no-go. I was debugging a problem where
an user record that is in two groups only shows one memberOf attribute
value whereas other users show the expected amount of memberOf values.
Now I'm looking into replacing the memberof overlay but it appears that
for my use case there is no replacement at all.
dynlist seems made to create dynamic groups or lists respectively but
everything in my DIT is a static group and static users. They are
created by a commercial product and I am unable to add further specific
URL attributes there when new entries are created.
I stumbled upon
https://www.mail-archive.com/openldap-technical@openldap.org/msg26067.html
via google search, but blindly copying the dynlist-attrset merely causes
the slapd to reply with
"/etc/ldap/slapd.conf: line 149: "dynlist-attrset <oc> [uri] <URL-ad>
[[<mapped-ad>:]<member-ad> ...]": unable to find AttributeDescription #0
"member+memberOf@groupOfNames"#012. " on startup and stopping
immediately. I suppose it needs some schema extension but of what I
don't understand and neither will I have a trigger objectClass unless I
could just use inetOrgPerson as trigger and have it work.
Is there a way to get back "synthetic" memberOf entries on static user
records (which are inetOrgPerson) with static groups (which are
groupOfNames) on openldap 2.4.49 without adding any special attributes
into users and/or groups themselves ?
Kind regards,
René
--
rene.gallati(a)ergon.ch
T +41 44 268 83 10
Ergon Informatik AG, Merkurstrasse 43, CH-8032 Zürich
www.ergon.ch
smart people – smart software
* * * * * * * * * * * * * * * * * * * * * * * * *
DELIVERING TECHNOLOGY ADVANTAGE SINCE 1984
8 months, 3 weeks
changes in own schema in multi provider setup
by Stefan Kania
Good morning,
we having a own schema with a lot of own attributes. We have a multi
provider replication of cn=config. What is the right way to add a new
attribute to our schema and get it into the configuration?
Stefan
8 months, 3 weeks
