Hi,
I've been out the LDAP loop for a bit but the recent discussion of the memberof overlay on 2.5 piqued my curiosity. Having upgraded a Dev box, removed the memberof elements from the database and replaced the memberof overlay with dynlist the queries appear to work as expected but are both a) slow and b) heavily CPU-intensive on the LDAP server.
2021-09-01T12:47:17.603513+01:00 bonsai.authorise-dev.is.ed.ac.uk slapd[30075]: conn=1019 fd=12 ACCEPT from IP=192.168.152.33:58738 (IP=129.215.17.9:636) 2021-09-01T12:47:17.687488+01:00 bonsai.authorise-dev.is.ed.ac.uk slapd[30075]: conn=1019 fd=12 TLS established tls_ssf=256 ssf=256 tls_proto=TLSv1.3 tls_cipher=TLS_AES_256_GCM_SHA384 2021-09-01T12:47:17.688032+01:00 bonsai.authorise-dev.is.ed.ac.uk slapd[30075]: conn=1019 op=0 SRCH base="" scope=0 deref=0 filter="(objectClass=*)" 2021-09-01T12:47:17.688470+01:00 bonsai.authorise-dev.is.ed.ac.uk slapd[30075]: conn=1019 op=0 SRCH attr=supportedSASLMechanisms 2021-09-01T12:47:17.688878+01:00 bonsai.authorise-dev.is.ed.ac.uk slapd[30075]: conn=1019 op=0 SEARCH RESULT tag=101 err=0 qtime=0.000014 etime=0.000214 nentries=1 text= 2021-09-01T12:47:17.811279+01:00 bonsai.authorise-dev.is.ed.ac.uk slapd[30075]: conn=1019 op=1 BIND dn="" method=163 2021-09-01T12:47:17.819249+01:00 bonsai.authorise-dev.is.ed.ac.uk slapd[30075]: conn=1019 op=1 RESULT tag=97 err=14 qtime=0.000030 etime=0.009084 text=SASL(0): successful result: 2021-09-01T12:47:17.908889+01:00 bonsai.authorise-dev.is.ed.ac.uk slapd[30075]: conn=1019 op=2 BIND dn="" method=163 2021-09-01T12:47:17.909836+01:00 bonsai.authorise-dev.is.ed.ac.uk slapd[30075]: conn=1019 op=2 RESULT tag=97 err=14 qtime=0.000031 etime=0.000181 text=SASL(0): successful result: 2021-09-01T12:47:17.938839+01:00 bonsai.authorise-dev.is.ed.ac.uk slapd[30075]: conn=1019 op=3 BIND dn="" method=163 2021-09-01T12:47:17.939621+01:00 bonsai.authorise-dev.is.ed.ac.uk slapd[30075]: conn=1019 op=3 BIND authcid="mcairney@EASE.ED.AC.UK" authzid="mcairney@EASE.ED.AC.UK" 2021-09-01T12:47:17.940213+01:00 bonsai.authorise-dev.is.ed.ac.uk slapd[30075]: conn=1019 op=3 BIND dn="uid=mcairney,ou=people,ou=central,dc=authorise-dev,dc=ed,dc=ac,dc=uk" mech=GSSAPI bind_ssf=256 ssf=256 2021-09-01T12:47:17.940616+01:00 bonsai.authorise-dev.is.ed.ac.uk slapd[30075]: conn=1019 op=3 RESULT tag=97 err=0 qtime=0.000024 etime=0.000409 text= 2021-09-01T12:47:18.227342+01:00 bonsai.authorise-dev.is.ed.ac.uk slapd[30075]: conn=1019 op=4 SRCH base="dc=authorise-dev,dc=ed,dc=ac,dc=uk" scope=2 deref=0 filter="(uid=mcairney)" 2021-09-01T12:47:18.227703+01:00 bonsai.authorise-dev.is.ed.ac.uk slapd[30075]: conn=1019 op=4 SRCH attr=* + 2021-09-01T12:47:31.392255+01:00 bonsai.authorise-dev.is.ed.ac.uk slapd[30075]: conn=1019 op=5 UNBIND 2021-09-01T12:47:31.460705+01:00 bonsai.authorise-dev.is.ed.ac.uk slapd[30075]: conn=1019 op=4 SEARCH RESULT tag=101 err=0 qtime=0.000031 etime=13.233679 nentries=1 text= 2021-09-01T12:47:31.461098+01:00 bonsai.authorise-dev.is.ed.ac.uk slapd[30075]: conn=1019 fd=12 closed
I'm guessing that as the values are computed that this will be heavier on the CPU but it seems a bit excessive? Has anyone else noticed any similar performance issues?
This is a relatively low-specced DEV server (2 vCPUs, 4GB RAM) so I guess this could be a factor but there's no io waiting on the server and no swapping?
The database is on a par in size with our Production service ( about 400K user objects with 1 group object per user and then about 80K actual groups of users)
The config for the primary DB (ACLs and rootPW redacted) is:
dn: olcDatabase={1}mdb,cn=config objectClass: olcDatabaseConfig objectClass: olcMdbConfig olcDatabase: {1}mdb olcDbDirectory: /opt/openldap/var/openldap-data/authorise olcSuffix: dc=authorise-dev,dc=ed,dc=ac,dc=uk
olcAddContentAcl: FALSE olcLastMod: TRUE olcMaxDerefDepth: 2 olcReadOnly: FALSE olcSecurity: ssf=1 olcSecurity: update_ssf=112 olcSecurity: simple_bind=64 olcSizeLimit: unlimited olcSyncUseSubentry: FALSE olcTimeLimit: unlimited olcMonitoring: TRUE olcDbEnvFlags: writemap olcDbEnvFlags: nometasync olcDbNoSync: FALSE olcDbIndex: objectClass eq olcDbIndex: entryUUID eq olcDbIndex: entryCSN eq olcDbIndex: cn pres,eq,sub olcDbIndex: uid pres,eq,sub olcDbIndex: uidNumber pres,eq olcDbIndex: gidNumber pres,eq olcDbIndex: eduniType eq olcDbIndex: gecos pres,eq,sub olcDbIndex: eduniCategory eq olcDbIndex: mail pres,eq,sub olcDbIndex: eduniSchoolCode eq olcDbIndex: eduniIDStatus eq olcDbIndex: eduniCollegeCode eq olcDbIndex: eduniOrgCode eq olcDbIndex: memberOf pres,eq olcDbIndex: eduniLibraryBarcode pres,eq olcDbIndex: eduniOrganisation pres,eq,sub olcDbIndex: eduniServiceCode pres,eq olcDbIndex: krbName pres,eq olcDbIndex: eduPersonAffiliation pres,eq olcDbIndex: eduPersonEntitlement pres,eq olcDbIndex: sn pres,eq,sub olcDbIndex: eduniIdmsId pres,eq olcDbIndex: member pres,eq olcDbIndex: memberUid pres,eq olcDbIndex: eduniRefNo pres,eq olcDbIndex: eduniTitle pres,eq olcDbIndex: title pres,eq,sub olcDbIndex: eduniCardNumber pres,eq olcDbIndex: eduniYearOfStudy eq olcDbIndex: description pres,eq,sub olcDbIndex: givenName pres,eq,sub olcDbIndex: aliasedObjectName eq olcDbIndex: yubiKeyId pres,eq olcDbIndex: isMemberOf pres,eq olcDbIndex: hasMember pres,eq olcDbIndex: proxyAddresses pres,eq,sub olcDbMaxReaders: 96 olcDbMaxSize: 32212254720 olcDbMode: 0600 olcDbSearchStack: 16 structuralObjectClass: olcMdbConfig
dn: olcOverlay={0}syncprov,olcDatabase={1}mdb,cn=config objectClass: olcOverlayConfig objectClass: olcConfig objectClass: top objectClass: olcSyncProvConfig olcOverlay: {0}syncprov structuralObjectClass: olcSyncProvConfig
dn: olcOverlay={1}accesslog,olcDatabase={1}mdb,cn=config objectClass: olcOverlayConfig objectClass: olcAccessLogConfig olcOverlay: {1}accesslog olcAccessLogDB: cn=accesslog olcAccessLogOps: writes olcAccessLogPurge: 02+00:00 00+04:00 olcAccessLogSuccess: TRUE structuralObjectClass: olcAccessLogConfig
dn: olcOverlay={2}dynlist,olcDatabase={1}mdb,cn=config objectClass: olcOverlayConfig objectClass: olcDynListConfig olcOverlay: {2}dynlist olcDynListAttrSet: {0}groupOfURLs memberURL member+memberOf@groupOfNames structuralObjectClass: olcDynListConfig
-- /****************************
Mark Cairney ITI Enterprise Services Information Services University of Edinburgh
Tel: 0131 650 6565 Email: Mark.Cairney@ed.ac.uk
*******************************/
The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. Is e buidheann carthannais a th’ ann an Oilthigh Dhùn Èideann, clàraichte an Alba, àireamh clàraidh SC005336.