5:07 a.m.
Hi,
I've been out the LDAP loop for a bit but the recent discussion of the
memberof overlay on 2.5 piqued my curiosity. Having upgraded a Dev box,
removed the memberof elements from the database and replaced the
memberof overlay with dynlist the queries appear to work as expected but
are both a) slow and b) heavily CPU-intensive on the LDAP server.
2021-09-01T12:47:17.603513+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 fd=12 ACCEPT from IP=192.168.152.33:58738
(IP=129.215.17.9:636)
2021-09-01T12:47:17.687488+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 fd=12 TLS established tls_ssf=256 ssf=256
tls_proto=TLSv1.3 tls_cipher=TLS_AES_256_GCM_SHA384
2021-09-01T12:47:17.688032+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=0 SRCH base="" scope=0 deref=0
filter="(objectClass=*)"
2021-09-01T12:47:17.688470+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=0 SRCH attr=supportedSASLMechanisms
2021-09-01T12:47:17.688878+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=0 SEARCH RESULT tag=101 err=0 qtime=0.000014
etime=0.000214 nentries=1 text=
2021-09-01T12:47:17.811279+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=1 BIND dn="" method=163
2021-09-01T12:47:17.819249+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=1 RESULT tag=97 err=14 qtime=0.000030
etime=0.009084 text=SASL(0): successful result:
2021-09-01T12:47:17.908889+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=2 BIND dn="" method=163
2021-09-01T12:47:17.909836+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=2 RESULT tag=97 err=14 qtime=0.000031
etime=0.000181 text=SASL(0): successful result:
2021-09-01T12:47:17.938839+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=3 BIND dn="" method=163
2021-09-01T12:47:17.939621+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=3 BIND authcid="mcairney(a)EASE.ED.AC.UK"
authzid="mcairney(a)EASE.ED.AC.UK"
2021-09-01T12:47:17.940213+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=3 BIND
dn="uid=mcairney,ou=people,ou=central,dc=authorise-dev,dc=ed,dc=ac,dc=uk"
mech=GSSAPI bind_ssf=256 ssf=256
2021-09-01T12:47:17.940616+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=3 RESULT tag=97 err=0 qtime=0.000024
etime=0.000409 text=
2021-09-01T12:47:18.227342+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=4 SRCH
base="dc=authorise-dev,dc=ed,dc=ac,dc=uk" scope=2 deref=0
filter="(uid=mcairney)"
2021-09-01T12:47:18.227703+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=4 SRCH attr=* +
2021-09-01T12:47:31.392255+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=5 UNBIND
2021-09-01T12:47:31.460705+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=4 SEARCH RESULT tag=101 err=0 qtime=0.000031
etime=13.233679 nentries=1 text=
2021-09-01T12:47:31.461098+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 fd=12 closed
I'm guessing that as the values are computed that this will be heavier
on the CPU but it seems a bit excessive? Has anyone else noticed any
similar performance issues?
This is a relatively low-specced DEV server (2 vCPUs, 4GB RAM) so I
guess this could be a factor but there's no io waiting on the server and
no swapping?
The database is on a par in size with our Production service ( about
400K user objects with 1 group object per user and then about 80K actual
groups of users)
The config for the primary DB (ACLs and rootPW redacted) is:
dn: olcDatabase={1}mdb,cn=config
objectClass: olcDatabaseConfig
objectClass: olcMdbConfig
olcDatabase: {1}mdb
olcDbDirectory: /opt/openldap/var/openldap-data/authorise
olcSuffix: dc=authorise-dev,dc=ed,dc=ac,dc=uk
olcAddContentAcl: FALSE
olcLastMod: TRUE
olcMaxDerefDepth: 2
olcReadOnly: FALSE
olcSecurity: ssf=1
olcSecurity: update_ssf=112
olcSecurity: simple_bind=64
olcSizeLimit: unlimited
olcSyncUseSubentry: FALSE
olcTimeLimit: unlimited
olcMonitoring: TRUE
olcDbEnvFlags: writemap
olcDbEnvFlags: nometasync
olcDbNoSync: FALSE
olcDbIndex: objectClass eq
olcDbIndex: entryUUID eq
olcDbIndex: entryCSN eq
olcDbIndex: cn pres,eq,sub
olcDbIndex: uid pres,eq,sub
olcDbIndex: uidNumber pres,eq
olcDbIndex: gidNumber pres,eq
olcDbIndex: eduniType eq
olcDbIndex: gecos pres,eq,sub
olcDbIndex: eduniCategory eq
olcDbIndex: mail pres,eq,sub
olcDbIndex: eduniSchoolCode eq
olcDbIndex: eduniIDStatus eq
olcDbIndex: eduniCollegeCode eq
olcDbIndex: eduniOrgCode eq
olcDbIndex: memberOf pres,eq
olcDbIndex: eduniLibraryBarcode pres,eq
olcDbIndex: eduniOrganisation pres,eq,sub
olcDbIndex: eduniServiceCode pres,eq
olcDbIndex: krbName pres,eq
olcDbIndex: eduPersonAffiliation pres,eq
olcDbIndex: eduPersonEntitlement pres,eq
olcDbIndex: sn pres,eq,sub
olcDbIndex: eduniIdmsId pres,eq
olcDbIndex: member pres,eq
olcDbIndex: memberUid pres,eq
olcDbIndex: eduniRefNo pres,eq
olcDbIndex: eduniTitle pres,eq
olcDbIndex: title pres,eq,sub
olcDbIndex: eduniCardNumber pres,eq
olcDbIndex: eduniYearOfStudy eq
olcDbIndex: description pres,eq,sub
olcDbIndex: givenName pres,eq,sub
olcDbIndex: aliasedObjectName eq
olcDbIndex: yubiKeyId pres,eq
olcDbIndex: isMemberOf pres,eq
olcDbIndex: hasMember pres,eq
olcDbIndex: proxyAddresses pres,eq,sub
olcDbMaxReaders: 96
olcDbMaxSize: 32212254720
olcDbMode: 0600
olcDbSearchStack: 16
structuralObjectClass: olcMdbConfig
dn: olcOverlay={0}syncprov,olcDatabase={1}mdb,cn=config
objectClass: olcOverlayConfig
objectClass: olcConfig
objectClass: top
objectClass: olcSyncProvConfig
olcOverlay: {0}syncprov
structuralObjectClass: olcSyncProvConfig
dn: olcOverlay={1}accesslog,olcDatabase={1}mdb,cn=config
objectClass: olcOverlayConfig
objectClass: olcAccessLogConfig
olcOverlay: {1}accesslog
olcAccessLogDB: cn=accesslog
olcAccessLogOps: writes
olcAccessLogPurge: 02+00:00 00+04:00
olcAccessLogSuccess: TRUE
structuralObjectClass: olcAccessLogConfig
dn: olcOverlay={2}dynlist,olcDatabase={1}mdb,cn=config
objectClass: olcOverlayConfig
objectClass: olcDynListConfig
olcOverlay: {2}dynlist
olcDynListAttrSet: {0}groupOfURLs memberURL member+memberOf@groupOfNames
structuralObjectClass: olcDynListConfig
--
/****************************
Mark Cairney
ITI Enterprise Services
Information Services
University of Edinburgh
Tel: 0131 650 6565
Email: Mark.Cairney(a)ed.ac.uk
*******************************/
The University of Edinburgh is a charitable body, registered in Scotland, with
registration number SC005336. Is e buidheann carthannais a th’ ann an Oilthigh Dhùn
Èideann, clàraichte an Alba, àireamh clàraidh SC005336.
8:17 a.m.
--On Wednesday, September 1, 2021 2:07 PM +0100 Mark Cairney
<Mark.Cairney(a)ed.ac.uk> wrote:
Hi,
I've been out the LDAP loop for a bit but the recent discussion of the
memberof overlay on 2.5 piqued my curiosity. Having upgraded a Dev box,
removed the memberof elements from the database and replaced the
memberof overlay with dynlist the queries appear to work as expected but
are both a) slow and b) heavily CPU-intensive on the LDAP server.
As an aside, I would note that you appear to be indexing "pres"
unnecessarily. Please read
<https://www.openldap.org/doc/admin25/tuning.html#Presence%20indexing>
If the group object is large you may be having slow searches due to indices
being collapsed to a range. You would need to run the search with trace
logging to determine if that's the case as was recently discussed on the
list.
Regards,
Quanah
--
Quanah Gibson-Mount
Product Architect
Symas Corporation
Packaged, certified, and supported LDAP solutions powered by OpenLDAP:
<http://www.symas.com>
3:48 a.m.
On 01/09/2021 17:17, Quanah Gibson-Mount wrote:
--On Wednesday, September 1, 2021 2:07 PM +0100 Mark Cairney
<Mark.Cairney(a)ed.ac.uk> wrote:
> Hi,
>
> I've been out the LDAP loop for a bit but the recent discussion of the
> memberof overlay on 2.5 piqued my curiosity. Having upgraded a Dev box,
> removed the memberof elements from the database and replaced the
> memberof overlay with dynlist the queries appear to work as expected but
> are both a) slow and b) heavily CPU-intensive on the LDAP server.
As an aside, I would note that you appear to be indexing "pres"
unnecessarily. Please read
<https://www.openldap.org/doc/admin25/tuning.html#Presence%20indexing>
If the group object is large you may be having slow searches due to indices
being collapsed to a range. You would need to run the search with trace
logging to determine if that's the case as was recently discussed on the
list.
Regards,
Quanah
Hi,
We are also observing very long query time involving memberOf when using dynlist migrating
a 2.4 OpenLDAP docker install to 2.5.11. We have followed community recommendation and
replaced the deprecated memberOf overlay with dynlist to calculate memberOf attributes,
and removed the memberof elements from the data before importing it.
Using dynlist, we have noticed an important drop of performance in queries involving the
memberOf attributes on (relatively) large database (about 36000 users and 9000 groups
dispatched in 150 branches) compared to 2.4: Queries on memberOf attributes that used to
be processed in 0.200 seconds now takes about 6-7 seconds. In the example below, the
search base is a branch with a relatively small number of users (about 350) and the
targeted group contains 260 users:
time ldapsearch -x -h 172.17.0.2 -D "cn=admin,dc=domain,dc=com" -w
"secret" -b "ou=people,dc=branch,dc=domain,dc=com"
"(memberof=cn=dev,ou=groups,dc=branch,dc=domain,dc=com)" uid
real 0m6,358s
user 0m0,024s
sys 0m0,020s
If the group used in memberOf has 1200 members, the query takes over 30 seconds. These
results are obtained using an OpenLDAP running in a simple docker container on a host with
8CPU/16GB RAM.
When using the memberOf overlay instead of dynlist to calculate memberOf, the performance
are back to what is used to be in 2.4 (< 0.200 second for the above ldapsearch).
We believe our config to be very standard: member attribute is indexed and dynlist is
loaded and configured as below
dn: olcOverlay={3}dynlist,olcDatabase={1}mdb,cn=config
objectClass: olcOverlayConfig
objectClass: olcDynListConfig
olcOverlay: {3}dynlist
olcDynListAttrSet: {0}groupOfURLs memberURL member+memberOf@groupOfNames
Are this performance issues an expected side-effect of switching to dynlist - as the
memberOf attributes are now dynamically calculated while the memberOf overlay used to
writes these attributes - or something is more likely wrong with our install or could be
tuned to improve these performances?
Some organizations have applications that relies on memberOf queries to be fast, and
expects to keep these performances when upgrading to 2.5. From what we observed, we can
only advise them to stick with the legacy memberOf overlay if they need these queries to
stay fast, or rethink their approach to avoid querying memberOf on large databases.
Regards,
--
Soisik Froger
Worteks | https://www.worteks.com
7:48 a.m.
Soisik Froger wrote:
On 01/09/2021 17:17, Quanah Gibson-Mount wrote:
>
>
> --On Wednesday, September 1, 2021 2:07 PM +0100 Mark Cairney
> <Mark.Cairney(a)ed.ac.uk> wrote:
>
>> Hi,
>>
>> I've been out the LDAP loop for a bit but the recent discussion of the
>> memberof overlay on 2.5 piqued my curiosity. Having upgraded a Dev box,
>> removed the memberof elements from the database and replaced the
>> memberof overlay with dynlist the queries appear to work as expected but
>> are both a) slow and b) heavily CPU-intensive on the LDAP server.
>
>
> As an aside, I would note that you appear to be indexing "pres"
> unnecessarily. Please read
> <https://www.openldap.org/doc/admin25/tuning.html#Presence%20indexing>
>
> If the group object is large you may be having slow searches due to indices
> being collapsed to a range. You would need to run the search with trace
> logging to determine if that's the case as was recently discussed on the
> list.
>
> Regards,
> Quanah
>
Hi,
We are also observing very long query time involving memberOf when using dynlist
migrating a 2.4 OpenLDAP docker install to 2.5.11. We have followed community
recommendation and replaced the deprecated memberOf overlay with dynlist to calculate
memberOf attributes, and removed the memberof elements from the data
before importing it.
Using dynlist, we have noticed an important drop of performance in queries involving the
memberOf attributes on (relatively) large database (about 36000 users
and 9000 groups dispatched in 150 branches) compared to 2.4: Queries on memberOf
attributes that used to be processed in 0.200 seconds now takes about 6-7
seconds. In the example below, the search base is a branch with a relatively small number
of users (about 350) and the targeted group contains 260 users:
time ldapsearch -x -h 172.17.0.2 -D "cn=admin,dc=domain,dc=com" -w
"secret" -b "ou=people,dc=branch,dc=domain,dc=com"
"(memberof=cn=dev,ou=groups,dc=branch,dc=domain,dc=com)" uid
real 0m6,358s
user 0m0,024s
sys 0m0,020s
If the group used in memberOf has 1200 members, the query takes over 30 seconds. These
results are obtained using an OpenLDAP running in a simple docker
container on a host with 8CPU/16GB RAM.
When using the memberOf overlay instead of dynlist to calculate memberOf, the performance
are back to what is used to be in 2.4 (< 0.200 second for the above
ldapsearch).
We believe our config to be very standard: member attribute is indexed and dynlist is
loaded and configured as below
dn: olcOverlay={3}dynlist,olcDatabase={1}mdb,cn=config
objectClass: olcOverlayConfig
objectClass: olcDynListConfig
olcOverlay: {3}dynlist
olcDynListAttrSet: {0}groupOfURLs memberURL member+memberOf@groupOfNames
Are this performance issues an expected side-effect of switching to dynlist - as the
memberOf attributes are now dynamically calculated while the memberOf
overlay used to writes these attributes - or something is more likely wrong with our
install or could be tuned to improve these performances?
That's just the cost of dynamically calculating these attributes, yes.
It's possible we can make some other optimizations to the overlay to reduce the amount
of work in calculating memberOf if you're only
using it in a filter and not returning all of its values. Looks like it could be a fairly
significant refactoring to achieve that though.
Some organizations have applications that relies on memberOf queries to be fast, and
expects to keep these performances when upgrading to 2.5. From what we
observed, we can only advise them to stick with the legacy memberOf overlay if they need
these queries to stay fast, or rethink their approach to avoid querying
memberOf on large databases.
Makes sense.
Regards,
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
7:52 a.m.
On May 11, 2022, at 5:48 AM, Soisik Froger
<soisik.froger(a)worteks.com> wrote:
If the group used in memberOf has 1200 members, the query takes over 30 seconds. These
results are obtained using an OpenLDAP running in a simple docker container on a host with
8CPU/16GB RAM.
When using the memberOf overlay instead of dynlist to calculate memberOf, the performance
are back to what is used to be in 2.4 (< 0.200 second for the above ldapsearch).
We believe our config to be very standard: member attribute is indexed and dynlist is
loaded and configured as below
dn: olcOverlay={3}dynlist,olcDatabase={1}mdb,cn=config
objectClass: olcOverlayConfig
objectClass: olcDynListConfig
olcOverlay: {3}dynlist
olcDynListAttrSet: {0}groupOfURLs memberURL member+memberOf@groupOfNames
Hi Soisik,
I’d like to recreate this in a test env in order to analyze and look for solutions. Can
you send me your (complete) test server config and specifics on the tests, like ldif,
sample data and operations?
Thanks
—
Shawn
7:51 p.m.
On 5/11/2022 3:48 AM, Soisik Froger wrote:
Are this performance issues an expected side-effect of switching to
dynlist - as the memberOf attributes are now dynamically calculated
while the memberOf overlay used to writes these attributes - or
I am also having ongoing sporadic issues with memberOf performance using
the new dynlist overlay. Initially, randomly a server would get into a
state where any query requesting the memberOf attribute would take in
excess of 30 seconds, whereas normally it would only take a fraction of
a second. The symptoms were the same, free memory, no swapping, but
insanely high read IO load.
I cranked up the memory, which not did resolve the issue, but did help,
it doesn't happen nearly as often. But still, every now and again, a
server demonstrates a high read IO rate and severely degraded memberOf
query performance. At this point, I just have a monitoring check that
alerts on slow query performance and high read I/O and go restart them
when it happens, as the additional memory made the issue go from a
couple of times a week to every month or three.
I did notice now that when the issue occurs the box with the slow
queries does have less memory available then when it is working
normally, but still a good gigabyte of free memory not being used.
Even when the systems don't completely blow up, there are occasional
slower than normal queries. Typically the test query I am doing
literally takes fractions of a second:
May 21 19:47:22 ldap-01 slapd[1223]: conn=849157 op=1 SEARCH RESULT
tag=101 err=0 qtime=0.000015 etime=0.198042 nentries=1 text=
Every now and again for no discernible reason it might take 5 to 10 seconds.
11:29 p.m.
New subject: Antw: [EXT] Re: dynlist vs memberof performance issues
>> "Paul B. Henson" <henson(a)acm.org> schrieb am
22.05.2022 um 04:51 in Nachricht
<5d343067-aef3-b499-63e3-996f3d68051e(a)acm.org>:
On 5/11/2022 3:48 AM, Soisik Froger wrote:
> Are this performance issues an expected side-effect of switching to
> dynlist - as the memberOf attributes are now dynamically calculated
> while the memberOf overlay used to writes these attributes - or
I am also having ongoing sporadic issues with memberOf performance using
the new dynlist overlay. Initially, randomly a server would get into a
state where any query requesting the memberOf attribute would take in
excess of 30 seconds, whereas normally it would only take a fraction of
a second. The symptoms were the same, free memory, no swapping, but
insanely high read IO load.
I'm wondering: If you'd make a core dump when the issue happens, how big would
such a core dump be with MDB?
I'm afraid it would be insanely large, containing the whole database. Am I wrong?
I cranked up the memory, which not did resolve the issue, but did help,
it doesn't happen nearly as often. But still, every now and again, a
server demonstrates a high read IO rate and severely degraded memberOf
query performance. At this point, I just have a monitoring check that
alerts on slow query performance and high read I/O and go restart them
when it happens, as the additional memory made the issue go from a
couple of times a week to every month or three.
I did notice now that when the issue occurs the box with the slow
queries does have less memory available then when it is working
normally, but still a good gigabyte of free memory not being used.
Even when the systems don't completely blow up, there are occasional
slower than normal queries. Typically the test query I am doing
literally takes fractions of a second:
May 21 19:47:22 ldap-01 slapd[1223]: conn=849157 op=1 SEARCH RESULT
tag=101 err=0 qtime=0.000015 etime=0.198042 nentries=1 text=
Every now and again for no discernible reason it might take 5 to 10 seconds.
1:10 a.m.
Hi Shawn,
I've just sent you sample slapd.conf and a data ldif that illustrate the long query
time on large database. Thank you very much to take a look at this !
--
Soisik Froger | Software Architect
Worteks | https://www.worteks.com
On 18/05/2022 16:52, Shawn McKinney wrote:
> Hi Soisik,
>
> I’d like to recreate this in a test env in order to analyze and look for solutions.
Can you send me your (complete) test server config and specifics on the tests, like ldif,
sample data and operations?
>
> Thanks
>
> —
> Shawn
9:04 a.m.
New subject: Antw: [EXT] Re: dynlist vs memberof performance issues
Ulrich Windl wrote:
>>> "Paul B. Henson" <henson(a)acm.org> schrieb
am 22.05.2022 um 04:51 in Nachricht
<5d343067-aef3-b499-63e3-996f3d68051e(a)acm.org>:
> On 5/11/2022 3:48 AM, Soisik Froger wrote:
>
>> Are this performance issues an expected side-effect of switching to
>> dynlist - as the memberOf attributes are now dynamically calculated
>> while the memberOf overlay used to writes these attributes - or
>
> I am also having ongoing sporadic issues with memberOf performance using
> the new dynlist overlay. Initially, randomly a server would get into a
> state where any query requesting the memberOf attribute would take in
> excess of 30 seconds, whereas normally it would only take a fraction of
> a second. The symptoms were the same, free memory, no swapping, but
> insanely high read IO load.
I'm wondering: If you'd make a core dump when the issue happens, how big would
such a core dump be with MDB?
I'm afraid it would be insanely large, containing the whole database. Am I wrong?
Yes, you are wrong.
mmap'd memory isn't included in core dumps by default.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
11:14 p.m.
New subject: Antw: [EXT] Re: dynlist vs memberof performance issues
>> Howard Chu <hyc(a)symas.com> schrieb am 23.05.2022 um
18:04 in Nachricht
<cc25e9ad-988c-fcd6-7bc0-0de6e3c530c7(a)symas.com>:
Ulrich Windl wrote:
>>>> "Paul B. Henson" <henson(a)acm.org> schrieb am 22.05.2022
um 04:51 in Nachricht
> <5d343067-aef3-b499-63e3-996f3d68051e(a)acm.org>:
>> On 5/11/2022 3:48 AM, Soisik Froger wrote:
>>
>>> Are this performance issues an expected side-effect of switching to
>>> dynlist - as the memberOf attributes are now dynamically calculated
>>> while the memberOf overlay used to writes these attributes - or
>>
>> I am also having ongoing sporadic issues with memberOf performance using
>> the new dynlist overlay. Initially, randomly a server would get into a
>> state where any query requesting the memberOf attribute would take in
>> excess of 30 seconds, whereas normally it would only take a fraction of
>> a second. The symptoms were the same, free memory, no swapping, but
>> insanely high read IO load.
>
> I'm wondering: If you'd make a core dump when the issue happens, how big
would such a core dump be with MDB?
> I'm afraid it would be insanely large, containing the whole database. Am I
wrong?
Yes, you are wrong.
mmap'd memory isn't included in core dumps by default.
OK, but that makes it hard to debug any MDB issues from the core dump unless significant
parts are copied in to private memory (which I think is not the idea behind MDB).
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
6:34 a.m.
New subject: Antw: [EXT] Re: dynlist vs memberof performance issues
Ulrich Windl wrote:
>>> Howard Chu <hyc(a)symas.com> schrieb am 23.05.2022 um
18:04 in Nachricht
<cc25e9ad-988c-fcd6-7bc0-0de6e3c530c7(a)symas.com>:
> Ulrich Windl wrote:
>>>>> "Paul B. Henson" <henson(a)acm.org> schrieb am
22.05.2022 um 04:51 in Nachricht
>> <5d343067-aef3-b499-63e3-996f3d68051e(a)acm.org>:
>>> On 5/11/2022 3:48 AM, Soisik Froger wrote:
>>>
>>>> Are this performance issues an expected side-effect of switching to
>>>> dynlist - as the memberOf attributes are now dynamically calculated
>>>> while the memberOf overlay used to writes these attributes - or
>>>
>>> I am also having ongoing sporadic issues with memberOf performance using
>>> the new dynlist overlay. Initially, randomly a server would get into a
>>> state where any query requesting the memberOf attribute would take in
>>> excess of 30 seconds, whereas normally it would only take a fraction of
>>> a second. The symptoms were the same, free memory, no swapping, but
>>> insanely high read IO load.
>>
>> I'm wondering: If you'd make a core dump when the issue happens, how big
> would such a core dump be with MDB?
>> I'm afraid it would be insanely large, containing the whole database. Am I
> wrong?
>
> Yes, you are wrong.
>
> mmap'd memory isn't included in core dumps by default.
OK, but that makes it hard to debug any MDB issues from the core dump unless significant
parts are copied in to private memory (which I think is not the idea behind MDB).
Thanks for your concern.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
10:38 a.m.
On May 23, 2022, at 3:10 AM, Soisik Froger
<soisik.froger(a)worteks.com> wrote:
I've just sent you sample slapd.conf and a data ldif that illustrate the long query
time on large database. Thank you very much to take a look at this !
Hi Soisik,
I've completed a review/test of dynlist, following your excellent example as sent off
list.
Here are my findings…
As a baseline, here’s an example of a 'normal' search across a well populated user
tree:
# time ldapsearch -x -LLL -H ldap://m01:389 -D "dc=example,dc=com" -W -s sub -b
'ou=people,dc=example,dc=com' "(cn=load01-TEST1-*)" | grep dn: | wc -l
17470
real 0m0.191s
user 0m0.128s
sys 0m0.132s
Compared with performing a search of the memberof attribute, generated by dynast. This
pulls back the members of a large group (about 25K members):
# time ldapsearch -x -LLL -H ldap://m01:389 -D "dc=example,dc=com" -W -s sub -b
'ou=people,dc=example,dc=com'
"(memberof=cn=load01-test1-2,ou=groups,dc=example,dc=com)" | grep dn: | wc -l
23869
real 4m6.167s
user 0m2.289s
sys 0m2.037s
That's 4 minutes to search across 100K users. The other search took < 200 ms.
As you (and others) have pointed out, there's a significant performance penalty for
searching attributes generated by dylist.
It's possible that we can get modest improvements within a 2.7 timeframe.
Unfortunately, we don't anticipate being able to get it back into range of a
'normal' search.
So, that gets us into looking at mitigation. One approach, focus on what the client does.
For example, instead of searching across the user tree for membership to a particular
group, it's far more efficient to just pull back that list from the group entry
itself.
This is a technique that of course we’re already familiar with. I bring it up here for the
sake of argument.
The question: can we sidestep the memberOf performance problems by focusing on the the
client?
Or, is there a usecase that I've missed for which there's no remedy?
Thanks
—
Shawn
11:01 a.m.
On 7/25/22 19:38, Shawn McKinney wrote:
So, that gets us into looking at mitigation. One approach, focus on
what the client does. For example, instead of searching across the
user tree for membership to a particular group, it's far more
efficient to just pull back that list from the group entry itself.
In my case
searching by 'memberOf' is not only done for determining the
users' group memberships. It is used to also effeciently retrieve the
users' attributes.
Example:
For Æ-DIR my aehostd uses 'memberOf' search to select only subsets of
users indirectly referenced via user groups by so-called service groups
(see https://www.ae-dir.com/docs.html#er-roles) to retrieve the passwd
map data.
Another approach I was thinking about is using deref control on
attribute 'member' when reading the group entry. But this would mean
that I potentially read user data several times if the user is member of
several groups referenced by same service groups (which is the case most
times). In opposite to that a filter (|(memberOf=..)(memberOf=..)) only
retrieves the user entry *once*.
And there were also issues with deref control and set-based ACLs:
https://bugs.openldap.org/show_bug.cgi?id=9800
I'm thinking about how to get rid of slapo-memberof for quite a while now...
BTW: You can access the Æ-DIR online demo via LDAPS and HTTPS and try
out yourself:
https://www.ae-dir.com/demo.html
https://demo.ae-dir.com/web2ldap
Ciao, Michael.
12:51 p.m.
Hello,
I have worked with Soisik on this particular topic.
As you mentionned, in many cases we can just query directly the group.
In our particular scenario, the software only allows one request for getting the mails of
the users present in the dynamic group.
Thus I think it would be great to have a server solution (if possible in 2.6), especially
if dynlist is officially the memberof replacement.
Regards,
David
Le 25 juillet 2022 19:38:52 GMT+02:00, Shawn McKinney <smckinney(a)symas.com> a
écrit :
> On May 23, 2022, at 3:10 AM, Soisik Froger <soisik.froger(a)worteks.com> wrote:
>
> I've just sent you sample slapd.conf and a data ldif that illustrate the long
query time on large database. Thank you very much to take a look at this !
Hi Soisik,
I've completed a review/test of dynlist, following your excellent example as sent off
list.
Here are my findings…
As a baseline, here’s an example of a 'normal' search across a well populated user
tree:
# time ldapsearch -x -LLL -H ldap://m01:389 -D "dc=example,dc=com" -W -s sub -b
'ou=people,dc=example,dc=com' "(cn=load01-TEST1-*)" | grep dn: | wc -l
17470
real 0m0.191s
user 0m0.128s
sys 0m0.132s
Compared with performing a search of the memberof attribute, generated by dynast. This
pulls back the members of a large group (about 25K members):
# time ldapsearch -x -LLL -H ldap://m01:389 -D "dc=example,dc=com" -W -s sub -b
'ou=people,dc=example,dc=com'
"(memberof=cn=load01-test1-2,ou=groups,dc=example,dc=com)" | grep dn: | wc -l
23869
real 4m6.167s
user 0m2.289s
sys 0m2.037s
That's 4 minutes to search across 100K users. The other search took < 200 ms.
As you (and others) have pointed out, there's a significant performance penalty for
searching attributes generated by dylist.
It's possible that we can get modest improvements within a 2.7 timeframe.
Unfortunately, we don't anticipate being able to get it back into range of a
'normal' search.
So, that gets us into looking at mitigation. One approach, focus on what the client does.
For example, instead of searching across the user tree for membership to a particular
group, it's far more efficient to just pull back that list from the group entry
itself.
This is a technique that of course we’re already familiar with. I bring it up here for the
sake of argument.
The question: can we sidestep the memberOf performance problems by focusing on the the
client?
Or, is there a usecase that I've missed for which there's no remedy?
Thanks
—
Shawn
3:04 p.m.
David Coutadeur wrote:
Hello,
I have worked with Soisik on this particular topic.
As you mentionned, in many cases we can just query directly the group.
In our particular scenario, the software only allows one request for getting the mails of
the users present in the dynamic group.
Have you already tried using the deref control to return a mail attribute instead?
Thus I think it would be great to have a server solution (if possible in 2.6), especially
if dynlist is officially the memberof replacement.
Regards,
David
Le 25 juillet 2022 19:38:52 GMT+02:00, Shawn McKinney <smckinney(a)symas.com> a
écrit :
On May 23, 2022, at 3:10 AM, Soisik Froger <soisik.froger(a)worteks.com>
wrote:
I've just sent you sample slapd.conf and a data ldif that illustrate the long
query time on large database. Thank you very much to take a look at this !
Hi Soisik,
I've completed a review/test of dynlist, following your excellent example as sent
off list.
Here are my findings…
As a baseline, here’s an example of a 'normal' search across a well populated
user tree:
# time ldapsearch -x -LLL -H ldap://m01:389 -D "dc=example,dc=com" -W -s
sub -b 'ou=people,dc=example,dc=com' "(cn=load01-TEST1-*)" | grep dn: |
wc -l
17470
real 0m0.191s
user 0m0.128s
sys 0m0.132s
Compared with performing a search of the memberof attribute, generated by dynast.
This pulls back the members of a large group (about 25K members):
# time ldapsearch -x -LLL -H ldap://m01:389 -D "dc=example,dc=com" -W -s
sub -b 'ou=people,dc=example,dc=com'
"(memberof=cn=load01-test1-2,ou=groups,dc=example,dc=com)" | grep dn: | wc -l
23869
real 4m6.167s
user 0m2.289s
sys 0m2.037s
That's 4 minutes to search across 100K users. The other search took < 200 ms.
As you (and others) have pointed out, there's a significant performance penalty
for searching attributes generated by dylist.
It's possible that we can get modest improvements within a 2.7 timeframe.
Unfortunately, we don't anticipate being able to get it back into range of a
'normal' search.
So, that gets us into looking at mitigation. One approach, focus on what the client
does. For example, instead of searching across the user tree for membership to a
particular group, it's far more efficient to just pull back that list from the group
entry itself.
This is a technique that of course we’re already familiar with. I bring it up here
for the sake of argument.
The question: can we sidestep the memberOf performance problems by focusing on the
the client?
Or, is there a usecase that I've missed for which there's no remedy?
Thanks
—
Shawn
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
4:16 p.m.
On 7/25/2022 10:38 AM, Shawn McKinney wrote:
As you (and others) have pointed out, there's a significant
performance penalty for searching attributes generated by dylist.
I'm still seeing performance issues with queries that simply return
memberOf, with no reference to it in the actual search filter.
For example, this query which searches on the static uid attribute and
returns memberOf:
time ldapsearch -H ldapi:/// uid=henson memberOf
Most of the time it completes in fractions of a second:
real 0m0.187s
user 0m0.005s
sys 0m0.003s
But sometimes it takes 5 seconds, 10 seconds, or even more. These
extremely slow response times coordinate with a high read I/O percentage
on the server and the high number of page faults on the slapd process.
When I first deployed 2.5, sometimes the server would get into a state
where every query that requested memberOf would take in excess of 30
seconds to return until the server was restarted. I cranked up the
memory on the servers and at this point I have had no more reoccurrences
of that behavior, but I am still regularly seeing occasional slow
performance on the queries and high read I/O percentages.
The servers have way more memory now than they should need to fully
cache the entire database:
# du -sh /var/symas/openldap-data-cpp
2.6G /var/symas/openldap-data-cpp
# free -m
total used free shared buff/cache
available
Mem: 4818 703 124 0 3991
3831
Swap: 2047 136 1911
I haven't been able to correlate the slow response times with any other
external criteria such as updates or query load. Sometimes it's just
slow 8-/. We never saw this problem under 2.4 which used the previous
implementation of dynlist to generate memberOf. I definitely appreciate
the ability to query on dynamic memberOf that was added in 2.5, but it
would be nice to sort out this performance issue.
1:56 a.m.
If you have the ability to do so and your kernel is v5+, out-of-hours experiment with
disabling system swap entirely (vm.swapiness=0 and 'swapoff -a' ) and then
simulate a run of requests you'd expect to encounter the page errors with and see if
it stops happening.
Depending on what else is going on this might not be a production-compatible strategy, but
it will either rule that out as a potential flaw/bug in 2.5 and/or give you a workaround
until it's fixed.
________________________________
From: Paul B. Henson <henson(a)acm.org>
Sent: 26 July 2022 00:16
To: openldap-technical(a)openldap.org <openldap-technical(a)openldap.org>
Subject: Re: dynlist vs memberof performance issues
On 7/25/2022 10:38 AM, Shawn McKinney wrote:
As you (and others) have pointed out, there's a significant
performance penalty for searching attributes generated by dylist.
I'm still seeing performance issues with queries that simply return
memberOf, with no reference to it in the actual search filter.
For example, this query which searches on the static uid attribute and
returns memberOf:
time ldapsearch -H ldapi:/// uid=henson memberOf
Most of the time it completes in fractions of a second:
real 0m0.187s
user 0m0.005s
sys 0m0.003s
But sometimes it takes 5 seconds, 10 seconds, or even more. These
extremely slow response times coordinate with a high read I/O percentage
on the server and the high number of page faults on the slapd process.
When I first deployed 2.5, sometimes the server would get into a state
where every query that requested memberOf would take in excess of 30
seconds to return until the server was restarted. I cranked up the
memory on the servers and at this point I have had no more reoccurrences
of that behavior, but I am still regularly seeing occasional slow
performance on the queries and high read I/O percentages.
The servers have way more memory now than they should need to fully
cache the entire database:
# du -sh /var/symas/openldap-data-cpp
2.6G /var/symas/openldap-data-cpp
# free -m
total used free shared buff/cache
available
Mem: 4818 703 124 0 3991
3831
Swap: 2047 136 1911
I haven't been able to correlate the slow response times with any other
external criteria such as updates or query load. Sometimes it's just
slow 8-/. We never saw this problem under 2.4 which used the previous
implementation of dynlist to generate memberOf. I definitely appreciate
the ability to query on dynamic memberOf that was added in 2.5, but it
would be nice to sort out this performance issue.
********** Internet Email Confidentiality Footer **********
Privileged/Confidential information may be contained in this message.
If you have received this message in error you should, without taking any copies,
immediately delete it from your system and kindly notify the sender by reply email.
Opinions, conclusions and other information in this message that do not relate to the
official business of this company shall be understood as neither given nor endorsed by it.
All contracts for goods and services are subject to the terms and conditions of our
relevant company, a copy of which is available on request or by visiting the company’s
website.
Amari Metals Ltd. Registered in England and Wales number 2023155.
Registered Office: Parkway House, Unit 6, Parkway Industrial Estate, Pacific Avenue,
Wednesbury, WS10 7WP.
3:23 a.m.
Hi both,
Just to let you know that with my dataset I see a consistently slow
(10-20 secs) lookup of a user's memberOf group membership using dynamic
dynlist overlays on 2.5 and 2.6 compared with the old "memberof" overlay.
My dynlist config is as below:
dn: olcOverlay={2}dynlist,olcDatabase={1}mdb,cn=config
objectClass: olcOverlayConfig
objectClass: olcDynListConfig
olcOverlay: {2}dynlist
olcDynListAttrSet: {0}groupOfURLs memberURL member+memberOf@groupOfNames
structuralObjectClass: olcDynListConfig
It's fairly common to use the memberOf attribute to look up the groups a
particular user is a member of e.g. for access control purposes. We use
it routinely for this purpose on our Shibboleth IdPs which use OpenLDAP
as a datasource.
In terms of data size/complexity we've currently got around 500K users
(each with their own unique group in a separate OU) with the bulk of our
group membership managed by Grouper. We've got about 100K groups in
there(!) ranging in size from a single "placeholder" member up to about
200K members for our applicant/alumni groups.
The total DIT size (dumped via slapcat) is just under 2GB and our
(freshly slapcat'ed and slapadd'ed mdb database size is currently
sitting at 5.1GB). We're running on Centos 7 boxes with 12vCPUs and 64GB
RAM each which I think should be enough CPU/memory resource for our usage?
On a related note for a production environment should we be targeting
the 2.5 or 2.6 branch? I'd like to upgrade our production servers off
2.4 before the start of next term.
Kind regards,
Mark
On 26/07/2022 00:16, Paul B. Henson wrote:
This email was sent to you by someone outside the University.
You should only click on links or attachments if you are certain that
the email is genuine and the content is safe.
On 7/25/2022 10:38 AM, Shawn McKinney wrote:
> As you (and others) have pointed out, there's a significant
> performance penalty for searching attributes generated by dylist.
I'm still seeing performance issues with queries that simply return
memberOf, with no reference to it in the actual search filter.
For example, this query which searches on the static uid attribute and
returns memberOf:
time ldapsearch -H ldapi:/// uid=henson memberOf
Most of the time it completes in fractions of a second:
real 0m0.187s
user 0m0.005s
sys 0m0.003s
But sometimes it takes 5 seconds, 10 seconds, or even more. These
extremely slow response times coordinate with a high read I/O percentage
on the server and the high number of page faults on the slapd process.
When I first deployed 2.5, sometimes the server would get into a state
where every query that requested memberOf would take in excess of 30
seconds to return until the server was restarted. I cranked up the
memory on the servers and at this point I have had no more reoccurrences
of that behavior, but I am still regularly seeing occasional slow
performance on the queries and high read I/O percentages.
The servers have way more memory now than they should need to fully
cache the entire database:
# du -sh /var/symas/openldap-data-cpp
2.6G /var/symas/openldap-data-cpp
# free -m
total used free shared buff/cache
available
Mem: 4818 703 124 0 3991
3831
Swap: 2047 136 1911
I haven't been able to correlate the slow response times with any other
external criteria such as updates or query load. Sometimes it's just
slow 8-/. We never saw this problem under 2.4 which used the previous
implementation of dynlist to generate memberOf. I definitely appreciate
the ability to query on dynamic memberOf that was added in 2.5, but it
would be nice to sort out this performance issue.
--
/****************************
Mark Cairney
ITI Enterprise Services
Information Services
University of Edinburgh
Tel: 0131 650 6565
Email: Mark.Cairney(a)ed.ac.uk
*******************************/
The University of Edinburgh is a charitable body, registered in Scotland, with
registration number SC005336. Is e buidheann carthannais a th’ ann an Oilthigh Dhùn
Èideann, clàraichte an Alba, àireamh clàraidh SC005336.
4:54 a.m.
Hello,
Le 26/07/2022 à 00:04, Howard Chu a écrit :
David Coutadeur wrote:
> Hello,
>
> I have worked with Soisik on this particular topic.
> As you mentionned, in many cases we can just query directly the group.
> In our particular scenario, the software only allows one request for getting the
mails of the users present in the dynamic group.
Have you already tried using the deref control to return a mail attribute instead?
Thanks for the idea.
I suppose it could work indeed. I have not tested it yet, but it could
be interresting to have the performance associated to such request.
However, in my specific use case, it is not possible to configure a LDAP
control in the software.
> Thus I think it would be great to have a server solution (if
possible in 2.6), especially if dynlist is officially the memberof replacement.
>
> Regards,
>
> David
>
>
>
> Le 25 juillet 2022 19:38:52 GMT+02:00, Shawn McKinney <smckinney(a)symas.com> a
écrit :
>
>
> On May 23, 2022, at 3:10 AM, Soisik Froger <soisik.froger(a)worteks.com>
wrote:
>
> I've just sent you sample slapd.conf and a data ldif that illustrate the
long query time on large database. Thank you very much to take a look at this !
>
>
> Hi Soisik,
>
> I've completed a review/test of dynlist, following your excellent example as
sent off list.
>
> Here are my findings…
>
> As a baseline, here’s an example of a 'normal' search across a well
populated user tree:
>
> # time ldapsearch -x -LLL -H ldap://m01:389 -D "dc=example,dc=com" -W
-s sub -b 'ou=people,dc=example,dc=com' "(cn=load01-TEST1-*)" | grep dn:
| wc -l
> 17470
>
> real 0m0.191s
> user 0m0.128s
> sys 0m0.132s
>
> Compared with performing a search of the memberof attribute, generated by
dynast. This pulls back the members of a large group (about 25K members):
> # time ldapsearch -x -LLL -H ldap://m01:389 -D "dc=example,dc=com" -W
-s sub -b 'ou=people,dc=example,dc=com'
"(memberof=cn=load01-test1-2,ou=groups,dc=example,dc=com)" | grep dn: | wc -l
> 23869
> real 4m6.167s
> user 0m2.289s
> sys 0m2.037s
>
> That's 4 minutes to search across 100K users. The other search took < 200
ms.
>
> As you (and others) have pointed out, there's a significant performance
penalty for searching attributes generated by dylist.
>
> It's possible that we can get modest improvements within a 2.7 timeframe.
Unfortunately, we don't anticipate being able to get it back into range of a
'normal' search.
>
> So, that gets us into looking at mitigation. One approach, focus on what the
client does. For example, instead of searching across the user tree for membership to a
particular group, it's far more efficient to just pull back that list from the group
entry itself.
>
> This is a technique that of course we’re already familiar with. I bring it up
here for the sake of argument.
>
> The question: can we sidestep the memberOf performance problems by focusing on
the the client?
>
> Or, is there a usecase that I've missed for which there's no remedy?
>
> Thanks
>
> —
> Shawn
>
--
David Coutadeur | IAM integrator
david.coutadeur(a)worteks.com
+33 7 88 46 85 34
137 boulevard de Magenta, Paris 75010
Worteks | https://www.worteks.com
7:25 a.m.
I second that, actually I was going to suggest doing exactly this test.
If possible, you should try to do it somewhere else that is not the
production server, maybe a VM after restoring the database?
That would allow you to carefully execute tests without interference
from other requests coming in.
If you're in Linux, strace might give you more details of what is
happening in terms of system calls, but it will generate a loot of
messages so you should try it as last resource.
Last, there are applications that might help you understand what is
going on:
http://cnmonitor.sourceforge.net/
https://github.com/jcollie/openldap_exporter (this will require to
install and configure Prometheus first)
Hope this helps.
Regards,
Alceu
On 26/07/2022 05:56, Wilkinson, Hugo (IT Dept) wrote:
If you have the ability to do so and your kernel is v5+, out-of-hours
experiment with disabling system swap entirely (vm.swapiness=0 and
'swapoff -a' ) and then simulate a run of requests you'd expect to
encounter the page errors with and see if it stops happening.
Depending on what else is going on this might not be a
production-compatible strategy, but it will either rule that out as a
potential flaw/bug in 2.5 and/or give you a workaround until it's fixed.
3:59 p.m.
On 7/26/2022 1:56 AM, Wilkinson, Hugo (IT Dept) wrote:
If you have the ability to do so and your kernel is v5+, out-of-hours
experiment with disabling system swap entirely (vm.swapiness=0 and
'swapoff -a' ) and then simulate a run of requests you'd expect to
encounter the page errors with and see if it stops happening.
We use to run our LDAP servers under Gentoo with a 5.4 kernel at the
time, but currently are using Rocky Linux 8 which comes with a modified
4.18 kernel.
I did actually run a production system with no swap for a while, it
still occasionally had slow responses for queries requesting the
memberOf attribute.
Part of the problem is I don't expect to ever see these slow response
times 8-/. As of yet I have not figured out what is going on when they
are slow, I definitely don't have a reproducible query test case to make
it happen <sigh>.
We were running 2.4 under rocky for a while before upgrading to 2.5 and
never saw this problem, so I'm pretty sure it's the result of some
change in the dynlist implementation.
9:44 a.m.
On Jul 25, 2022, at 2:51 PM, David Coutadeur
<david.coutadeur(a)gmail.com> wrote:
I have worked with Soisik on this particular topic.
As you mentionned, in many cases we can just query directly the group.
In our particular scenario, the software only allows one request for getting the mails of
the users present in the dynamic group.
Thus I think it would be great to have a server solution (if possible in 2.6), especially
if dynlist is officially the memberof replacement.
Hello,
Some work on the dynlist code of late to boost performance.
So, it might be worthwhile to give it a try. MR576[1] in the next release.
—
Shawn
[1] https://git.openldap.org/openldap/openldap/-/merge_requests/576
7:26 a.m.
On Jan 7, 2023, at 8:44 PM, Paul B. Henson <henson(a)acm.org>
wrote:
On Tue, Jan 03, 2023 at 11:44:30AM -0600, Shawn McKinney wrote:
> Some work on the dynlist code of late to boost performance.
>
> So, it might be worthwhile to give it a try. MR576[1] in the next release.
Is this going to hit the 2.5 LTS train or just 2.6?
Yes (both)
11:39 a.m.
On Mon, Jan 09, 2023 at 09:26:44AM -0600, Shawn McKinney wrote:
Yes (both)
Sweet :). We've had some performance issues with the new dynlist
implementation since we upgraded to 2.5, I look forward to trying this
out.
We're currently using dynlist to add the memberOf attribute to users.
One thing that was nice about the new dynlist implementation in 2.5 was
that it allows searching on the dynamic memberOf attribute which we
couldn't do in 2.4. Looking at the commit diff, there's one part that
says:
"To enable filter evaluation on the dynamic list, the configuration must
be changed to explicitly map the dynamic attributes to be filtered"
This just applies to groups created dynamically, right, not static
objects that get dynamic attributes added? Our current config is:
dynlist-attrset groupOfURLs memberURL member+memberOf@groupOfNames
It doesn't require any changes to keep working with a searchable
memberOf attribute?
Thanks much...
4:32 a.m.
On Jan 9, 2023, at 1:39 PM, Paul B. Henson <henson(a)acm.org>
wrote:
On Mon, Jan 09, 2023 at 09:26:44AM -0600, Shawn McKinney wrote:
> Yes (both)
Sweet :). We've had some performance issues with the new dynlist
implementation since we upgraded to 2.5, I look forward to trying this
out.
We're currently using dynlist to add the memberOf attribute to users.
One thing that was nice about the new dynlist implementation in 2.5 was
that it allows searching on the dynamic memberOf attribute which we
couldn't do in 2.4. Looking at the commit diff, there's one part that
says:
"To enable filter evaluation on the dynamic list, the configuration must
be changed to explicitly map the dynamic attributes to be filtered"
This just applies to groups created dynamically, right, not static
objects that get dynamic attributes added? Our current config is:
dynlist-attrset groupOfURLs memberURL member+memberOf@groupOfNames
It doesn't require any changes to keep working with a searchable
memberOf attribute?
Correct, no changes to the config of dynlist.
Thanks much...
4:41 a.m.
On Mon, Jan 09, 2023 at 11:39:47AM -0800, Paul B. Henson wrote:
We're currently using dynlist to add the memberOf attribute to
users.
One thing that was nice about the new dynlist implementation in 2.5 was
that it allows searching on the dynamic memberOf attribute which we
couldn't do in 2.4. Looking at the commit diff, there's one part that
says:
"To enable filter evaluation on the dynamic list, the configuration must
be changed to explicitly map the dynamic attributes to be filtered"
This just applies to groups created dynamically, right, not static
objects that get dynamic attributes added? Our current config is:
The latest manpage update should make it clearer how dynamic *lists*
differ from dynamic *groups*. And yes, no need to change config for
groups.
If you can give the current 2.5/2.6 branch a go, or if you decide to
wait until we get a call for testing out, feedback is always welcome.
Regards,
--
Ondřej Kuzník
Senior Software Engineer
Symas Corporation http://www.symas.com
Packaged, certified, and supported LDAP solutions powered by OpenLDAP
3:37 p.m.
On Tue, Jan 10, 2023 at 01:41:58PM +0100, Ondřej Kuzník wrote:
The latest manpage update should make it clearer how dynamic *lists*
differ from dynamic *groups*. And yes, no need to change config for
groups.
Cool, thank you both for the clarification.
If you can give the current 2.5/2.6 branch a go, or if you decide to
wait until we get a call for testing out, feedback is always welcome.
Will do, thanks again...
137
days inactive
634
days old
openldap-technical@openldap.org
27 comments
12 participants
participants (12)
-
Alceu Rodrigues de Freitas Junior -
David Coutadeur -
Howard Chu -
Mark Cairney -
Michael Ströder -
Ondřej Kuzník -
Paul B. Henson -
Quanah Gibson-Mount -
Shawn McKinney -
Soisik Froger -
Ulrich Windl -
Wilkinson, Hugo (IT Dept)