On May 23, 2022, at 3:10 AM, Soisik Froger soisik.froger@worteks.com wrote:
I've just sent you sample slapd.conf and a data ldif that illustrate the long query time on large database. Thank you very much to take a look at this !
Hi Soisik,
I've completed a review/test of dynlist, following your excellent example as sent off list.
Here are my findings…
As a baseline, here’s an example of a 'normal' search across a well populated user tree:
# time ldapsearch -x -LLL -H ldap://m01:389 -D "dc=example,dc=com" -W -s sub -b 'ou=people,dc=example,dc=com' "(cn=load01-TEST1-*)" | grep dn: | wc -l 17470
real 0m0.191s user 0m0.128s sys 0m0.132s
Compared with performing a search of the memberof attribute, generated by dynast. This pulls back the members of a large group (about 25K members): # time ldapsearch -x -LLL -H ldap://m01:389 -D "dc=example,dc=com" -W -s sub -b 'ou=people,dc=example,dc=com' "(memberof=cn=load01-test1-2,ou=groups,dc=example,dc=com)" | grep dn: | wc -l 23869 real 4m6.167s user 0m2.289s sys 0m2.037s
That's 4 minutes to search across 100K users. The other search took < 200 ms.
As you (and others) have pointed out, there's a significant performance penalty for searching attributes generated by dylist.
It's possible that we can get modest improvements within a 2.7 timeframe. Unfortunately, we don't anticipate being able to get it back into range of a 'normal' search.
So, that gets us into looking at mitigation. One approach, focus on what the client does. For example, instead of searching across the user tree for membership to a particular group, it's far more efficient to just pull back that list from the group entry itself.
This is a technique that of course we’re already familiar with. I bring it up here for the sake of argument.
The question: can we sidestep the memberOf performance problems by focusing on the the client?
Or, is there a usecase that I've missed for which there's no remedy?
Thanks
— Shawn