On May 23, 2022, at 3:10 AM, Soisik Froger <soisik.froger@worteks.com> wrote:

I've just sent you sample slapd.conf and a data ldif that illustrate the long query time on large database. Thank you very much to take a look at this !

Hi Soisik,

I've completed a review/test of dynlist, following your excellent example as sent off list.

Here are my findings… 

As a baseline, here’s an example of a 'normal' search across a well populated user tree:

# time ldapsearch -x -LLL -H ldap://m01:389 -D "dc=example,dc=com" -W -s sub -b 'ou=people,dc=example,dc=com' "(cn=load01-TEST1-*)" | grep dn: | wc -l
17470

real    0m0.191s
user    0m0.128s
sys     0m0.132s

Compared with performing a search of the memberof attribute, generated by dynast. This pulls back the members of a large group (about 25K members):
# time ldapsearch -x -LLL -H ldap://m01:389 -D "dc=example,dc=com" -W -s sub -b 'ou=people,dc=example,dc=com' "(memberof=cn=load01-test1-2,ou=groups,dc=example,dc=com)" | grep dn: | wc -l
23869
real    4m6.167s
user    0m2.289s
sys     0m2.037s

That's 4 minutes to search across 100K users. The other search took < 200 ms.

As you (and others) have pointed out, there's a significant performance penalty for searching attributes generated by dylist.

It's possible that we can get modest improvements within a 2.7 timeframe. Unfortunately, we don't anticipate being able to get it back into range of a 'normal' search.

So, that gets us into looking at mitigation.  One approach, focus on what the client does. For example, instead of searching across the user tree for membership to a particular group, it's far more efficient to just pull back that list from the group entry itself.

This is a technique that of course we’re already familiar with. I bring it up here for the sake of argument. 

The question: can we sidestep the memberOf performance problems by focusing on the the client?

Or, is there a usecase that I've missed for which there's no remedy?

Thanks

—
Shawn