Thank you for your quick response.
If idlexp is the accepted solution then I'd like to understand how to choose correct value for idlexp?
I have quickly tested with most values. With my quick tests I could not find any significant impact on performance. Setting idlexp to maximum 31 did however cause slapd to crash with segmentation fault on my system.
Thanks, Petteri ________________________________ From: Quanah Gibson-Mount quanah@symas.com Sent: Monday, August 16, 2021 23:19 To: Petteri Stenius Petteri.Stenius@ubisecure.com; openldap-technical@openldap.org openldap-technical@openldap.org Subject: Re: Index seems to return wrong amount of candidate causing really poor search performance
--On Monday, August 16, 2021 9:07 PM +0000 petteri.stenius@ubisecure.com wrote:
Hello,
We have experienced a similar issue. Unfortunately we too have a confidential dataset we cannot share.
Our dataset started slowing down at about 1.7m entries. With slapcat/slapadd tools I can reliably reproduce the issue:
- first I add about 1.7m entries with slapadd to an empty mdb database
- I test with a search operation that returns a single entry and it
performs as expected, about 0.06s - next I add a single entry with slapadd
- after this the exactly same search operation that returns a single
entry slows down significantly, about 1.37s
With the new idlexp parameter set to for example value 17 this slow down issue no longer happens with our dataset.
I don't understand idlexp parameter well enough. My fear is that idlexp tuning is not actually fixing this issue, instead the issue is simply postponed.
If setting idlexp fixed it, then the issue was that one additional entry caused the index to collapse to a range with the default idlexp value, which is why changing the setting had an effect since it would stop it from being a range at the higher idlexp value.
I added documentation as to what the idlexp command does to the admin guide for OpenLDAP 2.5.6. You may want to read it, it applies to OpenLDAP 2.4 as well.
https://www.openldap.org/doc/admin25/slapdconf2.html#MDB%20Database%20Directives
Section 5.2.6.1
--Quanah
--
Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com