Antw: [EXT] Re: Index seems to return wrong amount of candidate causing really poor search performance
Quanah Gibson-Mount quanah@symas.com schrieb am 16.08.2021 um 23:20 in
Nachricht <45379D5CBFA94DE3B1EA38E5@[192.168.1.4]>:
‑‑On Monday, August 16, 2021 10:00 PM +0000 Petteri Stenius Petteri.Stenius@ubisecure.com wrote:
Thank you for your quick response.
If idlexp is the accepted solution then I'd like to understand how to choose correct value for idlexp?
I have quickly tested with most values. With my quick tests I could not find any significant impact on performance. Setting idlexp to maximum 31 did however cause slapd to crash with segmentation fault on my system.
The appropriate value for any environment is entirely dependent on that environment's indexing and attribute value distribution for those indexed attributes. You generally want the minimum value for idlexp that allows searches to function without performance problems. Increasing the idlexp size increases slapd memory usage. Keep in mind that every increase in the
idlexp value increases the index slot range by a power of 2. The largest
Did you mean "to a power of 2", or do you really mean "by a power of 2"?
value I've ever needed for a massively large db with wide value distributions was 22.
...
Regards, Ulrich
--On Wednesday, August 18, 2021 9:15 AM +0200 Ulrich Windl Ulrich.Windl@rz.uni-regensburg.de wrote:
idlexp value increases the index slot range by a power of 2. The largest
Did you mean "to a power of 2", or do you really mean "by a power of 2"?
It increases by a power of 2, as documented in the man page.
idlexp _exp_ Specify a power of 2 for the maximum size of an index slot. The default is 16, yielding a maximum slot size of 2^16 or 65536. Once set, this option applies to every mdb database instance. The specified value must be in the range of 16-31.
--Quanah
--
Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
Quanah Gibson-Mount quanah@symas.com schrieb am 18.08.2021 um 17:34 in
Nachricht <1ACF53407B440BCD96A18A3F@[192.168.1.4]>:
‑‑On Wednesday, August 18, 2021 9:15 AM +0200 Ulrich Windl <Ulrich.Windl@rz.uni‑regensburg.de> wrote:
idlexp value increases the index slot range by a power of 2. The largest
Did you mean "to a power of 2", or do you really mean "by a power of 2"?
It increases by a power of 2, as documented in the man page.
idlexp _exp_ Specify a power of 2 for the maximum size of an index slot.The default is 16, yielding a maximum slot size of 2^16 or 65536. Once set, this option applies to every mdb database instance. The specified value must be in the range of 16‑31.
Still I'm slighly confused: If the default is 16 and you set it to (let's say) 18, by which power of two will the value be increased? 2^18 - 2^16 is not a power of 2, so you could increase the value three times by 2^16 (one to make 2^17, and another two to make 2^18). But I still guess the value is simply set to 2^18, whatever the increase from the default might be. Can you confirm?
Regards, Ulrich
‑‑Quanah
‑‑
Quanah Gibson‑Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
--On Thursday, August 19, 2021 9:18 AM +0200 Ulrich Windl Ulrich.Windl@rz.uni-regensburg.de wrote:
Still I'm slighly confused: If the default is 16 and you set it to (let's say) 18, by which power of two will the value be increased? 2^18 - 2^16 is not a power of 2, so you could increase the value three times by 2^16 (one to make 2^17, and another two to make 2^18). But I still guess the value is simply set to 2^18, whatever the increase from the default might be.
If the value is 16, it will be 2^16 = 65536. If the value is 18, it will be 2^18 which is 262,144. So yes, the difference between them isn't a power of 2, but the point was the max value is always a power of 2.
I would also note that every increment will cause slapd to require more memory. Larger values (such as 30) would require several terrabytes of RAM for slapd to function.
--Quanah
--
Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
openldap-technical@openldap.org
-
Quanah Gibson-Mount -
Ulrich Windl