Am 15.04.25 um 18:07 schrieb Howard Chu:
Norbert wrote:
Am 08.04.25 um 22:56 schrieb Norbert:
Hi,
Am 08.04.25 um 12:32 schrieb Ondřej Kuzník:
just a thought:
It looks like you also have a "sub"string index on that attribute, all indexes for a given attribute exist in the same namespace and a substring index generates a *lot* of items. So you'll get false positives competing for slapd's attention - have you enabled 64bit hashes already ("index_hash64 on")?
Should help with the contention if you haven't yet.
I did two further tests:
- olcIndexHash64: TRUE
- olcIndexHash64: TRUE and only keeping eq for almost_uniqe_attr
in both cases config and data was wiped and re-created with slapadd I confirmed that keysize is now 64bit in the index.
mdb_stat for the index with eq,sub and 32bit index keys from a running server Status of almost_uniqe_attr Tree depth: 3 Branch pages: 256 Leaf pages: 47269 Overflow pages: 0 Entries: 47486472
mdb_stat for index with eq only and 64bit index keys after fresh import Status of almost_uniqe_attr Tree depth: 4 Branch pages: 261 Leaf pages: 41908 Overflow pages: 0 Entries: 3931262
Unfortunately there was no change in runtime. The 1200 queries still take around 11s, might be even a tiny bit slower with 12s.
When running those 1200 filters and recording activity with perf in parallel I get at the top
The best way to diagnose this is to run a single search while gdb'ing slapd and check what two IDLs are being operated on in mdb_idl_intersection. Considering that 24% of CPU time is in the mdb_idl_next plt, you're seeing a ton of overhead simply from this backend being built as a dynamic module. You might be able to eliminate this overhead by adding -Bsymbolic to the linker invocation for back-mdb.
I managed to gdb it a bit. I put a breakpoint at line 727 in idl.c (2.5.19):
idmin = IDL_MAX( MDB_IDL_FIRST(a), MDB_IDL_FIRST(b) ); idmax = IDL_MIN( MDB_IDL_LAST(a), MDB_IDL_LAST(b) );
if ( idmin > idmax ) {
When almost_uniqe_attr has only 1 value then idmin == idmax. But in case of two values for a almost_uniqe_attr key I see that idmin = 1759457 and idmax = 2731413 so they are 971956 apart.