https://bugs.openldap.org/show_bug.cgi?id=9692
Issue ID: 9692 Summary: Insertion rate in large groups slows unexpectedly Product: OpenLDAP Version: 2.5.7 Hardware: All OS: All Status: UNCONFIRMED Keywords: needs_review Severity: normal Priority: --- Component: slapd Assignee: bugs@openldap.org Reporter: smckinney@symas.com Target Milestone: ---
Created attachment 843 --> https://bugs.openldap.org/attachment.cgi?id=843&action=edit slapd.conf
Observed during jmeter tests[1] that perform ldapmod operations, adding members into a group. The insertion speed decreases unexpectedly, as the size of the group increases.
A test run starts 10 jmeter threads, each doing 1000 mods = 100,000 members added altogether to a group.
At the beginning of the test, throughput is approximately 200/s. At the end, the mod rate slows down to < 10/s.
Multival and sortval are enabled (slapd.conf attached): sortvals member multival member 500,3
*** Server info Ubuntu20 2 CPU Cores 4GB RAM symas-openldap-server 2.5.7-1focal1
*** To verify multival is enabled:
``` data/dc=example,dc=com# mdb_stat -s id2v . Status of id2v Tree depth: 1 Branch pages: 0 Leaf pages: 1 Overflow pages: 0 Entries: 746316 ```
[1][ldap-load-gen](https://gitlab.symas.net/symas-public/ldap-load-gen)
https://bugs.openldap.org/show_bug.cgi?id=9692
--- Comment #1 from Shawn McKinney smckinney@symas.com ---
A test run starts 10 jmeter threads, each doing 1000 mods = 100,000 members added altogether to a group.
correction:
A test run starts 10 jmeter threads, each doing 10,000 mods = 100,000 members added altogether to a group.
https://bugs.openldap.org/show_bug.cgi?id=9692
--- Comment #2 from Shawn McKinney smckinney@symas.com --- Udpate:
Added this switch to slapd.conf:
index_hash64 on
Due to concern over (possible) hash collisions, as members with similar ids are inserted.
The load utility generates uid's with sequence numbers: {hostname-qualifier-1, hostname-qualifier-2, ... hostname-qualifier-n}
This change to the config didn't help/change the outcome of the test.
https://bugs.openldap.org/show_bug.cgi?id=9692
Quanah Gibson-Mount quanah@openldap.org changed:
What |Removed |Added ---------------------------------------------------------------------------- Target Milestone|--- |2.5.8 Keywords|needs_review |
https://bugs.openldap.org/show_bug.cgi?id=9692
Quanah Gibson-Mount quanah@openldap.org changed:
What |Removed |Added ---------------------------------------------------------------------------- Status|UNCONFIRMED |RESOLVED Resolution|--- |SUSPENDED Target Milestone|2.5.8 |---
--- Comment #3 from Quanah Gibson-Mount quanah@openldap.org --- OpenLDAP performs better in this area than other tested directory servers.
https://bugs.openldap.org/show_bug.cgi?id=9692
Quanah Gibson-Mount quanah@openldap.org changed:
What |Removed |Added ---------------------------------------------------------------------------- Status|RESOLVED |VERIFIED
https://bugs.openldap.org/show_bug.cgi?id=9692
--- Comment #4 from Ondřej Kuzník ondra@mistotebe.net --- For posterity, the assumption was that adding a new value to a large multival enabled attribute would cost on the order of O(log n) where n is the number of existing values, however testing showed that the time it took to actually perform this was at O(n).
Further analysis showed that the cost was due to back-mdb (just like other backends) having to materialise the whole entry before modification to facilitate schema and ACL checking. The time to perform actual insertion was dwarfed by this as would be expected.
-
openldap-its＠openldap.org