Hi,
I have a problem with mdb and modify operations on very large groups. Specifically deleting members from those groups. Removing 10 members from a group with 25000 members takes 23 seconds. Which also means, all other clients that want to do something hang. Deleting a user from multiple big groups takes minutes before it finishes. Adding members to a large group is quick though.
When this delete is running, the cpu goes also up to 100%.
It looks like it has to do with the index that I have on uniqueMember. If I remove the index on uniqueMember, the delete of members in big groups is fast.
System details are CentOS 6 64bit OpenLDAP 2.4.35 slapd.conf below
Is this something normal/exptected or is it maybe a bug?
Steps to reproduce - Install OpenLDAP 2.4.35 - Import a base ldif - Add a group with 25000 users - Remove 10 members from the group via ldapmodify
Example group dn: cn=x3zolgnanlmpzj1nfk21,ou=groups,dc=example,dc=com objectClass: top objectClass: groupOfUniqueNames ou: groups cn: x3zolgnanlmpzj1nfk21 description: aw88bob79vvqffv1fhii uniqueMember: uid=faefxus4e83ywhh7bbgw@ough6unnwjdx0zzqiy0i.e3j,ou=people,dc=example,dc=com ...
Below is the slapd.conf from a test system where I reproduced it. I used a very minimal config on the test system.
---- slapd.conf ---- include /usr/share/openldap2.4/schema/core.schema include /usr/share/openldap2.4/schema/cosine.schema include /usr/share/openldap2.4/schema/corba.schema include /usr/share/openldap2.4/schema/inetorgperson.schema include /usr/share/openldap2.4/schema/openldap.schema
pidfile /var/run/ldap2.4/slapd.pid argsfile /var/run/ldap2.4/slapd.args
modulepath /usr/lib64/oldap24/openldap2.4 moduleload back_monitor.la
loglevel stats sync
serverID 21 ldap://ldap.example.com
database mdb suffix "dc=example,dc=com" rootdn "cn=manager,dc=example,dc=com" rootpw secret directory /var/lib/ldap2.4/example.com dbnosync maxsize 107374182400
conn_max_pending_auth 2000
index objectClass eq index uniqueMember eq index entryCSN,entryUUID eq
monitoring on
database config rootdn "cn=admin,cn=config" rootpw secret
database monitor rootdn cn=monitor rootpw secret ---- slapd.conf ----
Thanks Marco
Marco Schirrmeister wrote:
Hi,
I have a problem with mdb and modify operations on very large groups. Specifically deleting members from those groups. Removing 10 members from a group with 25000 members takes 23 seconds. Which also means, all other clients that want to do something hang. Deleting a user from multiple big groups takes minutes before it finishes. Adding members to a large group is quick though.
When this delete is running, the cpu goes also up to 100%.
It looks like it has to do with the index that I have on uniqueMember. If I remove the index on uniqueMember, the delete of members in big groups is fast.
System details are CentOS 6 64bit OpenLDAP 2.4.35 slapd.conf below
Is this something normal/exptected or is it maybe a bug?
Read slapd.conf(5) manpage, sortvals keyword.
Hi!
It would be nice if there would exist a tool that reads (assuming those exist) slapd performance data and suggests modifications based on those. Of course that would requre timing statistics on specific attribute operations which do not exist ;-)
Howard Chu hyc@symas.com schrieb am 17.07.2013 um 01:08 in Nachricht
Marco Schirrmeister wrote:
Hi,
I have a problem with mdb and modify operations on very large groups.
Specifically deleting members from those groups.
Removing 10 members from a group with 25000 members takes 23 seconds. Which
also means, all other clients that want to do something hang.
Deleting a user from multiple big groups takes minutes before it finishes. Adding members to a large group is quick though.
When this delete is running, the cpu goes also up to 100%.
It looks like it has to do with the index that I have on uniqueMember. If I remove the index on uniqueMember, the delete of members in big groups
is fast.
System details are CentOS 6 64bit OpenLDAP 2.4.35 slapd.conf below
Is this something normal/exptected or is it maybe a bug?
Read slapd.conf(5) manpage, sortvals keyword.
You are suggesting to add "sortvals member"?
Regards, Ulrich
-- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
On Jul 17, 2013, at 1:08 AM, Howard Chu hyc@symas.com wrote:
Marco Schirrmeister wrote:
Hi,
I have a problem with mdb and modify operations on very large groups. Specifically deleting members from those groups. Removing 10 members from a group with 25000 members takes 23 seconds. Which also means, all other clients that want to do something hang. Deleting a user from multiple big groups takes minutes before it finishes. Adding members to a large group is quick though.
When this delete is running, the cpu goes also up to 100%.
It looks like it has to do with the index that I have on uniqueMember. If I remove the index on uniqueMember, the delete of members in big groups is fast.
System details are CentOS 6 64bit OpenLDAP 2.4.35 slapd.conf below
Is this something normal/exptected or is it maybe a bug?
Read slapd.conf(5) manpage, sortvals keyword.
Thanks for the hint Howard. Deleting members from a big group on the test server is now fast with sortvals.
Adding of members is a very little slower (20-30ms) compared to the adding without sortvals. I will see how it goes on the production environment with the real workload.
Thanks again Marco
Hi!
Just want to know: How does sortvals work? Are entries sorted in RAM only, or are the database entries (being stored) sorted? In the second case the delay (insert instead of append) would be explainable. For the first case there is no need to use an array implementation for the attributes: A balanced (search) tree would probably yield better insert performance.
Is there a reason not to combine a search tree implementation in RAM with an array implementation in the database?
Regards, Ulrich
Marco Schirrmeister marco@schirrmeister.net schrieb am 17.07.2013 um 11:32 in
Nachricht 51E33731-682C-4A55-8E1A-4F0D12588B61@schirrmeister.net:
On Jul 17, 2013, at 1:08 AM, Howard Chu hyc@symas.com wrote:
Marco Schirrmeister wrote:
Hi,
I have a problem with mdb and modify operations on very large groups.
Specifically deleting members from those groups.
Removing 10 members from a group with 25000 members takes 23 seconds. Which
also means, all other clients that want to do something hang.
Deleting a user from multiple big groups takes minutes before it finishes. Adding members to a large group is quick though.
When this delete is running, the cpu goes also up to 100%.
It looks like it has to do with the index that I have on uniqueMember. If I remove the index on uniqueMember, the delete of members in big groups
is fast.
System details are CentOS 6 64bit OpenLDAP 2.4.35 slapd.conf below
Is this something normal/exptected or is it maybe a bug?
Read slapd.conf(5) manpage, sortvals keyword.
Thanks for the hint Howard. Deleting members from a big group on the test server is now fast with sortvals.
Adding of members is a very little slower (20-30ms) compared to the adding without sortvals. I will see how it goes on the production environment with the real workload.
Thanks again Marco
--On Wednesday, July 17, 2013 11:32 AM +0200 Marco Schirrmeister marco@schirrmeister.net wrote:
On Jul 17, 2013, at 1:08 AM, Howard Chu hyc@symas.com wrote:
Marco Schirrmeister wrote:
Hi,
I have a problem with mdb and modify operations on very large groups. Specifically deleting members from those groups. Removing 10 members from a group with 25000 members takes 23 seconds. Which also means, all other clients that want to do something hang. Deleting a user from multiple big groups takes minutes before it finishes. Adding members to a large group is quick though.
When this delete is running, the cpu goes also up to 100%.
It looks like it has to do with the index that I have on uniqueMember. If I remove the index on uniqueMember, the delete of members in big groups is fast.
System details are CentOS 6 64bit OpenLDAP 2.4.35 slapd.conf below
Is this something normal/exptected or is it maybe a bug?
Read slapd.conf(5) manpage, sortvals keyword.
Thanks for the hint Howard. Deleting members from a big group on the test server is now fast with sortvals.
Adding of members is a very little slower (20-30ms) compared to the adding without sortvals. I will see how it goes on the production environment with the real workload.
Personally, I prefer dynamic groups. You may wish to investigate those as well.
--Quanah
--
Quanah Gibson-Mount Lead Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration
On Jul 17, 2013, at 7:08 PM, Quanah Gibson-Mount quanah@zimbra.com wrote:
--On Wednesday, July 17, 2013 11:32 AM +0200 Marco Schirrmeister marco@schirrmeister.net wrote:
On Jul 17, 2013, at 1:08 AM, Howard Chu hyc@symas.com wrote:
Marco Schirrmeister wrote:
Hi,
I have a problem with mdb and modify operations on very large groups. Specifically deleting members from those groups. Removing 10 members from a group with 25000 members takes 23 seconds. Which also means, all other clients that want to do something hang. Deleting a user from multiple big groups takes minutes before it finishes. Adding members to a large group is quick though.
When this delete is running, the cpu goes also up to 100%.
It looks like it has to do with the index that I have on uniqueMember. If I remove the index on uniqueMember, the delete of members in big groups is fast.
System details are CentOS 6 64bit OpenLDAP 2.4.35 slapd.conf below
Is this something normal/exptected or is it maybe a bug?
Read slapd.conf(5) manpage, sortvals keyword.
Thanks for the hint Howard. Deleting members from a big group on the test server is now fast with sortvals.
Adding of members is a very little slower (20-30ms) compared to the adding without sortvals. I will see how it goes on the production environment with the real workload.
Personally, I prefer dynamic groups. You may wish to investigate those as well.
We are already in the process of this. But various code needs to be updated which maintains right now the groups.
Those large groups are also relative new that we have them. That's why we did not notice the problem (or wrong config) earlier.
Marco
openldap-technical@openldap.org
-
Howard Chu -
Marco Schirrmeister -
Quanah Gibson-Mount -
Ulrich Windl