hello all,
I have a weird perf problem with a dynamic group. I wonder if it is normal, and if I can improve the situation. jump to the line marked "-------------> for the impatients, JUMP HERE :-)" if you want to skip the details
This is openldap 2.4.35 on debian 6 (built from source with debian defaults and patches)
here the relevant configuration: ------------ 8< ------------ dn: cn=module{0},cn=config ... olcModuleLoad: {4}dynlist.la
dn: olcDatabase={2}hdb,cn=config ... olcDbCacheSize: 4096 olcDbCheckpoint: 1024 10 olcDbConfig: {0}set_cachesize 2 0 0 olcDbConfig: {1}set_lk_max_objects 32768 olcDbConfig: {2}set_lk_max_locks 32768 olcDbConfig: {3}set_lk_max_lockers 1500 olcDbConfig: {4}set_lg_dir /var/lib/ldap/logs/dc=univ-reunion,dc=fr olcDbConfig: {5}set_lg_regionmax 262144 olcDbConfig: {6}set_lg_bsize 524288 olcDbIDLcacheSize: 12288
dn: olcOverlay={2}dynlist,olcDatabase={2}hdb,cn=config ... olcDlAttrSet: {0}groupOfURLs memberURL member ------------ 8< ------------
and the definition of the dynamic group: ------------ 8< ------------ dn: cn=users,ou=groups,ou=eduspot,dc=univ-reunion,dc=fr objectClass: groupOfURLs memberURL: ldap:///ou=users,ou=cas,dc=univ-reunion,dc=fr??one?(uid=*) memberURL: ldap:///ou=webaccounts,dc=univ-reunion,dc=fr??one?(&(uid=*)(runUnivCategorie=inconnu)(!(runUnivAuthorization=webaccounts:disabled))) ------------ 8< ------------ this result in a dynamic group with 45000 members
I dont know how the final application will use the dynamic group, but I did some tests on my own. now for the perf tests:
$ time ldapsearch -LLL -H "ldap://localhost" -x -D cn=admin,dc=univ-reunion,dc=fr -wadmin -b ou=users,ou=cas,dc=univ-reunion,dc=fr "(uid=*)" dn >/dev/null real 0m1.025s user 0m0.096s sys 0m0.196s
$ time ldapsearch -LLL -H "ldap://localhost" -x -D cn=admin,dc=univ-reunion,dc=fr -wadmin -b ou=webaccounts,dc=univ-reunion,dc=fr "(&(uid=*)(runUnivCategorie=inconnu)(!(runUnivAuthorization=webaccounts:disabled)))" dn >/dev/null real 0m0.043s user 0m0.004s sys 0m0.004s
$ time ldapsearch -LLL -H "ldap://localhost" -x -D cn=admin,dc=univ-reunion,dc=fr -wadmin -b ou=groups,ou=eduspot,dc=univ-reunion,dc=fr "(cn=users)" member >/dev/null real 0m34.334s user 0m0.024s sys 0m0.000s
-------------> for the impatients, JUMP HERE :-)
It needs more than 30 seconds to build the dynamic group! and this is the case every time I do the search
For the sake of it, I made a static group with 45000 member, and it takes 0.037 seconds for the search
So my question is: is it normal for the dynamic group to be so slow to be built? is there something I can do?
thanks in advance for your answers! best regards,
Am Tue, 07 May 2013 09:36:45 +0400 schrieb Jephte Clain jephte.clain@univ-reunion.fr:
hello all,
I have a weird perf problem with a dynamic group. I wonder if it is normal, and if I can improve the situation. jump to the line marked "-------------> for the impatients, JUMP HERE :-)" if you want to skip the details
This is openldap 2.4.35 on debian 6 (built from source with debian defaults and patches)
here the relevant configuration: ------------ 8< ------------ dn: cn=module{0},cn=config ... olcModuleLoad: {4}dynlist.la
dn: olcDatabase={2}hdb,cn=config ... olcDbCacheSize: 4096 olcDbCheckpoint: 1024 10 olcDbConfig: {0}set_cachesize 2 0 0 olcDbConfig: {1}set_lk_max_objects 32768 olcDbConfig: {2}set_lk_max_locks 32768 olcDbConfig: {3}set_lk_max_lockers 1500 olcDbConfig: {4}set_lg_dir /var/lib/ldap/logs/dc=univ-reunion,dc=fr olcDbConfig: {5}set_lg_regionmax 262144 olcDbConfig: {6}set_lg_bsize 524288 olcDbIDLcacheSize: 12288
dn: olcOverlay={2}dynlist,olcDatabase={2}hdb,cn=config ... olcDlAttrSet: {0}groupOfURLs memberURL member ------------ 8< ------------
and the definition of the dynamic group: ------------ 8< ------------ dn: cn=users,ou=groups,ou=eduspot,dc=univ-reunion,dc=fr objectClass: groupOfURLs memberURL: ldap:///ou=users,ou=cas,dc=univ-reunion,dc=fr??one?(uid=*) memberURL: ldap:///ou=webaccounts,dc=univ-reunion,dc=fr??one?(&(uid=*)(runUnivCategorie=inconnu)(!(runUnivAuthorization=webaccounts:disabled))) ------------ 8< ------------ this result in a dynamic group with 45000 members
I dont know how the final application will use the dynamic group, but I did some tests on my own. now for the perf tests:
$ time ldapsearch -LLL -H "ldap://localhost" -x -D cn=admin,dc=univ-reunion,dc=fr -wadmin -b ou=users,ou=cas,dc=univ-reunion,dc=fr "(uid=*)" dn >/dev/null real 0m1.025s user 0m0.096s sys 0m0.196s
$ time ldapsearch -LLL -H "ldap://localhost" -x -D cn=admin,dc=univ-reunion,dc=fr -wadmin -b ou=webaccounts,dc=univ-reunion,dc=fr "(&(uid=*)(runUnivCategorie=inconnu)(!(runUnivAuthorization=webaccounts:disabled)))" dn >/dev/null real 0m0.043s user 0m0.004s sys 0m0.004s
$ time ldapsearch -LLL -H "ldap://localhost" -x -D cn=admin,dc=univ-reunion,dc=fr -wadmin -b ou=groups,ou=eduspot,dc=univ-reunion,dc=fr "(cn=users)" member
/dev/null real 0m34.334s
user 0m0.024s sys 0m0.000s
-------------> for the impatients, JUMP HERE :-)
It needs more than 30 seconds to build the dynamic group! and this is the case every time I do the search
For the sake of it, I made a static group with 45000 member, and it takes 0.037 seconds for the search
So my question is: is it normal for the dynamic group to be so slow to be built? is there something I can do?
thanks in advance for your answers! best regards,
No, it is not normal. You probabely should search the base only and not a subtree. The result of some 30.000 entries: ldapsearch -x -H ldap://myhost -b cn=myDynamicGroup,o=avci,c=de -s base "(objectClass=*)" member > /dev/null
real 0m0.300s user 0m0.007s sys 0m0.003s
-Dieter
--On Tuesday, May 07, 2013 9:36 AM +0400 Jephte Clain jephte.clain@univ-reunion.fr wrote:
It needs more than 30 seconds to build the dynamic group! and this is the case every time I do the search
For the sake of it, I made a static group with 45000 member, and it takes 0.037 seconds for the search
So my question is: is it normal for the dynamic group to be so slow to be built? is there something I can do?
What about the second time you run a query on a subtree? For back-hdb, the first time it has load a bunch of data into the idl cache IIRC. Also, you do not show in your config if you have actually indexed the attributes to be used by the dynamic group. This of course would be a requirement. For example:
ldap:///ou=users,ou=cas,dc=univ-reunion,dc=fr??one?(uid=*)
pres index on uid
runUnivCategorie -> eq index runUnivAuthorizaion -> eq index
--Quanah
--
Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration
hello,
thanks for your answer. I will try it and post the results on friday.
2013/5/7 Dieter Klünter dieter@dkluenter.de:
Am Tue, 07 May 2013 09:36:45 +0400 schrieb Jephte Clain jephte.clain@univ-reunion.fr:
hello all,
I have a weird perf problem with a dynamic group. I wonder if it is normal, and if I can improve the situation. jump to the line marked "-------------> for the impatients, JUMP HERE :-)" if you want to skip the details
This is openldap 2.4.35 on debian 6 (built from source with debian defaults and patches)
here the relevant configuration: ------------ 8< ------------ dn: cn=module{0},cn=config ... olcModuleLoad: {4}dynlist.la
dn: olcDatabase={2}hdb,cn=config ... olcDbCacheSize: 4096 olcDbCheckpoint: 1024 10 olcDbConfig: {0}set_cachesize 2 0 0 olcDbConfig: {1}set_lk_max_objects 32768 olcDbConfig: {2}set_lk_max_locks 32768 olcDbConfig: {3}set_lk_max_lockers 1500 olcDbConfig: {4}set_lg_dir /var/lib/ldap/logs/dc=univ-reunion,dc=fr olcDbConfig: {5}set_lg_regionmax 262144 olcDbConfig: {6}set_lg_bsize 524288 olcDbIDLcacheSize: 12288
dn: olcOverlay={2}dynlist,olcDatabase={2}hdb,cn=config ... olcDlAttrSet: {0}groupOfURLs memberURL member ------------ 8< ------------
and the definition of the dynamic group: ------------ 8< ------------ dn: cn=users,ou=groups,ou=eduspot,dc=univ-reunion,dc=fr objectClass: groupOfURLs memberURL: ldap:///ou=users,ou=cas,dc=univ-reunion,dc=fr??one?(uid=*) memberURL: ldap:///ou=webaccounts,dc=univ-reunion,dc=fr??one?(&(uid=*)(runUnivCategorie=inconnu)(!(runUnivAuthorization=webaccounts:disabled))) ------------ 8< ------------ this result in a dynamic group with 45000 members
I dont know how the final application will use the dynamic group, but I did some tests on my own. now for the perf tests:
$ time ldapsearch -LLL -H "ldap://localhost" -x -D cn=admin,dc=univ-reunion,dc=fr -wadmin -b ou=users,ou=cas,dc=univ-reunion,dc=fr "(uid=*)" dn >/dev/null real 0m1.025s user 0m0.096s sys 0m0.196s
$ time ldapsearch -LLL -H "ldap://localhost" -x -D cn=admin,dc=univ-reunion,dc=fr -wadmin -b ou=webaccounts,dc=univ-reunion,dc=fr "(&(uid=*)(runUnivCategorie=inconnu)(!(runUnivAuthorization=webaccounts:disabled)))" dn >/dev/null real 0m0.043s user 0m0.004s sys 0m0.004s
$ time ldapsearch -LLL -H "ldap://localhost" -x -D cn=admin,dc=univ-reunion,dc=fr -wadmin -b ou=groups,ou=eduspot,dc=univ-reunion,dc=fr "(cn=users)" member
/dev/null real 0m34.334s
user 0m0.024s sys 0m0.000s
-------------> for the impatients, JUMP HERE :-)
It needs more than 30 seconds to build the dynamic group! and this is the case every time I do the search
For the sake of it, I made a static group with 45000 member, and it takes 0.037 seconds for the search
So my question is: is it normal for the dynamic group to be so slow to be built? is there something I can do?
thanks in advance for your answers! best regards,
No, it is not normal. You probabely should search the base only and not a subtree. The result of some 30.000 entries: ldapsearch -x -H ldap://myhost -b cn=myDynamicGroup,o=avci,c=de -s base "(objectClass=*)" member > /dev/null
real 0m0.300s user 0m0.007s sys 0m0.003s
-Dieter
-- Dieter Klünter | Systemberatung http://dkluenter.de GPG Key ID:DA147B05 53°37'09,95"N 10°08'02,42"E
openldap-technical@openldap.org
-
Dieter Klünter -
Jephte Clain -
Quanah Gibson-Mount