----------------------------------------
From: ctcard@hotmail.com To: quanah@zimbra.com Subject: RE: ldap query performance issue Date: Thu, 23 May 2013 17:37:18 +0000
Date: Thu, 23 May 2013 10:06:51 -0700 From: quanah@zimbra.com To: ctcard@hotmail.com; openldap-technical@openldap.org Subject: Re: ldap query performance issue
--On Thursday, May 23, 2013 4:40 PM +0000 Chris Card ctcard@hotmail.com wrote:
Hi all,
I have an openldap directory with about 7 million DNs, running openldap 2.4.31 with a BDB backend (4.6.21), running on CentOS 6.3.
The structure of the directory is like this, with suffix dc=x,dc=y
dc=x,dc=y account=a,dc=x,dc=y mail=m,account=a,dc=x,dc=y // Users .... licenceId=l,account=a,dc=x,dc=y // Licences, objectclass=licence .... group=g,account=a,dc=x,dc=y // Groups .... // etc.
account=b,dc=x,dc=y ....
Most of the DNs in the directory are users or groups, and the number of licences is small (<10) for each account.
If I do a query with basedn account=a,dc=x,dc=y and filter (objectclass=licence) I see wildly different performance, depending on how many users are under account a. For an account with ~30000 users the query takes 2 seconds at most, but for an account with ~60000 users the query takes 1 minute.
It only appears to be when I filter on objectclass=licence that I see that behaviour. If I filter on a different objectclass which matches a similar number of objects to the objectclass=licence filter, the performance doesn't seem to depend on the number of users.
There is an index on objectclass (of course), but the behaviour I'm seeing seems to indicate that for this query, at some point slapd stops using the index and just scans all the objects under the account.
Any ideas?
Increase the IDL range. This is how I do it:
--- openldap-2.4.35/servers/slapd/back-bdb/idl.h.orig 2011-02-17 16:32:02.598593211 -0800 +++ openldap-2.4.35/servers/slapd/back-bdb/idl.h 2011-02-17 16:32:08.937757993 -0800 @@ -20,7 +20,7 @@ /* IDL sizes - likely should be even bigger
- limiting factors: sizeof(ID), thread stack size
*/ -#define BDB_IDL_LOGN 16 /* DB_SIZE is 2^16, UM_SIZE is 2^17 */ +#define BDB_IDL_LOGN 17 /* DB_SIZE is 2^16, UM_SIZE is 2^17 */ #define BDB_IDL_DB_SIZE (1<<BDB_IDL_LOGN) #define BDB_IDL_UM_SIZE (1<<(BDB_IDL_LOGN+1)) #define BDB_IDL_UM_SIZEOF (BDB_IDL_UM_SIZE * sizeof(ID))
Thanks, that looks like it might be the issue. Unfortunately I only see the issue in production, so patching it might be a pain.
I've tried this change, but it made no difference to the performance of the query.
Chris
Chris Card wrote:
Any ideas?
Increase the IDL range. This is how I do it:
--- openldap-2.4.35/servers/slapd/back-bdb/idl.h.orig 2011-02-17 16:32:02.598593211 -0800 +++ openldap-2.4.35/servers/slapd/back-bdb/idl.h 2011-02-17 16:32:08.937757993 -0800 @@ -20,7 +20,7 @@ /* IDL sizes - likely should be even bigger
- limiting factors: sizeof(ID), thread stack size
*/ -#define BDB_IDL_LOGN 16 /* DB_SIZE is 2^16, UM_SIZE is 2^17 */ +#define BDB_IDL_LOGN 17 /* DB_SIZE is 2^16, UM_SIZE is 2^17 */ #define BDB_IDL_DB_SIZE (1<<BDB_IDL_LOGN) #define BDB_IDL_UM_SIZE (1<<(BDB_IDL_LOGN+1)) #define BDB_IDL_UM_SIZEOF (BDB_IDL_UM_SIZE * sizeof(ID))
Thanks, that looks like it might be the issue. Unfortunately I only see the issue in production, so patching it might be a pain.
I've tried this change, but it made no difference to the performance of the query.
You have to re-create all of the relevant indices as well. Also, it's always possible that some slots in your index are still too big, even for this increased size.
You should also test this query with your data loaded into back-mdb.
Hello,
had the same problem years ago and the patch worked for me. As I understood, this special problem exist in mdb too (http://www.openldap.org/lists/openldap-technical/201301/msg00185.html) Thats one reason, because I did not switch till now.
Thanks Meike
2013/5/24 Howard Chu hyc@symas.com:
Chris Card wrote:
Any ideas?
Increase the IDL range. This is how I do it:
--- openldap-2.4.35/servers/slapd/back-bdb/idl.h.orig 2011-02-17 16:32:02.598593211 -0800 +++ openldap-2.4.35/servers/slapd/back-bdb/idl.h 2011-02-17 16:32:08.937757993 -0800 @@ -20,7 +20,7 @@ /* IDL sizes - likely should be even bigger
- limiting factors: sizeof(ID), thread stack size
*/ -#define BDB_IDL_LOGN 16 /* DB_SIZE is 2^16, UM_SIZE is 2^17 */ +#define BDB_IDL_LOGN 17 /* DB_SIZE is 2^16, UM_SIZE is 2^17 */ #define BDB_IDL_DB_SIZE (1<<BDB_IDL_LOGN) #define BDB_IDL_UM_SIZE (1<<(BDB_IDL_LOGN+1)) #define BDB_IDL_UM_SIZEOF (BDB_IDL_UM_SIZE * sizeof(ID))
Thanks, that looks like it might be the issue. Unfortunately I only see the issue in production, so patching it might be a pain.
I've tried this change, but it made no difference to the performance of the query.
You have to re-create all of the relevant indices as well. Also, it's always possible that some slots in your index are still too big, even for this increased size.
You should also test this query with your data loaded into back-mdb.
-- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Sorry for top posting, google web client is hiding always the message while answering *grrr*
Meike
Meike Stone wrote:
Hello,
had the same problem years ago and the patch worked for me. As I understood, this special problem exist in mdb too (http://www.openldap.org/lists/openldap-technical/201301/msg00185.html) Thats one reason, because I did not switch till now.
Yes, back-mdb uses the same index design, but it is still inherently faster than BDB backends.
Thanks Meike
2013/5/24 Howard Chu hyc@symas.com:
Chris Card wrote:
Any ideas?
Increase the IDL range. This is how I do it:
--- openldap-2.4.35/servers/slapd/back-bdb/idl.h.orig 2011-02-17 16:32:02.598593211 -0800 +++ openldap-2.4.35/servers/slapd/back-bdb/idl.h 2011-02-17 16:32:08.937757993 -0800 @@ -20,7 +20,7 @@ /* IDL sizes - likely should be even bigger
- limiting factors: sizeof(ID), thread stack size
*/ -#define BDB_IDL_LOGN 16 /* DB_SIZE is 2^16, UM_SIZE is 2^17 */ +#define BDB_IDL_LOGN 17 /* DB_SIZE is 2^16, UM_SIZE is 2^17 */ #define BDB_IDL_DB_SIZE (1<<BDB_IDL_LOGN) #define BDB_IDL_UM_SIZE (1<<(BDB_IDL_LOGN+1)) #define BDB_IDL_UM_SIZEOF (BDB_IDL_UM_SIZE * sizeof(ID))
Thanks, that looks like it might be the issue. Unfortunately I only see the issue in production, so patching it might be a pain.
I've tried this change, but it made no difference to the performance of the query.
You have to re-create all of the relevant indices as well. Also, it's always possible that some slots in your index are still too big, even for this increased size.
You should also test this query with your data loaded into back-mdb.
-- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
--On Friday, May 24, 2013 5:38 PM +0200 Meike Stone meike.stone@googlemail.com wrote:
Hello,
had the same problem years ago and the patch worked for me. As I understood, this special problem exist in mdb too (http://www.openldap.org/lists/openldap-technical/201301/msg00185.html) Thats one reason, because I did not switch till now.
I haven't seen this issue with mdb myself. Or maybe it is just too fast for it to really manifest in the same way.
--Quanah
--
Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration
----------------------------------------
Date: Fri, 24 May 2013 07:48:55 -0700 From: hyc@symas.com To: ctcard@hotmail.com; openldap-technical@openldap.org Subject: Re: ldap query performance issue
Chris Card wrote:
Any ideas?
Increase the IDL range. This is how I do it:
--- openldap-2.4.35/servers/slapd/back-bdb/idl.h.orig 2011-02-17 16:32:02.598593211 -0800 +++ openldap-2.4.35/servers/slapd/back-bdb/idl.h 2011-02-17 16:32:08.937757993 -0800 @@ -20,7 +20,7 @@ /* IDL sizes - likely should be even bigger
- limiting factors: sizeof(ID), thread stack size
*/ -#define BDB_IDL_LOGN 16 /* DB_SIZE is 2^16, UM_SIZE is 2^17 */ +#define BDB_IDL_LOGN 17 /* DB_SIZE is 2^16, UM_SIZE is 2^17 */ #define BDB_IDL_DB_SIZE (1<<BDB_IDL_LOGN) #define BDB_IDL_UM_SIZE (1<<(BDB_IDL_LOGN+1)) #define BDB_IDL_UM_SIZEOF (BDB_IDL_UM_SIZE * sizeof(ID))
Thanks, that looks like it might be the issue. Unfortunately I only see the issue in production, so patching it might be a pain.
I've tried this change, but it made no difference to the performance of the query.
You have to re-create all of the relevant indices as well. Also, it's always possible that some slots in your index are still too big, even for this increased size.
Thanks Howard. I had to increase BDB_IDL_LOGN to 18 (also increasing the stack size) and rebuild the objectclass index, but the query is now sub-second.
You should also test this query with your data loaded into back-mdb.
I intend to look at using back-mdb instead of back-bdb soon, as well as upgrading to the latest openldap release. When do you expect 2.4.36 to be available?
Chris
openldap-technical@openldap.org
-
Chris Card -
Howard Chu -
Meike Stone -
Quanah Gibson-Mount