From: ctcard(a)hotmail.com To: quanah(a)zimbra.com Subject: RE: ldap query performance issue Date: Thu, 23 May 2013 17:37:18 +0000 ---------------------------------------- > Date: Thu, 23 May 2013 10:06:51 -0700 > From: quanah(a)zimbra.com > To: ctcard(a)hotmail.com; openldap-technical(a)openldap.org > Subject: Re: ldap query performance issue > > --On Thursday, May 23, 2013 4:40 PM +0000 Chris Card <ctcard(a)hotmail.com&gt; > wrote: > >> Hi all, >> >> I have an openldap directory with about 7 million DNs, running openldap >> 2.4.31 with a BDB backend (4.6.21), running on CentOS 6.3. >> >> The structure of the directory is like this, with suffix dc=x,dc=y >> >> dc=x,dc=y >> account=a,dc=x,dc=y >> mail=m,account=a,dc=x,dc=y // Users >> .... >> licenceId=l,account=a,dc=x,dc=y // Licences, >> objectclass=licence .... >> group=g,account=a,dc=x,dc=y // Groups >> .... >> // etc. >> >> account=b,dc=x,dc=y >> .... >> >> Most of the DNs in the directory are users or groups, and the number of >> licences is small (<10) for each account. >> >> If I do a query with basedn account=a,dc=x,dc=y and filter >> (objectclass=licence) I see wildly different performance, depending on >> how many users are under account a. For an account with ~30000 users the >> query takes 2 seconds at most, but for an account with ~60000 users the >> query takes 1 minute. >> >> It only appears to be when I filter on objectclass=licence that I see >> that behaviour. If I filter on a different objectclass which matches a >> similar number of objects to the objectclass=licence filter, the >> performance doesn't seem to depend on the number of users. >> >> There is an index on objectclass (of course), but the behaviour I'm >> seeing seems to indicate that for this query, at some point slapd stops >> using the index and just scans all the objects under the account. >> >> Any ideas? > > Increase the IDL range. This is how I do it: > > --- openldap-2.4.35/servers/slapd/back-bdb/idl.h.orig 2011-02-17 > 16:32:02.598593211 -0800 > +++ openldap-2.4.35/servers/slapd/back-bdb/idl.h 2011-02-17 > 16:32:08.937757993 -0800 > @@ -20,7 +20,7 @@ > /* IDL sizes - likely should be even bigger > * limiting factors: sizeof(ID), thread stack size > */ > -#define BDB_IDL_LOGN 16 /* DB_SIZE is 2^16, UM_SIZE is 2^17 > */ > +#define BDB_IDL_LOGN 17 /* DB_SIZE is 2^16, UM_SIZE is 2^17 > */ > #define BDB_IDL_DB_SIZE (1<<BDB_IDL_LOGN) > #define BDB_IDL_UM_SIZE (1<<(BDB_IDL_LOGN+1)) > #define BDB_IDL_UM_SIZEOF (BDB_IDL_UM_SIZE * sizeof(ID)) Thanks, that looks like it might be the issue. Unfortunately I only see the issue in production, so patching it might be a pain.