ldap query performance issue - openldap-technical

23 May 2013

      Hi all,
I have an openldap directory with about 7 million DNs, running openldap 2.4.31 with a BDB backend (4.6.21), running on CentOS 6.3.
The structure of the directory is like this, with suffix dc=x,dc=y
dc=x,dc=y 
   account=a,dc=x,dc=y
      mail=m,account=a,dc=x,dc=y           // Users
      ....   
      licenceId=l,account=a,dc=x,dc=y      // Licences, objectclass=licence
      ....
      group=g,account=a,dc=x,dc=y          // Groups
      ....
      // etc.

   account=b,dc=x,dc=y
      ....
Most of the DNs in the directory are users or groups, and the number of licences is small (<10) for each account.
If I do a query with basedn account=a,dc=x,dc=y and filter (objectclass=licence) I see wildly different performance, depending on how many users are under account a. For an account with ~30000 users the query takes 2 seconds at most, but for an account with ~60000 users 
the query takes 1 minute.
It only appears to be when I filter on objectclass=licence that I see that behaviour. If I filter on a different objectclass which matches a similar number of objects to the objectclass=licence filter, the performance doesn't seem to depend on the number of users.
There is an index on objectclass (of course), but the behaviour I'm seeing seems to indicate that for this query, at some point slapd stops using the index and just scans all the objects under the account.
Any ideas?
Chris