Thanks for responding, Howard. That does sound similar, but I don't think it's what I'm hitting. I'm using back-hdb. I thought that an indexing issue might be relevant so I did some further testing:

Ubuntu 18's slapd config comes with "olcDbIndex: cn,uid eq” so that’s what I’ve been using so far.

With that index in place: -b dc=example,dc=com 5487.20/sec -b ou=People,dc=example,dc=com 194.30/sec

With that index removed: -b dc=example,dc=com 1.00/sec -b ou=People,dc=example,dc=com 0.95/sec

I added a "uid eq" index and waited a few minutes until the CPU load from slapd returned to negligible - that's the only way I know to tell when an index is done rebuilding.

With the new index: -b dc=example,dc=com 5550.20/sec. -b ou=People,dc=example,dc=com 198.75/sec.

This is with the following config:

# {1}hdb, config dn: olcDatabase={1}hdb,cn=config objectClass: olcDatabaseConfig objectClass: olcHdbConfig olcDatabase: {1}hdb olcDbDirectory: /var/lib/ldap olcSuffix: dc=example,dc=com olcAccess: {0}to attrs=userPassword by self write by anonymous auth by * none olcAccess: {1}to attrs=shadowLastChange by self write by * read olcAccess: {2}to * by * read olcLastMod: TRUE olcRootDN: cn=admin,dc=carnegielearning,dc=com olcRootPW: {SSHA}HXcLFZzpcjFGgjV89OeR7rbfjBVGZHx0 olcDbCheckpoint: 512 30 olcDbConfig: {0}set_cachesize 0 2097152 0 olcDbConfig: {1}set_lk_max_objects 1500 olcDbConfig: {2}set_lk_max_locks 1500 olcDbConfig: {3}set_lk_max_lockers 1500 olcDbIndex: objectClass eq olcDbIndex: uidNumber,gidNumber eq olcDbIndex: member,memberUid eq olcDbIndex: uid eq

I can provide slapcats of both the config and the dataset, or just an LDIF of the 100K inetOrgPerson objects if anybody wants to see if they can replicate this.

From: Howard Chu hyc@symas.commailto:hyc@symas.com Date: September 19, 2018 at 10:29:00 AM To: Glover, Matthew mglover@carnegielearning.commailto:mglover@carnegielearning.com, openldap-technical@openldap.org openldap-technical@openldap.orgmailto:openldap-technical@openldap.org Subject: [Possible Spam] Re: Worse search performance on a branch than at the tree base

Glover, Matthew wrote:

I'm having a performance problem with searches on a branch and I'm hoping someone can explain it.

A search for "(uid=12345)" on basedn "dc=example,dc=com" with a subtree scope returns much more quickly than the same filter and scope on basedn "ou=People,dc=example,dc=com".

Sounds vaguely like ITS#7473 but that was fixed 5 years ago. Maybe your index is not configured?

To confirm this wasn't just a problem with my schema or configuration, I loaded Ubuntu 18 (slapd 2.4.45+dfsg-1ubuntu1), used ldctl to generate 100,000 inetOrgPerson records with uids from 00000 to 99999, loaded them in with ldapadd, then used ldctl to test performance of searches:

Both tests were scope subtree, 20 seconds, hitting the test server as fast as possible with randomly generated filters of "(uid=<random uid from 00000 to 99999>)". basedn: dc=example,dc=com - 5455.30/sec - total: 109106 successful searches basedn: ou=People,dc=example,dc=com - 198.80/sec - total: 3976 successful searches

Also, while testing against ou=People, the CPU load from slaps went through the roof. It feels rather like the index isn’t being used when the base isn’t the root.

I can provide the ldif of my test users and the parameters I used for ldctl tests if desired.

Thanks for any help you can offer, Matthew

-- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/