hume-ol@bofh.ca wrote:

Full_Name: Brandon Hume Version: -OPENLDAP_REL_ENG_2_4-0e03b13 OS: Redhat AS6 URL: http://den.bofh.ca/~hume/ldap_missing_results.tar.gz Submission from: (NULL) (2610:170:2c00:7:224:1ff:fed4:b4ff)

Thanks for the report and test case. Fixed now in git master.

Firstly, using RE24 built with the following ./configure:

#!/bin/sh env CC=gcc CFLAGS="-m64 -ggdb" LDFLAGS="-Wl,-rpath -Wl,/appl/ldap/lib -L/appl/ldap/lib" CPPFLAGS="-I/appl/ldap/include" \ ./configure --prefix=/appl/ldap \ --libexecdir=/appl/ldap/sbin --localstatedir=/var/run --datadir=/appl/ldap/data \ --sysconfdir=/appl/ldap/etc --mandir=/appl/ldap/man --with-subdir=no \ --disable-rlookups --with-tls=openssl \ --enable-slapd --enable-crypt --with-cyrus-sasl=no --enable-debug \ --enable-dynamic --enable-modules \ --enable-backends=mod --enable-overlays=mod --enable-sql=no \ --enable-perl=no --enable-ndb=no --enable-shell=no

Paired with BerkeleyDB built with the following:

#!/bin/sh env CC=gcc CFLAGS="-m64 -ggdb" LDFLAGS="-Wl,-rpath -Wl,/appl/ldap/lib -L/appl/ldap/lib" CPPFLAGS="-I/appl/ldap/include" \ ../dist/configure --prefix=/appl/ldap

DIT has 131207 entries. Search results will come and go as you shorten the search base. For example, when searching for "maillocaladdress=vbroken@dal.ca", which has the full DN of "cn=broken129251,cn=dal,cn=Mailmaps,cn=Services,dc=DAL,dc=CA", the following will occur:

 Searching with base=cn=dal,cn=Mailmaps,cn=services,dc=dal,dc=ca: Found.
 Searching with base=cn=Mailmaps,cn=services,dc=dal,dc=ca: NOT found.
 Searching with base=cn=services,dc=dal,dc=ca: NOT found.
 Searching with base=dc=dal,dc=ca: Found.

cn=dal has 123312 children, which is very high. However, breaking the branch up into subcontainers, none of which had more than 10k entries, did not cure the problem.

With olcLogLevel=any, the following is notable:

 Successful search:

Apr 5 21:09:01 kil-ds-3 slapd[17628]: => hdb_dn2idl("cn=dal,cn=mailmaps,cn=services,dc=dal,dc=ca") Apr 5 21:09:03 kil-ds-3 slapd[17628]: => bdb_filter_candidates Apr 5 21:09:03 kil-ds-3 slapd[17628]: #011AND Apr 5 21:09:03 kil-ds-3 slapd[17628]: => bdb_list_candidates 0xa0 Apr 5 21:09:03 kil-ds-3 slapd[17628]: => bdb_filter_candidates Apr 5 21:09:03 kil-ds-3 slapd[17628]: #011OR Apr 5 21:09:03 kil-ds-3 slapd[17628]: => bdb_list_candidates 0xa1 Apr 5 21:09:03 kil-ds-3 slapd[17628]: => bdb_filter_candidates Apr 5 21:09:03 kil-ds-3 slapd[17628]: #011EQUALITY Apr 5 21:09:03 kil-ds-3 slapd[17628]: => bdb_equality_candidates (objectClass) Apr 5 21:09:03 kil-ds-3 slapd[17628]: => key_read Apr 5 21:09:03 kil-ds-3 slapd[17628]: bdb_idl_fetch_key: [b49d1940] Apr 5 21:09:03 kil-ds-3 slapd[17628]:<= bdb_index_read: failed (-30988) Apr 5 21:09:03 kil-ds-3 slapd[17628]:<= bdb_equality_candidates: id=0, first=0, last=0 Apr 5 21:09:03 kil-ds-3 slapd[17628]:<= bdb_filter_candidates: id=0 first=0 last=0 Apr 5 21:09:03 kil-ds-3 slapd[17628]: => bdb_filter_candidates Apr 5 21:09:03 kil-ds-3 slapd[17628]: #011EQUALITY Apr 5 21:09:03 kil-ds-3 slapd[17628]: => bdb_equality_candidates (mailLocalAddress) Apr 5 21:09:03 kil-ds-3 slapd[17628]: => key_read Apr 5 21:09:03 kil-ds-3 slapd[17628]: bdb_idl_fetch_key: [217e141f] Apr 5 21:09:03 kil-ds-3 slapd[17628]:<= bdb_index_read 1 candidates Apr 5 21:09:03 kil-ds-3 slapd[17628]:<= bdb_equality_candidates: id=1, first=129301, last=129301 Apr 5 21:09:03 kil-ds-3 slapd[17628]:<= bdb_filter_candidates: id=1 first=129301 last=129301 Apr 5 21:09:03 kil-ds-3 slapd[17628]:<= bdb_list_candidates: id=1 first=129301 last=129301 Apr 5 21:09:03 kil-ds-3 slapd[17628]:<= bdb_filter_candidates: id=1 first=129301 last=129301 Apr 5 21:09:03 kil-ds-3 slapd[17628]:<= bdb_list_candidates: id=1 first=129301 last=129301 Apr 5 21:09:03 kil-ds-3 slapd[17628]:<= bdb_filter_candidates: id=1 first=129301 last=129301 Apr 5 21:09:03 kil-ds-3 slapd[17628]: bdb_search_candidates: id=1 first=129301 last=129301 Apr 5 21:09:03 kil-ds-3 slapd[17628]: entry_decode: "" Apr 5 21:09:03 kil-ds-3 slapd[17628]:<= entry_decode() Apr 5 21:09:03 kil-ds-3 slapd[17628]: => test_filter

 Unsuccessful search:

Apr 5 21:10:34 kil-ds-3 slapd[17628]: => hdb_dn2idl("cn=mailmaps,cn=services,dc=dal,dc=ca") Apr 5 21:10:34 kil-ds-3 slapd[17628]: => bdb_filter_candidates Apr 5 21:10:34 kil-ds-3 slapd[17628]: #011AND Apr 5 21:10:34 kil-ds-3 slapd[17628]: => bdb_list_candidates 0xa0 Apr 5 21:10:34 kil-ds-3 slapd[17628]: => bdb_filter_candidates Apr 5 21:10:34 kil-ds-3 slapd[17628]: #011OR Apr 5 21:10:34 kil-ds-3 slapd[17628]: => bdb_list_candidates 0xa1 Apr 5 21:10:34 kil-ds-3 slapd[17628]: => bdb_filter_candidates Apr 5 21:10:34 kil-ds-3 slapd[17628]: #011EQUALITY Apr 5 21:10:34 kil-ds-3 slapd[17628]: => bdb_equality_candidates (objectClass) Apr 5 21:10:34 kil-ds-3 slapd[17628]: => key_read Apr 5 21:10:34 kil-ds-3 slapd[17628]: bdb_idl_fetch_key: [b49d1940] Apr 5 21:10:34 kil-ds-3 slapd[17628]:<= bdb_index_read: failed (-30988) Apr 5 21:10:34 kil-ds-3 slapd[17628]:<= bdb_equality_candidates: id=0, first=0, last=0 Apr 5 21:10:34 kil-ds-3 slapd[17628]:<= bdb_filter_candidates: id=0 first=0 last=0 Apr 5 21:10:34 kil-ds-3 slapd[17628]: => bdb_filter_candidates Apr 5 21:10:34 kil-ds-3 slapd[17628]: #011EQUALITY Apr 5 21:10:34 kil-ds-3 slapd[17628]: => bdb_equality_candidates (mailLocalAddress) Apr 5 21:10:34 kil-ds-3 slapd[17628]: => key_read Apr 5 21:10:34 kil-ds-3 slapd[17628]: bdb_idl_fetch_key: [217e141f] Apr 5 21:10:34 kil-ds-3 slapd[17628]:<= bdb_index_read 1 candidates Apr 5 21:10:34 kil-ds-3 slapd[17628]:<= bdb_equality_candidates: id=1, first=129301, last=129301 Apr 5 21:10:34 kil-ds-3 slapd[17628]:<= bdb_filter_candidates: id=1 first=129301 last=129301 Apr 5 21:10:34 kil-ds-3 slapd[17628]:<= bdb_list_candidates: id=1 first=129301 last=129301 Apr 5 21:10:34 kil-ds-3 slapd[17628]:<= bdb_filter_candidates: id=1 first=129301 last=129301 Apr 5 21:10:34 kil-ds-3 slapd[17628]:<= bdb_list_candidates: id=0 first=3 last=0 Apr 5 21:10:34 kil-ds-3 slapd[17628]:<= bdb_filter_candidates: id=0 first=3 last=0 Apr 5 21:10:34 kil-ds-3 slapd[17628]: bdb_search_candidates: id=0 first=3 last=0 Apr 5 21:10:34 kil-ds-3 slapd[17628]: hdb_search: no candidates Apr 5 21:10:34 kil-ds-3 slapd[17628]: send_ldap_result: conn=1001 op=1 p=3 Apr 5 21:10:34 kil-ds-3 slapd[17628]: send_ldap_result: err=0 matched="" text="" Apr 5 21:10:34 kil-ds-3 slapd[17628]: send_ldap_response: msgid=2 tag=101 err=0 Apr 5 21:10:34 kil-ds-3 slapd[17628]: conn=1001 op=1 SEARCH RESULT tag=101 err=0 nentries=0 text=

I'm including a link to a tarball that contains: - My (simplified) config, that still produces the problem. I'm managed to remove locally-created schemas. - My test LDIF, containing the entries that can produce the problem. They've been anonymized, but it doesn't affect the problem. A server loaded with the above config and this DIT will be consistently unable to find the "maillocaladdress=vbroken@dal.ca" entry at cn=services,dc=dal,dc=ca search base. - My test script, test_ldap.sh, that searches for the above entry at the levels in between it and the base naming context (dc=dal,dc=ca).

I can provide more if needed, up to and including the compiled binaries.