Whenever I set olcTLSCACertificateFile to /etc/pki/tls/certs/ca-bundle.crt LDAP clients get cert errors connecting to the server. But it works fine when I point olcTLSCACertificateFile at the actual server cert instead of the CA bundle.

With olcTLSCACertificateFile pointed at /etc/pki/tls/certs/ca-bundle.crt:
# LDAPTLS_CACERT=/etc/pki/tls/certs/ca-bundle.crt ldapsearch -H 'ldap://fqdn.to.my.server' -ZZ -x -b '' -s base '+'
ldap_start_tls: Connect error (-11)
    additional info: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed (unable to get local issuer certificate)
SERVER (from "slapd -d conns"):
TLS: can't accept: error:14094418:SSL routines:SSL3_READ_BYTES:tlsv1 alert unknown ca.

Why is this happening. olcTLSCACertificateFile is supposed to control the certificates that OpenLDAP will recognize, not affect the certificate it gives to clients (right?).

This is OpenLDAP 2.4.31