So you problem is, that you have signed your server cert with a CA from a CA chain and your clients with another CA and you don't want clients to connect, not signed by your client CA?
This sounds more like a case for ACLs and matching rules, since you AFAIK you cannot tell ldap to only trust a CA for server cert verification purposes. A CA is trusted or not.