If the clients and servers are on the same machine, the servers
could listen to 'ldapi://<URL-escaped unique socket path>'
instead of to 'ldap://host:unique port/'. E.g. the users could
use 'ldapi://<escape $HOME/ldapi>' or '%2Ftmp%2Fldapi%2F<escape $USER>'.
However you'd need to upgrade OpenLDAP first, due to a security issue in
"ldapi://" implementations before 2.3.35. (In particular, ensure your
users won't use ldapi: with pre-2.3.35 clients against someone else's
server.)
--
Regards,
Hallvard