Referrals don't work like that. Read RFC4511: the <attrs> field is not mentioned. It mentions, indeed, the <filter> field, but OpenLDAP does not handle this. The behavior you possibly expect is not strictly specified, AFAIK.
I think you have a couple of options:
use ACLs to hide that entry to some specific clients
use a dummy proxy instead of a referral; the dummy proxy could
massage the request/response DNs, and the original server could use ACLs to hide that entry from the results returned to the proxy.
I tought OpenLDAP could support that kind of referrals. Now I think the best option is the best for my scenario.
The <attrs> issue makes little sense. The <filter> issue may be worth, although clear semantics need to be defined. I note that rfc 3296 appears to explicitly prohibit a filter in the LDAP URL, while rfc 4511 instructs the client about using the filter in the LDAP URL if present. This is not strictly a contradiction, because the server may modify the URL stored in the Named Subordinate Reference object before returning it to the client, e.g. by adding a filter.
For example, in order to obtain what I think you mean, a <filter> field in the Named Subordinate Reference object's URL, e.g. "(!(uid=admin))", would require the server, in case of a search with scope "subtree", base "ou=Paople,dc=example,dc=com" and filter "(attr=val)", to return a referral
ldap://host/ou=People,dc=example,dc=com??sub?(&(!(uid=admin))(attr=val))
The filter should be completely removed when performing other types of operations, in compliance with rfc3296.
If you think this makes sense and you would like to see it implemented, I suggest to file an ITS as a feature request, pending technical discussion http://www.openldap.org/its/.
p.