Unfortunately, I think the spec is broken here. There's no way for the consumer to record the references it receives, because they're not accompanied by the DN of the entry that triggered the reference. It would have made more sense for the provider to send the references uninterpreted, as regular LDAP entries.
Any suggestions?