Hello all,
We have a need to populate one of our replicas with mapped attributes long story short, using a ldap proxy to perform the mapping will not be sufficient since the attributes are being used by OpenLDAP itself.
I've found that when setting up a slapd-relay to virtualize the suffix and then use rwm to perform the mapping I get the desired view. However when trying to replicate the content using a replication account things start acting weird I think mostly because syncrepl processes changes via logs and update instructions from the provider to the consumer.
That being the case the syncrepl RefreshAndPersist actually changes the values from the original update instruction. Ergo the replica looking at the virtualized suffix (using relay) actually gets the attribute populated that I wanted mapped. Meaning the replica and the relay view don't agree rather the replica and the original data agree.
I've found that if I use the syncrepl RefreshOnly the replica yields the desired results however that is performed on intervals so the data isn't immediately available. So Have I missed something on the RefreshAndPersist method that would also yield the desired results? Or is RefreshAndPersist always going to see the original change being made even if the underlying relay view shows the mapped attributes?
Ugh the above is so complex to describe Below is an example to facilitate a quick explanation.
### Provider hdb database dn: cn=record,dc=suffix,dc=org cn: record attr1: value1 attr2: value2 attr3: value3
### Provider relay database dn: cn=record,dc=suffix,dc=org,cn=mapped cn: record attr1: value1 attr2: value3 | | attr3: value2 | Values switched |
#### { Replica Section } #####################################
### replica from cn=mapped using RefreshAndPersist (Undesired) dn: cn=record,dc=suffix,dc=org cn: record attr1: value1 attr2: value2 attr3: value3
### replica from cn=mapped using RefreshOnly (Desired) dn: cn=record,dc=suffix,dc=org,cn=mapped cn: record attr1: value1 attr2: value3 | | attr3: value2 | Values switched |
I hate to be answering my own posts, but I've had to abandon slapd-relay in favor of the (more advertised) slapd-ldap push replication method of which I can put the slapo-rwm overlay on top.
There seems to be some part of the syncrepl/syncprov that transmits the actual modification record instead of just telling the consumer to look at what it can actually see. This had the result of the attributes I was trying to map bypassing the rwm overlay rewrite rule and showing up after the initial correct sync.
the slapd-ldap + sycrepl + rwm seems to be working much better.
One problem I am having with the slapd-ldap syncrepl method is that I can't seem to integrate the slapd-ldap on the same instance as the consumer. (Which of course is not documented anyway) Trying to do so was causing any attempt to create or modify the syncrepl push proxy to hang. I also didn't want to place the proxy on the supplier server either so I settled for an additional proxy instance that sits in-between the provider and consumer. So far this has been stable in the functional testing (I haven't load tested yet).
I'm not sure if the syncrepl push proxy should be able to be on the same machine as the consumer or if that kind of configuration is undefined and not expected to work. If there is consensus of the former I'll create a ticket, but I don't want to waste time either.
Jeffrey
On 10/11/11 9:35 AM, Jeffrey Crawford wrote:
Hello all,
We have a need to populate one of our replicas with mapped attributes long story short, using a ldap proxy to perform the mapping will not be sufficient since the attributes are being used by OpenLDAP itself.
I've found that when setting up a slapd-relay to virtualize the suffix and then use rwm to perform the mapping I get the desired view. However when trying to replicate the content using a replication account things start acting weird I think mostly because syncrepl processes changes via logs and update instructions from the provider to the consumer.
That being the case the syncrepl RefreshAndPersist actually changes the values from the original update instruction. Ergo the replica looking at the virtualized suffix (using relay) actually gets the attribute populated that I wanted mapped. Meaning the replica and the relay view don't agree rather the replica and the original data agree.
I've found that if I use the syncrepl RefreshOnly the replica yields the desired results however that is performed on intervals so the data isn't immediately available. So Have I missed something on the RefreshAndPersist method that would also yield the desired results? Or is RefreshAndPersist always going to see the original change being made even if the underlying relay view shows the mapped attributes?
Ugh the above is so complex to describe Below is an example to facilitate a quick explanation.
### Provider hdb database dn: cn=record,dc=suffix,dc=org cn: record attr1: value1 attr2: value2 attr3: value3
### Provider relay database dn: cn=record,dc=suffix,dc=org,cn=mapped cn: record attr1: value1 attr2: value3 | | attr3: value2 | Values switched |
#### { Replica Section } #####################################
### replica from cn=mapped using RefreshAndPersist (Undesired) dn: cn=record,dc=suffix,dc=org cn: record attr1: value1 attr2: value2 attr3: value3
### replica from cn=mapped using RefreshOnly (Desired) dn: cn=record,dc=suffix,dc=org,cn=mapped cn: record attr1: value1 attr2: value3 | | attr3: value2 | Values switched |
Jeffrey Crawford wrote:
I hate to be answering my own posts, but I've had to abandon slapd-relay in favor of the (more advertised) slapd-ldap push replication method of which I can put the slapo-rwm overlay on top.
There seems to be some part of the syncrepl/syncprov that transmits the actual modification record instead of just telling the consumer to look at what it can actually see. This had the result of the attributes I was trying to map bypassing the rwm overlay rewrite rule and showing up after the initial correct sync.
the slapd-ldap + sycrepl + rwm seems to be working much better.
One problem I am having with the slapd-ldap syncrepl method is that I can't seem to integrate the slapd-ldap on the same instance as the consumer. (Which of course is not documented anyway) Trying to do so was causing any attempt to create or modify the syncrepl push proxy to hang. I also didn't want to place the proxy on the supplier server either so I settled for an additional proxy instance that sits in-between the provider and consumer. So far this has been stable in the functional testing (I haven't load tested yet).
I'm not sure if the syncrepl push proxy should be able to be on the same machine as the consumer or if that kind of configuration is undefined and not expected to work. If there is consensus of the former I'll create a ticket, but I don't want to waste time either.
No, that's not intended to work. Push replication using a slapd-ldap proxy was only developed to allow pushing to a slave on the other side of a firewall; as such the proxy would not normally reside in the same place as the replica.
Jeffrey
On 10/11/11 9:35 AM, Jeffrey Crawford wrote:
Hello all,
We have a need to populate one of our replicas with mapped attributes long story short, using a ldap proxy to perform the mapping will not be sufficient since the attributes are being used by OpenLDAP itself.
I've found that when setting up a slapd-relay to virtualize the suffix and then use rwm to perform the mapping I get the desired view. However when trying to replicate the content using a replication account things start acting weird I think mostly because syncrepl processes changes via logs and update instructions from the provider to the consumer.
That being the case the syncrepl RefreshAndPersist actually changes the values from the original update instruction. Ergo the replica looking at the virtualized suffix (using relay) actually gets the attribute populated that I wanted mapped. Meaning the replica and the relay view don't agree rather the replica and the original data agree.
I've found that if I use the syncrepl RefreshOnly the replica yields the desired results however that is performed on intervals so the data isn't immediately available. So Have I missed something on the RefreshAndPersist method that would also yield the desired results? Or is RefreshAndPersist always going to see the original change being made even if the underlying relay view shows the mapped attributes?
Ugh the above is so complex to describe Below is an example to facilitate a quick explanation.
### Provider hdb database dn: cn=record,dc=suffix,dc=org cn: record attr1: value1 attr2: value2 attr3: value3
### Provider relay database dn: cn=record,dc=suffix,dc=org,cn=mapped cn: record attr1: value1 attr2: value3 | | attr3: value2 | Values switched |
#### { Replica Section } #####################################
### replica from cn=mapped using RefreshAndPersist (Undesired) dn: cn=record,dc=suffix,dc=org cn: record attr1: value1 attr2: value2 attr3: value3
### replica from cn=mapped using RefreshOnly (Desired) dn: cn=record,dc=suffix,dc=org,cn=mapped cn: record attr1: value1 attr2: value3 | | attr3: value2 | Values switched |
openldap-technical@openldap.org