Hi there,
First of all, I'm new to this list, so, please, forgive me if this is a wrong place for the questions below, and feel free to redirect me wherever is more appropriate.
The issue I'm facing as stated above is regarding the syncrepl and attribute order. I'm writing a Perl module to be used with back_perl and noticed behaviour that seems strange. Making some modifications on the master server (provider) and looking on the relevant modifications on the slave I saw that they didn't quite match. There were some extra REPLACE operations made by syncrepl. Later investigation showed that the cause of those REPLACE's was different attribute order on the master and slave nodes. synrepl.c says that:
/* We assume that attributes are saved in the same order * in the remote and local databases. So if we walk through * the attributeDescriptions one by one they should match in * lock step. If not, look for an add or delete. */
Which seems strange to me. As a result syncrepl makes a REPLACE for every attribute whuch is "misplaced" in the local object.
This is probably not an issue if master and slave both use the same back-end module. Which is not true in my case.
So, the questions are:
Is that replacing of "misplaced" attributes by syncrepl is expected behaviour or just a side effect of its syncrepl_diff_entry diff'ing algorithm?
Does attribute order matter? Is it specified somehow (sorted by modification time?)?
Should this issue be reported as bug?
Thank you.
Evgeniy Kosov wrote:
Hi there,
First of all, I'm new to this list, so, please, forgive me if this is a wrong place for the questions below, and feel free to redirect me wherever is more appropriate.
The issue I'm facing as stated above is regarding the syncrepl and attribute order. I'm writing a Perl module to be used with back_perl and noticed behaviour that seems strange. Making some modifications on the master server (provider) and looking on the relevant modifications on the slave I saw that they didn't quite match. There were some extra REPLACE operations made by syncrepl. Later investigation showed that the cause of those REPLACE's was different attribute order on the master and slave nodes. synrepl.c says that:
/* We assume that attributes are saved in the same order * in the remote and local databases. So if we walk through * the attributeDescriptions one by one they should match in * lock step. If not, look for an add or delete. */
Which seems strange to me. As a result syncrepl makes a REPLACE for every attribute whuch is "misplaced" in the local object.
This is probably not an issue if master and slave both use the same back-end module. Which is not true in my case.
So, the questions are:
Is that replacing of "misplaced" attributes by syncrepl is expected behaviour or just a side effect of its syncrepl_diff_entry diff'ing algorithm?
Yes.
Does attribute order matter? Is it specified somehow (sorted by modification time?)?
No, attribute order in LDAP is unspecified.
Should this issue be reported as bug?
No. It is clearly working as designed.
On 18.07.2012 11:25, Nick Milas wrote:
On 17/7/2012 9:04 μμ, Evgeniy Kosov wrote:
The issue I'm facing as stated above is regarding the syncrepl and attribute order.
What version of Openldap are you using on provider and consumers?
Provider: openldap-server-2.4.23 on FreeBSD 8.1-RELEASE
Consumer: openldap-server-2.4.31 on FreeBSD 8.2-RELEASE
What backends are you using? Which versions thereof?
It's back_hdb on the provider and back_perl on the consumer. As for their versions, I'm not sure how do I check them. I thought backends inherit their versions from the openldap itself, don't they? If not, how do I know version of a backend?
On 18.07.2012 13:31, Howard Chu wrote:
[...]
Is that replacing of "misplaced" attributes by syncrepl is expected behaviour or just a side effect of its syncrepl_diff_entry diff'ing algorithm?
Yes.
Did this mean "Yes, this behaviour is expected (and intended)" or "Yes, that is a side effect"?
Does attribute order matter? Is it specified somehow (sorted by modification time?)?
No, attribute order in LDAP is unspecified.
Then why does syncrepl try to reorder attributes in a backend?
Should this issue be reported as bug?
No. It is clearly working as designed.
If attribute order is unspecified, so apparently it mostly depends on the backend implementation. On the other hand syncrepl makes an assumption regarding (unspecified) attribute order and the result (list of ops sent to a backend) depends on whether that assumption was right or not. In the end, all these order-dependent modifications affect my module, as I'm trying to hook on certain DIT changes and trigger corresponding logic.
Could your "working as designed" be interpreted as "That didn't matter so far"? Given my use case, could this design be thought over? If not, what would be you recommendations in my case?
Thank you.
Evgeniy Kosov wrote:
On 18.07.2012 13:31, Howard Chu wrote:
Does attribute order matter? Is it specified somehow (sorted by modification time?)?
No, attribute order in LDAP is unspecified.
Then why does syncrepl try to reorder attributes in a backend?
Syncrepl aims to make an entry on the consumer identical to the entry on the provider.
Should this issue be reported as bug?
No. It is clearly working as designed.
If attribute order is unspecified, so apparently it mostly depends on the backend implementation. On the other hand syncrepl makes an assumption regarding (unspecified) attribute order and the result (list of ops sent to a backend) depends on whether that assumption was right or not.
Yes. It is assumed that, while attribute order is unspecified, once an order has been set it will be preserved.
In the end, all these order-dependent modifications affect my module, as I'm trying to hook on certain DIT changes and trigger corresponding logic.
Could your "working as designed" be interpreted as "That didn't matter so far"? Given my use case, could this design be thought over? If not, what would be you recommendations in my case?
Use delta-syncrepl instead.
On 18.07.2012 15:49, Howard Chu wrote:
Evgeniy Kosov wrote:
On 18.07.2012 13:31, Howard Chu wrote:
Does attribute order matter? Is it specified somehow (sorted by modification time?)?
No, attribute order in LDAP is unspecified.
Then why does syncrepl try to reorder attributes in a backend?
Syncrepl aims to make an entry on the consumer identical to the entry on the provider.
The problem is that's impossible unless both backends (provider and consumer) do modifications to the object the same way (e.g. move modified attribute to the end of list). Is that (moving to the end) specified somewhere?
Btw, aren't the two objects below identical in terms of LDAP/LDIF?
dn: cn=johndoe,ou=users,o=example cn: johndoe sn: Doe givenName: John
dn: cn=johndoe,ou=users,o=example cn: johndoe givenName: John sn: Doe
Should this issue be reported as bug?
No. It is clearly working as designed.
If attribute order is unspecified, so apparently it mostly depends on the backend implementation. On the other hand syncrepl makes an assumption regarding (unspecified) attribute order and the result (list of ops sent to a backend) depends on whether that assumption was right or not.
Yes. It is assumed that, while attribute order is unspecified, once an order has been set it will be preserved.
In the end, all these order-dependent modifications affect my module, as I'm trying to hook on certain DIT changes and trigger corresponding logic.
Could your "working as designed" be interpreted as "That didn't matter so far"? Given my use case, could this design be thought over? If not, what would be you recommendations in my case?
Use delta-syncrepl instead.
I'll have a closer look at it. Thank you.
Evgeniy Kosov wrote:
On 18.07.2012 15:49, Howard Chu wrote:
Evgeniy Kosov wrote:
On 18.07.2012 13:31, Howard Chu wrote:
Does attribute order matter? Is it specified somehow (sorted by modification time?)?
No, attribute order in LDAP is unspecified.
Then why does syncrepl try to reorder attributes in a backend?
Syncrepl aims to make an entry on the consumer identical to the entry on the provider.
The problem is that's impossible unless both backends (provider and consumer) do modifications to the object the same way (e.g. move modified attribute to the end of list). Is that (moving to the end) specified somewhere?
Attribute order is unspecified. Don't ask redundant questions.
Btw, aren't the two objects below identical in terms of LDAP/LDIF?
dn: cn=johndoe,ou=users,o=example cn: johndoe sn: Doe givenName: John
dn: cn=johndoe,ou=users,o=example cn: johndoe givenName: John sn: Doe
Irrelevant.
"cn=johndoe" and "cn=JohnDoe" are also identical in terms of LDAP, but syncrepl attempts to recognize these changes as well. The design has to do more than the specifications cover, otherwise users will complain when their data is visually different.
Use delta-syncrepl instead.
I'll have a closer look at it. Thank you.
On 18.07.2012 16:42, Howard Chu wrote:
[...]
Btw, aren't the two objects below identical in terms of LDAP/LDIF?
dn: cn=johndoe,ou=users,o=example cn: johndoe sn: Doe givenName: John
dn: cn=johndoe,ou=users,o=example cn: johndoe givenName: John sn: Doe
Irrelevant.
"cn=johndoe" and "cn=JohnDoe" are also identical in terms of LDAP, but syncrepl attempts to recognize these changes as well. The design has to do more than the specifications cover, otherwise users will complain when their data is visually different.
Okay. Got your point! Thank you.
Use delta-syncrepl instead.
I'll have a closer look at it. Thank you.
openldap-technical@openldap.org
-
Evgeniy Kosov -
Howard Chu -
Nick Milas