3:54 p.m.
Hello,
I had thought I tested this beforehand but I seem to be able to reliably
reproduce the following situation:
We have an installation where the provider server has information that is
replicated to downstream replicas using the syncrepl protocol. The account
used to replicate is allowed to see records where certain attributes meet
specific values, a silly example is an attribute is set
dn: uid=somerecord,ou=people,dc=ucsc,dc=edu
replicateMe: TRUE
...
When an account has that attribute set it then replicates to the downstream
replica, however if later we set replicateMe to FALSE so that the account
used for replication can no longer see the entry, it seems to be orphaned
and is not removed in the replica.
We are using OpenLDAP 2.4.26 and I have the syncprov sessionlog set to 500
and the replica is set to refreshAndPersist.
Is this something that is simply not supported? or would a case like this
be expected to work and I've either got a configuration issue or a bug?
Thanks
Jeffrey
6:38 p.m.
Jeffrey Crawford wrote:
Hello,
I had thought I tested this beforehand but I seem to be able to reliably
reproduce the following situation:
We have an installation where the provider server has information that is
replicated to downstream replicas using the syncrepl protocol. The account
used to replicate is allowed to see records where certain attributes meet
specific values, a silly example is an attribute is set
dn: uid=somerecord,ou=people,dc=ucsc,dc=edu
replicateMe: TRUE
...
When an account has that attribute set it then replicates to the downstream
replica, however if later we set replicateMe to FALSE so that the account used
for replication can no longer see the entry, it seems to be orphaned and is
not removed in the replica.
We are using OpenLDAP 2.4.26 and I have the syncprov sessionlog set to 500 and
the replica is set to refreshAndPersist.
Is this something that is simply not supported? or would a case like this be
expected to work and I've either got a configuration issue or a bug?
Visibility changes due to ACL rules are not detected. syncprov only checks an
entry against the search parameters of the original sync search operation,
i.e., the base, scope, and filter. If an entry matches these params before the
modification, and no longer matches after the operation, syncprov will send a
delete message for that entry. (Likewise if an entry doesn't match before, but
matches after, syncprov will send an Add for the entry.)
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
10:54 p.m.
Thanks Howard,
Let me make sure I understand your response. I'm not changing any ACL's,
they are staying the same. Just the attributes in the record are changing.
Are you saying that syncprov looks at the account that is bound and sends
deletes if a record would become invisible after a modification? If that's
the case it doesn't seem to be working right.
If syncprov is being used and looks at which user is bound would it have
trouble if there are multiple replicas and replica accounts binding to get
replication data? we have two distinct downstream replicas that each have
different criteria for which records get replicated to them.
Jeffrey
On Thu, May 31, 2012 at 6:38 PM, Howard Chu <hyc(a)symas.com> wrote:
Jeffrey Crawford wrote:
> Hello,
>
> I had thought I tested this beforehand but I seem to be able to reliably
> reproduce the following situation:
>
> We have an installation where the provider server has information that is
> replicated to downstream replicas using the syncrepl protocol. The account
> used to replicate is allowed to see records where certain attributes meet
> specific values, a silly example is an attribute is set
>
> dn: uid=somerecord,ou=people,dc=**ucsc,dc=edu
> replicateMe: TRUE
> ...
>
> When an account has that attribute set it then replicates to the
> downstream
> replica, however if later we set replicateMe to FALSE so that the account
> used
> for replication can no longer see the entry, it seems to be orphaned and
> is
> not removed in the replica.
>
> We are using OpenLDAP 2.4.26 and I have the syncprov sessionlog set to
> 500 and
> the replica is set to refreshAndPersist.
>
> Is this something that is simply not supported? or would a case like this
> be
> expected to work and I've either got a configuration issue or a bug?
>
Visibility changes due to ACL rules are not detected. syncprov only checks
an entry against the search parameters of the original sync search
operation, i.e., the base, scope, and filter. If an entry matches these
params before the modification, and no longer matches after the operation,
syncprov will send a delete message for that entry. (Likewise if an entry
doesn't match before, but matches after, syncprov will send an Add for the
entry.)
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP
http://www.openldap.org/**project/<http://www.openldap.org/project/>
9:06 a.m.
On 1/6/2012 8:54 πμ, Jeffrey Crawford wrote:
Are you saying that syncprov looks at the account that is bound and
sends deletes if a record would become invisible after a modification?
I understand the opposite: syncprov will only send add/delete message
based on base/scope/filter and not on ACL-visibility. So in essence
Howard says that ACL-based filtering in replication does not result in
proper results to consumers.
This is tricky! (I didn't know either.) It means that we should *not*
design our replication based on ACL-filtering (which, unfortunately, we
have done too), but, on the contrary, that we should design our DIT so
that it can cover our replication needs based on consumer
base/scope/filter configuration, and we should design/adapt our ACLs
with the above rule in mind.
Please confirm the above thoughts.
Thanks,
Nick
9:26 a.m.
Humm and taking this one step further I'm guessing that the replication
account probably needs to see at least the entryUUID and entryCSN for all
accounts to make sure that it can see the records it needs to delete. Okay
at least I have some direction to go on now.
Jeffrey
On Fri, Jun 1, 2012 at 9:06 AM, Nick Milas <nick(a)eurobjects.com> wrote:
On 1/6/2012 8:54 πμ, Jeffrey Crawford wrote:
Are you saying that syncprov looks at the account that is bound and sends
> deletes if a record would become invisible after a modification?
>
I understand the opposite: syncprov will only send add/delete message
based on base/scope/filter and not on ACL-visibility. So in essence Howard
says that ACL-based filtering in replication does not result in proper
results to consumers.
This is tricky! (I didn't know either.) It means that we should *not*
design our replication based on ACL-filtering (which, unfortunately, we
have done too), but, on the contrary, that we should design our DIT so that
it can cover our replication needs based on consumer base/scope/filter
configuration, and we should design/adapt our ACLs with the above rule in
mind.
Please confirm the above thoughts.
Thanks,
Nick
--
I fly because it releases my mind from the tyranny of petty things . . .
— Antoine de Saint-Exupéry
Jeffrey E. Crawford
ITS Application Administrator (IDM)
831-459-4365
jeffreyc(a)ucsc.edu
12:16 p.m.
Ok I think I got this to work I didn't add a filter to the syncrepl
parameter so I'm using ACL's as before, however I changed the acls to
allow the replica account access to the attributes entry and entryUUID
only on every item in the directory, now setting attributes to values
so that they no longer match the access to the rest of the acls seem
to do what I wan't however I'm not quite sure how. so in short the
provider looks something like this:
olcAccess: to dn.subtree="dc=example,dc=org"
attrs=entry,entryUUID
by dn.base="cn=replica,ou=admins,dc=example,dc=org" ssf=128 read
by * none break
olcAccess: to dn.subtree="dc=example,dc=org"
filter="(replicateMe=TRUE)"
attrs=@inetOrgPerson,@posixAccount,entryCSN,contextCSN,createTimestamp,modifyTimestamp,structuralObjectClass
by dn.base="cn=replica,ou=admins,dc=example,dc=org" ssf=128 read
by * none break
And the replicas are configured as so:
dn: olcDatabase={1}hdb,cn=config
changeType: modify
replace: olcSyncrepl
olcSyncrepl: rid=1
provider=ldap://ldap-!!!!ENV!!!!-1.example.org:1389/
starttls=critical
tls_reqcert=never
bindmethod=simple
retry="60 10 900 92 86400 3"
binddn="cn=replica,ou=admins,dc=example,dc=org"
credentials=!!!!PW!!!!
schemachecking=off
searchbase="dc=example,dc=org"
type=refreshAndPersist
olcSyncrepl: rid=2
provider=ldap://ldap-!!!!ENV!!!!-2.example.org:1389/
starttls=critical
tls_reqcert=never
bindmethod=simple
retry="60 10 900 92 86400 3"
binddn="cn=replica,ou=admins,dc=example,dc=org"
credentials=!!!!PW!!!!
schemachecking=off
searchbase="dc=example,dc=org"
type=refreshAndPersist
now the following will add a record to the replica:
dn: uid=user,ou=people,dc=example,dc=org
changeType: modify
replace: replicateMe
replicateMe: TRUE
And the following will take it away (delete):
dn: uid=user,ou=people,dc=example,dc=org
changeType: modify
replace: replicateMe
replicateMe: FALSE
However I don't know enough about the underlying code to be sure the
following is working as designed. Meaning I don't want to be relying
on a bug that might get fixed later.
Thanks
Jeffrey
access:
On Fri, Jun 1, 2012 at 9:26 AM, Jeffrey Crawford <jeffreyc(a)ucsc.edu> wrote:
Humm and taking this one step further I'm guessing that the replication account
probably needs to see at least the entryUUID and entryCSN for all accounts to make sure
that it can see the records it needs to delete. Okay at least I have some direction to go
on now.
Jeffrey
On Fri, Jun 1, 2012 at 9:06 AM, Nick Milas <nick(a)eurobjects.com> wrote:
>
> On 1/6/2012 8:54 πμ, Jeffrey Crawford wrote:
>
>> Are you saying that syncprov looks at the account that is bound and sends deletes
if a record would become invisible after a modification?
>
>
> I understand the opposite: syncprov will only send add/delete message based on
base/scope/filter and not on ACL-visibility. So in essence Howard says that ACL-based
filtering in replication does not result in proper results to consumers.
>
> This is tricky! (I didn't know either.) It means that we should *not* design our
replication based on ACL-filtering (which, unfortunately, we have done too), but, on the
contrary, that we should design our DIT so that it can cover our replication needs based
on consumer base/scope/filter configuration, and we should design/adapt our ACLs with the
above rule in mind.
>
> Please confirm the above thoughts.
>
> Thanks,
> Nick
--
I fly because it releases my mind from the tyranny of petty things . . .
— Antoine de Saint-Exupéry
Jeffrey E. Crawford
ITS Application Administrator (IDM)
831-459-4365
jeffreyc(a)ucsc.edu
--
I fly because it releases my mind from the tyranny of petty things . . .
— Antoine de Saint-Exupéry
Jeffrey E. Crawford
ITS Application Administrator (IDM)
831-459-4365
jeffreyc(a)ucsc.edu
1:18 a.m.
On 1/6/2012 4:38 πμ, Howard Chu wrote:
Visibility changes due to ACL rules are not detected. syncprov only
checks an entry against the search parameters of the original sync
search operation, i.e., the base, scope, and filter. If an entry
matches these params before the modification, and no longer matches
after the operation, syncprov will send a delete message for that
entry. (Likewise if an entry doesn't match before, but matches after,
syncprov will send an Add for the entry.)
I would like a clarification on this, please:
Since the syncprov mechanism does a search based on base/scope/filter
from *a particular binddn* account, doesn't this mean that if visibility
*by that same binddn* of some entry (due to ACL restrictions) changes
after a modification, then effectively the same search (based on the
same base/scope/filter) will produce different results, which means that
the syncprov mechanism *should* generate an add/delete message accordingly?
In other words, syncprov does not produce messages based on the
differences between the results of standard ldapsearch'es? And if it
does not, shouldn't it?
Why syncprov should ignore ACL-based visibility? This seems unnatural
and does not assist conceptualization. At least it seems confusing to
me. Can you please provide more details on the syncprov mechanism with
regard to this?
Please advise!
Thanks,
Nick
6:49 a.m.
On 2/6/2012 11:18 πμ, Nick Milas wrote:
In other words, syncprov does not produce messages based on the
differences between the results of standard ldapsearch'es? And if it
does not, shouldn't it?
My tests (with v2.4.31 on both provider and consumer) show that syncrepl
(refreshAndPersist) works correctly when replicating based on ACL
restrictions. OpenLDAP consumer deletes correctly an entry from a branch
when the entry is moved to another, invisible by the consumer binddn,
branch, and it re-creates it correctly when it is moved back to a
visible (based on ACL) branch.
So the answer above is yes, syncprov *does* produce update messages
based on the differences between the results of standard ldapsearch'es.
BUT, I had problems in the past when replicating based on ACLs: There
might be scenarios - though I never had time to test exhaustively -
where replication stalls (I even had some crashes) when the consumer
binddn had -inadvertently- partial only privileges on some branches of
the provider (probably on entry/children pseudo-attrs only). I ceased to
have problems when I made sure that there existed *no* privileges *at
all* on branches / entries where the consumer binddn should NOT have
access (e.g. by explicitly declaring "by <consumer binddn> none").
I would like to invest some time to test such scenarios more (however,
not feasible in the immediate future).
Any other info by the developers might be insightful.
Regards,
Nick
4015
days inactive
4017
days old
openldap-technical@openldap.org
7 comments
3 participants
participants (3)
-
Howard Chu -
Jeffrey Crawford -
Nick Milas