---872237384-2137566121-1349092304=:39209 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable
Hi,=0AYes, I am able to reproduce the issue with 2.4.32=0A=0A=0AMaking sens= e of the logs for the exact reproduction is hard since it needs a lot of op= erations in a short time. But this will probably help:=0A=0A1. At the start= of the test, the group temp_group existed.=0A=0A2. I created a user temp_u= ser and added temp_user to temp_group.=0A=0A3. During replication, the grou= p was replicated first and I got an error 32 (NO_SUCH_OBJECT) when it tried= to modify the memberOf of the temp_user object (This does not exist in the= readslave yet).=0A=0A4. The temp_user object was replicated next, and as y= ou see, querying it does show a memberOf attribute, proving that this field= was replicated. (Note that I have run OpenLDAP with debug as -1 and verifi= ed that the replicated data has the memberOf field in it. I can provide thi= s too if needed.)=0A=0A5. The more serious problem occurs when the sequence= is reversed and the group has been deleted as a last operation - The user = is replicated first, but since the group is deleted, it is never replicated= and a stale memberOf entry stays with the user.=0A=0A=0ALOGS:=0A=0A5069157= c syncrepl_message_to_entry: rid=3D179 DN: ou=3Dtemp_group,ou=3Dgroup,dc=3D= example,dc=3Dcom, UUID: 31090ab1-f8a7-4363-83c2-c0ac0d3918d4=0A5069157c syn= crepl_entry: rid=3D179 LDAP_RES_SEARCH_ENTRY(LDAP_SYNC_ADD)=0A5069157c sync= repl_entry: rid=3D179 inserted UUID 31090ab1-f8a7-4363-83c2-c0ac0d3918d4=0A= 5069157c syncrepl_entry: rid=3D179 be_search (0)=0A5069157c syncrepl_entry:= rid=3D179 ou=3Dtemp_group,ou=3Dgroup,dc=3Dexample,dc=3Dcom=0A5069157c slap= _queue_csn: queing 0x4270c730 20121001040100.779862Z#000000#000#000000=0A50= 69157c slap_graduate_commit_csn: removing 0xfa035c0 20121001040100.779862Z#= 000000#000#000000=0A5069157c conn=3D-1 op=3D0: memberof_value_modify DN=3D"= uid=3Dtemp_user,dc=3Dexample,dc=3Dcom" add memberOf=3D"ou=3Dtemp_group,ou= =3Dgroup,dc=3Dexample,dc=3Dcom" failed err=3D32=0A5069157c syncrepl_entry: = rid=3D179 be_modify ou=3Dtemp_group,ou=3Dgroup,dc=3Dexample,dc=3Dcom (0)=0A= 5069157c syncrepl_message_to_entry: rid=3D179 DN: uid=3Dtemp_user,dc=3Dexam= ple,dc=3Dcom, UUID: 748bd1a9-6be3-450b-809c-5ea692aa073c=0A5069157c syncrep= l_entry: rid=3D179 LDAP_RES_SEARCH_ENTRY(LDAP_SYNC_ADD)=0A5069157c syncrepl= _entry: rid=3D179 inserted UUID 748bd1a9-6be3-450b-809c-5ea692aa073c=0A5069= 157c syncrepl_entry: rid=3D179 be_search (0)=0A5069157c syncrepl_entry: rid= =3D179 uid=3Dtemp_user,dc=3Dexample,dc=3Dcom=0A5069157c syncrepl_entry: rid= =3D179 be_add uid=3Dtemp_user,dc=3Dexample,dc=3Dcom (0)=0A=0AThe object tem= p_user:=0A=0Adn: uid=3Dtemp_user,dc=3Dexample,dc=3Dcom=0AmemberOf: ou=3Dtem= p_group,ou=3Dgroup,dc=3Dexample,dc=3Dcom=0A=0A=0AWhat is interesting is tha= t in this case, the memberOf field being replicated actually protects the s= lave from incorrect data as the temp_user entry was not present at the time= the group got replicated (The user entry was the second entry in the repli= cation order). On the other hand, a reversed path during replication causes= the mentioned bug.=0A=0AThanks=0A=0A=0A=0A________________________________= =0A From: Howard Chu hyc@symas.com=0ATo: arunkumar_1123@yahoo.com =0ACc: = openldap-its@openldap.org =0ASent: Sunday, 30 September 2012 8:51 PM=0ASubj= ect: Re: (ITS#7400) Memberof and Syncrepl incompatibility=0A =0Aarunkumar_1= 123@yahoo.com wrote:=0A> Full_Name: Arunkumar shanmugam=0A> Version: 2.4.29= =0A> OS: rhel5=0A> URL: ftp://ftp.openldap.org/incoming/=0A> Submission fro= m: (NULL) (203.83.248.32)=0A>=0A>=0A> Hi,=0A>=0A> I'm currently using Openl= dap 2.4.29 to model an Authorization platform. I=0A> noticed some inconsist= ent behavior with syncrepl and memberof overlays.=0A=0ADoes this issue occu= r with the current release, 2.4.32?=0A>=0A> The issue happens as follows:= =0A>=0A> If I Create groups with a large number of members and delete them = in quick=0A> succession on the writemaster, the data replicated to the read= slave is=0A> incorrect, in particular, the memberof fields of the User obje= cts.=0A>=0A> This seems to happen because the memberof field is getting rep= licated to the=0A> slave nodes, although the documentation states that it s= houldn't.=0A=0AIndeed. Do you have debug logs showing the replication traff= ic, and showing =0Athat the memberof attribute got replicated?=0A=0A> While= =0A> replicating, the User object is replicated inclusive of the memberof f= ields, but=0A> by the time the syncrepl search comes to the group object, i= t has already been=0A> deleted, and hence not replicated. This leaves a dan= gling memberof field in the=0A> read slave instance.=0A>=0A> I was wonderin= g if anyone has faced this issues (I did not see any ITS related=0A> to thi= s), and has a workaround.=0A>=0A> Thanks,=0A> Arunkumar=0A>=0A>=0A=0A=0A-- = =0A=A0 -- Howard Chu=0A=A0 CTO, Symas Corp.=A0 =A0 =A0 =A0 =A0 http://www= .symas.com=0A=A0 Director, Highland Sun=A0 =A0 http://highlandsun.com/hyc/= =0A=A0 Chief Architect, OpenLDAP=A0 http://www.openldap.org/project/ ---872237384-2137566121-1349092304=:39209 Content-Type: text/html; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable
<html><body><div style=3D"color:#000; background-color:#fff; font-family:ti= mes new roman, new york, times, serif;font-size:12pt"><div><span style=3D"c= olor: rgb(69, 69, 69); font-family: arial, helvetica, sans-serif; font-size= : 13px; background-color: transparent; ">Hi,</span></div><div style=3D"colo= r: rgb(69, 69, 69); font-size: 13px; font-family: arial, helvetica, sans-se= rif; background-color: transparent; font-style: normal; "><span style=3D"co= lor: rgb(69, 69, 69); font-family: arial, helvetica, sans-serif; font-size:= 13px; background-color: transparent; "><span class=3D"Apple-tab-span" styl= e=3D"white-space:pre">=09</span>Yes, I am able to reproduce the issue with = 2.4.32</span><br></div><div style=3D"color: rgb(69, 69, 69); font-size: 13p= x; font-family: arial, helvetica, sans-serif; background-color: transparent= ; font-style: normal; "><span><br style=3D"color: rgb(69, 69, 69); font-fam= ily: arial, helvetica, sans-serif; font-size: 13px; "><span style=3D"color:= rgb(69, 69, 69); font-family: arial, helvetica, sans-serif; font-size: 13px; ">Making = sense of the logs for the exact reproduction is hard since it needs a lot o= f operations in a short time. But this will probably help:</span><br style= =3D"color: rgb(69, 69, 69); font-family: arial, helvetica, sans-serif; font= -size: 13px; "><br style=3D"color: rgb(69, 69, 69); font-family: arial, hel= vetica, sans-serif; font-size: 13px; "><span style=3D"color: rgb(69, 69, 69= ); font-family: arial, helvetica, sans-serif; font-size: 13px; ">1. At the = start of the test, the group temp_group existed.</span><br style=3D"color: = rgb(69, 69, 69); font-family: arial, helvetica, sans-serif; font-size: 13px= ; "><br style=3D"color: rgb(69, 69, 69); font-family: arial, helvetica, san= s-serif; font-size: 13px; "><span style=3D"color: rgb(69, 69, 69); font-fam= ily: arial, helvetica, sans-serif; font-size: 13px; ">2. I created a user t= emp_user and added temp_user to temp_group.</span><br style=3D"color: rgb(6= 9, 69, 69); font-family: arial, helvetica, sans-serif; font-size: 13px; "><br sty= le=3D"color: rgb(69, 69, 69); font-family: arial, helvetica, sans-serif; fo= nt-size: 13px; "><span style=3D"color: rgb(69, 69, 69); font-family: arial,= helvetica, sans-serif; font-size: 13px; ">3. During replication, the group= was replicated first and I got an error 32 (NO_SUCH_OBJECT) when it tried = to modify the memberOf of the temp_user object (This does not exist in the = readslave yet).</span><br style=3D"color: rgb(69, 69, 69); font-family: ari= al, helvetica, sans-serif; font-size: 13px; "><br style=3D"color: rgb(69, 6= 9, 69); font-family: arial, helvetica, sans-serif; font-size: 13px; "><span= style=3D"color: rgb(69, 69, 69); font-family: arial, helvetica, sans-serif= ; font-size: 13px; ">4. The temp_user object was replicated next, and as yo= u see, querying it does show a memberOf attribute, proving that this field = was replicated. (Note that I have run OpenLDAP with debug as -1 and verifie= d that the replicated data has the memberOf field in it. I can provide this = too if needed.)</span><br style=3D"color: rgb(69, 69, 69); font-family: ari= al, helvetica, sans-serif; font-size: 13px; "><br style=3D"color: rgb(69, 6= 9, 69); font-family: arial, helvetica, sans-serif; font-size: 13px; "><span= style=3D"color: rgb(69, 69, 69); font-family: arial, helvetica, sans-serif= ; font-size: 13px; ">5. The more serious problem occurs when the sequence i= s reversed and the group has been deleted as a last operation - The user is= replicated first, but since the group is deleted, it is never replicated a= nd a stale memberOf entry stays with the user.</span><br style=3D"color: rg= b(69, 69, 69); font-family: arial, helvetica, sans-serif; font-size: 13px; = "><br style=3D"color: rgb(69, 69, 69); font-family: arial, helvetica, sans-= serif; font-size: 13px; "><br style=3D"color: rgb(69, 69, 69); font-family:= arial, helvetica, sans-serif; font-size: 13px; "><span style=3D"color: rgb= (69, 69, 69); font-family: arial, helvetica, sans-serif; font-size: 13px; ">LOG= S:</span><br style=3D"color: rgb(69, 69, 69); font-family: arial, helvetica= , sans-serif; font-size: 13px; "><br style=3D"color: rgb(69, 69, 69); font-= family: arial, helvetica, sans-serif; font-size: 13px; "><span style=3D"col= or: rgb(69, 69, 69); font-family: arial, helvetica, sans-serif; font-size: = 13px; ">5069157c syncrepl_message_to_entry: rid=3D179 DN: ou=3Dtemp_group,o= u=3Dgroup,dc=3Dexample,dc=3Dcom, UUID: 31090ab1-f8a7-4363-83c2-c0ac0d3918d4= </span><br style=3D"color: rgb(69, 69, 69); font-family: arial, helvetica, = sans-serif; font-size: 13px; "><span style=3D"color: rgb(69, 69, 69); font-= family: arial, helvetica, sans-serif; font-size: 13px; ">5069157c syncrepl_= entry: rid=3D179 LDAP_RES_SEARCH_ENTRY(LDAP_SYNC_ADD)</span><br style=3D"co= lor: rgb(69, 69, 69); font-family: arial, helvetica, sans-serif; font-size:= 13px; "><span style=3D"color: rgb(69, 69, 69); font-family: arial, helveti= ca, sans-serif; font-size: 13px; ">5069157c syncrepl_entry: rid=3D179 inserted UUID 31090a= b1-f8a7-4363-83c2-c0ac0d3918d4</span><br style=3D"color: rgb(69, 69, 69); f= ont-family: arial, helvetica, sans-serif; font-size: 13px; "><span style=3D= "color: rgb(69, 69, 69); font-family: arial, helvetica, sans-serif; font-si= ze: 13px; ">5069157c syncrepl_entry: rid=3D179 be_search (0)</span><br styl= e=3D"color: rgb(69, 69, 69); font-family: arial, helvetica, sans-serif; fon= t-size: 13px; "><span style=3D"color: rgb(69, 69, 69); font-family: arial, = helvetica, sans-serif; font-size: 13px; ">5069157c syncrepl_entry: rid=3D17= 9 ou=3Dtemp_group,ou=3Dgroup,dc=3Dexample,dc=3Dcom</span><br style=3D"color= : rgb(69, 69, 69); font-family: arial, helvetica, sans-serif; font-size: 13= px; "><span style=3D"color: rgb(69, 69, 69); font-family: arial, helvetica,= sans-serif; font-size: 13px; ">5069157c slap_queue_csn: queing 0x4270c730 = 20121001040100.779862Z#000000#000#000000</span><br style=3D"color: rgb(69, = 69, 69); font-family: arial, helvetica, sans-serif; font-size: 13px; "><span style=3D"color: rgb= (69, 69, 69); font-family: arial, helvetica, sans-serif; font-size: 13px; "=
5069157c slap_graduate_commit_csn: removing 0xfa035c0 20121001040100.77986=
2Z#000000#000#000000</span><br style=3D"color: rgb(69, 69, 69); font-family= : arial, helvetica, sans-serif; font-size: 13px; "><span style=3D"color: rg= b(69, 69, 69); font-family: arial, helvetica, sans-serif; font-size: 13px; = ">5069157c conn=3D-1 op=3D0: memberof_value_modify DN=3D"uid=3Dtemp_user,dc= =3Dexample,dc=3Dcom" add memberOf=3D"ou=3Dtemp_group,ou=3Dgroup,dc=3Dexampl= e,dc=3Dcom" failed err=3D32</span><br style=3D"color: rgb(69, 69, 69); font= -family: arial, helvetica, sans-serif; font-size: 13px; "><span style=3D"co= lor: rgb(69, 69, 69); font-family: arial, helvetica, sans-serif; font-size:= 13px; ">5069157c syncrepl_entry: rid=3D179 be_modify ou=3Dtemp_group,ou=3D= group,dc=3Dexample,dc=3Dcom (0)</span><br style=3D"color: rgb(69, 69, 69); = font-family: arial, helvetica, sans-serif; font-size: 13px; "><span style=3D"color: rgb(69, 69, 69); font= -family: arial, helvetica, sans-serif; font-size: 13px; ">5069157c syncrepl= _message_to_entry: rid=3D179 DN: uid=3Dtemp_user,dc=3Dexample,dc=3Dcom, UUI= D: 748bd1a9-6be3-450b-809c-5ea692aa073c</span><br style=3D"color: rgb(69, 6= 9, 69); font-family: arial, helvetica, sans-serif; font-size: 13px; "><span= style=3D"color: rgb(69, 69, 69); font-family: arial, helvetica, sans-serif= ; font-size: 13px; ">5069157c syncrepl_entry: rid=3D179 LDAP_RES_SEARCH_ENT= RY(LDAP_SYNC_ADD)</span><br style=3D"color: rgb(69, 69, 69); font-family: a= rial, helvetica, sans-serif; font-size: 13px; "><span style=3D"color: rgb(6= 9, 69, 69); font-family: arial, helvetica, sans-serif; font-size: 13px; ">5= 069157c syncrepl_entry: rid=3D179 inserted UUID 748bd1a9-6be3-450b-809c-5ea= 692aa073c</span><br style=3D"color: rgb(69, 69, 69); font-family: arial, he= lvetica, sans-serif; font-size: 13px; "><span style=3D"color: rgb(69, 69, 6= 9); font-family: arial, helvetica, sans-serif; font-size: 13px; ">5069157c syncrepl_entry: = rid=3D179 be_search (0)</span><br style=3D"color: rgb(69, 69, 69); font-fam= ily: arial, helvetica, sans-serif; font-size: 13px; "><span style=3D"color:= rgb(69, 69, 69); font-family: arial, helvetica, sans-serif; font-size: 13p= x; ">5069157c syncrepl_entry: rid=3D179 uid=3Dtemp_user,dc=3Dexample,dc=3Dc= om</span><br style=3D"color: rgb(69, 69, 69); font-family: arial, helvetica= , sans-serif; font-size: 13px; "><span style=3D"color: rgb(69, 69, 69); fon= t-family: arial, helvetica, sans-serif; font-size: 13px; ">5069157c syncrep= l_entry: rid=3D179 be_add uid=3Dtemp_user,dc=3Dexample,dc=3Dcom (0)</span><= br style=3D"color: rgb(69, 69, 69); font-family: arial, helvetica, sans-ser= if; font-size: 13px; "><br style=3D"color: rgb(69, 69, 69); font-family: ar= ial, helvetica, sans-serif; font-size: 13px; "><span style=3D"color: rgb(69= , 69, 69); font-family: arial, helvetica, sans-serif; font-size: 13px; ">Th= e object temp_user:</span><br style=3D"color: rgb(69, 69, 69); font-family: arial, = helvetica, sans-serif; font-size: 13px; "><br style=3D"color: rgb(69, 69, 6= 9); font-family: arial, helvetica, sans-serif; font-size: 13px; "><span sty= le=3D"color: rgb(69, 69, 69); font-family: arial, helvetica, sans-serif; fo= nt-size: 13px; ">dn: uid=3Dtemp_user,dc=3Dexample,dc=3Dcom</span><br style= =3D"color: rgb(69, 69, 69); font-family: arial, helvetica, sans-serif; font= -size: 13px; "><span style=3D"color: rgb(69, 69, 69); font-family: arial, h= elvetica, sans-serif; font-size: 13px; ">memberOf: ou=3Dtemp_group,ou=3Dgro= up,dc=3Dexample,dc=3Dcom</span><br style=3D"color: rgb(69, 69, 69); font-fa= mily: arial, helvetica, sans-serif; font-size: 13px; "><br style=3D"color: = rgb(69, 69, 69); font-family: arial, helvetica, sans-serif; font-size: 13px= ; "><br style=3D"color: rgb(69, 69, 69); font-family: arial, helvetica, san= s-serif; font-size: 13px; "><span style=3D"color: rgb(69, 69, 69); font-fam= ily: arial, helvetica, sans-serif; font-size: 13px; ">What is interesting is that in this case, t= he memberOf field being replicated actually protects the slave from incorre= ct data as the temp_user entry was not present at the time the group got re= plicated (The user entry was the second entry in the replication order). On= the other hand, a reversed path during replication causes the mentioned bu= g.</span><br style=3D"color: rgb(69, 69, 69); font-family: arial, helvetica= , sans-serif; font-size: 13px; "><br style=3D"color: rgb(69, 69, 69); font-= family: arial, helvetica, sans-serif; font-size: 13px; "><span style=3D"col= or: rgb(69, 69, 69); font-family: arial, helvetica, sans-serif; font-size: = 13px; ">Thanks</span></span></div><div><br></div> <div style=3D"font-famil= y: 'times new roman', 'new york', times, serif; font-size: 12pt; "> <div st= yle=3D"font-family: 'times new roman', 'new york', times, serif; font-size:= 12pt; "> <div dir=3D"ltr"> <font size=3D"2" face=3D"Arial"> <hr size=3D"1"=
<b><span
style=3D"font-weight:bold;">From:</span></b> Howard Chu <hyc@symas.com&= gt;<br> <b><span style=3D"font-weight: bold;">To:</span></b> arunkumar_1123= @yahoo.com <br><b><span style=3D"font-weight: bold;">Cc:</span></b> openlda= p-its@openldap.org <br> <b><span style=3D"font-weight: bold;">Sent:</span><= /b> Sunday, 30 September 2012 8:51 PM<br> <b><span style=3D"font-weight: bo= ld;">Subject:</span></b> Re: (ITS#7400) Memberof and Syncrepl incompatibili= ty<br> </font> </div> <br><a ymailto=3D"mailto:arunkumar_1123@yahoo.com" hr= ef=3D"mailto:arunkumar_1123@yahoo.com">arunkumar_1123@yahoo.com</a> wrote:<= br>> Full_Name: Arunkumar shanmugam<br>> Version: 2.4.29<br>> OS: = rhel5<br>> URL: <a href=3D"ftp://ftp.openldap.org/incoming/" target=3D"_= blank">ftp://ftp.openldap.org/incoming/</a><br>> Submission from: (NULL)= (203.83.248.32)<br>><br>><br>> Hi,<br>><br>> I'm currently = using Openldap 2.4.29 to model an Authorization platform. I<br>> noticed= some inconsistent behavior with syncrepl and memberof overlays.<br><br>Does thi= s issue occur with the current release, 2.4.32?<br>><br>> The issue h= appens as follows:<br>><br>> If I Create groups with a large number o= f members and delete them in quick<br>> succession on the writemaster, t= he data replicated to the readslave is<br>> incorrect, in particular, th= e memberof fields of the User objects.<br>><br>> This seems to happen= because the memberof field is getting replicated to the<br>> slave node= s, although the documentation states that it shouldn't.<br><br>Indeed. Do y= ou have debug logs showing the replication traffic, and showing <br>that th= e memberof attribute got replicated?<br><br>> While<br>> replicating,= the User object is replicated inclusive of the memberof fields, but<br>>= ; by the time the syncrepl search comes to the group object, it has already= been<br>> deleted, and hence not replicated. This leaves a dangling memberof field in the<br>> read slave instance.<br>><br>>= ; I was wondering if anyone has faced this issues (I did not see any ITS re= lated<br>> to this), and has a workaround.<br>><br>> Thanks,<br>&g= t; Arunkumar<br>><br>><br><br><br>-- <br> -- Howard Chu<br>&nb= sp; CTO, Symas Corp. <a href=3D"http://= www.symas.com/" target=3D"_blank">http://www.symas.com</a><br> Direc= tor, Highland Sun <a href=3D"http://highlandsun.com/hyc/" tar= get=3D"_blank">http://highlandsun.com/hyc/</a><br> Chief Architect, = OpenLDAP <a href=3D"http://www.openldap.org/project/" target=3D"_blan= k">http://www.openldap.org/project/</a><br><br><br> </div> </div> </div></= body></html> ---872237384-2137566121-1349092304=:39209--
-
arunkumar_1123＠yahoo.com