https://bugs.openldap.org/show_bug.cgi?id=9584 Quanah Gibson-Mount <quanah(a)openldap.org&gt; changed: What |Removed |Added ---------------------------------------------------------------------------- Target Milestone|--- |2.6.0 Assignee|bugs(a)openldap.org |hyc(a)openldap.org -- You are receiving this mail because: You are on the CC list for the issue.

https://bugs.openldap.org/show_bug.cgi?id=9584 --- Comment #1 from Howard Chu <hyc(a)openldap.org&gt; --- (In reply to Ondřej Kuzník from comment #0) I don't see how this could happen. In order for any consumer to write to cn=config, all other threads must be paused. There is no point where one consumer will have retrieved any entry from cn=config and then paused itself, giving any window for another consumer to rewrite the entry out from under it. In particular, the stacktrace for the crashed thread is: #0 0x0000557332297336 in nonpresent_callback (op=<optimized out>, rs=0x7ff39d52dcb0) at ../../../servers/slapd/syncrepl.c:5681 #1 0x0000557332244ef8 in slap_response_play (op=op@entry=0x7ff39d52e4f0, rs=rs@entry=0x7ff39d52dcb0) at ../../../servers/slapd/result.c:567 #2 0x0000557332246c30 in slap_send_search_entry (op=0x7ff39d52e4f0, rs=0x7ff39d52dcb0) at ../../../servers/slapd/result.c:1072 #3 0x000055733221c154 in config_send (op=op@entry=0x7ff39d52e4f0, rs=rs@entry=0x7ff39d52dcb0, ce=ce@entry=0x55733339fcd0, depth=depth@entry=0) at ../../../servers/slapd/bconfig.c:4719 #4 0x000055733221c790 in config_back_search (op=0x7ff39d52e4f0, rs=0x7ff39d52dcb0) at ../../../servers/slapd/bconfig.c:6794 #5 0x00005573322a57f1 in overlay_op_walk (op=op@entry=0x7ff39d52e4f0, rs=rs@entry=0x7ff39d52dcb0, which=which@entry=op_search, oi=oi@entry=0x7ff3941dcaf0, on=<optimized out>) at ../../../servers/slapd/backover.c:706 #6 0x00005573322a594e in over_op_func (op=0x7ff39d52e4f0, rs=0x7ff39d52dcb0, which=op_search) at ../../../servers/slapd/backover.c:766 #7 0x0000557332291b3d in syncrepl_del_nonpresent (op=op@entry=0x7ff39d52e4f0, si=si@entry=0x7ff388108640, uuids=uuids@entry=0x0, sc=sc@entry=0x7ff39d52dfc0, m=3) at ../../../servers/slapd/syncrepl.c:4625 #8 0x000055733229ea1a in do_syncrep2 (si=0x7ff388108640, op=0x7ff39d52e4f0) at ../../../servers/slapd/syncrepl.c:1840 #9 do_syncrepl (ctx=<optimized out>, arg=0x7ff388109cb0) at ../../../servers/slapd/syncrepl.c:2076 #10 0x00007ff3a0fe047e in ?? () #11 0x0000000000000000 in ?? () There is nowhere along the call path from frame #4 thru frame #0 where this thread will do a pausecheck. Which means any other consumer trying to modify the config DB necessarily waits for this call path to complete before it can proceed. Which means no other consumer could have changed any attributes out from under this thread. -- You are receiving this mail because: You are on the CC list for the issue.

https://bugs.openldap.org/show_bug.cgi?id=9584 Quanah Gibson-Mount <quanah(a)openldap.org&gt; changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords|replication | --- Comment #3 from Quanah Gibson-Mount <quanah(a)openldap.org&gt; --- Commits: • e1c90d09 by Howard Chu at 2021-07-27T16:12:14+01:00 ITS#9584 serialize refresh phase -- You are receiving this mail because: You are on the CC list for the issue.

https://bugs.openldap.org/show_bug.cgi?id=9584 --- Comment #5 from Howard Chu <hyc(a)openldap.org&gt; --- (In reply to Michael Ströder from comment #4) Yes. The overall time required will actually be shorter. Previously, without this serialization, all of the consumers would start at the same time and make redundant requests to their respective providers. Starting from the same cookie, they would each obtain the same content from their providers, so the same data is transmitted N times. The consumers go thru the entries one by one and end up ignoring the majority of the data. By serializing the start, one consumer will pull down the bulk of the data, and then when it finishes, the next consumer will be using the latest cookie when it talks to its provider, so no redundant data will be downloaded. -- You are receiving this mail because: You are on the CC list for the issue.

https://bugs.openldap.org/show_bug.cgi?id=9584 --- Comment #7 from Howard Chu <hyc(a)openldap.org&gt; --- (In reply to Ondřej Kuzník from comment #6) Agreed. It would have made more sense to have only a single task iterate through all of the configured consumers, at least for this startup refresh. I guess instead of the busy loop I can just exit the task and explicitly re-enable it later. Will look into it. -- You are receiving this mail because: You are on the CC list for the issue.

https://bugs.openldap.org/show_bug.cgi?id=9584 --- Comment #9 from Quanah Gibson-Mount <quanah(a)openldap.org&gt; --- Commits: • 79d33fe4 by Howard Chu at 2021-07-29T13:28:34+01:00 ITS#9584 avoid busy-loop while refresh is serialized -- You are receiving this mail because: You are on the CC list for the issue.

https://bugs.openldap.org/show_bug.cgi?id=9584 Ondřej Kuzník <ondra(a)mistotebe.net&gt; changed: What |Removed |Added ---------------------------------------------------------------------------- Status|VERIFIED |CONFIRMED Ever confirmed|0 |1 Resolution|FIXED |--- --- Comment #12 from Ondřej Kuzník <ondra(a)mistotebe.net&gt; --- The current fix in 2.6 abuses retry logic to handle SYNC_BUSY and that's going to cause us no end of trouble if we allow this into 2.5. Given it looks like we want to backport, we'll need to make some changes I planned to do eventually. My thinking is we make each syncinfo track whether it's active/paused, with all of them starting in paused state and we keep the existing cs_refreshing management as is. Also order probably matters but that just got fixed in ITS#9761. On startup, we kick off a task that picks the first paused syncinfo, marks it active and schedules accordingly. When a syncinfo gets a SYNC_BUSY - there is another active syncinfo in cs_refreshing - it makes itself paused. When a syncinfo drops itself from cs_refreshing it will pick the first paused syncinfo from the start and activates it as above. In this, it seems that syncinfos that run their retry counter to the end (going dead) stay marked as "active" and that looks safe, we don't actually want to reschedule them. Any cn=config prodding has to take the above into account, so it might have to schedule the kick-off task if required and some tracking to allow that will be needed. There is no active link from cn=monitor that can change these states/retries so that's fine. Decision whether or not to schedule the initial task could be based on cs_refreshing == NULL. Even if we spawn duplicate kick-off tasks, one of them will be able to take over cs_refreshing and others will get BUSY or find no more paused syncinfos. -- You are receiving this mail because: You are on the CC list for the issue.

https://bugs.openldap.org/show_bug.cgi?id=9584 Quanah Gibson-Mount <quanah(a)openldap.org&gt; changed: What |Removed |Added ---------------------------------------------------------------------------- Status|RESOLVED |VERIFIED -- You are receiving this mail because: You are on the CC list for the issue.

https://bugs.openldap.org/show_bug.cgi?id=9584 --- Comment #10 from Quanah Gibson-Mount <quanah(a)openldap.org&gt; --- Commits: • 75636a40 by Ondřej Kuzník at 2021-12-15T01:22:38+00:00 ITS#9584 Track refreshing status explicitly -- You are receiving this mail because: You are on the CC list for the issue.