openldap-bugs

openldap-bugs@openldap.org

1 participants
21065 discussions

[Issue 8591] back-perl: segmentation fault when loading SampleLDAP.pl
by openldap-its＠openldap.org 03 Aug '21

03 Aug '21

https://bugs.openldap.org/show_bug.cgi?id=8591 Quanah Gibson-Mount <quanah(a)openldap.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|RESOLVED |VERIFIED -- You are receiving this mail because: You are on the CC list for the issue.

1 0

[Issue 8958] 2nd cn=config update blocks slapd while adding subordinate index
by openldap-its＠openldap.org 03 Aug '21

03 Aug '21

https://bugs.openldap.org/show_bug.cgi?id=8958 --- Comment #31 from Hallvard Furuseth <h.b.furuseth(a)usit.uio.no> --- On 03.08.2021 14:42, openldap-its(a)openldap.org wrote: > https://bugs.openldap.org/show_bug.cgi?id=8958 > > --- Comment #26 from Howard Chu <hyc(a)openldap.org> --- > I don't think we should be changing anything else about how tpool > handles pauses. We should just be fixing this specific case of the > indexer being a slow task, by implementing checkpointing into the > indexer. I.e., when it detects a pause request it should save its > current progress and pause itself. If it gets resumed it can pick up > where it left off, or if a config change affects it it can abort or > or start over. A checkpointing mechanism is needed anyway, for the > case of a (clean) shutdown while the indexer is running. For fixing the observed problem: Improving the indexer sounds great in any case, go ahead:-) No idea how much work it is. tpool.c was code I knew how to change, so I did. Will it then be as reactive as ordinary tasks, also for large databases? Merely "much faster than now" might be very different from "fast enough to not be a problem". In general: I do think slapd should recognize that some tasks can be notably slower that others. Latency is in part a scheduling issue, and it can hit particularly hard at pauses. Currently tpool does not help with that. Setspeed can help. Except if a run-time config change removes the database/overlay, I'm not up to date about whether that can happen. But that's so for rescheduling the indexer as well. back-sock/back-shell can also be slow, so I expect they have the same issue. If they "fix" that with pool_idle() before reading results, I expect config changes could hose the operation badly. Same with setspeed() in the backend, it'd be too late. Maybe if the database declared itself "slow", then slapd could setspeed() before dispatching the operation to the database. Hallvard -- You are receiving this mail because: You are on the CC list for the issue.

1 0

[Issue 6138] Bad Cancel/Abandon/"internal abandon"/Syncprov interactions
by openldap-its＠openldap.org 03 Aug '21

03 Aug '21

https://bugs.openldap.org/show_bug.cgi?id=6138 Quanah Gibson-Mount <quanah(a)openldap.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Resolution|FIXED |TEST -- You are receiving this mail because: You are on the CC list for the issue.

1 0

[Issue 6138] Bad Cancel/Abandon/"internal abandon"/Syncprov interactions
by openldap-its＠openldap.org 03 Aug '21

03 Aug '21

https://bugs.openldap.org/show_bug.cgi?id=6138 Quanah Gibson-Mount <quanah(a)openldap.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|IN_PROGRESS |RESOLVED Resolution|--- |FIXED --- Comment #10 from Quanah Gibson-Mount <quanah(a)openldap.org> --- Commits: • d3bd4aa7 by Howard Chu at 2021-08-03T15:19:49+00:00 ITS#6138 don't allow redundant abandon/cancel ops • 5a61175d by Howard Chu at 2021-08-03T15:19:49+00:00 ITS#6138 add lock flag to connection_op_finish() • e9e6fd71 by Howard Chu at 2021-08-03T15:19:49+00:00 ITS#6138 don't overwrite rs->sr_err after sending response • 795add7b by Howard Chu at 2021-08-03T15:19:49+00:00 ITS#6138 use SLAPD_NO_REPLY for persistent searches The particular code doesn't matter, any result besides SLAP_CB_CONTINUE always halts overlay/response processing. • 658e526b by Howard Chu at 2021-08-03T15:19:49+00:00 ITS#6138 drop unnecessary use of o_abandon • 66ed15a2 by Howard Chu at 2021-08-03T15:19:49+00:00 ITS#6138 FIx exop handler to ignore SLAPD_ASYNCOPs • a54f9985 by Howard Chu at 2021-08-03T15:19:49+00:00 ITS#6138 let frontend handle all exop responses • 8311c71f by Howard Chu at 2021-08-03T15:19:49+00:00 ITS#6138 fix test timing issue -- You are receiving this mail because: You are on the CC list for the issue.

1 0

[Issue 8649] Issue when fetching the backend that will receive the contextCSN when we have glued databases
by openldap-its＠openldap.org 03 Aug '21

03 Aug '21

https://bugs.openldap.org/show_bug.cgi?id=8649 --- Comment #5 from Quanah Gibson-Mount <quanah(a)openldap.org> --- Commits: • bc29154c by Howard Chu at 2021-08-03T13:10:27+01:00 ITS#8649 syncrepl: fix backend selection in glued DBs -- You are receiving this mail because: You are on the CC list for the issue.

1 0

[Issue 8958] 2nd cn=config update blocks slapd while adding subordinate index
by openldap-its＠openldap.org 03 Aug '21

03 Aug '21

https://bugs.openldap.org/show_bug.cgi?id=8958 --- Comment #30 from Hallvard Furuseth <h.b.furuseth(a)usit.uio.no> --- I wrote wrote: > https://bugs.openldap.org/show_bug.cgi?id=8958 >> In patch #5 >> + ldap_pvt_thread_pool_setspeed( &connection_pool, ctx, 0 ); >> >> Shouldn't the minimum speed be 1, not 0? > > That's just the API. 0 = "slowest". I didn't want to export details > of the tpool implementation, which might get replaced. Could use 0.0 > so it looks different, if floating point numbers are OK in libldap. That is, I have a vague feeling that merely mentioning a floating point number would require libm in some C implementations. Don't remember. -- You are receiving this mail because: You are on the CC list for the issue.

1 0

[Issue 8958] 2nd cn=config update blocks slapd while adding subordinate index
by openldap-its＠openldap.org 03 Aug '21

03 Aug '21

https://bugs.openldap.org/show_bug.cgi?id=8958 --- Comment #29 from Hallvard Furuseth <h.b.furuseth(a)usit.uio.no> --- On 03.08.2021 14:25, openldap-its(a)openldap.org wrote: > --- Comment #24 from Howard Chu <hyc(a)openldap.org> --- > (In reply to Quanah Gibson-Mount from comment #23) >> Created attachment 799 [details] >> proposed fix > > In patch #5 > + ldap_pvt_thread_pool_setspeed( &connection_pool, ctx, 0 ); > > Shouldn't the minimum speed be 1, not 0? That's just the API. 0 = "slowest". I didn't want to export details of the tpool implementation, which might get replaced. Could use 0.0 so it looks different, if floating point numbers are OK in libldap. > Since you have > +enum { NOT_PAUSED = 0, WANT_PAUSE = LDAP_PVT_THREAD_POOL_SPEED_MAX+1, PAUSED > }; -- You are receiving this mail because: You are on the CC list for the issue.

1 0

[Issue 8958] 2nd cn=config update blocks slapd while adding subordinate index
by openldap-its＠openldap.org 03 Aug '21

03 Aug '21

https://bugs.openldap.org/show_bug.cgi?id=8958 --- Comment #28 from Hallvard Furuseth <h.b.furuseth(a)usit.uio.no> --- On 03.08.2021 14:36, openldap-its(a)openldap.org wrote: > https://bugs.openldap.org/show_bug.cgi?id=8958 > > --- Comment #25 from Howard Chu <hyc(a)openldap.org> --- > (In reply to Hallvard Furuseth from comment #17) > >> (...) A pause only stops tasks with speed < ltp_pause. >> In thread_pool_pause(), replace the WANT_PAUSE stage with >> >> while (++ltp_pause <= max speed) { >> wait until no more tasks with speed < ltp_pause; >> } >> >> Then fast tasks should breeze past slow ones when preparing >> to pause. Until all threads have slow tasks, anyway. > > I don't understand how this solves anything. If a slow indexing > task is currently running, and a fast config mod comes in, it's > still the case that the config change could pull the DB out from > under the indexer task. So there's nothing safe about letting the > fast task progress while the slow task is still running Fast tasks still wait for *running* slow tasks. And when there is no pause involved, slow tasks get scheduled normally. This is only about scheduling when something wants a pause. setspeed() does CHECK_PAUSE, standing aside for faster tasks. Then, a fast task which wants a pause (cn=config change #2) won't block other fast tasks while a slower task (indexer) is running. So normal tasks will keep getting scheduled, instead of slapd locking up for them. This all depends on there being only a few config changes/slow tasks at any time, since they do occupy a thread. -- You are receiving this mail because: You are on the CC list for the issue.

1 0

[Issue 8958] 2nd cn=config update blocks slapd while adding subordinate index
by openldap-its＠openldap.org 03 Aug '21

03 Aug '21

https://bugs.openldap.org/show_bug.cgi?id=8958 --- Comment #27 from Ondřej Kuzník <ondra(a)mistotebe.net> --- On Tue, Aug 03, 2021 at 12:42:01PM +0000, openldap-its(a)openldap.org wrote: > I don't think we should be changing anything else about how tpool > handles pauses. We should just be fixing this specific case of the > indexer being a slow task, by implementing checkpointing into the > indexer. I.e., when it detects a pause request it should save its > current progress and pause itself. If it gets resumed it can pick up > where it left off, or if a config change affects it it can abort or > or start over. A checkpointing mechanism is needed anyway, for the > case of a (clean) shutdown while the indexer is running. I'll put a suggestion here that we discussed previously: to support this checkpointing for pauses/shutdowns, the indexer could be writing to a "scratch" database (whatever that means for each backend) along with resume data and move them into place when finished. You mentioned that for liblmdb, this would need support for a database to be renamed. -- You are receiving this mail because: You are on the CC list for the issue.

1 0

[Issue 8958] 2nd cn=config update blocks slapd while adding subordinate index
by openldap-its＠openldap.org 03 Aug '21

03 Aug '21

https://bugs.openldap.org/show_bug.cgi?id=8958 --- Comment #26 from Howard Chu <hyc(a)openldap.org> --- (In reply to Howard Chu from comment #25) > (In reply to Hallvard Furuseth from comment #17) > > > Duuh, right. I got stuck looking for what's special about the > > indexing task and couldn't find it:-( I need to make it special. > > > > So, let tasks declare their expected speed until finish or > > between pausechecks. At FAST=1 (default) and SLOW=0. > > A pause only stops tasks with speed < ltp_pause. > > In thread_pool_pause(), replace the WANT_PAUSE stage with > > > > while (++ltp_pause <= max speed) { > > wait until no more tasks with speed < ltp_pause; > > } > > > > Then fast tasks should breeze past slow ones when preparing > > to pause. Until all threads have slow tasks, anyway. > > I don't understand how this solves anything. If a slow indexing > task is currently running, and a fast config mod comes in, it's > still the case that the config change could pull the DB out from > under the indexer task. So there's nothing safe about letting the > fast task progress while the slow task is still running. I don't think we should be changing anything else about how tpool handles pauses. We should just be fixing this specific case of the indexer being a slow task, by implementing checkpointing into the indexer. I.e., when it detects a pause request it should save its current progress and pause itself. If it gets resumed it can pick up where it left off, or if a config change affects it it can abort or or start over. A checkpointing mechanism is needed anyway, for the case of a (clean) shutdown while the indexer is running. -- You are receiving this mail because: You are on the CC list for the issue.

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs