openldap-bugs

openldap-bugs@openldap.org

1 participants
20965 discussions

Re: (ITS#8958) 2nd cn=config update blocks slapd while adding subordinate index
by hyc＠symas.com 20 Jan '19

20 Jan '19

Hallvard Breien Furuseth wrote: > On 1/19/19 8:04 PM, Howard Chu wrote: >> modifications to cn=3Dconfig require no other threads to be active. On= ce the background indexer >> starts running, its activity will of course block further cn=3Dconfig = modifications until it >> completes. >=20 > Nothing "of course" about it.=C2=A0 I don't know why running > the background indexer is considered a change to cn=3Dconfig. It is not. But the indexing rules can be changed by further config mods, = and we don't want the config to change out from under it. > The ldapmodify did the change to cn=3Dconfig and completed. >=20 > I can guess that it might block cn=3Dconfig simply because that's > the safe way to have both an indexer and dynamic config.=C2=A0 Or > I can read the code.=C2=A0 But "read the code" !=3D "of course". This was all discussed on the -devel mailing list when the feature was fi= rst implemented. http://www.openldap.org/lists/openldap-devel/200504/msg00090.html > So... back to start, I guess I was suggesting that the > indexing task should stop if another config mod comes in, > if it can resume afterwards.=C2=A0 And the config change need > only fail if it might prevent the indexer from resuming. > I suppose that gets complicated in a hurry though. Stopping and resuming is problematic, especially if other config changes = to the DB occur that would invalidate the in-progress indexing. We could arrange to return LDAP_BUSY to config mods if any background ind= exers are running. Or as I already mentioned, we could try to only do this for relevant conf= ig mods, e.g. to the active backend, or to global schema. --=20 -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8958) 2nd cn=config update blocks slapd while adding subordinate index
by h.b.furuseth＠usit.uio.no 19 Jan '19

19 Jan '19

On 1/19/19 8:04 PM, Howard Chu wrote: > modifications to cn=config require no other threads to be active. Once the background indexer > starts running, its activity will of course block further cn=config modifications until it > completes. Nothing "of course" about it. I don't know why running the background indexer is considered a change to cn=config. The ldapmodify did the change to cn=config and completed. I can guess that it might block cn=config simply because that's the safe way to have both an indexer and dynamic config. Or I can read the code. But "read the code" != "of course". So... back to start, I guess I was suggesting that the indexing task should stop if another config mod comes in, if it can resume afterwards. And the config change need only fail if it might prevent the indexer from resuming. I suppose that gets complicated in a hurry though.

1 0

Re: (ITS#8958) 2nd cn=config update blocks slapd while adding subordinate index
by h.b.furuseth＠usit.uio.no 19 Jan '19

19 Jan '19

On 1/20/19 5:26 AM, Howard Chu wrote: > Hallvard Breien Furuseth wrote: >> On 1/20/19 4:48 AM, Hallvard Breien Furuseth wrote: >>> On 1/20/19 2:14 AM, Howard Chu wrote: >>>> This was the safest approach, considering that a config change could alter/remove context out >>>> from under running threads otherwise. >>> >>> No, the safest approach is to fail the 2nd cn=config mod >>> unless the user says it's OK to block the server. > > How would the user say so? An LDAP control. Or the modify operation could modify an operational attribute which which doesn't exist in LDAP, it just changes the operation's behavior. Or only accept cn=config mods after the connection has added an ephemeral control entry which says how cn=config mods should behave in that connection. Maybe that's the least annoying way, since there won't be any special cases for the use to cope with. Either way, it sounds like there's no right answer for RE24's default behavior, and the only fix there would involve a config option which says which behavior to pick. (It'd be nice to be able to say things like "it's OK to hang if ldapi:// is the only listener", but I suppose then people have a hundred other conditions which would be nice too.) I don't like any of my own answers to this though. I just dislike server hangs more. >> Or fail the 1st cn=config mod too, if that's easier. > > Huh? That op has already completed. The indexer is started as a background task after > the mod op completes. I meant fail when the operation is issued, but never mind. That was silly, and cumbersome for the user to find out when he can safely issue the next operation. Either way, diagnosticMessage "RTFM: back-mdb(5) about olcDbIndex and slapd-config(5) about possible server hangs". And we should document when cn=config mods can be dangerous and how to use it safely without the users noticing. There should in any case be a method to ask the server "is cn=config very busy now, or is another cn=config mod safe?". Like failing the operation if it is not, like I said first.

1 0

Re: (ITS#8958) 2nd cn=config update blocks slapd while adding subordinate index
by hyc＠symas.com 19 Jan '19

19 Jan '19

Hallvard Breien Furuseth wrote: > On 1/20/19 4:48 AM, Hallvard Breien Furuseth wrote: >> On 1/20/19 2:14 AM, Howard Chu wrote: >>> This was the safest approach, considering that a config change could alter/remove context out >>> from under running threads otherwise. >> >> No, the safest approach is to fail the 2nd cn=config mod >> unless the user says it's OK to block the server. How would the user say so? > Or fail the 1st cn=config mod too, if that's easier. Huh? That op has already completed. The indexer is started as a background task after the mod op completes. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8958) 2nd cn=config update blocks slapd while adding subordinate index
by h.b.furuseth＠usit.uio.no 19 Jan '19

19 Jan '19

On 1/20/19 4:48 AM, Hallvard Breien Furuseth wrote: > On 1/20/19 2:14 AM, Howard Chu wrote: >> This was the safest approach, considering that a config change could >> alter/remove context out >> from under running threads otherwise. > > No, the safest approach is to fail the 2nd cn=config mod > unless the user says it's OK to block the server. Or fail the 1st cn=config mod too, if that's easier.

1 0

Re: (ITS#8958) 2nd cn=config update blocks slapd while adding subordinate index
by h.b.furuseth＠usit.uio.no 19 Jan '19

19 Jan '19

On 1/20/19 2:14 AM, Howard Chu wrote: > Ah, the 2nd cn=config mod has requested a pause, which can't succeed until all > background threads complete. And no new threads will be run while a pause is being requested. Exactly. > This was the safest approach, considering that a config change could alter/remove context out > from under running threads otherwise. No, the safest approach is to fail the 2nd cn=config mod unless the user says it's OK to block the server.

1 0

Re: (ITS#8958) 2nd cn=config update blocks slapd while adding subordinate index
by hyc＠symas.com 19 Jan '19

19 Jan '19

Hallvard Breien Furuseth wrote: > On 1/19/19 9:55 PM, h.b.furuseth(a)usit.uio.no wrote: >> On 1/19/19 8:04 PM, Howard Chu wrote: >>> The *server* is not hung. Only modifications to the cn=config DB are blocked. >> >> Yes it is. > > That is, the 2nd mod to cn=config blocks, and that's fine by me, except > _then_ the server is hung. Ah, the 2nd cn=config mod has requested a pause, which can't succeed until all background threads complete. And no new threads will be run while a pause is being requested. This was the safest approach, considering that a config change could alter/remove context out from under running threads otherwise. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8958) 2nd cn=config update blocks slapd while adding subordinate index
by h.b.furuseth＠usit.uio.no 19 Jan '19

19 Jan '19

On 1/19/19 9:55 PM, h.b.furuseth(a)usit.uio.no wrote: > On 1/19/19 8:04 PM, Howard Chu wrote: >> The *server* is not hung. Only modifications to the cn=config DB are blocked. > > Yes it is. That is, the 2nd mod to cn=config blocks, and that's fine by me, except _then_ the server is hung. > See the ldapwhoami in my example. And the 'time ldapwhoami' result in output.txt. Sorry, unreadable scripting on my part. Notice the single '&' in ldapmodify & { sleep; ldapwhoami; } That is, ldapmodify in the background and then ldapwhoami. I was pasting to/from the command line, and didn't want the ldapmodify to start until I pressed Return for the ldapwhoami. Thus the weird concatenation.

1 0

Re: (ITS#8958) 2nd cn=config update blocks slapd while adding subordinate index
by h.b.furuseth＠usit.uio.no 19 Jan '19

19 Jan '19

On 1/19/19 8:04 PM, Howard Chu wrote: > The *server* is not hung. Only modifications to the cn=config DB are blocked. Yes it is. See the ldapwhoami in my example.

1 0

Re: (ITS#8958) 2nd cn=config update blocks slapd while adding subordinate index
by hyc＠symas.com 19 Jan '19

19 Jan '19

Hallvard Breien Furuseth wrote: > Testing again 'subordinate' looks irrelevant. I wonder what I > did in my original and bigger testcase to make it seem relevant. >=20 > On 1/19/19 6:30 PM, Howard Chu wrote: >> This is not really surprising. >=20 > I don't know these interactions well enough to be unsurprised. >=20 > It's not hard to guess that modify(olcDbIndex) might be heavy and > had better be tested.=C2=A0 A simple test says it returns quickly.=C2=A0= I > don't know why the ongoing indexing keeps cn=3Dconfig locked after > the operation succeeded, instead of keeping something in the > indexed DB locked.=C2=A0 I wouldn't expect others to guess that either. modifications to cn=3Dconfig require no other threads to be active. Once = the background indexer starts running, its activity will of course block further cn=3Dconfig mod= ifications until it completes. >> Are you suggesting the indexing task should stop if another config mod= comes in? >=20 > I'm suggesting to fail the 2nd operation if that's what it takes. > Or only hang if the user passes some LDAP control. >=20 > I'm suggesting that config pauses should be brief.=C2=A0 Server hangs > are bad.=C2=A0 Even a crash can be better when other load-balanced > servers can take over.=C2=A0 Not that I'm suggesting that "fix":-) The *server* is not hung. Only modifications to the cn=3Dconfig DB are bl= ocked. You might make a case that background indexing on a backend shouldn't blo= ck writes to unrelated parts of the cn=3Dconfig tree. The current pause desi= gn wouldn't easily support that, but it might be worth working on. Also it's not clea= r which parts of the tree are actually unrelated. E.g., changing schema elements = could affect all backends at once. --=20 -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs