openldap-bugs

openldap-bugs@openldap.org

1 participants
20965 discussions

Re: (ITS#8959) duplicate symbols
by quanah＠symas.com 24 Jan '19

24 Jan '19

--On Thursday, January 24, 2019 6:26 PM +0000 ruga(a)protonmail.com wrote: > Full_Name: Rupert > Version: 2.4.47 > OS: Darwin alice 17.7.0 Darwin Kernel Version 17.7.0: Fri Nov 2 20:43:16 > PDT 2018; root:xnu-4570.71.17~1/RELEASE_X86_64 x86_64 URL: > ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (5.170.120.126) Builds fine for me using clang compiler on FreeBSD. Also appears to work fine for Apple, since it ships the OpenLDAP libraries. You'll need to provide more details as to exactly what you're doing to encounter this issue. --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

Re: (ITS#8960) code quality
by quanah＠symas.com 24 Jan '19

24 Jan '19

--On Thursday, January 24, 2019 6:47 PM +0000 ruga(a)protonmail.com wrote: > Full_Name: Rupert > Version: - > OS: - > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (5.170.120.126) > > > The source code of openldap went through lumnify's meat grinder, > receiving a score of 1 (fail) out of 5. Score 1 means "A lot of patch Sounds like lumnify is a poor quality product. You may wish to use something better written? In any case, the ITS system is for bug reports, this is clearly not a bug report. It will be closed. --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

(ITS#8960) code quality
by ruga＠protonmail.com 24 Jan '19

24 Jan '19

Full_Name: Rupert Version: - OS: - URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (5.170.120.126) The source code of openldap went through lumnify's meat grinder, receiving a score of 1 (fail) out of 5. Score 1 means "A lot of patch work and bandages, project is most likely in initial stage of development. Most of source code is highly unmaintainable and feels a bit chaotic. It is good starting point, but plenty of work ahead. Don't give up yet!". I see old bugs pending. Is the project active? [1] https://sysadmin.libhunt.com/openldap-alternatives [2] https://lumnify.com/grades/?rel=libhunt

1 0

(ITS#8959) duplicate symbols
by ruga＠protonmail.com 24 Jan '19

24 Jan '19

Full_Name: Rupert Version: 2.4.47 OS: Darwin alice 17.7.0 Darwin Kernel Version 17.7.0: Fri Nov 2 20:43:16 PDT 2018; root:xnu-4570.71.17~1/RELEASE_X86_64 x86_64 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (5.170.120.126) NOTE >make [...] duplicate symbol _ber_int_options in: /opt/src/openldap-2.4.47/libraries/liblber/.libs/liblber.a(options.o) ../../libraries/liblber/.libs/liblber.a(options.o) duplicate symbol _ber_set_option in: /opt/src/openldap-2.4.47/libraries/liblber/.libs/liblber.a(options.o) ../../libraries/liblber/.libs/liblber.a(options.o) duplicate symbol _ber_get_option in: /opt/src/openldap-2.4.47/libraries/liblber/.libs/liblber.a(options.o) ../../libraries/liblber/.libs/liblber.a(options.o) ld: 3 duplicate symbols for architecture x86_64 clang-7: error: linker command failed with exit code 1 (use -v to see invocation) make[2]: *** [apitest] Error 1 make[2]: *** Waiting for unfinished jobs.... duplicate symbol _ber_int_options in: /opt/src/openldap-2.4.47/libraries/liblber/.libs/liblber.a(options.o) ../../libraries/liblber/.libs/liblber.a(options.o) duplicate symbol _ber_set_option in: /opt/src/openldap-2.4.47/libraries/liblber/.libs/liblber.a(options.o) ../../libraries/liblber/.libs/liblber.a(options.o) duplicate symbol _ber_get_option in: /opt/src/openldap-2.4.47/libraries/liblber/.libs/liblber.a(options.o) ../../libraries/liblber/.libs/liblber.a(options.o) ld: 3 duplicate symbols for architecture x86_64 clang-7: error: linker command failed with exit code 1 (use -v to see invocation) make[2]: *** [dntest] Error 1 make[1]: *** [all-common] Error 1 make: *** [all-common] Error 1 COMMENT Silly bug, tripping the compiler.

1 0

Re: (ITS#8958) 2nd cn=config update blocks slapd while adding subordinate index
by ondra＠mistotebe.net 22 Jan '19

22 Jan '19

On Tue, Jan 22, 2019 at 05:49:12PM +0000, h.b.furuseth(a)usit.uio.no wrote: > On 1/21/19 7:49 PM, Ondřej Kuzník wrote: >> Except there are no locks as you know being the author of parts of >> code that deals with what I'm about to outline anyway: >> >> Whenever a cn=config op is about to be processed, a pause is requested. >> That tells worker threads to stop picking up new work and waits until >> they're all quiet. The indexing task is run by one of these worker >> threads. > > Duuh, right. I got stuck looking for what's special about the > indexing task and couldn't find it:-( I need to make it special. > > So, let tasks declare their expected speed until finish or > between pausechecks. At FAST=1 (default) and SLOW=0. > A pause only stops tasks with speed < ltp_pause. > In thread_pool_pause(), replace the WANT_PAUSE stage with > > while (++ltp_pause <= max speed) { > wait until no more tasks with speed < ltp_pause; > } > > Then fast tasks should breeze past slow ones when preparing > to pause. Until all threads have slow tasks, anyway. > > To mitigate that, we'd need to predeclare the speed when > submitting a task, and limit the number of parallel slow > tasks. pool_submit() could stash the rest in a "slow queue" > instead of submitting. But I don't want to go there yet. The problem is not that it's still scheduled when the pause is requested, scheduled tasks don't prevent a pause (and I think some of them need to be a bit more aware of that fact), but that it's been picked up by a worker thread and is running while cn=config waits for them to pause. What I was suggesting was that it could check every so often whether it might be holding up a pause and join it just like some other tasks do. But that's only a good idea if it's able to handle the changes that happen during the pause (indexes reconfigured again, the database going away, ...) as well as being able to release any locks temporarily. -- Ondřej Kuzník Senior Software Engineer Symas Corporation http://www.symas.com Packaged, certified, and supported LDAP solutions powered by OpenLDAP

1 0

Re: (ITS#8958) 2nd cn=config update blocks slapd while adding subordinate index
by h.b.furuseth＠usit.uio.no 22 Jan '19

22 Jan '19

On 1/21/19 7:49 PM, Ondřej Kuzník wrote: > Except there are no locks as you know being the author of parts of > code that deals with what I'm about to outline anyway: > > Whenever a cn=config op is about to be processed, a pause is requested. > That tells worker threads to stop picking up new work and waits until > they're all quiet. The indexing task is run by one of these worker > threads. Duuh, right. I got stuck looking for what's special about the indexing task and couldn't find it:-( I need to make it special. So, let tasks declare their expected speed until finish or between pausechecks. At FAST=1 (default) and SLOW=0. A pause only stops tasks with speed < ltp_pause. In thread_pool_pause(), replace the WANT_PAUSE stage with while (++ltp_pause <= max speed) { wait until no more tasks with speed < ltp_pause; } Then fast tasks should breeze past slow ones when preparing to pause. Until all threads have slow tasks, anyway. To mitigate that, we'd need to predeclare the speed when submitting a task, and limit the number of parallel slow tasks. pool_submit() could stash the rest in a "slow queue" instead of submitting. But I don't want to go there yet.

1 0

Re: (ITS#8958) 2nd cn=config update blocks slapd while adding subordinate index
by ondra＠mistotebe.net 21 Jan '19

21 Jan '19

On Mon, Jan 21, 2019 at 03:23:55PM +0000, h.b.furuseth(a)usit.uio.no wrote: > On 20.01.2019 19:52, Hallvard Breien Furuseth wrote: > > Can cn=config have 2 lock levels? That fixes this ITS (my > > ldapwhoami hang) without introducing cn=config failures. > > > > The outer lock serializes cn=config ops including the indexer. > > Other ops do not lock it. The inner lock blocks slapd like > > cn=config does now, and should only be held briefly. Always > > hold the outer lock when taking the inner lock. > > Correction again: while taking and holding the inner lock. > Except non-config ops. Except there are no locks as you know being the author of parts of code that deals with what I'm about to outline anyway: Whenever a cn=config op is about to be processed, a pause is requested. That tells worker threads to stop picking up new work and waits until they're all quiet. The indexing task is run by one of these worker threads. Some long-running processes (syncrepl) will regularly check to see if they should pause as well: https://www.openldap.org/devel/gitweb.cgi?p=openldap.git;a=blob;f=servers/s… Handling things in a similar way might be an option if we could survive anything changing in cn=config while we yield. That might be a big "if" to implement, though. -- Ondřej Kuzník Senior Software Engineer Symas Corporation http://www.symas.com Packaged, certified, and supported LDAP solutions powered by OpenLDAP

1 0

Re: (ITS#8958) 2nd cn=config update blocks slapd while adding subordinate index
by h.b.furuseth＠usit.uio.no 21 Jan '19

21 Jan '19

On 20.01.2019 19:52, Hallvard Breien Furuseth wrote: > Can cn=config have 2 lock levels? That fixes this ITS (my > ldapwhoami hang) without introducing cn=config failures. > > The outer lock serializes cn=config ops including the indexer. > Other ops do not lock it. The inner lock blocks slapd like > cn=config does now, and should only be held briefly. Always > hold the outer lock when taking the inner lock. Correction again: while taking and holding the inner lock. Except non-config ops. > Also, the inner lock could be taken and released several > times during a config op when feasible. That might improve > non-config op latency, for the price of slower config ops.

1 0

Re: (ITS#8958) 2nd cn=config update blocks slapd while adding subordinate index
by h.b.furuseth＠usit.uio.no 20 Jan '19

20 Jan '19

I wrote: > Limiting the problems beyond that is a bonus. Or maybe not. > A cn=config which fails in some obscure cases is flaky. (...) Correction: Obscure _situations_, not obscure cases. As in, operation X normally succeds, but can fail if you just did the somewhat-related operation Y.

1 0

Re: (ITS#8958) 2nd cn=config update blocks slapd while adding subordinate index
by h.b.furuseth＠usit.uio.no 20 Jan '19

20 Jan '19

Can cn=config have 2 lock levels? That fixes this ITS (my ldapwhoami hang) without introducing cn=config failures. The outer lock serializes cn=config ops including the indexer. Other ops do not lock it. The inner lock blocks slapd like cn=config does now, and should only be held briefly. Always hold the outer lock when taking the inner lock. Also, the inner lock could be taken and released several times during a config op when feasible. That might improve non-config op latency, for the price of slower config ops. On 1/20/19 4:42 PM, Howard Chu wrote: > Hallvard Breien Furuseth wrote: >> On 1/19/19 8:04 PM, Howard Chu wrote: >>> modifications to cn=config require no other threads to be active. Once the background indexer >>> starts running, its activity will of course block further cn=config modifications until it >>> completes. >> >> Nothing "of course" about it. I don't know why running >> the background indexer is considered a change to cn=config. > > It is not. But the indexing rules can be changed by further config mods, and we > don't want the config to change out from under it. OK. I was just parsing your worlds too literally. Also, I'm trying to think like an admin, not an OpenLDAP insider: >> The ldapmodify did the change to cn=config and completed. >> >> I can guess that it might block cn=config simply because that's >> the safe way to have both an indexer and dynamic config. Or >> I can read the code. But "read the code" != "of course". > > This was all discussed on the -devel mailing list when the feature was first implemented. > http://www.openldap.org/lists/openldap-devel/200504/msg00090.html Thanks. >> So... back to start, I guess I was suggesting that the >> indexing task should stop if another config mod comes in, >> if it can resume afterwards. And the config change need >> only fail if it might prevent the indexer from resuming. >> I suppose that gets complicated in a hurry though. > > Stopping and resuming is problematic, especially if other config changes to the DB occur > that would invalidate the in-progress indexing. OK. > We could arrange to return LDAP_BUSY to config mods if any background indexers are running. > Or as I already mentioned, we could try to only do this for relevant config mods, e.g. to > the active backend, or to global schema. I'm mostly interested in the worst expected case during normal cn=config use - whether hang or error. Limiting the problems beyond that is a bonus. Or maybe not. A cn=config which fails in some obscure cases is flaky. A cn=config which fails more frequently but in predictable ways can be annoying, but is easy to figure out how to operate. I suppose a simple test is how simple the documentation can be: If I want non-config operations to not hang, what do I do and what do I avoid? And same thing if want to write a config script which won't need to cope with expected failures.

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs