openldap-bugs

openldap-bugs@openldap.org

1 participants
20967 discussions

Re: (ITS#8875) [Patch] Performance problems in back-mdb with large DITs and many aliases
by hyc＠symas.com 04 Apr '19

04 Apr '19

henrik.bohnenkamp(a)ionos.com wrote: > On Fri, Jan 25, 2019 at 06:31:41AM -0800, Quanah Gibson-Mount wrote: >> --On Friday, January 25, 2019 1:51 PM +0000 hbohnenkamp(a)united-internet.de >> wrote: >> Hi, thanks for the report and investigation. I've briefly reviewed your patches. Please squash your #3 into #2, there's no reason to preserve that. We don't use "inline" in this code. If the compiler is smart enough it will inline automatically anyway. Please drop that from your patch. No need to keep the old mdb_idscope invocation commented out in patch #2. Just delete it, we have the git history. In patch #1 delete.c your patch line #145 is incorrect. You should be using is_entry_alias(e) instead of (op->ora_e.) The entry in question is not part of the op request, only its DN is in the Delete request. Not sure why you dropped nsubs from the diskNode definition. Are you sure you haven't broken the nsubs processing in this patch? > > Hi, > > I have now done quite a lot of testing in the way described in my > email from January (Followup 3). I used version 2.4.47, not the HEAD > of the openldap git master branch. > > I have found one bug, the one mentioned already in the same email. I > have added a fix for that bug to the patch directory in > https://github.com/hbo/openldap-mdb-deref-problem > > I have come across the regression with the back-hdb backend again, > which I also described in my previous email already. I am now > convinced that this regression is really a problem with the back-hdb > implementation, and I can show it without using my patched > implementation of back-mdb. The demonstration requires a bit more work > to be really presentable and should in any case go into a different > bug report. > > Apart from these two issues I have not seen any regressions, and I am > now quite confident that my patch would work reliable in the > environment of my company... no guarantees for anything else :-) > > I have seen that Quanah has suggested to look at ITS#8875 for the > 2.4.48 release. Whether there will be a fix in 2.4.48 and whether it > will be based on my patch is of course up to the lead developers (the > change in the DB schema is certainly a strong point against), but this > patch can perhaps serve as something to test the actual fix against. > > Best regards > Henrik > > > -- > Henrik Bohnenkamp > > > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#9003) Update slapd-ldap(5) idassert-authzfrom for policy difference
by quanah＠openldap.org 04 Apr '19

04 Apr '19

Full_Name: Quanah Gibson-Mount Version: 2.4.47 OS: N/A URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (47.208.144.40) The slapd-ldap(5) man page has the following statement: idassert-authzFrom <authz-regexp> if defined, selects what local identities are authorized to exploit the identity assertion feature. The string <authz- regexp> follows the rules defined for the authzFrom attribute. See slapd.conf(5), section related to authz-policy, for details on the syntax of this field. However, it deviates from the rules laid out in the authz-policy section in that the special case of "*" has a different meaning for slapd-ldap/slapd-meta. In their case, this *allows* anonymous, while in the authz-policy case, anonymous is denied. This exception to the normal behavior needs to be noted.

1 0

Re: (ITS#8875) [Patch] Performance problems in back-mdb with large DITs and many aliases
by henrik.bohnenkamp＠ionos.com 02 Apr '19

02 Apr '19

On Fri, Jan 25, 2019 at 06:31:41AM -0800, Quanah Gibson-Mount wrote: > --On Friday, January 25, 2019 1:51 PM +0000 hbohnenkamp(a)united-internet.de > wrote: > Hi, I have now done quite a lot of testing in the way described in my email from January (Followup 3). I used version 2.4.47, not the HEAD of the openldap git master branch. I have found one bug, the one mentioned already in the same email. I have added a fix for that bug to the patch directory in https://github.com/hbo/openldap-mdb-deref-problem I have come across the regression with the back-hdb backend again, which I also described in my previous email already. I am now convinced that this regression is really a problem with the back-hdb implementation, and I can show it without using my patched implementation of back-mdb. The demonstration requires a bit more work to be really presentable and should in any case go into a different bug report. Apart from these two issues I have not seen any regressions, and I am now quite confident that my patch would work reliable in the environment of my company... no guarantees for anything else :-) I have seen that Quanah has suggested to look at ITS#8875 for the 2.4.48 release. Whether there will be a fix in 2.4.48 and whether it will be based on my patch is of course up to the lead developers (the change in the DB schema is certainly a strong point against), but this patch can perhaps serve as something to test the actual fix against. Best regards Henrik -- Henrik Bohnenkamp

1 0

Re: (ITS#9002) Add option to slapcat to honor rtxnsize setting
by hyc＠symas.com 29 Mar '19

29 Mar '19

michael(a)stroeder.com wrote: > On 3/29/19 8:58 PM, quanah(a)openldap.org wrote: >> To work around this, slapcat could be given an option to honor the rtxnsize >> setting in slapd.conf/cn=config. >> [..] >> It should be noted in the man page section for this option that the value of >> such a backup is of dubious quality, since it is no longer an exact "point in >> time" representation of the database but rather a bunch of different "point in >> times" stitched together into a single file. > > Even if slapcat now outputs an exact "point in time" representation of > the database this does not say anything about consistency of related > LDAP entries anyway. > > But I wonder how rtxnsize is supposed to interop with > LDAP transactions in 2.5. There is no interaction. LDAP transactions only affect write ops. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#9002) Add option to slapcat to honor rtxnsize setting
by michael＠stroeder.com 29 Mar '19

29 Mar '19

On 3/29/19 8:58 PM, quanah(a)openldap.org wrote: > To work around this, slapcat could be given an option to honor the rtxnsize > setting in slapd.conf/cn=config. > [..] > It should be noted in the man page section for this option that the value of > such a backup is of dubious quality, since it is no longer an exact "point in > time" representation of the database but rather a bunch of different "point in > times" stitched together into a single file. Even if slapcat now outputs an exact "point in time" representation of the database this does not say anything about consistency of related LDAP entries anyway. But I wonder how rtxnsize is supposed to interop with LDAP transactions in 2.5.

1 0

(ITS#9002) Add option to slapcat to honor rtxnsize setting
by quanah＠openldap.org 29 Mar '19

29 Mar '19

Full_Name: Quanah Gibson-Mount Version: 2.4.47 OS: N/A URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (47.208.144.40) There is currently an issue with back-mdb, where if the database is quite large (10GB+) and the environment is experiencing a heavy write load, that if you start a backup (slapcat or mdb_copy), the database will heavily fragment and grow significantly in size due to the read lock from the slapcat/mdb_copy process. To work around this, slapcat could be given an option to honor the rtxnsize setting in slapd.conf/cn=config. This would allow the read lock to be released periodically and thus reduce the amount of fragmentation that occurs during the backup process. There is not (at this time) a solution proposed for mdb_copy. It should be noted in the man page section for this option that the value of such a backup is of dubious quality, since it is no longer an exact "point in time" representation of the database but rather a bunch of different "point in times" stitched together into a single file.

1 0

Re: (ITS#9001) try_read1msg() takes O(n) steps to lookup a matching request
by ondra＠mistotebe.net 29 Mar '19

29 Mar '19

On Fri, Mar 29, 2019 at 01:38:30PM +0000, ondra(a)openldap.org wrote: > Full_Name: Ondrej Kuznik > Version: re24/master > OS: > URL: https://github.com/mistotebe/openldap/tree/its9001 > Submission from: (NULL) (82.10.24.68) > > > If a client has thousands or more requests in flight at the same time, it has to > look up the right request every time it receives a response. As the requests are > tracked in a doubly-linked list, lookup tends to take O(n) steps whichever > direction we take and things generally get out of hand from there on. > > The linked branch puts them into a TAvl for quicker lookup, with another commit > to let people test as (t)avl code is part of liblutil at the moment. A trivial program to expose this behaviour is available at ftp://ftp.openldap.org/incoming/searcher.c The same program takes under a second to handle 100000 searches for cn=monitor and children when linked to the patched libldap. -- Ondřej Kuzník Senior Software Engineer Symas Corporation http://www.symas.com Packaged, certified, and supported LDAP solutions powered by OpenLDAP

1 0

(ITS#9001) try_read1msg() takes O(n) steps to lookup a matching request
by ondra＠openldap.org 29 Mar '19

29 Mar '19

Full_Name: Ondrej Kuznik Version: re24/master OS: URL: https://github.com/mistotebe/openldap/tree/its9001 Submission from: (NULL) (82.10.24.68) If a client has thousands or more requests in flight at the same time, it has to look up the right request every time it receives a response. As the requests are tracked in a doubly-linked list, lookup tends to take O(n) steps whichever direction we take and things generally get out of hand from there on. The linked branch puts them into a TAvl for quicker lookup, with another commit to let people test as (t)avl code is part of liblutil at the moment.

1 0

Re: (ITS#8997) openldap-nssov/back ldap segfault
by matt＠pallissard.net 28 Mar '19

28 Mar '19

--mcr7yvikhm7jn4cn Content-Type: text/plain; charset=us-ascii Content-Disposition: inline > Please try this patch. Right on. Ran some initial tests, so far so good. I'll bang on it for a few hours and follow up sometime in the late afternoon. Thanks a bunch! You guys rock! Matt Pallissard --mcr7yvikhm7jn4cn Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iHUEABYIAB0WIQTvIUMPApUGn6YFkXl1uof+t048SQUCXJzldgAKCRB1uof+t048 SaMGAP9G5n22WPLEVK/7TJ3lSGjl+BNRaj5AC0U3Im+DE60JewD/TrMXYDYlwpu8 kdOwSqY3UvRKHMdI13cEg8MG6iH3yAo= =chSM -----END PGP SIGNATURE----- --mcr7yvikhm7jn4cn--

1 0

Re: (ITS#8997) openldap-nssov/back ldap segfault
by hyc＠symas.com 28 Mar '19

28 Mar '19

This is a multi-part message in MIME format. --------------23850F02105CA62A388BCB06 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Pallissard, Matthew wrote: >> Actually now that I'm looking more closely at this output; the last line below looks interesting. >> >> (I probably should have tacked this on the backtrace) >> >> Thread 4 "slapd" received signal SIGSEGV, Segmentation fault. >> [Switching to Thread 0x7ffff0c7c700 (LWP 341)] >> nssov_dn2uid (op=0x7ffff0c7b730, ni=0x555555a24420, dn=0x7fffe811eab0, uid=0x7fffe8002450) at passwd.c:137 >> 137 passwd.c: No such file or directory. >> > > Sorry, that was from the wrong output, see below for the correct snippet. > > Thread 3 "slapd" received signal SIGSEGV, Segmentation fault. > [Switching to Thread 0x7ffff277f700 (LWP 170)] > nssov_dn2uid (op=0x7ffff277e730, ni=0x555555a24420, dn=0x7fffe4128430, uid=0x7fffe4002430) at passwd.c:137 > 137 passwd.c: No such file or directory. > > Matt Pallissard > Thanks that was a crucial piece. Please try this patch. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/ --------------23850F02105CA62A388BCB06 Content-Type: text/plain; charset=UTF-8; name="dif.txt" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="dif.txt" diff --git a/servers/slapd/back-ldap/search.c b/servers/slapd/back-ldap/search.c index 63345aed22..82e72aa8a7 100644 --- a/servers/slapd/back-ldap/search.c +++ b/servers/slapd/back-ldap/search.c @@ -1006,6 +1006,7 @@ retry: e = ldap_first_entry( lc->lc_ld, result ); if ( e == NULL ) { /* the entry exists, but it doesn't match the filter? */ + rc = LDAP_NO_RESULTS_RETURNED; goto cleanup; } --------------23850F02105CA62A388BCB06--

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs