openldap-bugs

openldap-bugs@openldap.org

1 participants
21065 discussions

Re: (ITS#8875) [Patch] Performance problems in back-mdb with large DITs and many aliases
by henrik.bohnenkamp＠ionos.com 02 Apr '19

02 Apr '19

On Fri, Jan 25, 2019 at 06:31:41AM -0800, Quanah Gibson-Mount wrote: > --On Friday, January 25, 2019 1:51 PM +0000 hbohnenkamp(a)united-internet.de > wrote: > Hi, I have now done quite a lot of testing in the way described in my email from January (Followup 3). I used version 2.4.47, not the HEAD of the openldap git master branch. I have found one bug, the one mentioned already in the same email. I have added a fix for that bug to the patch directory in https://github.com/hbo/openldap-mdb-deref-problem I have come across the regression with the back-hdb backend again, which I also described in my previous email already. I am now convinced that this regression is really a problem with the back-hdb implementation, and I can show it without using my patched implementation of back-mdb. The demonstration requires a bit more work to be really presentable and should in any case go into a different bug report. Apart from these two issues I have not seen any regressions, and I am now quite confident that my patch would work reliable in the environment of my company... no guarantees for anything else :-) I have seen that Quanah has suggested to look at ITS#8875 for the 2.4.48 release. Whether there will be a fix in 2.4.48 and whether it will be based on my patch is of course up to the lead developers (the change in the DB schema is certainly a strong point against), but this patch can perhaps serve as something to test the actual fix against. Best regards Henrik -- Henrik Bohnenkamp

1 0

Re: (ITS#9002) Add option to slapcat to honor rtxnsize setting
by hyc＠symas.com 29 Mar '19

29 Mar '19

michael(a)stroeder.com wrote: > On 3/29/19 8:58 PM, quanah(a)openldap.org wrote: >> To work around this, slapcat could be given an option to honor the rtxnsize >> setting in slapd.conf/cn=config. >> [..] >> It should be noted in the man page section for this option that the value of >> such a backup is of dubious quality, since it is no longer an exact "point in >> time" representation of the database but rather a bunch of different "point in >> times" stitched together into a single file. > > Even if slapcat now outputs an exact "point in time" representation of > the database this does not say anything about consistency of related > LDAP entries anyway. > > But I wonder how rtxnsize is supposed to interop with > LDAP transactions in 2.5. There is no interaction. LDAP transactions only affect write ops. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#9002) Add option to slapcat to honor rtxnsize setting
by michael＠stroeder.com 29 Mar '19

29 Mar '19

On 3/29/19 8:58 PM, quanah(a)openldap.org wrote: > To work around this, slapcat could be given an option to honor the rtxnsize > setting in slapd.conf/cn=config. > [..] > It should be noted in the man page section for this option that the value of > such a backup is of dubious quality, since it is no longer an exact "point in > time" representation of the database but rather a bunch of different "point in > times" stitched together into a single file. Even if slapcat now outputs an exact "point in time" representation of the database this does not say anything about consistency of related LDAP entries anyway. But I wonder how rtxnsize is supposed to interop with LDAP transactions in 2.5.

1 0

(ITS#9002) Add option to slapcat to honor rtxnsize setting
by quanah＠openldap.org 29 Mar '19

29 Mar '19

Full_Name: Quanah Gibson-Mount Version: 2.4.47 OS: N/A URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (47.208.144.40) There is currently an issue with back-mdb, where if the database is quite large (10GB+) and the environment is experiencing a heavy write load, that if you start a backup (slapcat or mdb_copy), the database will heavily fragment and grow significantly in size due to the read lock from the slapcat/mdb_copy process. To work around this, slapcat could be given an option to honor the rtxnsize setting in slapd.conf/cn=config. This would allow the read lock to be released periodically and thus reduce the amount of fragmentation that occurs during the backup process. There is not (at this time) a solution proposed for mdb_copy. It should be noted in the man page section for this option that the value of such a backup is of dubious quality, since it is no longer an exact "point in time" representation of the database but rather a bunch of different "point in times" stitched together into a single file.

1 0

Re: (ITS#9001) try_read1msg() takes O(n) steps to lookup a matching request
by ondra＠mistotebe.net 29 Mar '19

29 Mar '19

On Fri, Mar 29, 2019 at 01:38:30PM +0000, ondra(a)openldap.org wrote: > Full_Name: Ondrej Kuznik > Version: re24/master > OS: > URL: https://github.com/mistotebe/openldap/tree/its9001 > Submission from: (NULL) (82.10.24.68) > > > If a client has thousands or more requests in flight at the same time, it has to > look up the right request every time it receives a response. As the requests are > tracked in a doubly-linked list, lookup tends to take O(n) steps whichever > direction we take and things generally get out of hand from there on. > > The linked branch puts them into a TAvl for quicker lookup, with another commit > to let people test as (t)avl code is part of liblutil at the moment. A trivial program to expose this behaviour is available at ftp://ftp.openldap.org/incoming/searcher.c The same program takes under a second to handle 100000 searches for cn=monitor and children when linked to the patched libldap. -- Ondřej Kuzník Senior Software Engineer Symas Corporation http://www.symas.com Packaged, certified, and supported LDAP solutions powered by OpenLDAP

1 0

(ITS#9001) try_read1msg() takes O(n) steps to lookup a matching request
by ondra＠openldap.org 29 Mar '19

29 Mar '19

Full_Name: Ondrej Kuznik Version: re24/master OS: URL: https://github.com/mistotebe/openldap/tree/its9001 Submission from: (NULL) (82.10.24.68) If a client has thousands or more requests in flight at the same time, it has to look up the right request every time it receives a response. As the requests are tracked in a doubly-linked list, lookup tends to take O(n) steps whichever direction we take and things generally get out of hand from there on. The linked branch puts them into a TAvl for quicker lookup, with another commit to let people test as (t)avl code is part of liblutil at the moment.

1 0

Re: (ITS#8997) openldap-nssov/back ldap segfault
by matt＠pallissard.net 28 Mar '19

28 Mar '19

--mcr7yvikhm7jn4cn Content-Type: text/plain; charset=us-ascii Content-Disposition: inline > Please try this patch. Right on. Ran some initial tests, so far so good. I'll bang on it for a few hours and follow up sometime in the late afternoon. Thanks a bunch! You guys rock! Matt Pallissard --mcr7yvikhm7jn4cn Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iHUEABYIAB0WIQTvIUMPApUGn6YFkXl1uof+t048SQUCXJzldgAKCRB1uof+t048 SaMGAP9G5n22WPLEVK/7TJ3lSGjl+BNRaj5AC0U3Im+DE60JewD/TrMXYDYlwpu8 kdOwSqY3UvRKHMdI13cEg8MG6iH3yAo= =chSM -----END PGP SIGNATURE----- --mcr7yvikhm7jn4cn--

1 0

Re: (ITS#8997) openldap-nssov/back ldap segfault
by hyc＠symas.com 28 Mar '19

28 Mar '19

This is a multi-part message in MIME format. --------------23850F02105CA62A388BCB06 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Pallissard, Matthew wrote: >> Actually now that I'm looking more closely at this output; the last line below looks interesting. >> >> (I probably should have tacked this on the backtrace) >> >> Thread 4 "slapd" received signal SIGSEGV, Segmentation fault. >> [Switching to Thread 0x7ffff0c7c700 (LWP 341)] >> nssov_dn2uid (op=0x7ffff0c7b730, ni=0x555555a24420, dn=0x7fffe811eab0, uid=0x7fffe8002450) at passwd.c:137 >> 137 passwd.c: No such file or directory. >> > > Sorry, that was from the wrong output, see below for the correct snippet. > > Thread 3 "slapd" received signal SIGSEGV, Segmentation fault. > [Switching to Thread 0x7ffff277f700 (LWP 170)] > nssov_dn2uid (op=0x7ffff277e730, ni=0x555555a24420, dn=0x7fffe4128430, uid=0x7fffe4002430) at passwd.c:137 > 137 passwd.c: No such file or directory. > > Matt Pallissard > Thanks that was a crucial piece. Please try this patch. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/ --------------23850F02105CA62A388BCB06 Content-Type: text/plain; charset=UTF-8; name="dif.txt" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="dif.txt" diff --git a/servers/slapd/back-ldap/search.c b/servers/slapd/back-ldap/search.c index 63345aed22..82e72aa8a7 100644 --- a/servers/slapd/back-ldap/search.c +++ b/servers/slapd/back-ldap/search.c @@ -1006,6 +1006,7 @@ retry: e = ldap_first_entry( lc->lc_ld, result ); if ( e == NULL ) { /* the entry exists, but it doesn't match the filter? */ + rc = LDAP_NO_RESULTS_RETURNED; goto cleanup; } --------------23850F02105CA62A388BCB06--

1 0

Re: (ITS#8997) openldap-nssov/back ldap segfault
by matt＠pallissard.net 28 Mar '19

28 Mar '19

--mb3a5envygzqum4d Content-Type: text/plain; charset=us-ascii Content-Disposition: inline > Actually now that I'm looking more closely at this output; the last line below looks interesting. > > (I probably should have tacked this on the backtrace) > > Thread 4 "slapd" received signal SIGSEGV, Segmentation fault. > [Switching to Thread 0x7ffff0c7c700 (LWP 341)] > nssov_dn2uid (op=0x7ffff0c7b730, ni=0x555555a24420, dn=0x7fffe811eab0, uid=0x7fffe8002450) at passwd.c:137 > 137 passwd.c: No such file or directory. > Sorry, that was from the wrong output, see below for the correct snippet. Thread 3 "slapd" received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7ffff277f700 (LWP 170)] nssov_dn2uid (op=0x7ffff277e730, ni=0x555555a24420, dn=0x7fffe4128430, uid=0x7fffe4002430) at passwd.c:137 137 passwd.c: No such file or directory. Matt Pallissard --mb3a5envygzqum4d Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iHUEABYIAB0WIQTvIUMPApUGn6YFkXl1uof+t048SQUCXJzYtgAKCRB1uof+t048 ST8nAP9UWBuG0358vqbQnGRxp4y4YMol81iU7UJswPBjPaBzgwD9HNk+aOAJJ1rn Czboi8ftlPsM0xnB34jmHtIYLz6EWww= =Vmu0 -----END PGP SIGNATURE----- --mb3a5envygzqum4d--

1 0

Re: (ITS#8997) openldap-nssov/back ldap segfault
by matt＠pallissard.net 28 Mar '19

28 Mar '19

--vpit5odwafhd55ju Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2019-03-28T06:54:35, Pallissard, Matthew wrote: > On 2019-03-21T15:13:30, Pallissard, Matthew wrote: > > On 2019-03-20T22:01:40, Howard Chu wrote: > > > matt(a)pallissard.net wrote: > > > > > > > 2. Turning the log level to 0 /seems/ to make the issue go away. I'= ll report > > > > back once I can confirm that. >=20 > Follow up, turning the log level to 0 does *not* prevent these segfaults. >=20 > Matt Pallissard Actually now that I'm looking more closely at this output; the last line be= low looks interesting. (I probably should have tacked this on the backtrace) =20 Thread 4 "slapd" received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7ffff0c7c700 (LWP 341)] nssov_dn2uid (op=3D0x7ffff0c7b730, ni=3D0x555555a24420, dn=3D0x7fffe811eab0= , uid=3D0x7fffe8002450) at passwd.c:137 137 passwd.c: No such file or directory. Matt Pallissard --vpit5odwafhd55ju Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iHUEABYIAB0WIQTvIUMPApUGn6YFkXl1uof+t048SQUCXJzXNQAKCRB1uof+t048 SXioAQDKoJWdiIpuasjmkaDnSI/bPGhCdBEXuYQfyZV5pqe++AD9GvYTSddxicKP wD0noDfTRI7oS5heRZHOnYO/ZK83RQc= =djIS -----END PGP SIGNATURE----- --vpit5odwafhd55ju--

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs