openldap-bugs July 2019

openldap-bugs@openldap.org

18 participants
59 discussions

Re: (ITS#8875) [Patch] Performance problems in back-mdb with large DITs and many aliases
by hyc＠symas.com 15 Jul '19

15 Jul '19

Henrik Bohnenkamp wrote: > On Mon, Jul 15, 2019 at 02:26:59PM +0100, Howard Chu wrote: >> >> Fyi, on our problematic test database with 11M entries and 3.7M aliases, a search with -a always , starting from the >> DB suffix, took 4 minutes without this patch, and 1235 minutes with this patch. >> >> Needless to say, that's not looking good. Still checking other test cases. > > Interesting, so the behavior is reversed now :-). I assume you have > found an alternative approach to solve the problem. That's fine with > me, I want the problem solved, not my patch integrated. I'm of course > interested in how you do it. Surely you did not get the 4 minutes with > a stock 2.4.48 slapd? For this size of DB we needed the ITS#8977 patches to accommodate larger IDLs. (I used 24 bits for IDLs, 16.7M slots) Also at this size, the IDL processing itself is the main bottleneck now. We would need to switch to bitmaps or trees to avoid this bottleneck, but that's also a much larger change than we can consider for this release. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8875) [Patch] Performance problems in back-mdb with large DITs and many aliases
by henrik.bohnenkamp＠ionos.com 15 Jul '19

15 Jul '19

On Mon, Jul 15, 2019 at 02:26:59PM +0100, Howard Chu wrote: >=20 > Fyi, on our problematic test database with 11M entries and 3.7M aliases, = a search with -a always , starting from the > DB suffix, took 4 minutes without this patch, and 1235 minutes with this = patch. >=20 > Needless to say, that's not looking good. Still checking other test cases. Interesting, so the behavior is reversed now :-). I assume you have found an alternative approach to solve the problem. That's fine with me, I want the problem solved, not my patch integrated. I'm of course interested in how you do it. Surely you did not get the 4 minutes with a stock 2.4.48 slapd?

1 0

Re: Regression after ITS#8427 fix with back-ldap
by ondra＠mistotebe.net 15 Jul '19

15 Jul '19

On Wed, Jul 10, 2019 at 07:02:48PM +0100, Howard Chu wrote: > Ondřej Kuzník wrote: >> On Wed, Jul 10, 2019 at 04:48:55PM +0000, hyc(a)symas.com wrote: >>> ondra(a)mistotebe.net wrote: >>>> On Thu, Jun 27, 2019 at 08:08:19PM +0000, a.chelouah(a)gmail.com wrote: >>>>> Hello, >>>>> >>>>> Commit 6f623dfa1ca65698c19ccc6c058cd170e633384e fixing ITS#8427 (Set up >>>>> TLS settings on each reconnection) introduce a regression when the proxy >>>>> connect to the**Backend ldap server via ldaps:// >>>>> >>>>> The relevent part of my config is: >>>>> >>>>> dn: olcDatabase={2}ldap,cn=config >>>>> objectClass: olcDatabaseConfig >>>>> objectClass: olcLDAPConfig >>>>> olcDatabase: {2}ldap >>>>> olcSuffix: dc=local >>>>> olcDbURI: ldaps://ldap.local >>>>> olcDbChaseReferrals: TRUE >>>>> olcDbRebindAsUser: TRUE >>>>> olcDbIDAssertBind: bindmethod=none tls_cacert=/etc/pki/tls/certs/ca.crt >>>>> olcDbIDAssertAuthzFrom: "*" >>>>> >>>>> (I also tried by setting LDAPTLS_CACERT env var when starting slapd) >>>>> >>>>> On backend ldap server logs, I get the message "TLS negociation failure" >>>> >>>> I've set up a test script here >>>> https://github.com/mistotebe/openldap/tree/its8427-regression >>>> >>>> This runs without issues but if you replace olcDbStartTLS with an >>>> analogous olcDbIDAssertBind in the configs, it seems the CA certificate >>>> is not set for the connection. >>> >>> Then this is a new bug. Clearly the idassert-bind option takes tls_cacert >>> as a parameter, so if it is provided it is expected to be used. >> >> Sure, on idassert connections only, though. When does back-ldap use one? >> I want to edit the linked script to do exercise that so we have a decent >> test for this now. > > idassert is used when you want back-ldap to propagate the incoming client's > identity to the remote server. It affects every operation that a client issues. Had to have a look at the code, so this is what seems to happen and I think the code does what it should regardless of the ITS#8427 patch, see updated test at the same place: https://github.com/mistotebe/openldap/tree/its8427-regression There are three completely separate types of connections as reflected in the configuration: - acl connections ("private" in terms of the code), used for internal operations and anything issued by the rootdn - idassert connections, used when the client identity should be asserted over the other side (bindmethod setting applies or proxyauth is in effect through other means), also currently used for acl connections if acl is unconfigured - normal connections (configured with the tls keyword) All of this should default to the slapd's internal client TLS settings, not sure how exactly that works yet and whether it reflects later changes to the slapd's client settings. So if anything should be shared between all connections, the preferred way should be using global TLS* options if I understand correctly. If different settings need to apply to different ldap backends, they need to be copied over to each option (acl, idassert and tls) that's in effect. There was a slight issue with how late configuration of olcStartTLS might not take effect, that is also handled in the above branch (and master soon after both full make test and make its finish). -- Ondřej Kuzník Senior Software Engineer Symas Corporation http://www.symas.com Packaged, certified, and supported LDAP solutions powered by OpenLDAP

1 0

Re: (ITS#8875) [Patch] Performance problems in back-mdb with large DITs and many aliases
by hyc＠symas.com 15 Jul '19

15 Jul '19

henrik.bohnenkamp(a)ionos.com wrote: > On Thu, Jun 27, 2019 at 10:34:19PM -0700, Quanah Gibson-Mount wrote: >> --On Sunday, April 07, 2019 10:15 AM +0000 henrik.bohnenkamp(a)ionos.com >> wrote: >> >>> On Thu, Apr 04, 2019 at 05:32:16PM +0100, Howard Chu wrote: >> >> Had a customer who was hitting this issue try out these patches -- It >> greatly decreases the search time (from unknown/infinite to 1 minute). >> Unfortunately slapd then segv's. Working on getting a test database to >> reproduce the issue with for a good backtrace. > > oh, that's too bad. I'd be happy to participate in the bug hunt. May I > ask: which version did your customer use? I have modified the patch to > work with the recent changes on master, but I did not dig in very deep > to understand these changes. So I must admit that these modifications > to the patch are completely untested (except the tests in the tests > subdir). > > Things might work better with the 2.4.46/47 patch, if that is an > option for your customer. > > Wrt getting a test-tree, maybe the python script I have in the github > repo for the patch can provide > one. (https://github.com/hbo/openldap-mdb-deref-problem/blob/master/scripts/write…) > > I'm of course very interested to keep this patch alive, so if you have > details on the segv, please let me know. Fyi, on our problematic test database with 11M entries and 3.7M aliases, a search with -a always , starting from the DB suffix, took 4 minutes without this patch, and 1235 minutes with this patch. Needless to say, that's not looking good. Still checking other test cases. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

答复: (ITS#9053) Ldap synchronization problem
by feng.wang＠iquantex.com 14 Jul '19

14 Jul '19

--_B51161C3-08D0-4E0E-AFBF-4338AF61897D_ Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Thank you very much for your advice, I will go to implement it. =E5=8F=91=E9=80=81=E8=87=AA Windows 10 =E7=89=88=E9=82=AE=E4=BB=B6=E5=BA=94= =E7=94=A8 =E5=8F=91=E4=BB=B6=E4=BA=BA: Quanah Gibson-Mount =E5=8F=91=E9=80=81=E6=97=B6=E9=97=B4: 2019=E5=B9=B47=E6=9C=8815=E6=97=A5 11= :53 =E6=94=B6=E4=BB=B6=E4=BA=BA: feng.wang(a)iquantex.com; openldap-its(a)OpenLDAP.= org =E4=B8=BB=E9=A2=98: Re: (ITS#9053) Ldap synchronization problem --On Monday, July 15, 2019 3:42 AM +0000 feng.wang(a)iquantex.com wrote: > Sorry, take up your precious time. I encountered a problem. In the > openldap HA environment, the update user group found that it was not > synchronized to the client in time (8 hosts, 5 stations, 3 syncs, not > synchronized, waiting for half an hour to complete synchronization). Has > anyone encountered such a problem? And would like to know how to > troubleshoot such problems, I use openldap soon. thank you very much! Hello, The ITS system is for bug reports, not usage issues. Please direct your=20 questions to the openldap-technical @openldap.org mailing list. In=20 addition the version in use, please note whether you are using standard=20 syncrepl or delta-syncrepl, and any relevant configuration information. I= =20 would note that numerous bugs with MMR have been fixed since 2.4.44 and=20 that it is strongly advised to use delta-syncrepl as the replication=20 mechanism if stability is desired. Regards, Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com> --_B51161C3-08D0-4E0E-AFBF-4338AF61897D_ Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset="utf-8" <html xmlns:o=3D"urn:schemas-microsoft-com:office:office" xmlns:w=3D"urn:sc= hemas-microsoft-com:office:word" xmlns:m=3D"http://schemas.microsoft.com/of= fice/2004/12/omml" xmlns=3D"http://www.w3.org/TR/REC-html40"><head><meta ht= tp-equiv=3DContent-Type content=3D"text/html; charset=3Dutf-8"><meta name= =3DGenerator content=3D"Microsoft Word 15 (filtered medium)"><style></style></head><body lang=3DZH-CN link=3Dblue vlink=3D"#954F72"><div cla= ss=3DWordSection1> = Thank you very much for your advice, I will go to implement it.<span= lang=3DEN-US style=3D'font-size:12.0pt;font-family:"Arial",sans-serif;colo= r:#777777'><o:p></o:p><o:p> </o:p><= span lang=3DEN-US><o:p> </o:p>=E5=8F= =91=E9=80=81=E8=87=AA Windows 10 =E7=89=88<a href=3D"https://go.microsoft.com/fwlink/?LinkId=3D550986"><s= pan lang=3DEN-US>=E9=82=AE=E4=BB=B6</a></s= pan>=E5=BA=94=E7=94=A8<o:p> </o:p><div style= =3D'mso-element:para-border-div;border:none;border-top:solid #E1E1E1 1.0pt;= padding:3.0pt 0cm 0cm 0cm'>=E5=8F=91=E4=BB=B6=E4=BA=BA: <a href=3D"mailto:quanah@symas.com">Quanah Gibson-Mount</a><br= >=E5=8F=91=E9=80=81=E6=97=B6=E9=97=B4: = 2019=E5=B9=B47=E6= =9C=8815=E6=97=A5 11:53 </s= pan>=E6=94=B6=E4=BB=B6=E4=BA=BA: <a href=3D"mailto:feng.wang@iquantex.com">feng.wang(a)iquantex.com<= /a>; <a href=3D"mailto:openldap-its@OpenLDAP.org">openldap-its(a)OpenLDAP.org= </a> =E4=B8=BB=E9=A2=98: Re: (ITS#9053) Ldap synchronization problem</div><o:p> </o:p>--On Monday, July 15, 2019 3:42 AM +0000 feng.wang(a)iquantex.com wrote:</sp= an><o:p> </o:p>= > Sorry, take up your precious t= ime. I encountered a problem. In the> openldap HA environment, the update user group found that = it was not> synchroni= zed to the client in time (8 hosts, 5 stations, 3 syncs, not> synchronized, waiting for half an = hour to complete synchronization). Has<span= lang=3DEN-US>> anyone encountered such a problem? And would like to kno= w how to> troubleshoo= t such problems, I use openldap soon. thank you very much!<o:p> </o:p>Hello,<o:p> </o:p>The ITS system is for bug reports, not usage issues.=C2=A0 Please direct = your questions to the op= enldap-technical @openldap.org mailing list.=C2=A0 In addition the version in use, please note wh= ether you are using standard syncrepl or delta-syncrepl, and any relevant configuration information= .=C2=A0 I would note tha= t numerous bugs with MMR have been fixed since 2.4.44 and that it is strongly advised to use delta-= syncrepl as the replication mechanism if stability is desired.<span= lang=3DEN-US><o:p> </o:p>Regards,Quanah<= /span><o:p> </o:p><= /p><o:p> </o:p><o:p> </o:p>--<o:p> </o:p>Quanah Gibson-MountPro= duct ArchitectSymas Corp= orationPackaged, certifi= ed, and supported LDAP solutions powered by OpenLDAP:<http://www.symas.com><o:p> </o:p><o:p> </o:p></div></body></html>= --_B51161C3-08D0-4E0E-AFBF-4338AF61897D_--

1 0

Re: (ITS#9053) Ldap synchronization problem
by quanah＠symas.com 14 Jul '19

14 Jul '19

--On Monday, July 15, 2019 3:42 AM +0000 feng.wang(a)iquantex.com wrote: > Sorry, take up your precious time. I encountered a problem. In the > openldap HA environment, the update user group found that it was not > synchronized to the client in time (8 hosts, 5 stations, 3 syncs, not > synchronized, waiting for half an hour to complete synchronization). Has > anyone encountered such a problem? And would like to know how to > troubleshoot such problems, I use openldap soon. thank you very much! Hello, The ITS system is for bug reports, not usage issues. Please direct your questions to the openldap-technical @openldap.org mailing list. In addition the version in use, please note whether you are using standard syncrepl or delta-syncrepl, and any relevant configuration information. I would note that numerous bugs with MMR have been fixed since 2.4.44 and that it is strongly advised to use delta-syncrepl as the replication mechanism if stability is desired. Regards, Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

(ITS#9053) Ldap synchronization problem
by feng.wang＠iquantex.com 14 Jul '19

14 Jul '19

Full_Name: wangfeng Version: 2.4.44 OS: linux URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (2408:84e1:103:3d23:7c55:cd40:7721:3c1e) Sorry, take up your precious time. I encountered a problem. In the openldap HA environment, the update user group found that it was not synchronized to the client in time (8 hosts, 5 stations, 3 syncs, not synchronized, waiting for half an hour to complete synchronization). Has anyone encountered such a problem? And would like to know how to troubleshoot such problems, I use openldap soon. thank you very much!

1 0

Re: Regression after ITS#8427 fix with back-ldap
by hyc＠symas.com 10 Jul '19

10 Jul '19

Ond=C5=99ej Kuzn=C3=ADk wrote: > On Wed, Jul 10, 2019 at 04:48:55PM +0000, hyc(a)symas.com wrote: >> ondra(a)mistotebe.net wrote: >>> On Thu, Jun 27, 2019 at 08:08:19PM +0000, a.chelouah(a)gmail.com wrote: >>>> Hello, >>>> >>>> Commit 6f623dfa1ca65698c19ccc6c058cd170e633384e fixing ITS#8427 (Set= up=20 >>>> TLS settings on each reconnection) introduce a regression when the p= roxy=20 >>>> connect to the**Backend ldap server via ldaps:// >>>> >>>> The relevent part of my config is: >>>> >>>> dn: olcDatabase=3D{2}ldap,cn=3Dconfig >>>> objectClass: olcDatabaseConfig >>>> objectClass: olcLDAPConfig >>>> olcDatabase: {2}ldap >>>> olcSuffix: dc=3Dlocal >>>> olcDbURI: ldaps://ldap.local >>>> olcDbChaseReferrals: TRUE >>>> olcDbRebindAsUser: TRUE >>>> olcDbIDAssertBind: bindmethod=3Dnone tls_cacert=3D/etc/pki/tls/certs= /ca.crt >>>> olcDbIDAssertAuthzFrom: "*" >>>> >>>> (I also tried by setting LDAPTLS_CACERT env var when starting slapd) >>>> >>>> On backend ldap server logs, I get the message "TLS negociation fail= ure" >>> >>> I've set up a test script here >>> https://github.com/mistotebe/openldap/tree/its8427-regression >>> >>> This runs without issues but if you replace olcDbStartTLS with an >>> analogous olcDbIDAssertBind in the configs, it seems the CA certifica= te >>> is not set for the connection. >> >> Then this is a new bug. Clearly the idassert-bind option takes tls_cac= ert >> as a parameter, so if it is provided it is expected to be used. >=20 > Sure, on idassert connections only, though. When does back-ldap use one= ? > I want to edit the linked script to do exercise that so we have a decen= t > test for this now. idassert is used when you want back-ldap to propagate the incoming client= 's identity to the remote server. It affects every operation that a client i= ssues. --=20 -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: Regression after ITS#8427 fix with back-ldap
by ondra＠mistotebe.net 10 Jul '19

10 Jul '19

On Wed, Jul 10, 2019 at 04:48:55PM +0000, hyc(a)symas.com wrote: > ondra(a)mistotebe.net wrote: >> On Thu, Jun 27, 2019 at 08:08:19PM +0000, a.chelouah(a)gmail.com wrote: >>> Hello, >>> >>> Commit 6f623dfa1ca65698c19ccc6c058cd170e633384e fixing ITS#8427 (Set up >>> TLS settings on each reconnection) introduce a regression when the proxy >>> connect to the**Backend ldap server via ldaps:// >>> >>> The relevent part of my config is: >>> >>> dn: olcDatabase={2}ldap,cn=config >>> objectClass: olcDatabaseConfig >>> objectClass: olcLDAPConfig >>> olcDatabase: {2}ldap >>> olcSuffix: dc=local >>> olcDbURI: ldaps://ldap.local >>> olcDbChaseReferrals: TRUE >>> olcDbRebindAsUser: TRUE >>> olcDbIDAssertBind: bindmethod=none tls_cacert=/etc/pki/tls/certs/ca.crt >>> olcDbIDAssertAuthzFrom: "*" >>> >>> (I also tried by setting LDAPTLS_CACERT env var when starting slapd) >>> >>> On backend ldap server logs, I get the message "TLS negociation failure" >> >> I've set up a test script here >> https://github.com/mistotebe/openldap/tree/its8427-regression >> >> This runs without issues but if you replace olcDbStartTLS with an >> analogous olcDbIDAssertBind in the configs, it seems the CA certificate >> is not set for the connection. > > Then this is a new bug. Clearly the idassert-bind option takes tls_cacert > as a parameter, so if it is provided it is expected to be used. Sure, on idassert connections only, though. When does back-ldap use one? I want to edit the linked script to do exercise that so we have a decent test for this now. >> I guess we've introduced a behaviour change with ITS#8427, not sure what >> the documentation implies should happen in these cases, whether the new >> behaviour is inconsistent with it or you've been relying on incorrect >> behaviour that has since been corrected. -- Ondřej Kuzník Senior Software Engineer Symas Corporation http://www.symas.com Packaged, certified, and supported LDAP solutions powered by OpenLDAP

1 0

Re: Regression after ITS#8427 fix with back-ldap
by hyc＠symas.com 10 Jul '19

10 Jul '19

ondra(a)mistotebe.net wrote: > On Thu, Jun 27, 2019 at 08:08:19PM +0000, a.chelouah(a)gmail.com wrote: >> Hello, >> >> Commit 6f623dfa1ca65698c19ccc6c058cd170e633384e fixing ITS#8427 (Set up >> TLS settings on each reconnection) introduce a regression when the proxy >> connect to the**Backend ldap server via ldaps:// >> >> The relevent part of my config is: >> >> dn: olcDatabase={2}ldap,cn=config >> objectClass: olcDatabaseConfig >> objectClass: olcLDAPConfig >> olcDatabase: {2}ldap >> olcSuffix: dc=local >> olcDbURI: ldaps://ldap.local >> olcDbChaseReferrals: TRUE >> olcDbRebindAsUser: TRUE >> olcDbIDAssertBind: bindmethod=none tls_cacert=/etc/pki/tls/certs/ca.crt >> olcDbIDAssertAuthzFrom: "*" >> >> (I also tried by setting LDAPTLS_CACERT env var when starting slapd) >> >> On backend ldap server logs, I get the message "TLS negociation failure" > > I've set up a test script here > https://github.com/mistotebe/openldap/tree/its8427-regression > > This runs without issues but if you replace olcDbStartTLS with an > analogous olcDbIDAssertBind in the configs, it seems the CA certificate > is not set for the connection. Then this is a new bug. Clearly the idassert-bind option takes tls_cacert as a parameter, so if it is provided it is expected to be used. > > I guess we've introduced a behaviour change with ITS#8427, not sure what > the documentation implies should happen in these cases, whether the new > behaviour is inconsistent with it or you've been relying on incorrect > behaviour that has since been corrected. > > Regards, > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs July 2019