openldap-bugs May 2018

openldap-bugs@openldap.org

18 participants
45 discussions

Re: (ITS#8852) slapd memory use grows continuously with non-delta syncrepl and modifying groups
by ryan＠nardis.ca 12 May '18

12 May '18

bisect identifies c365ac359e9c9b483b934c2a1f0bc552645c32fa as the commit that introduced this behaviour. 003dfbda574f37bbf1a2240f530ff9fa35ab0801 on RE24 (2.4.20) commit c365ac359e9c9b483b934c2a1f0bc552645c32fa Author: Howard Chu <hyc(a)openldap.org> Date: Sun Nov 22 04:42:00 2009 +0000 ITS#6368 use dup'd entries in response queue

1 0

(ITS#8852) slapd memory use grows continuously with non-delta syncrepl and modifying groups
by ryan＠openldap.org 11 May '18

11 May '18

Full_Name: Ryan Tandy Version: 2.4.46 OS: Debian URL: ftp://ftp.openldap.org/incoming/20180511_rtandy_syncrepl-memory-consumer.tgz Submission from: (NULL) (70.66.128.207) Submitted by: ryan When running object-based syncrepl, and making changes to groups, the provider slapd uses more and more memory, apparently without bound. We've discussed this issue before but there was no ITS tracking it specifically. original Debian bug: https://bugs.debian.org/725091 and a possibly related openldap-technical post: https://www.openldap.org/lists/openldap-technical/201503/msg00206.html reproducer: ftp://ftp.openldap.org/incoming/20180511_rtandy_syncrepl-memory-consumer.tgz ./prepare ./runslapd (backgrounds a provider slapd and a consumer slapd) ./modify (makes a number of modifications on the provider) ./clean (kills both slapds and cleans databases) Run top in another terminal and watch the memory growth. On my system, the provider grows to over 3 GB resident and does not shrink even after replication completes. with delta-syncrepl enabled, the producer's RSS is only around 10 MB. Reproduced on Debian unstable with 2.4.46 and glibc malloc (glibc 2.27-3) and tcmalloc_minimal (2.7-1).

1 0

(ITS#8851) Severe warnings
by pbrilius＠gmail.com 10 May '18

10 May '18

Full_Name: Povilas Brilius Version: 2.4.45 OS: Gentoo URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (217.17.80.180) Excerpt from Gentoo net-nds/openldap build log: * QA Notice: Package triggers severe warnings which indicate that it * may exhibit random runtime failures. * /var/tmp/portage/net-nds/openldap-2.4.45/work/openldap-2.4.45/libraries/libldap/open.c:251:7: warning: implicit declaration of function ldap_is_ldapc_url; did you mean ldap_is_ldapi_url? [-Wimplicit-function-declaration] * /var/tmp/portage/net-nds/openldap-2.4.45/work/openldap-2.4.45/libraries/libldap_r/thr_posix.c:93:9: warning: implicit declaration of function pthread_setconcurrency; did you mean pthread_setcanceltype? [-Wimplicit-function-declaration] * /var/tmp/portage/net-nds/openldap-2.4.45/work/openldap-2.4.45/libraries/libldap_r/thr_posix.c:107:9: warning: implicit declaration of function pthread_getconcurrency; did you mean ldap_pvt_thread_get_concurrency? [-Wimplicit-function-declaration] * open.c:251:7: warning: implicit declaration of function ldap_is_ldapc_url; did you mean ldap_is_ldapi_url? [-Wimplicit-function-declaration] * /var/tmp/portage/net-nds/openldap-2.4.45/work/openldap-2.4.45/libraries/libldap/open.c:251:7: warning: implicit declaration of function ldap_is_ldapc_url; did you mean ldap_is_ldapi_url? [-Wimplicit-function-declaration] * /var/tmp/portage/net-nds/openldap-2.4.45/work/openldap-2.4.45/libraries/libldap_r/thr_posix.c:93:9: warning: implicit declaration of function pthread_setconcurrency; did you mean pthread_setcanceltype? [-Wimplicit-function-declaration] * /var/tmp/portage/net-nds/openldap-2.4.45/work/openldap-2.4.45/libraries/libldap_r/thr_posix.c:107:9: warning: implicit declaration of function pthread_getconcurrency; did you mean ldap_pvt_thread_get_concurrency? [-Wimplicit-function-declaration] * open.c:251:7: warning: implicit declaration of function ldap_is_ldapc_url; did you mean ldap_is_ldapi_url? [-Wimplicit-function-declaration] * Please do not file a Gentoo bug and instead report the above QA * issues directly to the upstream developers of this software. * Homepage: http://www.OpenLDAP.org/

1 0

(ITS#8850) Split thread pool shutdown and destruction
by ondra＠openldap.org 09 May '18

09 May '18

Full_Name: Ondrej Kuznik Version: master OS: URL: https://github.com/mistotebe/openldap/tree/ITS8850 Submission from: (NULL) (82.10.24.68) At the moment, libldap_r only supports one thread pool. Closing and freeing that pool is currently done in a single step. If there is a side thread that wants to schedule work into this global pool, there is currently a (short) window when the thread pool is being destroyed and that thread has not been notified yet (which only happens after the main slapd threads have already finished). If a thread tries to schedule more work to be done, it might trample onto memory/mutexes that have already been released. The linked patch outlines a solution to the problem by separating ldap_pvt_thread_pool_destroy into a part that shuts down the pool and the actual deconstruction. slapd is adapted to call the shutdown at the shutdown point and only destroy the pool after all backends have been closed. Tries to maintain backward compatibility. Any feedback is welcome.

1 0

(ITS#8849) Notify backends of server (un)pause
by ondra＠openldap.org 09 May '18

09 May '18

Full_Name: Ondrej Kuznik Version: master OS: URL: https://github.com/mistotebe/openldap/tree/ITS8849 Submission from: (NULL) (82.10.24.68) If a backend needs to handle its own long-running threads on the side, it might need to be notified to pause while a config modification is underway and when it is finished. This is the case of the load balancer module, which registers itself as a module and maintains its own threads and needs to know when a server (or even its own) reconfiguration is in progress. The linked patches provide one of the possible implementations, they introduce the hooks to pause and unpause the server, use them in bconfig.c and add two new backend hooks that are consulted while this is happening.

1 0

Re: (ITS#8845) Cannot preserve existing controls with new extended operations
by ondra＠mistotebe.net 09 May '18

09 May '18

On Tue, May 08, 2018 at 04:09:12PM +0000, hyc(a)symas.com wrote: > ondra(a)mistotebe.net wrote: >> The issue is actually allowing existing controls to be used with newly >> registered exops, since the only way to declare the compatibility would >> be by re-registering the control. Problem there is that built-in >> controls have everything static to controls.c, so it's not even possible >> to do that. >> >> A solution would be to add another piece of API that receives a pair of >> (control OID, exop OID), finds the control and adds the OID to the >> sc_extendedops(bv) lists. >> >> Going to do just that (register_control_exop() a decent name?). This implementation is in the ITS8845 branch of my GitHub repo: https://github.com/mistotebe/openldap/tree/ITS8845 -- Ondřej Kuzník Senior Software Engineer Symas Corporation http://www.symas.com Packaged, certified, and supported LDAP solutions powered by OpenLDAP

1 0

RE: (ITS#8846) Patch to introduce new LDAP option to ignore hostname checking while verifying certificates in TLS mode
by sudhir.singam＠nokia.com 08 May '18

08 May '18

1 0

RE: (ITS#8846) Patch to introduce new LDAP option to ignore hostname checking while verifying certificates in TLS mode
by sudhir.singam＠nokia.com 08 May '18

08 May '18

SGksDQoNClRoYW5rcyBmb3IgeW91ciByZXNwb25zZS4NCg0KQmFzaWNhbGx5IG91ciB1c2UgY2Fz ZSBjYW4gYmUgYmVsb3cgb3IgYW55IHNpbWlsYXIgb25lIHdoZXJlIHdlIHdhbnQgdG8gZGlzYWJs ZSBob3N0bmFtZSBjaGVja2luZy4NCg0KYSkJV2Ugd2FudCB1c2UgdGhlIHNhbWUgY2VydHMgZnJv bSBtdWx0aSBkb21haW5zDQpiKQlXZSB3YW50IHVzZSB0aGUgc2FtZSBjZXJ0cyBmb3IgYWxsIHN1 YiBkb21haW5zIHVuZGVyIGEgbWFpbiBkb21haW4uDQpjKQlETlMgc2VydmVyIGlzIG5vdCBzZXQg dXAgICBJLmUuLCB0aGUgY2VydGlmaWNhdGUgY291bGQgYmUgaXNzdWVkIHdpdGggYSBuYW1lIGxp a2Ug4oCcbmV0YWN0Lm9wZXJhdG9y4oCdLCBidXQgd2XigJlkIGJlIHVzaW5nIDEwLjIuMy43LCBh bmQgRE5TIGhhcyBub3QgYmVlbiBzZXR1cCBpbiB0aGUgb3BlcmF0b3IgaW50ZXJuYWwgbmV0d29y aw0KDQpCdXQgd2hhdCB3ZSBmZWVsIGlzIHRoYXQgdGhlcmUgc2hvdWxkIGJlIGFuIG9wdGlvbiB0 byBiZSBjaG9zZW4gYnkgdXNlciB0byBlaXRoZXIgaWdub3JlIG9yIGVuYWJsZSBob3N0bmFtZSBj aGVja2luZy4gQWxyZWFkeSB3ZSBrbm93IHRoYXQgSFRUUCBjbGllbnRzLCBmb3IgZXhhbXBsZSwg YnJvd3NlcnMgcHJvdmlkZSBzdWNoIG9wdGlvbiB0byB1c2VyIGFuZCBpdCdzIHVwIHRvIHRoZSB1 c2VyIHRoYXQgd2hldGhlciB0byBjb250aW51ZSBjb21tdW5pY2F0aW9uIHRvIHRoZSBzZXJ2ZXIg b3Igbm90LCBpZiBob3N0bmFtZSBtaXNtYXRjaCBvY2N1cnMuDQoNClJGQyBodHRwOi8vd3d3Lmll dGYub3JnL3JmYy9yZmMyODE4LnR4dCAoZm9yIEhUVFApIHNheXMgYmVsb3csIHdoaWNoIG1lYW5z IGZvciBpbnRlcmFjdGl2ZSBvciBBVVRPTUFURUQgY2xpZW50cyBvcHRpb24gc2hvdWxkIGJlIHBy b3ZpZGVkLg0KDQoiICAgSWYgdGhlIGhvc3RuYW1lIGRvZXMgbm90IG1hdGNoIHRoZSBpZGVudGl0 eSBpbiB0aGUgY2VydGlmaWNhdGUsIHVzZXINCiAgIG9yaWVudGVkIGNsaWVudHMgTVVTVCBlaXRo ZXIgbm90aWZ5IHRoZSB1c2VyIChjbGllbnRzIE1BWSBnaXZlIHRoZQ0KICAgdXNlciB0aGUgb3Bw b3J0dW5pdHkgdG8gY29udGludWUgd2l0aCB0aGUgY29ubmVjdGlvbiBpbiBhbnkgY2FzZSkgb3IN CiAgIHRlcm1pbmF0ZSB0aGUgY29ubmVjdGlvbiB3aXRoIGEgYmFkIGNlcnRpZmljYXRlIGVycm9y LiBBdXRvbWF0ZWQNCiAgIGNsaWVudHMgTVVTVCBsb2cgdGhlIGVycm9yIHRvIGFuIGFwcHJvcHJp YXRlIGF1ZGl0IGxvZyAoaWYgYXZhaWxhYmxlKQ0KICAgYW5kIFNIT1VMRCB0ZXJtaW5hdGUgdGhl IGNvbm5lY3Rpb24gKHdpdGggYSBiYWQgY2VydGlmaWNhdGUgZXJyb3IpLg0KICAgQXV0b21hdGVk IGNsaWVudHMgTUFZIHByb3ZpZGUgYSBjb25maWd1cmF0aW9uIHNldHRpbmcgdGhhdCBkaXNhYmxl cw0KICAgdGhpcyBjaGVjaywgYnV0IE1VU1QgcHJvdmlkZSBhIHNldHRpbmcgd2hpY2ggZW5hYmxl cyBpdC4NCiINCg0KUkZDIGh0dHBzOi8vdG9vbHMuaWV0Zi5vcmcvaHRtbC9yZmM0NTEzIChMREFQ KSBzYXlzIGJlbG93LCB0aGF0IEFVVE9NQVRFRCBjbGllbnQgc2hvdWxkIGxvZyBlcnJvciBidXQg bm90IHByb3ZpZGUgYW55IGNvbmZpZ3VyYWJsZSBvcHRpb24uDQoNCuKAnA0KDQpJZiB0aGUgc2Vy dmVyIGlkZW50aXR5IGNoZWNrIGZhaWxzLCB1c2VyLW9yaWVudGVkIGNsaWVudHMgU0hPVUxEDQog ICBlaXRoZXIgbm90aWZ5IHRoZSB1c2VyIChjbGllbnRzIG1heSBnaXZlIHRoZSB1c2VyIHRoZSBv cHBvcnR1bml0eSB0bw0KICAgY29udGludWUgd2l0aCB0aGUgTERBUCBzZXNzaW9uIGluIHRoaXMg Y2FzZSkgb3IgY2xvc2UgdGhlIHRyYW5zcG9ydA0KICAgY29ubmVjdGlvbiBhbmQgaW5kaWNhdGUg dGhhdCB0aGUgc2VydmVyJ3MgaWRlbnRpdHkgaXMgc3VzcGVjdC4NCiAgIEF1dG9tYXRlZCBjbGll bnRzIFNIT1VMRCBjbG9zZSB0aGUgdHJhbnNwb3J0IGNvbm5lY3Rpb24gYW5kIHRoZW4NCiAgIHJl dHVybiBvciBsb2cgYW4gZXJyb3IgaW5kaWNhdGluZyB0aGF0IHRoZSBzZXJ2ZXIncyBpZGVudGl0 eSBpcw0KICAgc3VzcGVjdCBvciBib3RoLg0K4oCcDQoNCk5vdCBzdXJlIGhvdyB0aGUgQVVUT01B VEVEIGNsaWVudHMgZGlmZmVyIGZvciBIVFRQIG9yIExEQVAuDQoNClNvIEkgd291bGQgbGlrZSB0 byBrbm93IHdoeSB0aGlzIGlzIG5vdCBpbXBsZW1lbnRlZCBvciBjYW4ndCBiZSBpbXBsZW1lbnRl ZCBmb3Igb3BlbkxEQVAuDQoNClJlZ2FyZHMsDQpTdWRoaXIgU2luZ2FtDQoNCkRFTElWRVJJTkcg QkVTVC1JTi1DTEFTUyBQTEFURk9STSBpcyBvdXIgdmlzaW9uDQoNCi0tLS0tT3JpZ2luYWwgTWVz c2FnZS0tLS0tDQpGcm9tOiBIb3dhcmQgQ2h1IFttYWlsdG86aHljQHN5bWFzLmNvbV0gDQpTZW50 OiBTdW5kYXksIE1heSAwNiwgMjAxOCAxMTo0NSBQTQ0KVG86IFNpbmdhbSwgU3VkaGlyIChOb2tp YSAtIElOL0JhbmdhbG9yZSkgPHN1ZGhpci5zaW5nYW1Abm9raWEuY29tPjsgb3BlbmxkYXAtaXRz QE9wZW5MREFQLm9yZw0KU3ViamVjdDogUmU6IChJVFMjODg0NikgUGF0Y2ggdG8gaW50cm9kdWNl IG5ldyBMREFQIG9wdGlvbiB0byBpZ25vcmUgaG9zdG5hbWUgY2hlY2tpbmcgd2hpbGUgdmVyaWZ5 aW5nIGNlcnRpZmljYXRlcyBpbiBUTFMgbW9kZQ0KDQpzdWRoaXIuc2luZ2FtQG5va2lhLmNvbSB3 cm90ZToNCj4gRnVsbF9OYW1lOiBTaW5nYW0gU3VkaGlyIFJlZGR5DQo+IFZlcnNpb246IG1hc3Rl ciBicmFuY2gNCj4gT1M6IGZlZG9yYQ0KPiBVUkw6IGZ0cDovL2Z0cC5vcGVubGRhcC5vcmcvaW5j b21pbmcvc3VkaGlyc2luZ2FtLTE4MDUwNi5wYXRjaA0KPiBTdWJtaXNzaW9uIGZyb206IChOVUxM KSAoNjEuMS4yMzIuMTU0KQ0KPiANCj4gDQo+IFRoZSBhdHRhY2hlZCBmaWxlIGlzIGRlcml2ZWQg ZnJvbSBPcGVuTERBUCBTb2Z0d2FyZS4gQWxsIG9mIHRoZSBtb2RpZmljYXRpb25zIHRvDQo+IE9w ZW5MREFQIFNvZnR3YXJlIHJlcHJlc2VudGVkIGluIHRoZSBmb2xsb3dpbmcgcGF0Y2goZXMpIHdl cmUgZGV2ZWxvcGVkIGJ5DQo+IE5PS0lBLiBOT0tJQSBoYXMgbm90IGFzc2lnbmVkIHJpZ2h0cyBh bmQvb3IgaW50ZXJlc3QgaW4gdGhpcyB3b3JrIHRvIGFueSBwYXJ0eS4NCj4gSSwgU0lOR0FNIFNV REhJUiBSRUREWSBhdXRob3JpemVkIGJ5IE5PS0lBLCBteSBlbXBsb3llciwgdG8gcmVsZWFzZSB0 aGlzIHdvcmsNCj4gdW5kZXIgdGhlIGZvbGxvd2luZyB0ZXJtcy4NCj4gDQo+IE5PS0lBIGhlcmVi eSBwbGFjZSB0aGUgZm9sbG93aW5nIG1vZGlmaWNhdGlvbnMgdG8gT3BlbkxEQVAgU29mdHdhcmUg KGFuZCBvbmx5DQo+IHRoZXNlIG1vZGlmaWNhdGlvbnMpIGludG8gdGhlIHB1YmxpYyBkb21haW4u IEhlbmNlLCB0aGVzZSBtb2RpZmljYXRpb25zIG1heSBiZQ0KPiBmcmVlbHkgdXNlZCBhbmQvb3Ig cmVkaXN0cmlidXRlZCBmb3IgYW55IHB1cnBvc2Ugd2l0aCBvciB3aXRob3V0IGF0dHJpYnV0aW9u DQo+IGFuZC9vciBvdGhlciBub3RpY2UuDQo+IA0KPiAqKioqDQo+IERlc2NyaXB0aW9uOg0KPiAN Cj4gVGhpcyBpcyBtaW5vciBlbmhhbmNlbWVudCB0byBpbnRyb2R1Y2UgYSBuZXcgTERBUCBvcHRp b24NCj4gIkxEQVBfT1BUX1hfVExTX0RFTUFORF9FWENMX0hPU1ROQU1FX0NIRUNLIiB0byBpZ25v cmUgaG9zdG5hbWUgY2hlY2tpbmcgYnkNCj4gY2xpZW50IGluIFRMUyBjb21tdW5pY2F0aW9uIG1v ZGUuIFRoaXMgaXMgdmVyeSBzaW1pbGFyIHRvDQo+ICJMREFQX09QVF9YX1RMU19ERU1BTkQiIExE QVAgb3B0aW9uIGV4Y2VwdCB0aGF0IEhPU1ROQU1FIGNoZWNraW5nIGlzIGlnbm9yZWQuDQo+IA0K PiBUaGlzIG9wdGlvbiBjYW4gYmUgc2V0IGJ5IGNsaWVudCBlaXRoZXIgYnkgdXNpbmcgTERBUCBB UEkgImxkYXBfc2V0X29wdGlvbiIgb3INCj4gY2FuIGJlIGdsb2JhbGx5IHNldCBpbiB0aGUgY29u ZmlndXJhdGlvbiBmaWxlIC9ldGMvb3BlbmxkYXAvbGRhcC5jb25mIGxpa2UNCj4gYmVsb3cuDQo+ IA0KPiBUTFNfUkVRQ0VSVCBkZW1hbmRfZXhjbF9ob3N0bmFtZV9jaGVjaw0KPiANCj4gUHVycG9z ZToNCj4gDQo+IEdlbmVyYWxseSBvcGVyYXRvcnMgdXNlIHNhbWUgc2V0IG9mIGNlcnRpZmljYXRl cyBmb3IgZGlmZmVyZW50IHNlcnZpY2VzIChmcm9tDQo+IGRpZmZlcmVudCBob3N0cykgd2hpY2gg c3VwcG9ydCBUTFMgY29tbXVuaWNhdGlvbi4gV2hlbiBzdWNoIGNlcnRpZmljYXRlcyBhcmUNCj4g dXNlZCwgdGhpcyBvcHRpb24gZ2l2ZXMgZmFjaWxpdHkgZm9yIG9wZW5sZGFwIGJhc2VkIHNlcnZp Y2VzIHRvIGlnbm9yZSBob3N0bmFtZQ0KPiBjaGVja2luZyBhdCBjbGllbnQgc2lkZS4NCg0KTm8u IElmIHlvdSdyZSB1c2luZyBhIHNpbmdsZSBzZXQgb2YgY2VydGlmaWNhdGVzIGZvciBtdWx0aXBs ZSBob3N0cyB5b3Ugc2hvdWxkIA0KYmUgdXNpbmcgYSB3aWxkY2FyZCBjZXJ0LiBDbG9zaW5nIHRo aXMgSVRTLg0KDQotLSANCiAgIC0tIEhvd2FyZCBDaHUNCiAgIENUTywgU3ltYXMgQ29ycC4gICAg ICAgICAgIGh0dHA6Ly93d3cuc3ltYXMuY29tDQogICBEaXJlY3RvciwgSGlnaGxhbmQgU3VuICAg ICBodHRwOi8vaGlnaGxhbmRzdW4uY29tL2h5Yy8NCiAgIENoaWVmIEFyY2hpdGVjdCwgT3BlbkxE QVAgIGh0dHA6Ly93d3cub3BlbmxkYXAub3JnL3Byb2plY3QvDQo=

1 0

Re: (ITS#8845) Cannot preserve existing controls with new extended operations
by hyc＠symas.com 08 May '18

08 May '18

ondra(a)mistotebe.net wrote: > On Fri, May 04, 2018 at 09:21:59PM +0000, quanah(a)openldap.org wrote: >> As noted in the OpenLDAP source >> (http://www.openldap.org/devel/gitweb.cgi?p=openldap.git;a=blob;f=servers/sl…) >> there is an issue with controls being reset with a new extended operation. This >> is problematic for a proxy auth mechanism being developed, as it needs to have >> the controls preserved while passing an extended op to slapd. >> >> A possibility would be to re-register the control, adding the corresponding >> extended operations, but the parsing function resets them so this unfortunately >> is not an option. > > The issue is actually allowing existing controls to be used with newly > registered exops, since the only way to declare the compatibility would > be by re-registering the control. Problem there is that built-in > controls have everything static to controls.c, so it's not even possible > to do that. > > A solution would be to add another piece of API that receives a pair of > (control OID, exop OID), finds the control and adds the OID to the > sc_extendedops(bv) lists. > > Going to do just that (register_control_exop() a decent name?). Makes sense. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8845) Cannot preserve existing controls with new extended operations
by ondra＠mistotebe.net 08 May '18

08 May '18

On Fri, May 04, 2018 at 09:21:59PM +0000, quanah(a)openldap.org wrote: > As noted in the OpenLDAP source > (http://www.openldap.org/devel/gitweb.cgi?p=openldap.git;a=blob;f=servers/sl…) > there is an issue with controls being reset with a new extended operation. This > is problematic for a proxy auth mechanism being developed, as it needs to have > the controls preserved while passing an extended op to slapd. > > A possibility would be to re-register the control, adding the corresponding > extended operations, but the parsing function resets them so this unfortunately > is not an option. The issue is actually allowing existing controls to be used with newly registered exops, since the only way to declare the compatibility would be by re-registering the control. Problem there is that built-in controls have everything static to controls.c, so it's not even possible to do that. A solution would be to add another piece of API that receives a pair of (control OID, exop OID), finds the control and adds the OID to the sc_extendedops(bv) lists. Going to do just that (register_control_exop() a decent name?). Cheers, -- Ondřej Kuzník Senior Software Engineer Symas Corporation http://www.symas.com Packaged, certified, and supported LDAP solutions powered by OpenLDAP

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs May 2018