openldap-bugs October 2017

openldap-bugs@openldap.org

29 participants
41 discussions

(ITS#8755) invalid file descriptor when closing tls connection
by info＠christianknueppel.de 12 Oct '17

12 Oct '17

Full_Name: Christian Knueppel Version: 2.4.45 OS: Ubuntu 16.04.3 LTS URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (141.12.132.220) I currently developing on a c software which is using Openldap with TLS authentication. My software is working fine, but when i test it with valgrind, i always get an invalid file descriptor when closing the connection. Here is the stacktrace from valgrind: ==17517== Warning: invalid file descriptor -1 in syscall write() ==17517== at 0x4E4A4BD: ??? (syscall-template.S:84) ==17517== by 0x61BF5E7: sb_debug_write (in /usr/lib/x86_64-linux-gnu/liblber-2.4.so.2.10.5) ==17517== by 0x7261644: _gnutls_writev_emu (gnutls_buffers.c:447) ==17517== by 0x7261644: _gnutls_writev (gnutls_buffers.c:505) ==17517== by 0x7261644: _gnutls_io_write_flush (gnutls_buffers.c:699) ==17517== by 0x725BDFF: _gnutls_send_tlen_int (gnutls_record.c:464) ==17517== by 0x727E0D7: _gnutls_send_int (gnutls_record.h:43) ==17517== by 0x727E0D7: gnutls_alert_send (gnutls_alert.c:165) ==17517== by 0x725DCD8: gnutls_bye (gnutls_record.c:289) ==17517== by 0x5F9F181: tlsg_sb_close (in /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2.10.5) ==17517== by 0x61C07B1: ber_int_sb_close (in /usr/lib/x86_64-linux-gnu/liblber-2.4.so.2.10.5) ==17517== by 0x61C08A3: ber_sockbuf_free (in /usr/lib/x86_64-linux-gnu/liblber-2.4.so.2.10.5) ==17517== by 0x5F815CD: ldap_ld_free (in /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2.10.5) ==17517== by 0x41618F: ldap_close_handle (ldap_connection.c:481) --> In function ldap_close_handle i call ldap_unbind_ext_s(ld, NULL, NULL). The connection is built with ldap_initialize(&ld, config.ldap_url) and ldap_start_tls_s(ld, NULL, NULL). Options set with ldap_set_option() are LDAP_OPT_X_TLS_REQUIRE_CERT to 2 (LDAP_OPT_X_TLS_DEMAND) and LDAP_OPT_X_TLS_CACERTFILE are set to all SSL CA-Certificates (/etc/ssl/certs/ca-certificates.crt). I run the ldap_unbind_ext_s command (for test purpose) shortly after the start_tls command is finished. When i use ldap_sasl_interactive_bind_s with DIGEST-MD5 instead of ldap_start_tls_s, the warning doesn't appear. When i use both, tls and sasl, the warning also appears. My computer running on Ubuntu 16.04.3 LTS (uname: 4.4.0-97-generic x86_64) with libldap-2.4-2 (2.4.42+dfsg-2ubuntu3.2) and libgnutls30 (3.4.10-4ubuntu1.4). I also tested it with the newest Ubuntu Artful Aardvark and the newest openldap (2.4.45+dfsg-1ubuntu1) and gnutls(3.5.8-6ubuntu3) release, but it didn't has any effect in my case. I also tryed to compiled openldap against openssl to see, if it might be a gnutls bug, but the invalid file descriptor occurs again. The lower valgrind stacktrace is done with openldap 2.4.45 and openssl 1.0.2g on the newest Artful Aardvark 17.10. ==2638== Warning: invalid file descriptor -1 in syscall write() ==2638== at 0x4E4DCC0: write (write.c:26) ==2638== by 0x61C6E87: sb_debug_write (sockbuf.c:854) ==2638== by 0x5FAAB4A: tlso_bio_write.part.8 (tls_o.c:790) ==2638== by 0x516B94A: BIO_write (bio_lib.c:243) ==2638== by 0x5B20C61: ssl3_write_pending (s3_pkt.c:1105) ==2638== by 0x5B22E82: ssl3_dispatch_alert (s3_pkt.c:1733) ==2638== by 0x5B1EAE6: ssl3_shutdown (s3_lib.c:4372) ==2638== by 0x5FAA749: tlso_sb_close (tls_o.c:899) ==2638== by 0x61C7D8A: ber_int_sb_close (sockbuf.c:383) ==2638== by 0x61C7E73: ber_sockbuf_free (sockbuf.c:74) ==2638== by 0x5F8D006: ldap_ld_free (unbind.c:134) ==2638== by 0x1277B7: ldap_close_handle (ldap_connection.c:481)

1 0

(ITS#8754) ldpasearch IPv5 only
by dpa-openldap＠aegee.org 11 Oct '17

11 Oct '17

Full_Name: &#1044;&#1080;&#1083;&#1103;&#1085; &#1055;&#1072;&#1083;&#1072;&#1091;&#1079;&#1086;&#1074; Version: 2.4.45 OS: Linux URL: Submission from: (NULL) (87.193.154.82) After `./configure --disable-ipv6` I expect, that `ldapsearch` will not contact severs over IPv6, which is achieved with the fix below. diff --git a/libraries/libldap/os-ip.c b/libraries/libldap/os-ip.c --- a/libraries/libldap/os-ip.c +++ b/libraries/libldap/os-ip.c @@ -623,6 +623,9 @@ ldap_connect_to_host(LDAP *ld, Sockbuf *sb, continue; } +#ifndef LDAP_PF_INET6 + if (sai->ai_family == AF_INET6) continue; +#endif /* we assume AF_x and PF_x are equal for all x */ s = ldap_int_socket( ld, sai->ai_family, socktype ); if ( s == AC_SOCKET_INVALID ) {

1 0

(ITS#8753) Public key pinning support in libldap
by ondra＠openldap.org 10 Oct '17

10 Oct '17

Full_Name: Ondrej Kuznik Version: master OS: URL: https://github.com/mistotebe/openldap/tree/its8753 Submission from: (NULL) (82.10.24.68) Some programs might want to pin the server's public key instead of/in addition to certificate validation. The patch linked implements this option and provides OpenSSL/GnuTLS support code. A new libldap option LDAP_OPT_X_TLS_PEERKEY_HASH that accepts a string 'hashname/base64_hash_of_public_key'. If a TLS session is already present on the main connection, it is also checked immediately. It introduces a dependency on liblutil by depending on the symbol lutil_b64_pton. Somehow, this breaks the build for the ldap* tools, not sure why or how to fix that yet.

1 0

Re: (ITS#8609) segfault in mods.c - modify_add_values
by henson＠acm.org 06 Oct '17

06 Oct '17

On Fri, Sep 15, 2017 at 07:54:22PM -0700, Paul B. Henson wrote: > I've been running 2.4.44 with the test patch since May and haven't seen > a crash since, for what that's worth. We upgraded to 2.4.45 last week, > without that patch. I'm going to see if the crash shows up again, and if > it does, apply the patch to 2.4.45 and see if it goes away again. Well, since upgrading, I've now seen two crashes; I don't have it compiled with debugging right now so the trace is limited: #0 0x0000000000483eb9 in modify_add_values () #1 0x00007f7b80d45bc3 in mdb_modify_internal () from /usr/lib64/openldap/openldap/back_mdb.so #2 0x00007f7b80d46f55 in mdb_modify () from /usr/lib64/openldap/openldap/back_mdb.so #3 0x000000000049a6d2 in overlay_op_walk () #4 0x000000000049a82a in ?? () #5 0x000000000048bbef in ?? () #6 0x000000000048f8cf in ?? () #7 0x0000000000493ae3 in ?? () #8 0x00007f7b858327f2 in ?? () from /usr/lib64/libldap_r-2.4.so.2 #9 0x00007f7b84bb03e4 in start_thread () from /lib64/libpthread.so.0 #10 0x00007f7b848f23ed in clone () from /lib64/libc.so.6 but it looks like the same issue to me. I'm going to go ahead and recompile 2.4.45 with the ignore-invalid-accesslog-ops.patch and run that for a while and see what happens. Thanks...

1 0

(ITS#8752) MMR delta-sync deadlock using slapd.conf
by quanah＠openldap.org 04 Oct '17

04 Oct '17

Full_Name: Quanah Gibson-Mount Version: 2.4.45 OS: Linux URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (47.208.148.239) There is a reproducible lockup with delta-sync based multimaster replication. The more masters exist the easier it is to trigger, with 100% reproducibility at 4 masters. Generally: Load a database onto a server with no syncprov/accesslog configuration. Stop slapd Reconfigure slapd.conf to include syncprov/accesslog overlays, and cn=accesslog DB for changes, and replication agreements with other servers Configure other servers for syncprov/accesslog, but no database start slapd on all servers execute a MOD operation sequentially across the servers (1, 2, 3, 4) slapd will deadlock on server 3 or 4 100% of the time

1 0

(ITS#8751) deref aliases with back meta goes wrong in db_cache_find_ndn()
by paul＠subsignal.org 04 Oct '17

04 Oct '17

Full_Name: Paul K.lle Version: 24.44 OS: omnios URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (141.43.201.48) Disclaimer: I don't know C very well and I'm a complete noob wrt OpenLDAP source meta (dc=btu,dc=local) with two olcMetaSub URI: 0 -> olcDbURI ldap://localhost/ou=Fak6,dc=btu,dc=local olcDbRewrite {0}suffixmassage "ou=Fak6,dc=btu,dc=local" "ou=Fak6,dc=IT,dc=local" 1 -> olcDbURI ldap://ldap.xxxxx.de/ou=People,dc=btu,dc=local olcDbRewrite » {0}suffixmassage "ou=People,dc=btu,dc=local" "ou=People,o=XXXXXXXXXXXXX XXXXX,c=De" no deref search: [root@smb01 ~]# ldapsearch -x -h 192.168.70.101 -b ou=users,ou=Fak6,dc=btu,dc=local '(cn=Paul*)' '*' # extended LDIF # # LDAPv3 # base <ou=users,ou=Fak6,dc=btu,dc=local> with scope subtree # filter: (cn=Paul*) # requesting: * # # Paul Koelle, Users, Fak6, btu.local dn: cn=Paul Koelle,ou=Users,ou=Fak6,dc=btu,dc=local aliasedObjectName: cn=Paul Koelle,ou=People,dc=btu,dc=local cn: Paul Koelle objectClass: alias objectClass: extensibleObject # search result search: 2 result: 0 Success # numResponses: 2 # numEntries: 1 with deref: [root@smb01 ~]# ldapsearch -a always -x -h 192.168.70.101 -b ou=users,ou=Fak6,dc=btu,dc=local '(cn=Paul*)' '*' # extended LDIF # # LDAPv3 # base <ou=users,ou=Fak6,dc=btu,dc=local> with scope subtree # filter: (cn=Paul*) # requesting: * # # search result search: 2 result: 0 Success # numResponses: 1 debug log reveals: 59d38a50 bdb_idl_fetch_key: [01872a84] 59d38a50 <= bdb_index_read 1 candidates 59d38a50 <= bdb_equality_candidates: id=1, first=19, last=19 59d38a50 <= bdb_filter_candidates: id=1 first=19 last=19 59d38a50 => hdb_dn2idl("ou=users,ou=fak6,dc=it,dc=local") 59d38a50 bdb_dn2entry("cn=paul koelle,ou=people,dc=btu,dc=local") 59d38a50 => hdb_dn2id("c=btu,dc=local") 59d38a50 <= hdb_dn2id: get failed: DB_NOTFOUND: No matching key/data pair found (-30988) 59d38a50 bdb_search_candidates: failed (rc=33) breakpoint for hdb_dn2id (gdb) bt #0 hdb_dn2id (op=0xa05510, in=0xfffffd7ffc87f208, ei=0xfffffd7ffc87f1f0, txn=0x8e6950, cursor=0xfffffd7ffc87f1e0) at dn2id.c:705 #1 0x000000000059beed in hdb_cache_find_ndn (op=0xa05510, txn=0x8e6950, ndn=0xfffffd7ffc87f350, res=0xfffffd7ffc87f2e0) at cache.c:462 #2 0x00000000005a3d81 in hdb_dn2entry (op=0xa05510, tid=0x8e6950, dn=0xfffffd7ffc87f350, e=0xfffffd7ffc87f348, matched=0, lock=0xfffffd7ffc87f330) at dn2entry.c:47 #3 0x000000000053756c in deref_base (op=0xa05510, rs=0xfffffd7ffc9ffb40, e=0x91c0b8, matched=0xfffffd7ffc87f460, txn=0x8e6950, lock=0xfffffd7ffc87f3c0, tmp=0x10e3010, visited=0xf63010) at search.c:104 #4 0x0000000000537be7 in search_aliases (op=0xa05510, rs=0xfffffd7ffc9ffb40, e=0xfffffd7ffc87f750, txn=0x8e6950, ids=0xfffffd7ffc8ff7a0, scopes=0xfffffd7ffc87f7a0, stack=0xde3010) at search.c:250 #5 0x000000000053ab3d in search_candidates (op=0xa05510, rs=0xfffffd7ffc9ffb40, e=0xfffffd7ffc87f750, txn=0x8e6950, ids=0xfffffd7ffc8ff7a0, scopes=0xfffffd7ffc87f7a0) at search.c:1258 #6 0x0000000000538e27 in hdb_search (op=0xa05510, rs=0xfffffd7ffc9ffb40) at search.c:616 #7 0x000000000048438d in fe_op_search (op=0xa05510, rs=0xfffffd7ffc9ffb40) at search.c:402 #8 0x0000000000483bcc in do_search (op=0xa05510, rs=0xfffffd7ffc9ffb40) at search.c:247 #9 0x0000000000480537 in connection_operation (ctx=0xfffffd7ffc9ffc70, arg_v=0xa05510) at connection.c:1158 #10 0x0000000000480acc in connection_read_thread (ctx=0xfffffd7ffc9ffc70, argv=0xf) at connection.c:1294 #11 0x0000000000622afd in ldap_int_thread_pool_wrapper (xpool=0x7d8840) at tpool.c:696 #12 0xfffffd7ffeb9e43a in _thrp_setup () from /lib/64/libc.so.1 #13 0xfffffd7ffeb9e750 in ?? () from /lib/64/libc.so.1 #14 0x0000000000000000 in ?? () if you look at the arguments in (h|b)db_cache_find_ndn there are mixed values from pre/post meta rewrite? #1 0x000000000059beed in hdb_cache_find_ndn (op=0xa05510, txn=0x8e6950, ndn=0xfffffd7ffc87f350, res=0xfffffd7ffc87f2e0) at cache.c:462 462 rc = bdb_dn2id( op, &ei.bei_nrdn, &ei, txn, &cursor ); (gdb) p *ndn $8 = {bv_len = 40, bv_val = 0x9fdd06 "cn=paul koelle,ou=people,dc=btu,dc=local"} (gdb) p *res $9 = (EntryInfo *) 0x0 (gdb) p ndn->bv_val $13 = 0x9fdd06 "cn=paul koelle,ou=people,dc=btu,dc=local" (gdb) p op->o_bd->be_nsuffix[0] $16 = {bv_len = 14, bv_val = 0x9f17d0 "dc=it,dc=local"} and the code in bdb_cache_find_ndn constructs a new BV from *ndn and *op which have different bases (cache.c line 426): /* we're searching a full DN from the root */ ptr = ndn->bv_val + ndn->bv_len - op->o_bd->be_nsuffix[0].bv_len; ei.bei_nrdn.bv_val = ptr; ei.bei_nrdn.bv_len = op->o_bd->be_nsuffix[0].bv_len; where op->o_bd_>be_nsuffix[0] is the post-rewrite suffix and ndn has the pre-rewrite suffix?? continuing lead to the following debug output: 59d38434 => hdb_dn2id("c=btu,dc=local") 59d38434 <= hdb_dn2id: get failed: DB_NOTFOUND: No matching key/data pair found (-30988) note the incomplete "c=btu,dc=local" probably because "dc=it,dc=local" is one char less than "dc=btu,dc=local"

1 0

Re: (ITS#8747) LDAP load balancer daemon (lloadd)
by ondra＠mistotebe.net 02 Oct '17

02 Oct '17

On Thu, Sep 28, 2017 at 04:13:46PM +0000, ondra(a)openldap.org wrote: > The 'lloadd' branch linked above contains the load balancer code that > is now ready for review. > > This adds a new server to the OpenLDAP project, a load balancing proxy > (prototype). To summarise the reasons for this project to exist and highlight its features, limitations: - most LDAP load balancers pin a connection to another server and then just ship data, lloadd can distribute operations from a single client connection across LDAP servers - to make the above possible, lloadd sets up connections to the backend servers on startup and manages them as per configuration (independent on the clients) - bind operations are forwarded over dedicated bind connections or using the VC exop if feature 'vc' is enabled in its config, the designated identity is then passed on with the operations with proxyauth control (if feature 'proxyauthz' is enabled) - it is expected that all backends are indistinguishable (same features, suffixes, data) - no SASL bind support yet - if an operation cannot be processed or forwarded for any reason (overload, connection loss, ...), it is never re-sent, client will however still be sent an appropriate result in that case The lloadd.8 and lloadd.conf.5 manpages are provided, Admin guide and further documentation will come as the implementation matures. -- Ondřej Kuzník Senior Software Engineer Symas Corporation http://www.symas.com Packaged, certified, and supported LDAP solutions powered by OpenLDAP

1 0

Re: AW: (ITS#8749) Proxy: LDAP-querry doesn't work for e.g(userAccountControl:1.2.840.113556.1.4.803:=2)
by michael＠stroeder.com 01 Oct '17

01 Oct '17

steffen.krahl(a)nexio.de wrote: > Regarding segmentation fault: that's true, but I have to investigate > further Please make sure to install with debug symbols and read how to use gdb to obtain a stack back trace: https://www.openldap.org/faq/data/cache/59.html Ciao, Michael.

1 0

AW: (ITS#8749) Proxy: LDAP-querry doesn't work for e.g (userAccountControl:1.2.840.113556.1.4.803:=2)
by Steffen.Krahl＠nexio.de 01 Oct '17

01 Oct '17

RGVhciBNaWNoYWVsLA0KDQptYW55IHRoYW5rcyBmb3IgeW91ciBleHBsYWluYXRpb24uIFJlZ2FyZGlu ZyBzZWdtZW50YXRpb24gZmF1bHQ6IHRoYXQncyB0cnVlLCBidXQgSSBoYXZlIHRvIGludmVzdGlnYXRl IGZ1cnRoZXINCg0KQlINClN0ZWZmZW4gDQoNCi0tLS0tVXJzcHLDvG5nbGljaGUgTmFjaHJpY2h0LS0t LS0NClZvbjogTWljaGFlbCBTdHLDtmRlciBbbWFpbHRvOm1pY2hhZWxAc3Ryb2VkZXIuY29tXSANCkdl c2VuZGV0OiBTb25udGFnLCAxLiBPa3RvYmVyIDIwMTcgMTA6MjUNCkFuOiBLcmFobCwgU3RlZmZlbjsg b3BlbmxkYXAtaXRzQE9wZW5MREFQLm9yZw0KQmV0cmVmZjogUmU6IChJVFMjODc0OSkgUHJveHk6IExE QVAtcXVlcnJ5IGRvZXNuJ3Qgd29yayBmb3IgZS5nICh1c2VyQWNjb3VudENvbnRyb2w6MS4yLjg0MC4x MTM1NTYuMS40LjgwMzo9MikNCg0Kc3RlZmZlbi5rcmFobEBuZXhpby5kZSB3cm90ZToNCj4gSSdtIHVz aW5nIE9wZW5MREFQIHdpdGggTERBUC1iYWNrZW5kIGFzIHByb3h5IGZvciBBY3RpdmVEaXJlY3Rvcnkg DQo+IEl0J3Mgd29ya2luZyB3ZWxsIHNvIGZhciwgb25seSBMREFQLXF1ZXJpZXMgd2hpY2ggc2hvdWxk IGV4Y2x1ZGUNCj4gZGVhY3RpdmF0ZWQgdXNlcnMgZG9uJ3Qgd29yay4gSXQgc2VlbXMgc2xhcGQgZG9l cyBub3QgYWNjZXB0IHF1ZXJpZXMNCj4gbGlrZSAoYXR0cmlidXRlOk9JRDo9dmFsdWUpDQoNCk9wZW5M REFQIGRvZXMgc3VwcG9ydCBleHRlbmRlZCBmaWx0ZXJzIHdpdGggYSBtYXRjaGluZyBydWxlLiBCdXQg b25seQ0Kd2l0aCBtYXRjaGluZyBydWxlcyBpbXBsZW1lbnRlZCBpbiBPcGVuTERBUC4NCg0KPiBpbiBw YXJ0aWN1bGFyDQo+ICghKHVzZXJBY2NvdW50Q29udHJvbDoxLjIuODQwLjExMzU1Ni4xLjQuODAzOj0y KSkpDQoNClRoZSBtYXRjaGluZyBydWxlIGRlZmluZWQgYnkgMS4yLjg0MC4xMTM1NTYuMS40LjgwMyBp cyBhIHByb3ByaWV0YXJ5DQptYXRjaGluZyBydWxlIGRlZmluZWQgYnkgTWljcm9zb2Z0IGZvciBiaXQt d2lzZSBtYXRjaGluZy4gQUZBSUNTIHRoZXkNCm5ldmVyIHdyb3RlIGEgcHVibGljIGZvcm1hbCBzcGVj IGZvciBpdC4gU28gdGhpcyBwYXJ0aWN1bGFyIG1hdGNoaW5nDQpydWxlIGlzIG5vdCBpbXBsZW1lbnRl ZCBpbiBPcGVuTERBUC4NCg0KPiBwZXJmb3JtaW5nIHVwcGVyIHF1ZXJ5IGdldHM6IE9jdCAgMSAwMDo0 NTozMyBueGxkMDEgc2xhcGRbMzAwMl06DQo+IHN0cjJmaWx0ZXIgIigmKHNBTUFjY291bnRUeXBlPSA4 MDUzMDYzNjgpKD89ZXJyb3IpKSIgT2N0ICAxIDAwOjQ1OjMzDQo+IG54bGQwMSBrZXJuZWw6IFs0OTQz Ni45MzM3MzVdIHNsYXBkWzMwMDVdOiBzZWdmYXVsdCBhdCAxOCBpcCANCj4gMDAwMDdmZjRmNzgzZDUx MiBzcCAwMDAwN2ZmNGYxYWZjODEwIGVycm9yIDQgaW4gDQo+IGxpYmMtMi4yMy5zb1s3ZmY0Zjc3Yjkw MDArMWMwMDAwXQ0KDQpEb2VzIHRoYXQgbWVhbiBzbGFwZCBzZWcgZmF1bHRzPyBJdCBzaG91bGRuJ3Qu DQoNCj4gcGVyZm9ybWluZyB0aGUgZm9sbG93aW5nIHF1ZXJ5DQo+ICAoJihvYmplY3RDbGFzcz0qKSh1 c2VyQWNjb3VudENvbnRyb2w6MS4yLjg0MC4xMTM1NTYuMS40LjgwMzo9MikpDQo+IHdpbGwgZ2V0IGZv bGxvd2luZyBsb2cgd250cnk6DQo+IE9jdCAgMSAwMDo0OTowNyBueGxkMDEgc2xhcGRbMzAzM106IHN0 cjJmaWx0ZXINCj4gIigmKG9iamVjdENsYXNzPSopKCEob2JqZWN0Q2xhc3M9KikpKSINCg0KSU1PIGl0 IG1ha2VzIHBlcmZlY3Qgc2Vuc2UgdG8gdHJlYXQgZXh0ZW5kZWQgZmlsdGVyIHBhcnQgd2l0aCBhDQpu b24tc3VwcG9ydGVkIG1hdGNoaW5nIHJ1bGUgYXMgYSBmaWx0ZXIgd2hpY2ggYWx3YXlzIGV2YWx1YXRl cyB0byBGYWxzZS4NCg0KQ2lhbywgTWljaGFlbC4NCg0KCgpTaXR6IGRlciBHZXNlbGxzY2hhZnQ6IE5p ZWRlci1PbG0gCkFtdHNnZXJpY2h0IE1haW56LCBIUkIgNzE4NQpVU3QuLUlEOiBERSAyMDggMzAzIDY2 NgpHZXNjaMOkZnRzZsO8aHJlcjogQW5kcmVhIE1hbmRhbmljaQoKRGllc2UgRS1NYWlsIGVudGjDpGx0 IHZlcnRyYXVsaWNoZSBvZGVyIHJlY2h0bGljaCBnZXNjaMO8dHp0ZSBJbmZvcm1hdGlvbmVuLiBXZW5u IFNpZSBuaWNodCBkZXIgcmljaHRpZ2UgQWRyZXNzYXQgc2luZCBvZGVyIGRpZXNlIEUtTWFpbCBpcnJ0 w7xtbGljaCBlcmhhbHRlbiBoYWJlbiwgaW5mb3JtaWVyZW4gU2llIGJpdHRlIHNvZm9ydCBkZW4gQWJz ZW5kZXIgdW5kIHZlcm5pY2h0ZW4gU2llIGRpZXNlIEUtTWFpbC4gRGFzIHVuZXJsYXVidGUgS29waWVy ZW4gc293aWUgZGllIHVuYmVmdWd0ZSBXZWl0ZXJnYWJlIGRpZXNlciBFLU1haWwgaXN0IG5pY2h0IGdl c3RhdHRldC4KClRoaXMgRS1NYWlsIG1heSBjb250YWluIGNvbmZpZGVudGlhbCBhbmQvb3IgcHJpdmls ZWdlZCBpbmZvcm1hdGlvbi4gSWYgeW91IGFyZSBub3QgdGhlIGludGVuZGVkIHJlY2lwaWVudCAob3Ig aGF2ZSByZWNlaXZlZCB0aGlzIEUtTWFpbCBpbiBlcnJvcikgcGxlYXNlIG5vdGlmeSB0aGUgc2VuZGVy IGltbWVkaWF0ZWx5IGFuZCBkZXN0cm95IHRoaXMgRS1NYWlsLiBBbnkgdW5hdXRob3Jpc2VkIGNvcHlp bmcsIGRpc2Nsb3N1cmUgb3IgZGlzdHJpYnV0aW9uIG9mIHRoZSBtYXRlcmlhbCBpbiB0aGlzIEUtTWFp bCBpcyBzdHJpY3RseSBmb3JiaWRkZW4u

1 0

(ITS#8750) About the implementation of mutex lock(li_log_mutex) in accesslog_purge()
by yos-nishino＠ys.jp.nec.com 01 Oct '17

01 Oct '17

Full_Name: Yoshinori Nishino Version: 2.4.45 OS: CentOS 7.3.1611 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (210.143.35.20) Dear sir, I saw the deadlock between accesslog_purge() and accesslog_responce(), both of which are implemented in servers/slapd/overlays/accesslog.c. The deadlock was caused by __db_hybrid_mutex_suspend() in libdb-5.3.so(hdb). ==[the output of pstack]== Thread 22 (Thread 0x7f1c845ec700 (LWP 23001)): #0 0x00007f2e17344d13 in epoll_wait () from /lib64/libc.so.6 #1 0x00007f2e19062ee8 in slapd_daemon_task () #2 0x00007f2e17e81dc5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f2e1734473d in clone () from /lib64/libc.so.6 Thread 21 (Thread 0x7f1c82dea700 (LWP 23002)): #0 0x00007f2e17344d13 in epoll_wait () from /lib64/libc.so.6 #1 0x00007f2e19062ee8 in slapd_daemon_task () #2 0x00007f2e17e81dc5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f2e1734473d in clone () from /lib64/libc.so.6 Thread 20 (Thread 0x7f1c825e9700 (LWP 23003)): #0 0x00007f2e17344d13 in epoll_wait () from /lib64/libc.so.6 #1 0x00007f2e19062ee8 in slapd_daemon_task () #2 0x00007f2e17e81dc5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f2e1734473d in clone () from /lib64/libc.so.6 Thread 19 (Thread 0x7f1c81de8700 (LWP 23004)): #0 0x00007f2e17344d13 in epoll_wait () from /lib64/libc.so.6 #1 0x00007f2e19062ee8 in slapd_daemon_task () #2 0x00007f2e17e81dc5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f2e1734473d in clone () from /lib64/libc.so.6 Thread 18 (Thread 0x7f1c815e7700 (LWP 23005)): #0 0x00007f2e17344d13 in epoll_wait () from /lib64/libc.so.6 #1 0x00007f2e19062ee8 in slapd_daemon_task () #2 0x00007f2e17e81dc5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f2e1734473d in clone () from /lib64/libc.so.6 Thread 17 (Thread 0x7f1c80de6700 (LWP 23006)): #0 0x00007f2e17344d13 in epoll_wait () from /lib64/libc.so.6 #1 0x00007f2e19062ee8 in slapd_daemon_task () #2 0x00007f2e17e81dc5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f2e1734473d in clone () from /lib64/libc.so.6 Thread 16 (Thread 0x7f1c477f0700 (LWP 23007)): #0 0x00007f2e17344d13 in epoll_wait () from /lib64/libc.so.6 #1 0x00007f2e19062ee8 in slapd_daemon_task () #2 0x00007f2e17e81dc5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f2e1734473d in clone () from /lib64/libc.so.6 Thread 15 (Thread 0x7f1c46fef700 (LWP 23009)): #0 0x00007f2e17e856d5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f2e1860b2f3 in __db_hybrid_mutex_suspend () from /lib64/libdb-5.3.so #2 0x00007f2e1860a640 in __db_tas_mutex_lock () from /lib64/libdb-5.3.so #3 0x00007f2e186b4d3a in __lock_get_internal () from /lib64/libdb-5.3.so #4 0x00007f2e186b5820 in __lock_get () from /lib64/libdb-5.3.so #5 0x00007f2e186e1142 in __db_lget () from /lib64/libdb-5.3.so #6 0x00007f2e18612602 in __bamc_writelock () from /lib64/libdb-5.3.so #7 0x00007f2e186cd079 in __dbc_idel () from /lib64/libdb-5.3.so #8 0x00007f2e186d1113 in __dbc_del () from /lib64/libdb-5.3.so #9 0x00007f2e186dc3db in __dbc_del_pp () from /lib64/libdb-5.3.so #10 0x00007f2e1913b382 in hdb_dn2id_delete () #11 0x00007f2e191394ca in hdb_delete () #12 0x00007f2e190d6996 in overlay_op_walk () #13 0x00007f2e190d6b04 in over_op_func () #14 0x00007f2e110f3399 in accesslog_purge () from /usr/lib64/openldap/accesslog-2.4.so.2 #15 0x00007f2e18bb9fba in ldap_int_thread_pool_wrapper () from /lib64/libldap_r-2.4.so.2 #16 0x00007f2e17e81dc5 in start_thread () from /lib64/libpthread.so.0 #17 0x00007f2e1734473d in clone () from /lib64/libc.so.6 Thread 14 (Thread 0x7f1c467ee700 (LWP 23008)): #0 0x00007f2e17344d13 in epoll_wait () from /lib64/libc.so.6 #1 0x00007f2e19062ee8 in slapd_daemon_task () #2 0x00007f2e17e81dc5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f2e1734473d in clone () from /lib64/libc.so.6 Thread 13 (Thread 0x7f1c45fed700 (LWP 23010)): #0 0x00007f2e17344d13 in epoll_wait () from /lib64/libc.so.6 #1 0x00007f2e19062ee8 in slapd_daemon_task () #2 0x00007f2e17e81dc5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f2e1734473d in clone () from /lib64/libc.so.6 Thread 12 (Thread 0x7f1c457ec700 (LWP 23011)): #0 0x00007f2e17e856d5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f2e18bba00b in ldap_int_thread_pool_wrapper () from /lib64/libldap_r-2.4.so.2 #2 0x00007f2e17e81dc5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f2e1734473d in clone () from /lib64/libc.so.6 Thread 11 (Thread 0x7f1c44feb700 (LWP 23012)): #0 0x00007f2e17344d13 in epoll_wait () from /lib64/libc.so.6 #1 0x00007f2e19062ee8 in slapd_daemon_task () #2 0x00007f2e17e81dc5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f2e1734473d in clone () from /lib64/libc.so.6 Thread 10 (Thread 0x7f1c447ea700 (LWP 23013)): #0 0x00007f2e17e856d5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f2e18bba00b in ldap_int_thread_pool_wrapper () from /lib64/libldap_r-2.4.so.2 #2 0x00007f2e17e81dc5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f2e1734473d in clone () from /lib64/libc.so.6 Thread 9 (Thread 0x7f1c43fe9700 (LWP 23014)): #0 0x00007f2e17344d13 in epoll_wait () from /lib64/libc.so.6 #1 0x00007f2e19062ee8 in slapd_daemon_task () #2 0x00007f2e17e81dc5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f2e1734473d in clone () from /lib64/libc.so.6 Thread 8 (Thread 0x7f1c437e8700 (LWP 23015)): #0 0x00007f2e17344d13 in epoll_wait () from /lib64/libc.so.6 #1 0x00007f2e19062ee8 in slapd_daemon_task () #2 0x00007f2e17e81dc5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f2e1734473d in clone () from /lib64/libc.so.6 Thread 7 (Thread 0x7f1c42fe7700 (LWP 23016)): #0 0x00007f2e17e856d5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f2e1860b2f3 in __db_hybrid_mutex_suspend () from /lib64/libdb-5.3.so #2 0x00007f2e1860a640 in __db_tas_mutex_lock () from /lib64/libdb-5.3.so #3 0x00007f2e186b4d3a in __lock_get_internal () from /lib64/libdb-5.3.so #4 0x00007f2e186b5820 in __lock_get () from /lib64/libdb-5.3.so #5 0x00007f2e186e1142 in __db_lget () from /lib64/libdb-5.3.so #6 0x00007f2e18628605 in __bam_search () from /lib64/libdb-5.3.so #7 0x00007f2e18613256 in __bamc_search () from /lib64/libdb-5.3.so #8 0x00007f2e1861790f in __bamc_put () from /lib64/libdb-5.3.so #9 0x00007f2e186ce514 in __dbc_iput () from /lib64/libdb-5.3.so #10 0x00007f2e186c9a0e in __db_put () from /lib64/libdb-5.3.so #11 0x00007f2e186defa4 in __db_put_pp () from /lib64/libdb-5.3.so #12 0x00007f2e1913acec in hdb_dn2id_add () #13 0x00007f2e1913261b in hdb_add () #14 0x00007f2e190d6996 in overlay_op_walk () #15 0x00007f2e190d6b04 in over_op_func () #16 0x00007f2e110f44e4 in accesslog_response () from /usr/lib64/openldap/accesslog-2.4.so.2 #17 0x00007f2e190d5d48 in over_back_response () #18 0x00007f2e19078243 in slap_response_play () #19 0x00007f2e19078800 in send_ldap_response () #20 0x00007f2e19079392 in slap_send_ldap_result () #21 0x00007f2e190f438d in hdb_modify () #22 0x00007f2e190d6996 in overlay_op_walk () #23 0x00007f2e190d6b04 in over_op_func () #24 0x00007f2e190c7bae in syncrepl_message_to_op () #25 0x00007f2e190cab57 in do_syncrep2 () #26 0x00007f2e190cf95b in do_syncrepl () #27 0x00007f2e190689f1 in connection_read_thread () #28 0x00007f2e18bb9fba in ldap_int_thread_pool_wrapper () from /lib64/libldap_r-2.4.so.2 #29 0x00007f2e17e81dc5 in start_thread () from /lib64/libpthread.so.0 #30 0x00007f2e1734473d in clone () from /lib64/libc.so.6 Thread 6 (Thread 0x7f1c427e6700 (LWP 23018)): #0 0x00007f2e17344d13 in epoll_wait () from /lib64/libc.so.6 #1 0x00007f2e19062ee8 in slapd_daemon_task () #2 0x00007f2e17e81dc5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f2e1734473d in clone () from /lib64/libc.so.6 Thread 5 (Thread 0x7f1c41fe5700 (LWP 23017)): #0 0x00007f2e17e881bd in __lll_lock_wait () from /lib64/libpthread.so.0 #1 0x00007f2e17e83d02 in _L_lock_791 () from /lib64/libpthread.so.0 #2 0x00007f2e17e83c08 in pthread_mutex_lock () from /lib64/libpthread.so.0 #3 0x00007f2e110f3f54 in accesslog_response () from /usr/lib64/openldap/accesslog-2.4.so.2 #4 0x00007f2e190d5d48 in over_back_response () #5 0x00007f2e19078243 in slap_response_play () #6 0x00007f2e19078800 in send_ldap_response () #7 0x00007f2e19079392 in slap_send_ldap_result () #8 0x00007f2e190f438d in hdb_modify () #9 0x00007f2e190d6996 in overlay_op_walk () #10 0x00007f2e190d6b04 in over_op_func () #11 0x00007f2e1909d7d3 in passwd_extop () #12 0x00007f2e1909bccc in fe_extended () #13 0x00007f2e1909ba7d in do_extended () #14 0x00007f2e19067d0c in connection_operation () #15 0x00007f2e1906807b in connection_read_thread () #16 0x00007f2e18bb9fba in ldap_int_thread_pool_wrapper () from /lib64/libldap_r-2.4.so.2 #17 0x00007f2e17e81dc5 in start_thread () from /lib64/libpthread.so.0 #18 0x00007f2e1734473d in clone () from /lib64/libc.so.6 Thread 4 (Thread 0x7f1c417e4700 (LWP 23019)): #0 0x00007f2e17344d13 in epoll_wait () from /lib64/libc.so.6 #1 0x00007f2e19062ee8 in slapd_daemon_task () #2 0x00007f2e17e81dc5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f2e1734473d in clone () from /lib64/libc.so.6 Thread 3 (Thread 0x7f1c40fe3700 (LWP 23020)): #0 0x00007f2e17344d13 in epoll_wait () from /lib64/libc.so.6 #1 0x00007f2e19062ee8 in slapd_daemon_task () #2 0x00007f2e17e81dc5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f2e1734473d in clone () from /lib64/libc.so.6 Thread 2 (Thread 0x7f1c3bfff700 (LWP 23021)): #0 0x00007f2e17344d13 in epoll_wait () from /lib64/libc.so.6 #1 0x00007f2e19062ee8 in slapd_daemon_task () #2 0x00007f2e17e81dc5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f2e1734473d in clone () from /lib64/libc.so.6 Thread 1 (Thread 0x7f2e1900d740 (LWP 22992)): #0 0x00007f2e17e82ef7 in pthread_join () from /lib64/libpthread.so.0 #1 0x00007f2e19064de1 in slapd_daemon () #2 0x00007f2e1904bb62 in main () ==== ==[the output of "db_stat -C A"]== Default locking region information: 9 Last allocated locker ID 0x7fffffff Current maximum unused locker ID 9 Number of lock modes 3600 Initial number of locks allocated 0 Initial number of lockers allocated 3600 Initial number of lock objects allocated 26000 Maximum number of locks possible 26000 Maximum number of lockers possible 26000 Maximum number of lock objects possible 3600 Current number of locks allocated 18 Current number of lockers allocated 3600 Current number of lock objects allocated 720 Number of lock object partitions 32771 Size of object hash table 15 Number of current locks 2044 Maximum number of locks at any one time 4 Maximum number of locks in any one bucket 1 Maximum number of locks stolen by for an empty partition 1 Maximum number of locks stolen for any one partition 17 Number of current lockers 17 Maximum number of lockers at any one time 11 Number of current lock objects 1456 Maximum number of lock objects at any one time 3 Maximum number of lock objects in any one bucket 0 Maximum number of objects stolen by for an empty partition 0 Maximum number of objects stolen for any one partition 301M Total number of locks requested (301292352) 301M Total number of locks released (301292334) 0 Total number of locks upgraded 7 Total number of locks downgraded 3 Lock requests not available due to conflicts, for which we waited 2 Lock requests not available due to conflicts, for which we did not wait 0 Number of deadlocks 0 Lock timeout value 0 Number of locks that have timed out 0 Transaction timeout value 0 Number of transactions that have timed out 2GB 524MB 688KB Region size 0 The number of partition locks that required waiting (0%) 0 The maximum number of times any partition lock was waited for (0%) 0 The number of object queue operations that required waiting (0%) 0 The number of locker allocations that required waiting (0%) 0 The number of region locks that required waiting (0%) 3 Maximum hash bucket length =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Lock REGINFO information: Environment Region type 1 Region ID /var/lib/ldap/accesslog_db/__db.001 Region name 0x7f0ffe75b000 Region address 0x7f0ffe75b0a0 Region allocation head 0x7f0ffe95b5b0 Region primary address 0 Region maximum allocation 0 Region allocated Region allocations: 1032145 allocations, 0 failures, 1030659 frees, 1 longest Allocations by power-of-two sizes: 1KB 1032140 2KB 0 4KB 0 8KB 0 16KB 0 32KB 0 64KB 1 128KB 0 256KB 0 512KB 0 1024KB 4 REGION_SHARED Region flags =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Lock region parameters: 2 Lock region region mutex [0/11M 0% 6376/139709380257600] 32771 locker table size 32771 object table size 2099280 obj_off 5570968 locker_off 0 need_dd =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Lock conflict matrix: =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Locks grouped by lockers: Locker Mode Count Status ----------------- Object --------------- 1 dd= 0 locks held 1 write locks 0 pid/thread 22992/139835964708672 flags 10 priority 100 1 READ 1 HELD id2entry.bdb handle 0 2 dd= 0 locks held 1 write locks 0 pid/thread 22992/139835964708672 flags 10 priority 100 2 READ 1 HELD dn2id.bdb handle 0 3 dd= 0 locks held 0 write locks 0 pid/thread 22992/139835964708672 flags 0 priority 100 4 dd= 0 locks held 1 write locks 0 pid/thread 22992/139760456615680 flags 10 priority 100 4 READ 1 HELD entryCSN.bdb handle 0 5 dd= 0 locks held 0 write locks 0 pid/thread 22992/139759426926336 flags 0 priority 100 6 dd= 0 locks held 1 write locks 0 pid/thread 22992/139759426926336 flags 10 priority 100 6 READ 1 HELD objectClass.bdb handle 0 7 dd= 0 locks held 1 write locks 0 pid/thread 22992/139759426926336 flags 10 priority 100 7 READ 1 HELD reqStart.bdb handle 0 8 dd= 0 locks held 1 write locks 0 pid/thread 22992/139759384962816 flags 10 priority 100 8 READ 1 HELD reqEnd.bdb handle 0 9 dd= 0 locks held 1 write locks 0 pid/thread 22992/139759384962816 flags 10 priority 100 9 READ 1 HELD reqResult.bdb handle 0 80000003 dd= 0 locks held 0 write locks 0 pid/thread 22992/139835964708672 flags 0 priority 100 80000004 dd= 0 locks held 0 write locks 0 pid/thread 22992/139835964708672 flags 0 priority 100 80000006 dd= 0 locks held 0 write locks 0 pid/thread 22992/139759384962816 flags 0 priority 100 80000007 dd= 0 locks held 0 write locks 0 pid/thread 22992/139759426926336 flags 0 priority 100 8007dd08 dd= 0 locks held 2 write locks 0 pid/thread 22992/139759359784704 flags 0 priority 100 8007dd08 READ 1 HELD 0x652a90 len: 9 data: 010000000000000000 8007dd08 READ 1 HELD dn2id.bdb page 3 8007dd09 dd= 0 locks held 2 write locks 0 pid/thread 22992/139759426926336 flags 0 priority 100 8007dd09 READ 1 HELD 0x652a90 len: 9 data: 010000000000000000 8007dd09 READ 1 HELD 0x6716e8 len: 9 data: cfea8d000000000000 8007dd0a dd= 0 locks held 2 write locks 0 pid/thread 22992/139759426926336 flags 0 priority 100 8007dd0a WRITE 1 WAIT dn2id.bdb page 3 8007dd0a READ 1 HELD dn2id.bdb page 3 8007dd0a READ 1 HELD dn2id.bdb page 444591 8007dd0b dd= 0 locks held 0 write locks 0 pid/thread 22992/139759359784704 flags 0 priority 100 8007dd0b WRITE 1 WAIT dn2id.bdb page 3 =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Locks grouped by object: Locker Mode Count Status ----------------- Object --------------- 8007dd08 READ 1 HELD dn2id.bdb page 3 8007dd0a READ 1 HELD dn2id.bdb page 3 8007dd0b WRITE 1 WAIT dn2id.bdb page 3 8007dd0a WRITE 1 WAIT dn2id.bdb page 3 2 READ 1 HELD dn2id.bdb handle 0 1 READ 1 HELD id2entry.bdb handle 0 4 READ 1 HELD entryCSN.bdb handle 0 8007dd09 READ 1 HELD 0x652a90 len: 9 data: 010000000000000000 8007dd08 READ 1 HELD 0x652a90 len: 9 data: 010000000000000000 9 READ 1 HELD reqResult.bdb handle 0 8 READ 1 HELD reqEnd.bdb handle 0 8007dd0a READ 1 HELD dn2id.bdb page 444591 7 READ 1 HELD reqStart.bdb handle 0 8007dd09 READ 1 HELD 0x6716e8 len: 9 data: cfea8d000000000000 6 READ 1 HELD objectClass.bdb handle 0 ==== #pid/thread 22992/139759426926336:Thread 7f1c46fef700 - accesslog_purge() #pid/thread 22992/139759359784704:Thread 7f1c42fe7700 - accesslog_responce() I am going to consider the tentative fix for openldap because I anticipate it would take so long time to pursue the fix for bdb(hdb), The tentative fix is like the following. Would you let me know whethere or not there are any concern about the fix? ==[accesslog_purge() in accesslog.c]== for (i=0; i<pd.used; i++) { op->o_req_dn = pd.dn[i]; op->o_req_ndn = pd.ndn[i]; if ( !slapd_shutdown ) { ldap_pvt_thread_mutex_lock( &li->li_log_mutex ); rs_reinit( &rs, REP_RESULT ); op->o_bd->be_delete( op, &rs ); ldap_pvt_thread_mutex_unlock( &li->li_log_mutex ); } ch_free( pd.ndn[i].bv_val ); ch_free( pd.dn[i].bv_val ); ldap_pvt_thread_pool_pausecheck( &connection_pool ); } ==== I know openldap itself currently recommend NOT using hdb/bdb as backend database. However, I should seek any workaround in the hdb backend case. Best Regards,

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs October 2017