(ITS#8755) invalid file descriptor when closing tls connection
by info＠christianknueppel.de
Full_Name: Christian Knueppel
Version: 2.4.45
OS: Ubuntu 16.04.3 LTS
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (141.12.132.220)
I currently developing on a c software which is using Openldap with TLS
authentication. My software is working fine, but when i test it with valgrind, i
always get an invalid file descriptor when closing the connection.
Here is the stacktrace from valgrind:
==17517== Warning: invalid file descriptor -1 in syscall write()
==17517== at 0x4E4A4BD: ??? (syscall-template.S:84)
==17517== by 0x61BF5E7: sb_debug_write (in
/usr/lib/x86_64-linux-gnu/liblber-2.4.so.2.10.5)
==17517== by 0x7261644: _gnutls_writev_emu (gnutls_buffers.c:447)
==17517== by 0x7261644: _gnutls_writev (gnutls_buffers.c:505)
==17517== by 0x7261644: _gnutls_io_write_flush (gnutls_buffers.c:699)
==17517== by 0x725BDFF: _gnutls_send_tlen_int (gnutls_record.c:464)
==17517== by 0x727E0D7: _gnutls_send_int (gnutls_record.h:43)
==17517== by 0x727E0D7: gnutls_alert_send (gnutls_alert.c:165)
==17517== by 0x725DCD8: gnutls_bye (gnutls_record.c:289)
==17517== by 0x5F9F181: tlsg_sb_close (in
/usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2.10.5)
==17517== by 0x61C07B1: ber_int_sb_close (in
/usr/lib/x86_64-linux-gnu/liblber-2.4.so.2.10.5)
==17517== by 0x61C08A3: ber_sockbuf_free (in
/usr/lib/x86_64-linux-gnu/liblber-2.4.so.2.10.5)
==17517== by 0x5F815CD: ldap_ld_free (in
/usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2.10.5)
==17517== by 0x41618F: ldap_close_handle (ldap_connection.c:481)
--> In function ldap_close_handle i call ldap_unbind_ext_s(ld, NULL, NULL).
The connection is built with ldap_initialize(&ld, config.ldap_url) and
ldap_start_tls_s(ld, NULL, NULL). Options set with ldap_set_option() are
LDAP_OPT_X_TLS_REQUIRE_CERT to 2 (LDAP_OPT_X_TLS_DEMAND) and
LDAP_OPT_X_TLS_CACERTFILE are set to all SSL CA-Certificates
(/etc/ssl/certs/ca-certificates.crt). I run the ldap_unbind_ext_s command (for
test purpose) shortly after the start_tls command is finished.
When i use ldap_sasl_interactive_bind_s with DIGEST-MD5 instead of
ldap_start_tls_s, the warning doesn't appear. When i use both, tls and sasl, the
warning also appears.
My computer running on Ubuntu 16.04.3 LTS (uname: 4.4.0-97-generic x86_64) with
libldap-2.4-2 (2.4.42+dfsg-2ubuntu3.2) and libgnutls30 (3.4.10-4ubuntu1.4). I
also tested it with the newest Ubuntu Artful Aardvark and the newest openldap
(2.4.45+dfsg-1ubuntu1) and gnutls(3.5.8-6ubuntu3) release, but it didn't has any
effect in my case.
I also tryed to compiled openldap against openssl to see, if it might be a
gnutls bug, but the invalid file descriptor occurs again. The lower valgrind
stacktrace is done with openldap 2.4.45 and openssl 1.0.2g on the newest Artful
Aardvark 17.10.
==2638== Warning: invalid file descriptor -1 in syscall write()
==2638== at 0x4E4DCC0: write (write.c:26)
==2638== by 0x61C6E87: sb_debug_write (sockbuf.c:854)
==2638== by 0x5FAAB4A: tlso_bio_write.part.8 (tls_o.c:790)
==2638== by 0x516B94A: BIO_write (bio_lib.c:243)
==2638== by 0x5B20C61: ssl3_write_pending (s3_pkt.c:1105)
==2638== by 0x5B22E82: ssl3_dispatch_alert (s3_pkt.c:1733)
==2638== by 0x5B1EAE6: ssl3_shutdown (s3_lib.c:4372)
==2638== by 0x5FAA749: tlso_sb_close (tls_o.c:899)
==2638== by 0x61C7D8A: ber_int_sb_close (sockbuf.c:383)
==2638== by 0x61C7E73: ber_sockbuf_free (sockbuf.c:74)
==2638== by 0x5F8D006: ldap_ld_free (unbind.c:134)
==2638== by 0x1277B7: ldap_close_handle (ldap_connection.c:481)
5 years, 11 months
(ITS#8754) ldpasearch IPv5 only
by dpa-openldap＠aegee.org
Full_Name: Дилян Палаузов
Version: 2.4.45
OS: Linux
URL:
Submission from: (NULL) (87.193.154.82)
After `./configure --disable-ipv6` I expect, that `ldapsearch` will not contact
severs over IPv6, which is achieved with the fix below.
diff --git a/libraries/libldap/os-ip.c b/libraries/libldap/os-ip.c
--- a/libraries/libldap/os-ip.c
+++ b/libraries/libldap/os-ip.c
@@ -623,6 +623,9 @@ ldap_connect_to_host(LDAP *ld, Sockbuf *sb,
continue;
}
+#ifndef LDAP_PF_INET6
+ if (sai->ai_family == AF_INET6) continue;
+#endif
/* we assume AF_x and PF_x are equal for all x */
s = ldap_int_socket( ld, sai->ai_family, socktype );
if ( s == AC_SOCKET_INVALID ) {
5 years, 11 months
(ITS#8753) Public key pinning support in libldap
by ondra＠openldap.org
Full_Name: Ondrej Kuznik
Version: master
OS:
URL: https://github.com/mistotebe/openldap/tree/its8753
Submission from: (NULL) (82.10.24.68)
Some programs might want to pin the server's public key instead of/in addition
to certificate validation. The patch linked implements this option and provides
OpenSSL/GnuTLS support code.
A new libldap option LDAP_OPT_X_TLS_PEERKEY_HASH that accepts a string
'hashname/base64_hash_of_public_key'. If a TLS session is already present on the
main connection, it is also checked immediately.
It introduces a dependency on liblutil by depending on the symbol
lutil_b64_pton. Somehow, this breaks the build for the ldap* tools, not sure why
or how to fix that yet.
5 years, 11 months
Re: (ITS#8609) segfault in mods.c - modify_add_values
by henson＠acm.org
On Fri, Sep 15, 2017 at 07:54:22PM -0700, Paul B. Henson wrote:
> I've been running 2.4.44 with the test patch since May and haven't seen
> a crash since, for what that's worth. We upgraded to 2.4.45 last week,
> without that patch. I'm going to see if the crash shows up again, and if
> it does, apply the patch to 2.4.45 and see if it goes away again.
Well, since upgrading, I've now seen two crashes; I don't have it
compiled with debugging right now so the trace is limited:
#0 0x0000000000483eb9 in modify_add_values ()
#1 0x00007f7b80d45bc3 in mdb_modify_internal () from
/usr/lib64/openldap/openldap/back_mdb.so
#2 0x00007f7b80d46f55 in mdb_modify () from
/usr/lib64/openldap/openldap/back_mdb.so
#3 0x000000000049a6d2 in overlay_op_walk ()
#4 0x000000000049a82a in ?? ()
#5 0x000000000048bbef in ?? ()
#6 0x000000000048f8cf in ?? ()
#7 0x0000000000493ae3 in ?? ()
#8 0x00007f7b858327f2 in ?? () from /usr/lib64/libldap_r-2.4.so.2
#9 0x00007f7b84bb03e4 in start_thread () from /lib64/libpthread.so.0
#10 0x00007f7b848f23ed in clone () from /lib64/libc.so.6
but it looks like the same issue to me. I'm going to go ahead and
recompile 2.4.45 with the ignore-invalid-accesslog-ops.patch and run
that for a while and see what happens.
Thanks...
5 years, 12 months
(ITS#8752) MMR delta-sync deadlock using slapd.conf
by quanah＠openldap.org
Full_Name: Quanah Gibson-Mount
Version: 2.4.45
OS: Linux
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (47.208.148.239)
There is a reproducible lockup with delta-sync based multimaster replication.
The more masters exist the easier it is to trigger, with 100% reproducibility at
4 masters.
Generally:
Load a database onto a server with no syncprov/accesslog configuration.
Stop slapd
Reconfigure slapd.conf to include syncprov/accesslog overlays, and cn=accesslog
DB for changes, and replication agreements with other servers
Configure other servers for syncprov/accesslog, but no database
start slapd on all servers
execute a MOD operation sequentially across the servers (1, 2, 3, 4)
slapd will deadlock on server 3 or 4 100% of the time
5 years, 12 months
(ITS#8751) deref aliases with back meta goes wrong in db_cache_find_ndn()
by paul＠subsignal.org
Full_Name: Paul K.lle
Version: 24.44
OS: omnios
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (141.43.201.48)
Disclaimer: I don't know C very well and I'm a complete noob wrt OpenLDAP
source
meta (dc=btu,dc=local) with two olcMetaSub URI:
0 -> olcDbURI ldap://localhost/ou=Fak6,dc=btu,dc=local
olcDbRewrite {0}suffixmassage "ou=Fak6,dc=btu,dc=local"
"ou=Fak6,dc=IT,dc=local"
1 -> olcDbURI ldap://ldap.xxxxx.de/ou=People,dc=btu,dc=local
olcDbRewrite » {0}suffixmassage "ou=People,dc=btu,dc=local"
"ou=People,o=XXXXXXXXXXXXX XXXXX,c=De"
no deref search:
[root@smb01 ~]# ldapsearch -x -h 192.168.70.101 -b
ou=users,ou=Fak6,dc=btu,dc=local '(cn=Paul*)' '*'
# extended LDIF
#
# LDAPv3
# base <ou=users,ou=Fak6,dc=btu,dc=local> with scope subtree
# filter: (cn=Paul*)
# requesting: *
#
# Paul Koelle, Users, Fak6, btu.local
dn: cn=Paul Koelle,ou=Users,ou=Fak6,dc=btu,dc=local
aliasedObjectName: cn=Paul Koelle,ou=People,dc=btu,dc=local
cn: Paul Koelle
objectClass: alias
objectClass: extensibleObject
# search result
search: 2
result: 0 Success
# numResponses: 2
# numEntries: 1
with deref:
[root@smb01 ~]# ldapsearch -a always -x -h 192.168.70.101 -b
ou=users,ou=Fak6,dc=btu,dc=local '(cn=Paul*)' '*'
# extended LDIF
#
# LDAPv3
# base <ou=users,ou=Fak6,dc=btu,dc=local> with scope subtree
# filter: (cn=Paul*)
# requesting: *
#
# search result
search: 2
result: 0 Success
# numResponses: 1
debug log reveals:
59d38a50 bdb_idl_fetch_key: [01872a84]
59d38a50 <= bdb_index_read 1 candidates
59d38a50 <= bdb_equality_candidates: id=1, first=19, last=19
59d38a50 <= bdb_filter_candidates: id=1 first=19 last=19
59d38a50 => hdb_dn2idl("ou=users,ou=fak6,dc=it,dc=local")
59d38a50 bdb_dn2entry("cn=paul koelle,ou=people,dc=btu,dc=local")
59d38a50 => hdb_dn2id("c=btu,dc=local")
59d38a50 <= hdb_dn2id: get failed: DB_NOTFOUND: No matching key/data pair found
(-30988)
59d38a50 bdb_search_candidates: failed (rc=33)
breakpoint for hdb_dn2id
(gdb) bt
#0 hdb_dn2id (op=0xa05510, in=0xfffffd7ffc87f208, ei=0xfffffd7ffc87f1f0,
txn=0x8e6950, cursor=0xfffffd7ffc87f1e0) at dn2id.c:705
#1 0x000000000059beed in hdb_cache_find_ndn (op=0xa05510, txn=0x8e6950,
ndn=0xfffffd7ffc87f350, res=0xfffffd7ffc87f2e0) at cache.c:462
#2 0x00000000005a3d81 in hdb_dn2entry (op=0xa05510, tid=0x8e6950,
dn=0xfffffd7ffc87f350, e=0xfffffd7ffc87f348, matched=0,
lock=0xfffffd7ffc87f330) at dn2entry.c:47
#3 0x000000000053756c in deref_base (op=0xa05510, rs=0xfffffd7ffc9ffb40,
e=0x91c0b8, matched=0xfffffd7ffc87f460, txn=0x8e6950,
lock=0xfffffd7ffc87f3c0, tmp=0x10e3010, visited=0xf63010) at search.c:104
#4 0x0000000000537be7 in search_aliases (op=0xa05510, rs=0xfffffd7ffc9ffb40,
e=0xfffffd7ffc87f750, txn=0x8e6950, ids=0xfffffd7ffc8ff7a0,
scopes=0xfffffd7ffc87f7a0, stack=0xde3010) at search.c:250
#5 0x000000000053ab3d in search_candidates (op=0xa05510,
rs=0xfffffd7ffc9ffb40, e=0xfffffd7ffc87f750, txn=0x8e6950,
ids=0xfffffd7ffc8ff7a0, scopes=0xfffffd7ffc87f7a0) at search.c:1258
#6 0x0000000000538e27 in hdb_search (op=0xa05510, rs=0xfffffd7ffc9ffb40)
at search.c:616
#7 0x000000000048438d in fe_op_search (op=0xa05510, rs=0xfffffd7ffc9ffb40)
at search.c:402
#8 0x0000000000483bcc in do_search (op=0xa05510, rs=0xfffffd7ffc9ffb40)
at search.c:247
#9 0x0000000000480537 in connection_operation (ctx=0xfffffd7ffc9ffc70,
arg_v=0xa05510) at connection.c:1158
#10 0x0000000000480acc in connection_read_thread (ctx=0xfffffd7ffc9ffc70,
argv=0xf) at connection.c:1294
#11 0x0000000000622afd in ldap_int_thread_pool_wrapper (xpool=0x7d8840)
at tpool.c:696
#12 0xfffffd7ffeb9e43a in _thrp_setup () from /lib/64/libc.so.1
#13 0xfffffd7ffeb9e750 in ?? () from /lib/64/libc.so.1
#14 0x0000000000000000 in ?? ()
if you look at the arguments in (h|b)db_cache_find_ndn there are mixed values
from pre/post meta rewrite?
#1 0x000000000059beed in hdb_cache_find_ndn (op=0xa05510, txn=0x8e6950,
ndn=0xfffffd7ffc87f350, res=0xfffffd7ffc87f2e0) at cache.c:462
462 rc = bdb_dn2id( op, &ei.bei_nrdn, &ei, txn,
&cursor );
(gdb) p *ndn
$8 = {bv_len = 40,
bv_val = 0x9fdd06 "cn=paul koelle,ou=people,dc=btu,dc=local"}
(gdb) p *res
$9 = (EntryInfo *) 0x0
(gdb) p ndn->bv_val
$13 = 0x9fdd06 "cn=paul koelle,ou=people,dc=btu,dc=local"
(gdb) p op->o_bd->be_nsuffix[0]
$16 = {bv_len = 14, bv_val = 0x9f17d0 "dc=it,dc=local"}
and the code in bdb_cache_find_ndn constructs a new BV from *ndn and *op which
have different bases (cache.c line 426):
/* we're searching a full DN from the root */
ptr = ndn->bv_val + ndn->bv_len - op->o_bd->be_nsuffix[0].bv_len;
ei.bei_nrdn.bv_val = ptr;
ei.bei_nrdn.bv_len = op->o_bd->be_nsuffix[0].bv_len;
where op->o_bd_>be_nsuffix[0] is the post-rewrite suffix and ndn has the
pre-rewrite suffix??
continuing lead to the following debug output:
59d38434 => hdb_dn2id("c=btu,dc=local")
59d38434 <= hdb_dn2id: get failed: DB_NOTFOUND: No matching key/data pair found
(-30988)
note the incomplete "c=btu,dc=local" probably because "dc=it,dc=local" is one
char less than "dc=btu,dc=local"
5 years, 12 months
Re: (ITS#8747) LDAP load balancer daemon (lloadd)
by ondra＠mistotebe.net
On Thu, Sep 28, 2017 at 04:13:46PM +0000, ondra(a)openldap.org wrote:
> The 'lloadd' branch linked above contains the load balancer code that
> is now ready for review.
>
> This adds a new server to the OpenLDAP project, a load balancing proxy
> (prototype).
To summarise the reasons for this project to exist and highlight its
features, limitations:
- most LDAP load balancers pin a connection to another server and
then just ship data, lloadd can distribute operations from a single
client connection across LDAP servers
- to make the above possible, lloadd sets up connections to the backend
servers on startup and manages them as per configuration (independent
on the clients)
- bind operations are forwarded over dedicated bind connections or using
the VC exop if feature 'vc' is enabled in its config, the designated
identity is then passed on with the operations with proxyauth control
(if feature 'proxyauthz' is enabled)
- it is expected that all backends are indistinguishable (same features,
suffixes, data)
- no SASL bind support yet
- if an operation cannot be processed or forwarded for any reason
(overload, connection loss, ...), it is never re-sent, client will
however still be sent an appropriate result in that case
The lloadd.8 and lloadd.conf.5 manpages are provided, Admin guide and
further documentation will come as the implementation matures.
--
Ondřej Kuzník
Senior Software Engineer
Symas Corporation http://www.symas.com
Packaged, certified, and supported LDAP solutions powered by OpenLDAP
5 years, 12 months
AW: (ITS#8749) Proxy: LDAP-querry doesn't work for e.g (userAccountControl:1.2.840.113556.1.4.803:=2)
by Steffen.Krahl＠nexio.de
RGVhciBNaWNoYWVsLA0KDQptYW55IHRoYW5rcyBmb3IgeW91ciBleHBsYWluYXRpb24uIFJlZ2FyZGlu
ZyBzZWdtZW50YXRpb24gZmF1bHQ6IHRoYXQncyB0cnVlLCBidXQgSSBoYXZlIHRvIGludmVzdGlnYXRl
IGZ1cnRoZXINCg0KQlINClN0ZWZmZW4gDQoNCi0tLS0tVXJzcHLDvG5nbGljaGUgTmFjaHJpY2h0LS0t
LS0NClZvbjogTWljaGFlbCBTdHLDtmRlciBbbWFpbHRvOm1pY2hhZWxAc3Ryb2VkZXIuY29tXSANCkdl
c2VuZGV0OiBTb25udGFnLCAxLiBPa3RvYmVyIDIwMTcgMTA6MjUNCkFuOiBLcmFobCwgU3RlZmZlbjsg
b3BlbmxkYXAtaXRzQE9wZW5MREFQLm9yZw0KQmV0cmVmZjogUmU6IChJVFMjODc0OSkgUHJveHk6IExE
QVAtcXVlcnJ5IGRvZXNuJ3Qgd29yayBmb3IgZS5nICh1c2VyQWNjb3VudENvbnRyb2w6MS4yLjg0MC4x
MTM1NTYuMS40LjgwMzo9MikNCg0Kc3RlZmZlbi5rcmFobEBuZXhpby5kZSB3cm90ZToNCj4gSSdtIHVz
aW5nIE9wZW5MREFQIHdpdGggTERBUC1iYWNrZW5kIGFzIHByb3h5IGZvciBBY3RpdmVEaXJlY3Rvcnkg
DQo+IEl0J3Mgd29ya2luZyB3ZWxsIHNvIGZhciwgb25seSBMREFQLXF1ZXJpZXMgd2hpY2ggc2hvdWxk
IGV4Y2x1ZGUNCj4gZGVhY3RpdmF0ZWQgdXNlcnMgZG9uJ3Qgd29yay4gSXQgc2VlbXMgc2xhcGQgZG9l
cyBub3QgYWNjZXB0IHF1ZXJpZXMNCj4gbGlrZSAoYXR0cmlidXRlOk9JRDo9dmFsdWUpDQoNCk9wZW5M
REFQIGRvZXMgc3VwcG9ydCBleHRlbmRlZCBmaWx0ZXJzIHdpdGggYSBtYXRjaGluZyBydWxlLiBCdXQg
b25seQ0Kd2l0aCBtYXRjaGluZyBydWxlcyBpbXBsZW1lbnRlZCBpbiBPcGVuTERBUC4NCg0KPiBpbiBw
YXJ0aWN1bGFyDQo+ICghKHVzZXJBY2NvdW50Q29udHJvbDoxLjIuODQwLjExMzU1Ni4xLjQuODAzOj0y
KSkpDQoNClRoZSBtYXRjaGluZyBydWxlIGRlZmluZWQgYnkgMS4yLjg0MC4xMTM1NTYuMS40LjgwMyBp
cyBhIHByb3ByaWV0YXJ5DQptYXRjaGluZyBydWxlIGRlZmluZWQgYnkgTWljcm9zb2Z0IGZvciBiaXQt
d2lzZSBtYXRjaGluZy4gQUZBSUNTIHRoZXkNCm5ldmVyIHdyb3RlIGEgcHVibGljIGZvcm1hbCBzcGVj
IGZvciBpdC4gU28gdGhpcyBwYXJ0aWN1bGFyIG1hdGNoaW5nDQpydWxlIGlzIG5vdCBpbXBsZW1lbnRl
ZCBpbiBPcGVuTERBUC4NCg0KPiBwZXJmb3JtaW5nIHVwcGVyIHF1ZXJ5IGdldHM6IE9jdCAgMSAwMDo0
NTozMyBueGxkMDEgc2xhcGRbMzAwMl06DQo+IHN0cjJmaWx0ZXIgIigmKHNBTUFjY291bnRUeXBlPSA4
MDUzMDYzNjgpKD89ZXJyb3IpKSIgT2N0ICAxIDAwOjQ1OjMzDQo+IG54bGQwMSBrZXJuZWw6IFs0OTQz
Ni45MzM3MzVdIHNsYXBkWzMwMDVdOiBzZWdmYXVsdCBhdCAxOCBpcCANCj4gMDAwMDdmZjRmNzgzZDUx
MiBzcCAwMDAwN2ZmNGYxYWZjODEwIGVycm9yIDQgaW4gDQo+IGxpYmMtMi4yMy5zb1s3ZmY0Zjc3Yjkw
MDArMWMwMDAwXQ0KDQpEb2VzIHRoYXQgbWVhbiBzbGFwZCBzZWcgZmF1bHRzPyBJdCBzaG91bGRuJ3Qu
DQoNCj4gcGVyZm9ybWluZyB0aGUgZm9sbG93aW5nIHF1ZXJ5DQo+ICAoJihvYmplY3RDbGFzcz0qKSh1
c2VyQWNjb3VudENvbnRyb2w6MS4yLjg0MC4xMTM1NTYuMS40LjgwMzo9MikpDQo+IHdpbGwgZ2V0IGZv
bGxvd2luZyBsb2cgd250cnk6DQo+IE9jdCAgMSAwMDo0OTowNyBueGxkMDEgc2xhcGRbMzAzM106IHN0
cjJmaWx0ZXINCj4gIigmKG9iamVjdENsYXNzPSopKCEob2JqZWN0Q2xhc3M9KikpKSINCg0KSU1PIGl0
IG1ha2VzIHBlcmZlY3Qgc2Vuc2UgdG8gdHJlYXQgZXh0ZW5kZWQgZmlsdGVyIHBhcnQgd2l0aCBhDQpu
b24tc3VwcG9ydGVkIG1hdGNoaW5nIHJ1bGUgYXMgYSBmaWx0ZXIgd2hpY2ggYWx3YXlzIGV2YWx1YXRl
cyB0byBGYWxzZS4NCg0KQ2lhbywgTWljaGFlbC4NCg0KCgpTaXR6IGRlciBHZXNlbGxzY2hhZnQ6IE5p
ZWRlci1PbG0gCkFtdHNnZXJpY2h0IE1haW56LCBIUkIgNzE4NQpVU3QuLUlEOiBERSAyMDggMzAzIDY2
NgpHZXNjaMOkZnRzZsO8aHJlcjogQW5kcmVhIE1hbmRhbmljaQoKRGllc2UgRS1NYWlsIGVudGjDpGx0
IHZlcnRyYXVsaWNoZSBvZGVyIHJlY2h0bGljaCBnZXNjaMO8dHp0ZSBJbmZvcm1hdGlvbmVuLiBXZW5u
IFNpZSBuaWNodCBkZXIgcmljaHRpZ2UgQWRyZXNzYXQgc2luZCBvZGVyIGRpZXNlIEUtTWFpbCBpcnJ0
w7xtbGljaCBlcmhhbHRlbiBoYWJlbiwgaW5mb3JtaWVyZW4gU2llIGJpdHRlIHNvZm9ydCBkZW4gQWJz
ZW5kZXIgdW5kIHZlcm5pY2h0ZW4gU2llIGRpZXNlIEUtTWFpbC4gRGFzIHVuZXJsYXVidGUgS29waWVy
ZW4gc293aWUgZGllIHVuYmVmdWd0ZSBXZWl0ZXJnYWJlIGRpZXNlciBFLU1haWwgaXN0IG5pY2h0IGdl
c3RhdHRldC4KClRoaXMgRS1NYWlsIG1heSBjb250YWluIGNvbmZpZGVudGlhbCBhbmQvb3IgcHJpdmls
ZWdlZCBpbmZvcm1hdGlvbi4gSWYgeW91IGFyZSBub3QgdGhlIGludGVuZGVkIHJlY2lwaWVudCAob3Ig
aGF2ZSByZWNlaXZlZCB0aGlzIEUtTWFpbCBpbiBlcnJvcikgcGxlYXNlIG5vdGlmeSB0aGUgc2VuZGVy
IGltbWVkaWF0ZWx5IGFuZCBkZXN0cm95IHRoaXMgRS1NYWlsLiBBbnkgdW5hdXRob3Jpc2VkIGNvcHlp
bmcsIGRpc2Nsb3N1cmUgb3IgZGlzdHJpYnV0aW9uIG9mIHRoZSBtYXRlcmlhbCBpbiB0aGlzIEUtTWFp
bCBpcyBzdHJpY3RseSBmb3JiaWRkZW4u
6 years
(ITS#8750) About the implementation of mutex lock(li_log_mutex) in accesslog_purge()
by yos-nishino＠ys.jp.nec.com
Full_Name: Yoshinori Nishino
Version: 2.4.45
OS: CentOS 7.3.1611
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (210.143.35.20)
Dear sir,
I saw the deadlock between accesslog_purge() and accesslog_responce(), both of
which are implemented in servers/slapd/overlays/accesslog.c.
The deadlock was caused by __db_hybrid_mutex_suspend() in libdb-5.3.so(hdb).
==[the output of pstack]==
Thread 22 (Thread 0x7f1c845ec700 (LWP 23001)):
#0 0x00007f2e17344d13 in epoll_wait () from /lib64/libc.so.6
#1 0x00007f2e19062ee8 in slapd_daemon_task ()
#2 0x00007f2e17e81dc5 in start_thread () from /lib64/libpthread.so.0
#3 0x00007f2e1734473d in clone () from /lib64/libc.so.6
Thread 21 (Thread 0x7f1c82dea700 (LWP 23002)):
#0 0x00007f2e17344d13 in epoll_wait () from /lib64/libc.so.6
#1 0x00007f2e19062ee8 in slapd_daemon_task ()
#2 0x00007f2e17e81dc5 in start_thread () from /lib64/libpthread.so.0
#3 0x00007f2e1734473d in clone () from /lib64/libc.so.6
Thread 20 (Thread 0x7f1c825e9700 (LWP 23003)):
#0 0x00007f2e17344d13 in epoll_wait () from /lib64/libc.so.6
#1 0x00007f2e19062ee8 in slapd_daemon_task ()
#2 0x00007f2e17e81dc5 in start_thread () from /lib64/libpthread.so.0
#3 0x00007f2e1734473d in clone () from /lib64/libc.so.6
Thread 19 (Thread 0x7f1c81de8700 (LWP 23004)):
#0 0x00007f2e17344d13 in epoll_wait () from /lib64/libc.so.6
#1 0x00007f2e19062ee8 in slapd_daemon_task ()
#2 0x00007f2e17e81dc5 in start_thread () from /lib64/libpthread.so.0
#3 0x00007f2e1734473d in clone () from /lib64/libc.so.6
Thread 18 (Thread 0x7f1c815e7700 (LWP 23005)):
#0 0x00007f2e17344d13 in epoll_wait () from /lib64/libc.so.6
#1 0x00007f2e19062ee8 in slapd_daemon_task ()
#2 0x00007f2e17e81dc5 in start_thread () from /lib64/libpthread.so.0
#3 0x00007f2e1734473d in clone () from /lib64/libc.so.6
Thread 17 (Thread 0x7f1c80de6700 (LWP 23006)):
#0 0x00007f2e17344d13 in epoll_wait () from /lib64/libc.so.6
#1 0x00007f2e19062ee8 in slapd_daemon_task ()
#2 0x00007f2e17e81dc5 in start_thread () from /lib64/libpthread.so.0
#3 0x00007f2e1734473d in clone () from /lib64/libc.so.6
Thread 16 (Thread 0x7f1c477f0700 (LWP 23007)):
#0 0x00007f2e17344d13 in epoll_wait () from /lib64/libc.so.6
#1 0x00007f2e19062ee8 in slapd_daemon_task ()
#2 0x00007f2e17e81dc5 in start_thread () from /lib64/libpthread.so.0
#3 0x00007f2e1734473d in clone () from /lib64/libc.so.6
Thread 15 (Thread 0x7f1c46fef700 (LWP 23009)):
#0 0x00007f2e17e856d5 in pthread_cond_wait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1 0x00007f2e1860b2f3 in __db_hybrid_mutex_suspend () from /lib64/libdb-5.3.so
#2 0x00007f2e1860a640 in __db_tas_mutex_lock () from /lib64/libdb-5.3.so
#3 0x00007f2e186b4d3a in __lock_get_internal () from /lib64/libdb-5.3.so
#4 0x00007f2e186b5820 in __lock_get () from /lib64/libdb-5.3.so
#5 0x00007f2e186e1142 in __db_lget () from /lib64/libdb-5.3.so
#6 0x00007f2e18612602 in __bamc_writelock () from /lib64/libdb-5.3.so
#7 0x00007f2e186cd079 in __dbc_idel () from /lib64/libdb-5.3.so
#8 0x00007f2e186d1113 in __dbc_del () from /lib64/libdb-5.3.so
#9 0x00007f2e186dc3db in __dbc_del_pp () from /lib64/libdb-5.3.so
#10 0x00007f2e1913b382 in hdb_dn2id_delete ()
#11 0x00007f2e191394ca in hdb_delete ()
#12 0x00007f2e190d6996 in overlay_op_walk ()
#13 0x00007f2e190d6b04 in over_op_func ()
#14 0x00007f2e110f3399 in accesslog_purge () from
/usr/lib64/openldap/accesslog-2.4.so.2
#15 0x00007f2e18bb9fba in ldap_int_thread_pool_wrapper () from
/lib64/libldap_r-2.4.so.2
#16 0x00007f2e17e81dc5 in start_thread () from /lib64/libpthread.so.0
#17 0x00007f2e1734473d in clone () from /lib64/libc.so.6
Thread 14 (Thread 0x7f1c467ee700 (LWP 23008)):
#0 0x00007f2e17344d13 in epoll_wait () from /lib64/libc.so.6
#1 0x00007f2e19062ee8 in slapd_daemon_task ()
#2 0x00007f2e17e81dc5 in start_thread () from /lib64/libpthread.so.0
#3 0x00007f2e1734473d in clone () from /lib64/libc.so.6
Thread 13 (Thread 0x7f1c45fed700 (LWP 23010)):
#0 0x00007f2e17344d13 in epoll_wait () from /lib64/libc.so.6
#1 0x00007f2e19062ee8 in slapd_daemon_task ()
#2 0x00007f2e17e81dc5 in start_thread () from /lib64/libpthread.so.0
#3 0x00007f2e1734473d in clone () from /lib64/libc.so.6
Thread 12 (Thread 0x7f1c457ec700 (LWP 23011)):
#0 0x00007f2e17e856d5 in pthread_cond_wait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1 0x00007f2e18bba00b in ldap_int_thread_pool_wrapper () from
/lib64/libldap_r-2.4.so.2
#2 0x00007f2e17e81dc5 in start_thread () from /lib64/libpthread.so.0
#3 0x00007f2e1734473d in clone () from /lib64/libc.so.6
Thread 11 (Thread 0x7f1c44feb700 (LWP 23012)):
#0 0x00007f2e17344d13 in epoll_wait () from /lib64/libc.so.6
#1 0x00007f2e19062ee8 in slapd_daemon_task ()
#2 0x00007f2e17e81dc5 in start_thread () from /lib64/libpthread.so.0
#3 0x00007f2e1734473d in clone () from /lib64/libc.so.6
Thread 10 (Thread 0x7f1c447ea700 (LWP 23013)):
#0 0x00007f2e17e856d5 in pthread_cond_wait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1 0x00007f2e18bba00b in ldap_int_thread_pool_wrapper () from
/lib64/libldap_r-2.4.so.2
#2 0x00007f2e17e81dc5 in start_thread () from /lib64/libpthread.so.0
#3 0x00007f2e1734473d in clone () from /lib64/libc.so.6
Thread 9 (Thread 0x7f1c43fe9700 (LWP 23014)):
#0 0x00007f2e17344d13 in epoll_wait () from /lib64/libc.so.6
#1 0x00007f2e19062ee8 in slapd_daemon_task ()
#2 0x00007f2e17e81dc5 in start_thread () from /lib64/libpthread.so.0
#3 0x00007f2e1734473d in clone () from /lib64/libc.so.6
Thread 8 (Thread 0x7f1c437e8700 (LWP 23015)):
#0 0x00007f2e17344d13 in epoll_wait () from /lib64/libc.so.6
#1 0x00007f2e19062ee8 in slapd_daemon_task ()
#2 0x00007f2e17e81dc5 in start_thread () from /lib64/libpthread.so.0
#3 0x00007f2e1734473d in clone () from /lib64/libc.so.6
Thread 7 (Thread 0x7f1c42fe7700 (LWP 23016)):
#0 0x00007f2e17e856d5 in pthread_cond_wait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1 0x00007f2e1860b2f3 in __db_hybrid_mutex_suspend () from /lib64/libdb-5.3.so
#2 0x00007f2e1860a640 in __db_tas_mutex_lock () from /lib64/libdb-5.3.so
#3 0x00007f2e186b4d3a in __lock_get_internal () from /lib64/libdb-5.3.so
#4 0x00007f2e186b5820 in __lock_get () from /lib64/libdb-5.3.so
#5 0x00007f2e186e1142 in __db_lget () from /lib64/libdb-5.3.so
#6 0x00007f2e18628605 in __bam_search () from /lib64/libdb-5.3.so
#7 0x00007f2e18613256 in __bamc_search () from /lib64/libdb-5.3.so
#8 0x00007f2e1861790f in __bamc_put () from /lib64/libdb-5.3.so
#9 0x00007f2e186ce514 in __dbc_iput () from /lib64/libdb-5.3.so
#10 0x00007f2e186c9a0e in __db_put () from /lib64/libdb-5.3.so
#11 0x00007f2e186defa4 in __db_put_pp () from /lib64/libdb-5.3.so
#12 0x00007f2e1913acec in hdb_dn2id_add ()
#13 0x00007f2e1913261b in hdb_add ()
#14 0x00007f2e190d6996 in overlay_op_walk ()
#15 0x00007f2e190d6b04 in over_op_func ()
#16 0x00007f2e110f44e4 in accesslog_response () from
/usr/lib64/openldap/accesslog-2.4.so.2
#17 0x00007f2e190d5d48 in over_back_response ()
#18 0x00007f2e19078243 in slap_response_play ()
#19 0x00007f2e19078800 in send_ldap_response ()
#20 0x00007f2e19079392 in slap_send_ldap_result ()
#21 0x00007f2e190f438d in hdb_modify ()
#22 0x00007f2e190d6996 in overlay_op_walk ()
#23 0x00007f2e190d6b04 in over_op_func ()
#24 0x00007f2e190c7bae in syncrepl_message_to_op ()
#25 0x00007f2e190cab57 in do_syncrep2 ()
#26 0x00007f2e190cf95b in do_syncrepl ()
#27 0x00007f2e190689f1 in connection_read_thread ()
#28 0x00007f2e18bb9fba in ldap_int_thread_pool_wrapper () from
/lib64/libldap_r-2.4.so.2
#29 0x00007f2e17e81dc5 in start_thread () from /lib64/libpthread.so.0
#30 0x00007f2e1734473d in clone () from /lib64/libc.so.6
Thread 6 (Thread 0x7f1c427e6700 (LWP 23018)):
#0 0x00007f2e17344d13 in epoll_wait () from /lib64/libc.so.6
#1 0x00007f2e19062ee8 in slapd_daemon_task ()
#2 0x00007f2e17e81dc5 in start_thread () from /lib64/libpthread.so.0
#3 0x00007f2e1734473d in clone () from /lib64/libc.so.6
Thread 5 (Thread 0x7f1c41fe5700 (LWP 23017)):
#0 0x00007f2e17e881bd in __lll_lock_wait () from /lib64/libpthread.so.0
#1 0x00007f2e17e83d02 in _L_lock_791 () from /lib64/libpthread.so.0
#2 0x00007f2e17e83c08 in pthread_mutex_lock () from /lib64/libpthread.so.0
#3 0x00007f2e110f3f54 in accesslog_response () from
/usr/lib64/openldap/accesslog-2.4.so.2
#4 0x00007f2e190d5d48 in over_back_response ()
#5 0x00007f2e19078243 in slap_response_play ()
#6 0x00007f2e19078800 in send_ldap_response ()
#7 0x00007f2e19079392 in slap_send_ldap_result ()
#8 0x00007f2e190f438d in hdb_modify ()
#9 0x00007f2e190d6996 in overlay_op_walk ()
#10 0x00007f2e190d6b04 in over_op_func ()
#11 0x00007f2e1909d7d3 in passwd_extop ()
#12 0x00007f2e1909bccc in fe_extended ()
#13 0x00007f2e1909ba7d in do_extended ()
#14 0x00007f2e19067d0c in connection_operation ()
#15 0x00007f2e1906807b in connection_read_thread ()
#16 0x00007f2e18bb9fba in ldap_int_thread_pool_wrapper () from
/lib64/libldap_r-2.4.so.2
#17 0x00007f2e17e81dc5 in start_thread () from /lib64/libpthread.so.0
#18 0x00007f2e1734473d in clone () from /lib64/libc.so.6
Thread 4 (Thread 0x7f1c417e4700 (LWP 23019)):
#0 0x00007f2e17344d13 in epoll_wait () from /lib64/libc.so.6
#1 0x00007f2e19062ee8 in slapd_daemon_task ()
#2 0x00007f2e17e81dc5 in start_thread () from /lib64/libpthread.so.0
#3 0x00007f2e1734473d in clone () from /lib64/libc.so.6
Thread 3 (Thread 0x7f1c40fe3700 (LWP 23020)):
#0 0x00007f2e17344d13 in epoll_wait () from /lib64/libc.so.6
#1 0x00007f2e19062ee8 in slapd_daemon_task ()
#2 0x00007f2e17e81dc5 in start_thread () from /lib64/libpthread.so.0
#3 0x00007f2e1734473d in clone () from /lib64/libc.so.6
Thread 2 (Thread 0x7f1c3bfff700 (LWP 23021)):
#0 0x00007f2e17344d13 in epoll_wait () from /lib64/libc.so.6
#1 0x00007f2e19062ee8 in slapd_daemon_task ()
#2 0x00007f2e17e81dc5 in start_thread () from /lib64/libpthread.so.0
#3 0x00007f2e1734473d in clone () from /lib64/libc.so.6
Thread 1 (Thread 0x7f2e1900d740 (LWP 22992)):
#0 0x00007f2e17e82ef7 in pthread_join () from /lib64/libpthread.so.0
#1 0x00007f2e19064de1 in slapd_daemon ()
#2 0x00007f2e1904bb62 in main ()
====
==[the output of "db_stat -C A"]==
Default locking region information:
9 Last allocated locker ID
0x7fffffff Current maximum unused locker ID
9 Number of lock modes
3600 Initial number of locks allocated
0 Initial number of lockers allocated
3600 Initial number of lock objects allocated
26000 Maximum number of locks possible
26000 Maximum number of lockers possible
26000 Maximum number of lock objects possible
3600 Current number of locks allocated
18 Current number of lockers allocated
3600 Current number of lock objects allocated
720 Number of lock object partitions
32771 Size of object hash table
15 Number of current locks
2044 Maximum number of locks at any one time
4 Maximum number of locks in any one bucket
1 Maximum number of locks stolen by for an empty partition
1 Maximum number of locks stolen for any one partition
17 Number of current lockers
17 Maximum number of lockers at any one time
11 Number of current lock objects
1456 Maximum number of lock objects at any one time
3 Maximum number of lock objects in any one bucket
0 Maximum number of objects stolen by for an empty partition
0 Maximum number of objects stolen for any one partition
301M Total number of locks requested (301292352)
301M Total number of locks released (301292334)
0 Total number of locks upgraded
7 Total number of locks downgraded
3 Lock requests not available due to conflicts, for which we waited
2 Lock requests not available due to conflicts, for which we did not wait
0 Number of deadlocks
0 Lock timeout value
0 Number of locks that have timed out
0 Transaction timeout value
0 Number of transactions that have timed out
2GB 524MB 688KB Region size
0 The number of partition locks that required waiting (0%)
0 The maximum number of times any partition lock was waited for (0%)
0 The number of object queue operations that required waiting (0%)
0 The number of locker allocations that required waiting (0%)
0 The number of region locks that required waiting (0%)
3 Maximum hash bucket length
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Lock REGINFO information:
Environment Region type
1 Region ID
/var/lib/ldap/accesslog_db/__db.001 Region name
0x7f0ffe75b000 Region address
0x7f0ffe75b0a0 Region allocation head
0x7f0ffe95b5b0 Region primary address
0 Region maximum allocation
0 Region allocated
Region allocations: 1032145 allocations, 0 failures, 1030659 frees, 1 longest
Allocations by power-of-two sizes:
1KB 1032140
2KB 0
4KB 0
8KB 0
16KB 0
32KB 0
64KB 1
128KB 0
256KB 0
512KB 0
1024KB 4
REGION_SHARED Region flags
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Lock region parameters:
2 Lock region region mutex [0/11M 0% 6376/139709380257600]
32771 locker table size
32771 object table size
2099280 obj_off
5570968 locker_off
0 need_dd
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Lock conflict matrix:
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Locks grouped by lockers:
Locker Mode Count Status ----------------- Object ---------------
1 dd= 0 locks held 1 write locks 0 pid/thread 22992/139835964708672
flags 10 priority 100
1 READ 1 HELD id2entry.bdb handle 0
2 dd= 0 locks held 1 write locks 0 pid/thread 22992/139835964708672
flags 10 priority 100
2 READ 1 HELD dn2id.bdb handle 0
3 dd= 0 locks held 0 write locks 0 pid/thread 22992/139835964708672
flags 0 priority 100
4 dd= 0 locks held 1 write locks 0 pid/thread 22992/139760456615680
flags 10 priority 100
4 READ 1 HELD entryCSN.bdb handle 0
5 dd= 0 locks held 0 write locks 0 pid/thread 22992/139759426926336
flags 0 priority 100
6 dd= 0 locks held 1 write locks 0 pid/thread 22992/139759426926336
flags 10 priority 100
6 READ 1 HELD objectClass.bdb handle 0
7 dd= 0 locks held 1 write locks 0 pid/thread 22992/139759426926336
flags 10 priority 100
7 READ 1 HELD reqStart.bdb handle 0
8 dd= 0 locks held 1 write locks 0 pid/thread 22992/139759384962816
flags 10 priority 100
8 READ 1 HELD reqEnd.bdb handle 0
9 dd= 0 locks held 1 write locks 0 pid/thread 22992/139759384962816
flags 10 priority 100
9 READ 1 HELD reqResult.bdb handle 0
80000003 dd= 0 locks held 0 write locks 0 pid/thread 22992/139835964708672
flags 0 priority 100
80000004 dd= 0 locks held 0 write locks 0 pid/thread 22992/139835964708672
flags 0 priority 100
80000006 dd= 0 locks held 0 write locks 0 pid/thread 22992/139759384962816
flags 0 priority 100
80000007 dd= 0 locks held 0 write locks 0 pid/thread 22992/139759426926336
flags 0 priority 100
8007dd08 dd= 0 locks held 2 write locks 0 pid/thread 22992/139759359784704
flags 0 priority 100
8007dd08 READ 1 HELD 0x652a90 len: 9 data: 010000000000000000
8007dd08 READ 1 HELD dn2id.bdb page 3
8007dd09 dd= 0 locks held 2 write locks 0 pid/thread 22992/139759426926336
flags 0 priority 100
8007dd09 READ 1 HELD 0x652a90 len: 9 data: 010000000000000000
8007dd09 READ 1 HELD 0x6716e8 len: 9 data: cfea8d000000000000
8007dd0a dd= 0 locks held 2 write locks 0 pid/thread 22992/139759426926336
flags 0 priority 100
8007dd0a WRITE 1 WAIT dn2id.bdb page 3
8007dd0a READ 1 HELD dn2id.bdb page 3
8007dd0a READ 1 HELD dn2id.bdb page 444591
8007dd0b dd= 0 locks held 0 write locks 0 pid/thread 22992/139759359784704
flags 0 priority 100
8007dd0b WRITE 1 WAIT dn2id.bdb page 3
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Locks grouped by object:
Locker Mode Count Status ----------------- Object ---------------
8007dd08 READ 1 HELD dn2id.bdb page 3
8007dd0a READ 1 HELD dn2id.bdb page 3
8007dd0b WRITE 1 WAIT dn2id.bdb page 3
8007dd0a WRITE 1 WAIT dn2id.bdb page 3
2 READ 1 HELD dn2id.bdb handle 0
1 READ 1 HELD id2entry.bdb handle 0
4 READ 1 HELD entryCSN.bdb handle 0
8007dd09 READ 1 HELD 0x652a90 len: 9 data: 010000000000000000
8007dd08 READ 1 HELD 0x652a90 len: 9 data: 010000000000000000
9 READ 1 HELD reqResult.bdb handle 0
8 READ 1 HELD reqEnd.bdb handle 0
8007dd0a READ 1 HELD dn2id.bdb page 444591
7 READ 1 HELD reqStart.bdb handle 0
8007dd09 READ 1 HELD 0x6716e8 len: 9 data: cfea8d000000000000
6 READ 1 HELD objectClass.bdb handle 0
====
#pid/thread 22992/139759426926336:Thread 7f1c46fef700 - accesslog_purge()
#pid/thread 22992/139759359784704:Thread 7f1c42fe7700 - accesslog_responce()
I am going to consider the tentative fix for openldap because I anticipate it
would take so long time to pursue the fix for bdb(hdb),
The tentative fix is like the following.
Would you let me know whethere or not there are any concern about the fix?
==[accesslog_purge() in accesslog.c]==
for (i=0; i<pd.used; i++) {
op->o_req_dn = pd.dn[i];
op->o_req_ndn = pd.ndn[i];
if ( !slapd_shutdown ) {
ldap_pvt_thread_mutex_lock( &li->li_log_mutex
);
rs_reinit( &rs, REP_RESULT );
op->o_bd->be_delete( op, &rs );
ldap_pvt_thread_mutex_unlock( &li->li_log_mutex
);
}
ch_free( pd.ndn[i].bv_val );
ch_free( pd.dn[i].bv_val );
ldap_pvt_thread_pool_pausecheck( &connection_pool );
}
====
I know openldap itself currently recommend NOT using hdb/bdb as backend
database.
However, I should seek any workaround in the hdb backend case.
Best Regards,
6 years
