[Issue 10330] New: TIMEOUT and NETWORK_TIMEOUT not respected when receiving bad data during TLS negotiation
https://bugs.openldap.org/show_bug.cgi?id=10330
Issue ID: 10330 Summary: TIMEOUT and NETWORK_TIMEOUT not respected when receiving bad data during TLS negotiation Product: OpenLDAP Version: 2.6.9 Hardware: x86_64 OS: Linux Status: UNCONFIRMED Keywords: needs_review Severity: normal Priority: --- Component: libraries Assignee: bugs@openldap.org Reporter: michael.kourlas@solace.com Target Milestone: ---
Created attachment 1063 --> https://bugs.openldap.org/attachment.cgi?id=1063&action=edit Test program
This seems related to bug 8047.
Steps to reproduce:
1. Setup a netcat server on a host: "nc -l -k -p 636".
2. On a different host, attempt to connect to the host running "nc" as if it were an LDAP server via ldaps: "ldapsearch -o NETWORK_TIMEOUT=5 -o TIMEOUT=5 -H ldaps://<ip>:636".
3. During the 5 second timeout period, switch back to the netcat server and transmit a newline by pressing enter.
4. ldapsearch will hang forever until the TCP connection is closed (e.g. by killing the netcat server).
My expectation would be that ldapsearch would exit after 5 seconds, per the NETWORK_TIMEOUT and TIMEOUT options.
I'm using the following version of ldapsearch on Fedora 41 (x86-64):
ldapsearch: @(#) $OpenLDAP: ldapsearch 2.6.9 (Mar 27 2025 00:00:00) $ openldap (LDAP library: OpenLDAP 20609)
This problem is also observable when directly using the OpenLDAP C API. This is more of an issue, since any application using the API could become unresponsive if these timeout values aren't respected.
I've attached a short test program which can be used instead of ldapsearch. If I abort the test program while it is stuck in this state, the traceback looks like this:
#0 0x00007f50b7a25811 in __GI___libc_read (fd=3, buf=0x2aaf9ac5, nbytes=3) at ../sysdeps/unix/sysv/linux/read.c:26 #1 0x00007f50b792f8b9 in sb_debug_read (sbiod=0x2aadf390, buf=0x2aaf9ac5, len=3) at /usr/src/debug/openldap-2.6.9-1.fc41.x86_64/openldap-2.6.9/libraries/liblber/sockbuf.c:829 #2 0x00007f50b7b61156 in tlso_bio_read (b=0x2aadf9c0, buf=0x2aaf9ac5 "", len=3) at /usr/src/debug/openldap-2.6.9-1.fc41.x86_64/openldap-2.6.9/libraries/libldap/tls_o.c:1279 #3 0x00007f50b7221ea3 in bread_conv (bio=<optimized out>, data=<optimized out>, datal=<optimized out>, readbytes=0x7ffd1cf2e3f0) at crypto/bio/bio_meth.c:121 #4 0x00007f50b7226047 in bio_read_intern (b=b@entry=0x2aadf9c0, data=0x2aaf9ac5, data@entry=0x555f3588, dlen=3, dlen@entry=18446744072487067717, readbytes=readbytes@entry=0x7ffd1cf2e3f0) at crypto/bio/bio_lib.c:285 #5 0x00007f50b72261db in BIO_read (b=0x2aadf9c0, data=0x555f3588, dlen=-1222483899) at crypto/bio/bio_lib.c:311 #6 BIO_read (b=b@entry=0x2aadf9c0, data=data@entry=0x2aaf9ac5, dlen=dlen@entry=3) at crypto/bio/bio_lib.c:303 #7 0x00007f50b783ce03 in tls_default_read_n (rl=0x2aaec1c0, n=5, max=<optimized out>, extend=<optimized out>, clearold=<optimized out>, readbytes=0x7ffd1cf2e4b8) at ssl/record/methods/tls_common.c:406 #8 0x00007f50b784151b in tls_get_more_records (rl=0x2aaec1c0) at ssl/record/methods/tls_common.c:583 #9 0x00007f50b783b8ea in tls_read_record (rl=0x2aaec1c0, rechandle=0x2aaeb600, rversion=0x2aaeb608, type=0x2aaeb60c "", data=0x2aaeb610, datalen=0x2aaeb620, epoch=0x0, seq_num=0x0) at ssl/record/methods/tls_common.c:1130 #10 0x00007f50b783969a in ssl3_read_bytes (ssl=<optimized out>, type=22 '\026', recvd_type=0x7ffd1cf2e684 "", buf=0x2aaee480 "\001", len=4, peek=0, readbytes=0x7ffd1cf2e688) at ssl/record/rec_layer_s3.c:689 #11 0x00007f50b784f5a7 in tls_get_message_header (s=0x2aaea980, mt=<synthetic pointer>) at ssl/statem/statem_lib.c:1554 --Type <RET> for more, q to quit, c to continue without paging-- #12 read_state_machine (s=0x2aaea980) at ssl/statem/statem.c:625 #13 state_machine (s=<optimized out>, server=0) at ssl/statem/statem.c:479 #14 0x00007f50b7b615c3 in tlso_session_connect (ld=<optimized out>, sess=0x2aaea980, name_in=<optimized out>) at /usr/src/debug/openldap-2.6.9-1.fc41.x86_64/openldap-2.6.9/libraries/libldap/tls_o.c:693 #15 0x00007f50b7b65bf2 in ldap_int_tls_connect (ld=ld@entry=0x2a9b2430, conn=conn@entry=0x2a9b25d0, host=host@entry=0x2a9b2550 "192.168.133.56") at /usr/src/debug/openldap-2.6.9-1.fc41.x86_64/openldap-2.6.9/libraries/libldap/tls2.c:425 #16 0x00007f50b7b6636f in ldap_int_tls_start (ld=0x2a9b2430, conn=0x2a9b25d0, srv=<optimized out>) at /usr/src/debug/openldap-2.6.9-1.fc41.x86_64/openldap-2.6.9/libraries/libldap/tls2.c:1245 #17 0x00007f50b7b3d4c2 in ldap_int_open_connection (ld=0x2a9b2430, conn=0x2a9b25d0, srv=0x2a9b24d0, async=0) at /usr/src/debug/openldap-2.6.9-1.fc41.x86_64/openldap-2.6.9/libraries/libldap/open.c:515 #18 0x00007f50b7b5212d in ldap_new_connection (ld=0x2a9b2430, srvlist=0x2a9b2878, use_ldsb=1, connect=<optimized out>, bind=0x0, m_req=0, m_res=0) at /usr/src/debug/openldap-2.6.9-1.fc41.x86_64/openldap-2.6.9/libraries/libldap/request.c:491 #19 0x00007f50b7b3c7b4 in ldap_open_defconn (ld=0x2a9b2430) at /usr/src/debug/openldap-2.6.9-1.fc41.x86_64/openldap-2.6.9/libraries/libldap/open.c:42 #20 0x00007f50b7b52ed8 in ldap_send_initial_request (ld=0x2a9b2430, msgtype=96, dn=0x4023a9 "cn=admin,dc=solace,dc=com", ber=0x2a9b2570, msgid=1) at /usr/src/debug/openldap-2.6.9-1.fc41.x86_64/openldap-2.6.9/libraries/libldap/request.c:131 #21 0x00007f50b7b429b9 in ldap_sasl_bind (ld=0x2a9b2430, dn=0x4023a9 "cn=admin,dc=solace,dc=com", mechanism=0x0, cred=0x7ffd1cf2eb90, sctrls=0x0, cctrls=0x0, msgidp=0x7ffd1cf2eb8c) at /usr/src/debug/openldap-2.6.9-1.fc41.x86_64/openldap-2.6.9/libraries/libldap/sasl.c:164
https://bugs.openldap.org/show_bug.cgi?id=10330
--- Comment #1 from Michael Kourlas michael.kourlas@solace.com --- Created attachment 1064 --> https://bugs.openldap.org/attachment.cgi?id=1064&action=edit Test program (LDAP_OPT_CONNECT_ASYNC)
This behaviour can also be reproduced without sending any data from the nc server during the TLS negotiation.
To reproduce it consistently in this context, I have to configure the test application to use LDAP_OPT_CONNECT_ASYNC. The revised test application hangs with this traceback:
#0 0x00007f16f76b3811 in __GI___libc_read (fd=3, buf=0x34307b73, nbytes=5) at ../sysdeps/unix/sysv/linux/read.c:26 #1 0x00007f16f75bd8b9 in sb_debug_read (sbiod=0x342ed440, buf=0x34307b73, len=5) at /usr/src/debug/openldap-2.6.9-1.fc41.x86_64/openldap-2.6.9/libraries/liblber/sockbuf.c:829 #2 0x00007f16f77ef156 in tlso_bio_read (b=0x342eda70, buf=0x34307b73 "", len=5) at /usr/src/debug/openldap-2.6.9-1.fc41.x86_64/openldap-2.6.9/libraries/libldap/tls_o.c:1279 #3 0x00007f16f6e21ea3 in bread_conv (bio=<optimized out>, data=<optimized out>, datal=<optimized out>, readbytes=0x7fff73b5d730) at crypto/bio/bio_meth.c:121 #4 0x00007f16f6e26047 in bio_read_intern (b=b@entry=0x342eda70, data=0x34307b73, data@entry=0x6860f6e6, dlen=5, dlen@entry=18446744073556615239, readbytes=readbytes@entry=0x7fff73b5d730) at crypto/bio/bio_lib.c:285 #5 0x00007f16f6e261db in BIO_read (b=0x342eda70, data=0x6860f6e6, dlen=-152936377) at crypto/bio/bio_lib.c:311 #6 BIO_read (b=b@entry=0x342eda70, data=data@entry=0x34307b73, dlen=dlen@entry=5) at crypto/bio/bio_lib.c:303 #7 0x00007f16f74cae03 in tls_default_read_n (rl=0x342fa270, n=5, max=<optimized out>, extend=<optimized out>, clearold=<optimized out>, readbytes=0x7fff73b5d7f8) at ssl/record/methods/tls_common.c:406 #8 0x00007f16f74cf51b in tls_get_more_records (rl=0x342fa270) at ssl/record/methods/tls_common.c:583 #9 0x00007f16f74c98ea in tls_read_record (rl=0x342fa270, rechandle=0x342f96b0, rversion=0x342f96b8, type=0x342f96bc "", data=0x342f96c0, datalen=0x342f96d0, epoch=0x0, seq_num=0x0) at ssl/record/methods/tls_common.c:1130 #10 0x00007f16f74c769a in ssl3_read_bytes (ssl=<optimized out>, type=22 '\026', recvd_type=0x7fff73b5d9c4 "\001", buf=0x342fc530 "\001", len=4, peek=0, readbytes=0x7fff73b5d9c8) at ssl/record/rec_layer_s3.c:689 #11 0x00007f16f74dd5a7 in tls_get_message_header (s=0x342f8a30, mt=<synthetic pointer>) at ssl/statem/statem_lib.c:1554 #12 read_state_machine (s=0x342f8a30) at ssl/statem/statem.c:625 #13 state_machine (s=<optimized out>, server=0) at ssl/statem/statem.c:479 #14 0x00007f16f77ef5c3 in tlso_session_connect (ld=<optimized out>, sess=0x342f8a30, name_in=<optimized out>) at /usr/src/debug/openldap-2.6.9-1.fc41.x86_64/openldap-2.6.9/libraries/libldap/tls_o.c:693 #15 0x00007f16f77f3bf2 in ldap_int_tls_connect (ld=ld@entry=0x341c0430, conn=conn@entry=0x341c05d0, host=host@entry=0x341c0460 "192.168.133.56") at /usr/src/debug/openldap-2.6.9-1.fc41.x86_64/openldap-2.6.9/libraries/libldap/tls2.c:425 #16 0x00007f16f77f41b4 in ldap_int_tls_start (ld=0x341c0430, conn=0x341c05d0, srv=<optimized out>) at /usr/src/debug/openldap-2.6.9-1.fc41.x86_64/openldap-2.6.9/libraries/libldap/tls2.c:1179 #17 0x00007f16f77cb65c in ldap_int_check_async_open (sd=<optimized out>, ld=0x341c0430) at /usr/src/debug/openldap-2.6.9-1.fc41.x86_64/openldap-2.6.9/libraries/libldap/open.c:673 #18 ldap_int_check_async_open (ld=0x341c0430, sd=<optimized out>) at /usr/src/debug/openldap-2.6.9-1.fc41.x86_64/openldap-2.6.9/libraries/libldap/open.c:645 #19 0x00007f16f77e0f5b in ldap_send_initial_request (ld=0x341c0430, msgtype=96, dn=0x4023de "cn=admin,dc=solace,dc=com", ber=0x341bdd10, msgid=1322) at /usr/src/debug/openldap-2.6.9-1.fc41.x86_64/openldap-2.6.9/libraries/libldap/request.c:138 #20 0x00007f16f77d09b9 in ldap_sasl_bind (ld=0x341c0430, dn=0x4023de "cn=admin,dc=solace,dc=com", mechanism=0x0, cred=0x7fff73b5ddf0, sctrls=0x0, cctrls=0x0, msgidp=0x7fff73b5ddec) at /usr/src/debug/openldap-2.6.9-1.fc41.x86_64/openldap-2.6.9/libraries/libldap/sasl.c:164 #21 0x000000000040138d in main ()
https://bugs.openldap.org/show_bug.cgi?id=10330
--- Comment #2 from Ondřej Kuzník ondra@mistotebe.net --- On Thu, Apr 24, 2025 at 09:39:52PM +0000, openldap-its@openldap.org wrote:
This behaviour can also be reproduced without sending any data from the nc server during the TLS negotiation.
To reproduce it consistently in this context, I have to configure the test application to use LDAP_OPT_CONNECT_ASYNC. The revised test application hangs with this traceback:
Hi Michael, yes, this is related to ITS#8047, the fix there is partial, it allowed the initial connection set up to honour timeouts properly, but as acknowledged in there, we don't have an API that lets you do an async TLS handshake like this.
Your only option right now seems to be doing what lloadd does, make the socket non-blocking and manage this internally through ber_sockbuf_ctrl(sb, LBER_SB_OPT_NEEDS_READ/WRITE) + epoll().
If you want to contribute the polling inside libldap mid-handshake (handling the LDAP_OPT_CONNECT_ASYNC on/off cases), we would certainly consider it for inclusion in 2.7.
Thanks,
https://bugs.openldap.org/show_bug.cgi?id=10330
Quanah Gibson-Mount quanah@openldap.org changed:
What |Removed |Added ---------------------------------------------------------------------------- Keywords|needs_review | Target Milestone|--- |2.7.0
https://bugs.openldap.org/show_bug.cgi?id=10330
--- Comment #3 from Michael Kourlas michael.kourlas@solace.com --- Created attachment 1074 --> https://bugs.openldap.org/attachment.cgi?id=1074&action=edit 0001-ITS-10209-do-not-poll-socket-in-ldap_int_tls_start-i.patch
The attached modifications to OpenLDAP Software are subject to the following notice: Copyright 2025 Solace Corporation Redistribution and use in source and binary forms, with or without modification, are permitted only as authorized by the OpenLDAP Public License.
https://bugs.openldap.org/show_bug.cgi?id=10330
--- Comment #4 from Michael Kourlas michael.kourlas@solace.com --- Created attachment 1075 --> https://bugs.openldap.org/attachment.cgi?id=1075&action=edit 0002-ITS-10209-keep-socket-non-blocking-during-polling-in.patch
The attached modifications to OpenLDAP Software are subject to the following notice: Copyright 2025 Solace Corporation Redistribution and use in source and binary forms, with or without modification, are permitted only as authorized by the OpenLDAP Public License.
https://bugs.openldap.org/show_bug.cgi?id=10330
--- Comment #5 from Michael Kourlas michael.kourlas@solace.com --- I've submitted a couple of patches in the prior two comments. These patches address the bug described in the original description. Unfortunately, they do not address the issue observed with LDAP_OPT_CONNECT_ASYNC.
Below is the required notice for the patches:
The patches in the prior two comments are derived from OpenLDAP Software. All of the modifications to OpenLDAP Software represented in the patches were developed by Solace Corporation. Solace Corporation has not assigned rights and/or interest in this work to any party. I, Michael Kourlas, am authorized by Solace Corporation, my employer, to release this work under the terms listed in the prior comments.
https://bugs.openldap.org/show_bug.cgi?id=10330
--- Comment #6 from Ondřej Kuzník ondra@mistotebe.net --- On Fri, May 30, 2025 at 10:08:30PM +0000, openldap-its@openldap.org wrote:
I've submitted a couple of patches in the prior two comments. These patches address the bug described in the original description. Unfortunately, they do not address the issue observed with LDAP_OPT_CONNECT_ASYNC.
Below is the required notice for the patches:
The patches in the prior two comments are derived from OpenLDAP Software. All of the modifications to OpenLDAP Software represented in the patches were developed by Solace Corporation. Solace Corporation has not assigned rights and/or interest in this work to any party. I, Michael Kourlas, am authorized by Solace Corporation, my employer, to release this work under the terms listed in the prior comments.
Hi Michael, thanks for the patches, they look good to me. I've adjusted the commit messages (they mentioned ITS#10209 for some reason?) and forwarded them to Gitlab here: https://git.openldap.org/openldap/openldap/-/merge_requests/774
Thanks,
-
openldap-its＠openldap.org