https://bugs.openldap.org/show_bug.cgi?id=10096
Issue ID: 10096 Summary: SIGSEGV in try_read1msg() while using referrals with AD Product: OpenLDAP Version: 2.6.2 Hardware: x86_64 OS: Linux Status: UNCONFIRMED Keywords: needs_review Severity: normal Priority: --- Component: libraries Assignee: bugs@openldap.org Reporter: simon.pichugin@gmail.com Target Milestone: ---
The double free happens during a referral chasing with AD.
SSSD usage Background (But I think the issue can happen even without SSSD): Referral chasing with AD and Kerberos based GSSAPI/GSS-SPNEGO authentication will never work based in the fact that AD will return domain names instead of the names of AD DC in the referral. That with with 'id_provider = ad' (SSSD setting) there is 'ldap_referrals = false' as default. For 'id_provider = ldap' we expect a generic LDAP server (not AD) which returns proper referrals with fully-qualified hostname or where is simple bind is used, either anonymous or with bind DN and password (which is expected to be the same on all involved LDAP servers and which is not the case with AD since the AD DC from a different domain won't know the given bind DN).
So the issue is an unusual one, but still, as it's a crash, I think it deserves a look at. The stack trace for the issue:
#0 __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44 #1 0x00007f34182a15b3 in __pthread_kill_internal (signo=6, threadid=<optimized out>) at pthread_kill.c:78 #2 0x00007f3418254d46 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26 #3 0x00007f34182287f3 in __GI_abort () at abort.c:79 #4 0x00007f3418229130 in __libc_message (fmt=<optimized out>, fmt@entry=0x7f34183bb6bd "%s\n") at ../sysdeps/posix/libc_fatal.c:150 #5 0x00007f34182ab617 in malloc_printerr (str=str@entry=0x7f34183be2d0 "double free or corruption (!prev)") at malloc.c:5515 #6 0x00007f34182ad30c in _int_free (av=0x7f34183fac80 <main_arena>, p=0x560fc4b23130, have_lock=<optimized out>) at malloc.c:4458 #7 0x00007f34182af955 in __GI___libc_free (mem=<optimized out>) at malloc.c:3258 #8 0x00007f34173b6f52 in try_read1msg (result=0x7ffea107bb88, lc=0x560fc4ac20e0, all=0, msgid=-1, ld=0x560fc4a77450) at /usr/src/debug/openldap-2.6.2-3.el9.x86_64/openldap-2.6.2/libraries/libldap/result.c:919 #9 wait4msg (result=0x7ffea107bb88, timeout=<optimized out>, all=0, msgid=-1, ld=0x560fc4a77450) at /usr/src/debug/openldap-2.6.2-3.el9.x86_64/openldap-2.6.2/libraries/libldap/result.c:369 #10 ldap_result (ld=0x560fc4a77450, msgid=msgid@entry=-1, all=all@entry=0, timeout=timeout@entry=0x7ffea107bb90, result=result@entry=0x7ffea107bb88) at /usr/src/debug/openldap-2.6.2-3.el9.x86_64/openldap-2.6.2/libraries/libldap/result.c:120 #11 0x00007f3415b6f7be in sdap_process_result (ev=0x560fc49e2420, pvt=<optimized out>) at src/providers/ldap/sdap_async.c:233 #12 0x00007f3418433675 in tevent_common_invoke_fd_handler (fde=fde@entry=0x560fc4b07ce0, flags=1, removed=removed@entry=0x0) at ../../tevent_fd.c:142 #13 0x00007f34184373cf in epoll_event_loop (tvalp=0x7ffea107bc30, epoll_ev=0x560fc49e2700) at ../../tevent_epoll.c:737 #14 epoll_event_loop_once (ev=<optimized out>, location=<optimized out>) at ../../tevent_epoll.c:938 #15 0x00007f341842f6ab in std_event_loop_once (ev=0x560fc49e2420, location=0x7f3418575f10 "src/util/server.c:787") at ../../tevent_standard.c:110 #16 0x00007f3418431b88 in _tevent_loop_once (ev=ev@entry=0x560fc49e2420, location=location@entry=0x7f3418575f10 "src/util/server.c:787") at ../../tevent.c:825 #17 0x00007f3418431c7b in tevent_common_loop_wait (ev=0x560fc49e2420, location=0x7f3418575f10 "src/util/server.c:787") at ../../tevent.c:948 #18 0x00007f341842f71b in std_event_loop_wait (ev=0x560fc49e2420, location=0x7f3418575f10 "src/util/server.c:787") at ../../tevent_standard.c:141 #19 0x00007f3418552237 in server_loop (main_ctx=0x560fc49e2790) at src/util/server.c:787 #20 0x0000560fc371ebda in main (argc=8, argv=<optimized out>) at src/providers/data_provider_be.c:811
(gdb) frame 11 #11 0x00007f3415b6f7be in sdap_process_result (ev=0x560fc49e2420, pvt=<optimized out>) at src/providers/ldap/sdap_async.c:233 233 ret = ldap_result(sh->ldap, LDAP_RES_ANY, 0, &no_timeout, &msg);
(gdb) frame 10 #10 ldap_result (ld=0x560fc4a77450, msgid=msgid@entry=-1, all=all@entry=0, timeout=timeout@entry=0x7ffea107bb90, result=result@entry=0x7ffea107bb88) at /usr/src/debug/openldap-2.6.2-3.el9.x86_64/openldap-2.6.2/libraries/libldap/result.c:120 120 rc = wait4msg( ld, msgid, all, timeout, result );
(gdb) frame 9 #9 wait4msg (result=0x7ffea107bb88, timeout=<optimized out>, all=0, msgid=-1, ld=0x560fc4a77450) at /usr/src/debug/openldap-2.6.2-3.el9.x86_64/openldap-2.6.2/libraries/libldap/result.c:369 369 rc = try_read1msg( ld, msgid, all, lc, result );
(gdb) frame 8 #8 0x00007f34173b6f52 in try_read1msg (result=0x7ffea107bb88, lc=0x560fc4ac20e0, all=0, msgid=-1, ld=0x560fc4a77450) at /usr/src/debug/openldap-2.6.2-3.el9.x86_64/openldap-2.6.2/libraries/libldap/result.c:919 919 ldap_return_request( ld, lr, 0 );
(gdb) frame 7 #7 0x00007f34182af955 in __GI___libc_free (mem=<optimized out>) at malloc.c:3258 3258 _int_free (ar_ptr, p, 0);
(gdb) frame 6 #6 0x00007f34182ad30c in _int_free (av=0x7f34183fac80 <main_arena>, p=0x560fc4b23130, have_lock=<optimized out>) at malloc.c:4458 4458 malloc_printerr ("double free or corruption (!prev)");
(gdb) frame 8 #8 0x00007f34173b6f52 in try_read1msg (result=0x7ffea107bb88, lc=0x560fc4ac20e0, all=0, msgid=-1, ld=0x560fc4a77450) at /usr/src/debug/openldap-2.6.2-3.el9.x86_64/openldap-2.6.2/libraries/libldap/result.c:919 919 ldap_return_request( ld, lr, 0 ); (gdb) list 914 } 915 } 916 917 if ( lr != NULL ) { 918 if ( lr != &dummy_lr ) { 919 ldap_return_request( ld, lr, 0 ); 920 } 921 lr = NULL; 922 } 923 (gdb) p *ld $6 = {ldc = 0x560fc4b01110, ld_errno = 10, ld_error = 0x0, ld_matched = 0x0, ld_referrals = 0x0} (gdb) p *lr $8 = {lr_msgid = -995099056, lr_status = 22031, lr_refcnt = -995206016, lr_outrefcnt = 22030, lr_abandoned = 0, lr_origid = 40, lr_parentcnt = 4, lr_res_msgtype = 101, lr_res_errno = 0, lr_res_error = 0x0, lr_res_matched = 0x0, lr_ber = 0x0, lr_conn = 0x560fc4ac20e0, lr_dn = {bv_len = 20, bv_val = 0x560fc4affe6b "\304\017V"}, lr_parent = 0x0, lr_child = 0x560fc4adb650, lr_refnext = 0x0, lr_prev = 0x0, lr_next = 0x90}
Thank you! Simon
https://bugs.openldap.org/show_bug.cgi?id=10096
--- Comment #1 from Ondřej Kuzník ondra@mistotebe.net --- On Mon, Aug 28, 2023 at 08:58:28PM +0000, openldap-its@openldap.org wrote:
The double free happens during a referral chasing with AD.
SSSD usage Background (But I think the issue can happen even without SSSD): Referral chasing with AD and Kerberos based GSSAPI/GSS-SPNEGO authentication will never work based in the fact that AD will return domain names instead of the names of AD DC in the referral. That with with 'id_provider = ad' (SSSD setting) there is 'ldap_referrals = false' as default. For 'id_provider = ldap' we expect a generic LDAP server (not AD) which returns proper referrals with fully-qualified hostname or where is simple bind is used, either anonymous or with bind DN and password (which is expected to be the same on all involved LDAP servers and which is not the case with AD since the AD DC from a different domain won't know the given bind DN).
So the issue is an unusual one, but still, as it's a crash, I think it deserves a look at.
Hi Simon, given you can repro it and seems you're suggesting it's hard to reproduce without an AD, can you: - try running it under valgrind's memcheck and post any errors reported, tracing memory origins as well (I suspect the request has been freed already) - enable libldap's TRACE logging and post the logs here?
That's assuming you can't wrap something up that we could use to reproduce the issue ourselves, that would obviously be preferable.
Thanks,
https://bugs.openldap.org/show_bug.cgi?id=10096
--- Comment #2 from Simon Pichugin simon.pichugin@gmail.com --- (In reply to Ondřej Kuzník from comment #1)
Hi Simon, given you can repro it and seems you're suggesting it's hard to reproduce without an AD, can you:
- try running it under valgrind's memcheck and post any errors reported, tracing memory origins as well (I suspect the request has been freed already)
- enable libldap's TRACE logging and post the logs here?
That's assuming you can't wrap something up that we could use to reproduce the issue ourselves, that would obviously be preferable.
Thanks,
Hi Ondřej, Unfortunately, the source's a customer case, and it's closed now (as the correct settings are 'ldap_referrals = false' for AD config). Hence, I wasn't able to extract more information.
I've tried to debug it myself first, but the most suspicious place I found is this one: https://git.openldap.org/openldap/openldap/-/blob/master/libraries/libldap/r...
But I'm not sure how we could end up there when we chase the referral while it's AD (and somehow a parent is involved?).
Valgrind and debug logging was my next suggestion, but the work with the customer was successfully finished.
So unless you have more ideas looking at the stack trace, I'm not sure how to proceed, sorry...
https://bugs.openldap.org/show_bug.cgi?id=10096
Quanah Gibson-Mount quanah@openldap.org changed:
What |Removed |Added ---------------------------------------------------------------------------- Resolution|--- |SUSPENDED Status|UNCONFIRMED |RESOLVED Keywords|needs_review |
--- Comment #3 from Quanah Gibson-Mount quanah@openldap.org --- Suspending until there is a path to reproduction.
-
openldap-its＠openldap.org