https://bugs.openldap.org/show_bug.cgi?id=10513
Issue ID: 10513 Summary: slapd segfault during shutdown when syncrepl is in retry mode Product: OpenLDAP Version: 2.6.10 Hardware: x86_64 OS: Linux Status: UNCONFIRMED Keywords: needs_review Severity: normal Priority: --- Component: slapd Assignee: bugs@openldap.org Reporter: Marcelo.DeCastroLoebens@windriver.com Target Milestone: ---
## Problem slapd crashes with a segfault during shutdown. The crash occurs when the syncrepl provider is unreachable (syncrepl in retry mode). The issue has been observed twice under the same conditions.
## How to Reproduce 1. Configure slapd with syncrepl+TLS pointing to an unreachable host 2. Start slapd (syncrepl enters retry loop) 3. Send SIGTERM to slapd
The crash is a race condition and does not occur often, but is reproducible with repeated attempts.
## Environment - OpenLDAP: 2.6.10 - OS: Debian 13 (Trixie), Linux 6.12, amd64 - OpenSSL: 3.5.1 - glibc: 2.41
## Configuration slapd with back-mdb, syncrepl configured in refreshAndPersist mode with TLS, pointing to a provider that is unreachable (syncrepl in retry loop). mirrormode enabled, syncprov overlay loaded.
Relevant slapd.conf excerpt:
database mdb directory /var/lib/ldap
syncrepl rid=000 provider=ldaps://peer-host tls_reqsan=demand type=refreshAndPersist retry="5 5 300 +" searchbase="dc=example,dc=local" bindmethod=simple binddn="cn=admin,dc=example,dc=local" credentials=secret
mirrormode TRUE overlay syncprov
## Stack Traces
Two threads crash simultaneously: Two threads crash simultaneously:
Thread 1 (main thread, PID 6946): #0 0x00007fe467381d91 (libcrypto.so.3 + 0x137d91) #1 0x00007fe467382303 (libcrypto.so.3 + 0x138303) #2 0x00007fe467382133 (libcrypto.so.3 + 0x138133) #3 0x00007fe4673822a5 (libcrypto.so.3 + 0x1382a5) #4 0x00007fe467382133 (libcrypto.so.3 + 0x138133) #5 0x00007fe467382303 (libcrypto.so.3 + 0x138303) #6 0x00007fe467382133 (libcrypto.so.3 + 0x138133) #7 0x00007fe467382209 ASN1_item_free (libcrypto.so.3 + 0x138209) #8 0x00007fe467634ce9 X509_OBJECT_free (libcrypto.so.3 + 0x3eace9) #9 0x00007fe467601970 OPENSSL_sk_pop_free (libcrypto.so.3 + 0x3b7970) #10 0x00007fe46763528d X509_STORE_free (libcrypto.so.3 + 0x3eb28d) #11 0x00007fe4678c914e SSL_CTX_free (libssl.so.3 + 0x4614e) #12 0x0000563ee1c599b1 bindconf_free (/usr/sbin/slapd + 0x3d9b1) #13 0x0000563ee1ccf0e2 syncinfo_free (/usr/sbin/slapd + 0xb30e2) #14 0x0000563ee1c70d1c backend_destroy_one (/usr/sbin/slapd + 0x54d1c) #15 0x0000563ee1c70e6a backend_destroy (/usr/sbin/slapd + 0x54e6a) #16 0x0000563ee1c931c1 slap_destroy (/usr/sbin/slapd + 0x771c1) #17 0x0000563ee1c444c3 main (/usr/sbin/slapd + 0x284c3) #18 0x00007fe4679bbca8 (libc.so.6 + 0x29ca8) #19 0x00007fe4679bbd65 __libc_start_main (libc.so.6 + 0x29d65) #20 0x0000563ee1c45431 _start (/usr/sbin/slapd + 0x29431)
Thread 2 (worker, TID 6948): #0 0x00007fe466dc9ae9 (back_mdb.so.2 + 0x29ae9) #1 0x00007fe467a22271 (libc.so.6 + 0x90271) #2 0x00007fe467a24a4e (libc.so.6 + 0x92a4e) #3 0x00007fe467aa27b8 (libc.so.6 + 0x1107b8)
Kernel log: slapd[6948]: segfault at 7fe467c900c8 ip 00007fe466dc9ae9 sp 00007fe425bf4e80 error 7 in back_mdb.so.2.0.200
https://bugs.openldap.org/show_bug.cgi?id=10513
--- Comment #1 from Howard Chu hyc@openldap.org --- Looks like more of an OpenSSL bug.
Have you reproduced this on a current OpenLDAP release?
Not sure it's worth investigating since the server is already shutting down.
https://bugs.openldap.org/show_bug.cgi?id=10513
--- Comment #2 from Marcelo.DeCastroLoebens@windriver.com --- Hello, thanks for the prompt answer:
- Regarding "Looks like more of an OpenSSL bug."
The segfault is in Thread 2 (back_mdb.so), not in OpenSSL. From kernel info: slapd[6948]: segfault at 7fe467c900c8 ip 00007fe466dc9ae9 in back_mdb.so.2
- "Have you reproduced this on a current OpenLDAP release?" Unfortunately, I'm currently stuck with Debian Trixie's stable version (2.6.10). I can pick a patch if a code change is identified, but not upversion to 2.6.13.
- Regarding "not worth investigating" From my side, when the issue occurs, the crash generates a coredump, which triggers monitoring alerts and fails system health checks. While I think a clean shutdown shouldn't segfault (i.e. I believe this to be a genuine issue), I understand that the impact is minor from your perspective.
https://bugs.openldap.org/show_bug.cgi?id=10513
--- Comment #3 from Howard Chu hyc@openldap.org --- A more detailed backtrace with full debug symbols would be helpful.
On a fresh build of 2.6.10 using this slapd.conf:
tests> cat testrun/slapd.4.conf
include ./schema/core.schema include ./schema/cosine.schema include ./schema/inetorgperson.schema include ./schema/openldap.schema include ./schema/nis.schema
modulepath ../servers/slapd/overlays/ moduleload syncprov.la modulepath ../servers/slapd/back-ldap/ moduleload back_ldap.la
TLSCertificateFile localhost.crt TLSCertificateKeyFile localhost.key TLSCACertificateFile cacert.pem
database mdb suffix "dc=example,dc=com" rootdn "cn=consumer,dc=example,dc=com" rootpw secret directory testrun/db.4.a index objectClass eq index cn,sn,uid pres,eq,sub index entryUUID,entryCSN eq
syncrepl rid=1 provider=ldaps://localhost:9011/ binddn="cn=Manager,dc=example,dc=com" bindmethod=simple credentials=secret tls_reqsan=demand searchbase="dc=example,dc=com" type=refreshAndPersist retry="5 5 300 +"
mirrormode true overlay syncprov
database monitor ####
And issuing this command: ../servers/slapd/slapd -f testrun/slapd.4.conf -h ldap://:9014 -s0 -dsync & sleep 4 ; kill %
I just see a normal shutdown. You'll need to provide detailed exact steps to reproduce it.
https://bugs.openldap.org/show_bug.cgi?id=10513
--- Comment #4 from Marcelo.DeCastroLoebens@windriver.com --- Created attachment 1151 --> https://bugs.openldap.org/attachment.cgi?id=1151&action=edit Backtrace with symbols
Our reproduction happens during a system shutdown, where we have syncrepl enabled but the target host is not available.
The issue is not frequent, it will be challenging to reproduce it at will. We saw it 3 times in total, running daily sanity tests in the span of ~3 months.
https://bugs.openldap.org/show_bug.cgi?id=10513
Howard Chu hyc@openldap.org changed:
What |Removed |Added ---------------------------------------------------------------------------- Status|UNCONFIRMED |RESOLVED Resolution|--- |DUPLICATE
--- Comment #5 from Howard Chu hyc@openldap.org --- Based on the backtrace, this appears to be the same as ITS#8901
*** This issue has been marked as a duplicate of issue 8901 ***
https://bugs.openldap.org/show_bug.cgi?id=10513
Quanah Gibson-Mount quanah@openldap.org changed:
What |Removed |Added ---------------------------------------------------------------------------- Keywords|needs_review | Status|RESOLVED |VERIFIED