https://bugs.openldap.org/show_bug.cgi?id=9362
Issue ID: 9362 Summary: slapd crashed with a segmentation fault in syncprov Product: OpenLDAP Version: 2.4.44 Hardware: All OS: Linux Status: UNCONFIRMED Severity: normal Priority: --- Component: slapd Assignee: bugs@openldap.org Reporter: simon.pichugin@gmail.com Target Milestone: ---
Description of problem: Two times on slapd crashed with segmentation faults and cores were outputted. The following messages were outputted to /var/log/messages.
-------------------- Jun 23 19:42:51 GC001CVFC101 kernel: slapd[1985]: segfault at 2 ip 00005647e0543414 sp 00007f6c088b5de0 error 4 in slapd[5647e04df000+1c0000] Jun 23 19:42:53 GC001CVFC101 systemd: slapd.service: main process exited, code=killed, status=11/SEGV Jun 23 19:42:53 GC001CVFC101 systemd: Unit slapd.service entered failed state. Jun 23 19:42:53 GC001CVFC101 systemd: slapd.service failed. ..... Jun 23 23:56:48 GC001CVFC101 kernel: slapd[8759]: segfault at 3 ip 000055d24efbc414 sp 00007fe9a8ffae80 error 4 in slapd[55d24ef58000+1c0000] Jun 23 23:56:49 GC001CVFC101 systemd: slapd.service: main process exited, code=killed, status=11/SEGV Jun 23 23:56:50 GC001CVFC101 systemd: Unit slapd.service entered failed state. Jun 23 23:56:50 GC001CVFC101 systemd: slapd.service failed. --------------------
The followings are the logs that are outputted to /var/log/slapd.log-20200624 Each segmentation fault occurred during modify and delete requests for ldap entries.
------------- Jun 23 19:42:50 GC001CVFC101 slapd[2281]: conn=7028108 op=391 MOD dn="uid=user,ou=users,dc=example,dc=com" ..... Jun 23 23:56:48 GC001CVFC101 slapd[32641]: conn=1552 op=144 DEL dn="cn=group1,ou=groups,dc=example,dc=com" -------------
The followings are the backtraces of two cores.
1) core-slapd.2281: -------------------- (gdb) bt #0 test_filter (op=op@entry=0x7f6c088b6130, e=0x7f6bee9e37c8, f=0x2) at filterentry.c:69 ^^^^^*1 #1 0x00007f6cb379e598 in syncprov_matchops (op=op@entry=0x7f6be01191d0, opc=opc@entry=0x7f6bec001710, saveit=saveit@entry=1) at syncprov.c:1334 #2 0x00007f6cb379ec63 in syncprov_op_mod (op=0x7f6be01191d0, rs=<optimized out>) at syncprov.c:2201 #3 0x00005647e0591e8a in overlay_op_walk (op=op@entry=0x7f6be01191d0, rs=0x7f6c088b7960, which=op_modify, oi=0x5647e1633dd0, on=0x5647e1638d00) at backover.c:661 #4 0x00005647e0592034 in over_op_func (op=0x7f6be01191d0, rs=<optimized out>, which=<optimized out>) at backover.c:730 #5 0x00005647e053b2f9 in fe_op_modify (op=0x7f6be01191d0, rs=0x7f6c088b7960) at modify.c:303 #6 0x00005647e053d2ed in do_modify (op=0x7f6be01191d0, rs=0x7f6c088b7960) at modify.c:177 #7 0x00005647e0522e7c in connection_operation (ctx=ctx@entry=0x7f6c088b7bd0, arg_v=arg_v@entry=0x7f6be01191d0) at connection.c:1158 #8 0x00005647e05231eb in connection_read_thread (ctx=0x7f6c088b7bd0, argv=0x67) at connection.c:1294 #9 0x00007f6cbaad82fa in ldap_int_thread_pool_wrapper () from debug/lib64/libldap_r-2.4.so.2 #10 0x00007f6cb9d9ae25 in start_thread (arg=0x7f6c088b8700) at pthread_create.c:308 #11 0x00007f6cb925c34d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113 (gdb) --------------------
2) core-slapd.32641 -------------------- (gdb) bt #0 test_filter (op=op@entry=0x7fe9a8ffb1d0, e=0x7fe9c7ea8888, f=0x3) at filterentry.c:69 ^^^^^*1 #1 0x00007feab3f22598 in syncprov_matchops (op=op@entry=0x7fe99c000950, opc=opc@entry=0x7fe99c001808, saveit=saveit@entry=1) at syncprov.c:1334 #2 0x00007feab3f22c63 in syncprov_op_mod (op=0x7fe99c000950, rs=<optimized out>) at syncprov.c:2201 #3 0x000055d24f00ae8a in overlay_op_walk (op=op@entry=0x7fe99c000950, rs=0x7fe9a8ffb960, which=op_delete, oi=0x55d250c54dd0, on=0x55d250c59d00) at backover.c:661 #4 0x000055d24f00b034 in over_op_func (op=0x7fe99c000950, rs=<optimized out>, which=<optimized out>) at backover.c:730 #5 0x000055d24efb6bf6 in fe_op_delete (op=0x7fe99c000950, rs=0x7fe9a8ffb960) at delete.c:174 #6 0x000055d24efb68d6 in do_delete (op=0x7fe99c000950, rs=0x7fe9a8ffb960) at delete.c:95 #7 0x000055d24ef9be7c in connection_operation (ctx=ctx@entry=0x7fe9a8ffbbd0, arg_v=arg_v@entry=0x7fe99c000950) at connection.c:1158 #8 0x000055d24ef9c1eb in connection_read_thread (ctx=0x7fe9a8ffbbd0, argv=0x3a) at connection.c:1294 #9 0x00007feabb25c2fa in ldap_int_thread_pool_wrapper () from debug/lib64/libldap_r-2.4.so.2 #10 0x00007feaba51ee25 in start_thread (arg=0x7fe9a8ffc700) at pthread_create.c:308 #11 0x00007feab99e034d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113 (gdb) --------------------
The two cores seem to be the same, only the difference between modify and delete request. Both are considered to cause a segmentation fault because the value of the third parameter(*1) of test_filter is invalid. The third parameter(*1) of test_filter should be an address, like the source below.
Excerpt from filterentry.c: -------------------- 60 int 61 test_filter( 62 Operation *op, 63 Entry *e, 64 Filter *f ) ** the third parameter of test_filter 65 { 66 int rc; 67 Debug( LDAP_DEBUG_FILTER, "=> test_filter\n", 0, 0, 0 ); 68 69 if ( f->f_choice & SLAPD_FILTER_UNDEFINED ) { ** referenced here 70 Debug( LDAP_DEBUG_FILTER, " UNDEFINED\n", 0, 0, 0 ); --------------------
Version-Release number of selected component (if applicable):
How reproducible: Two times
Steps to Reproduce: Unknown
Actual results: slapd crashes with a segmentation fault and a core is outputted.
Expected results: slapd doesn't crash and a core isn't outputted.
Thanks! Simon
https://bugs.openldap.org/show_bug.cgi?id=9362
--- Comment #1 from Quanah Gibson-Mount quanah@openldap.org --- Hello,
The 2.4.44 release is over 4 years old. A multitude of issues have been fixed since that release. Please upgrade to a current, stable release.
https://bugs.openldap.org/show_bug.cgi?id=9362
--- Comment #2 from Quanah Gibson-Mount quanah@openldap.org --- Description of problem: Two times on slapd crashed with segmentation faults and cores were outputted. The following messages were outputted to /var/log/messages.
-------------------- Jun 23 19:42:51 XXXX kernel: slapd[1985]: segfault at 2 ip 00005647e0543414 sp 00007f6c088b5de0 error 4 in slapd[5647e04df000+1c0000] Jun 23 19:42:53 XXXX systemd: slapd.service: main process exited, code=killed, status=11/SEGV Jun 23 19:42:53 XXXX systemd: Unit slapd.service entered failed state. Jun 23 19:42:53 XXXX systemd: slapd.service failed. ..... Jun 23 23:56:48 XXXX kernel: slapd[8759]: segfault at 3 ip 000055d24efbc414 sp 00007fe9a8ffae80 error 4 in slapd[55d24ef58000+1c0000] Jun 23 23:56:49 XXXX systemd: slapd.service: main process exited, code=killed, status=11/SEGV Jun 23 23:56:50 XXXX systemd: Unit slapd.service entered failed state. Jun 23 23:56:50 XXXX systemd: slapd.service failed. --------------------
The followings are the logs that are outputted to /var/log/slapd.log-20200624 Each segmentation fault occurred during modify and delete requests for ldap entries.
------------- Jun 23 19:42:50 XXXX slapd[2281]: conn=7028108 op=391 MOD dn="uid=user,ou=users,dc=example,dc=com" ..... Jun 23 23:56:48 XXXX slapd[32641]: conn=1552 op=144 DEL dn="cn=group1,ou=groups,dc=example,dc=com" -------------
The followings are the backtraces of two cores.
1) core-slapd.2281: -------------------- (gdb) bt #0 test_filter (op=op@entry=0x7f6c088b6130, e=0x7f6bee9e37c8, f=0x2) at filterentry.c:69 ^^^^^*1 #1 0x00007f6cb379e598 in syncprov_matchops (op=op@entry=0x7f6be01191d0, opc=opc@entry=0x7f6bec001710, saveit=saveit@entry=1) at syncprov.c:1334 #2 0x00007f6cb379ec63 in syncprov_op_mod (op=0x7f6be01191d0, rs=<optimized out>) at syncprov.c:2201 #3 0x00005647e0591e8a in overlay_op_walk (op=op@entry=0x7f6be01191d0, rs=0x7f6c088b7960, which=op_modify, oi=0x5647e1633dd0, on=0x5647e1638d00) at backover.c:661 #4 0x00005647e0592034 in over_op_func (op=0x7f6be01191d0, rs=<optimized out>, which=<optimized out>) at backover.c:730 #5 0x00005647e053b2f9 in fe_op_modify (op=0x7f6be01191d0, rs=0x7f6c088b7960) at modify.c:303 #6 0x00005647e053d2ed in do_modify (op=0x7f6be01191d0, rs=0x7f6c088b7960) at modify.c:177 #7 0x00005647e0522e7c in connection_operation (ctx=ctx@entry=0x7f6c088b7bd0, arg_v=arg_v@entry=0x7f6be01191d0) at connection.c:1158 #8 0x00005647e05231eb in connection_read_thread (ctx=0x7f6c088b7bd0, argv=0x67) at connection.c:1294 #9 0x00007f6cbaad82fa in ldap_int_thread_pool_wrapper () from debug/lib64/libldap_r-2.4.so.2 #10 0x00007f6cb9d9ae25 in start_thread (arg=0x7f6c088b8700) at pthread_create.c:308 #11 0x00007f6cb925c34d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113 (gdb) --------------------
2) core-slapd.32641 -------------------- (gdb) bt #0 test_filter (op=op@entry=0x7fe9a8ffb1d0, e=0x7fe9c7ea8888, f=0x3) at filterentry.c:69 ^^^^^*1 #1 0x00007feab3f22598 in syncprov_matchops (op=op@entry=0x7fe99c000950, opc=opc@entry=0x7fe99c001808, saveit=saveit@entry=1) at syncprov.c:1334 #2 0x00007feab3f22c63 in syncprov_op_mod (op=0x7fe99c000950, rs=<optimized out>) at syncprov.c:2201 #3 0x000055d24f00ae8a in overlay_op_walk (op=op@entry=0x7fe99c000950, rs=0x7fe9a8ffb960, which=op_delete, oi=0x55d250c54dd0, on=0x55d250c59d00) at backover.c:661 #4 0x000055d24f00b034 in over_op_func (op=0x7fe99c000950, rs=<optimized out>, which=<optimized out>) at backover.c:730 #5 0x000055d24efb6bf6 in fe_op_delete (op=0x7fe99c000950, rs=0x7fe9a8ffb960) at delete.c:174 #6 0x000055d24efb68d6 in do_delete (op=0x7fe99c000950, rs=0x7fe9a8ffb960) at delete.c:95 #7 0x000055d24ef9be7c in connection_operation (ctx=ctx@entry=0x7fe9a8ffbbd0, arg_v=arg_v@entry=0x7fe99c000950) at connection.c:1158 #8 0x000055d24ef9c1eb in connection_read_thread (ctx=0x7fe9a8ffbbd0, argv=0x3a) at connection.c:1294 #9 0x00007feabb25c2fa in ldap_int_thread_pool_wrapper () from debug/lib64/libldap_r-2.4.so.2 #10 0x00007feaba51ee25 in start_thread (arg=0x7fe9a8ffc700) at pthread_create.c:308 #11 0x00007feab99e034d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113 (gdb) --------------------
The two cores seem to be the same, only the difference between modify and delete request. Both are considered to cause a segmentation fault because the value of the third parameter(*1) of test_filter is invalid. The third parameter(*1) of test_filter should be an address, like the source below.
Excerpt from filterentry.c: -------------------- 60 int 61 test_filter( 62 Operation *op, 63 Entry *e, 64 Filter *f ) ** the third parameter of test_filter 65 { 66 int rc; 67 Debug( LDAP_DEBUG_FILTER, "=> test_filter\n", 0, 0, 0 ); 68 69 if ( f->f_choice & SLAPD_FILTER_UNDEFINED ) { ** referenced here 70 Debug( LDAP_DEBUG_FILTER, " UNDEFINED\n", 0, 0, 0 ); --------------------
Version-Release number of selected component (if applicable):
How reproducible: Two times
Steps to Reproduce: Unknown
Actual results: slapd crashes with a segmentation fault and a core is outputted.
Expected results: slapd doesn't crash and a core isn't outputted.
Thanks! Simon
https://bugs.openldap.org/show_bug.cgi?id=9362
Quanah Gibson-Mount quanah@openldap.org changed:
What |Removed |Added ---------------------------------------------------------------------------- Keywords| |replication
--- Comment #3 from Quanah Gibson-Mount quanah@openldap.org --- Hi Simon,
Have you been able to confirm if you continue to see this issue with a current OpenLDAP release? Again, numerous issues have been fixed since the 2.4.44 release.
Regards, Quanah
https://bugs.openldap.org/show_bug.cgi?id=9362
--- Comment #4 from Simon Pichugin simon.pichugin@gmail.com --- (In reply to Quanah Gibson-Mount from comment #3)
Hi Simon,
Have you been able to confirm if you continue to see this issue with a current OpenLDAP release? Again, numerous issues have been fixed since the 2.4.44 release.
Regards, Quanah
Hi Quanah! We can't update the machine where the crash has happened, unfortunately.
At the same time, the issue hasn't happened since... It looks like either it's really rare naturally, either something has triggered the issue in a very unique way.
We've set 'timelimit' on the machine as a less impacting relief.
If it'll happen and we'll able to reproduce it - we will try the same procedure on a current release. But for now, it is what it is...
Thank you, Simon
https://bugs.openldap.org/show_bug.cgi?id=9362
--- Comment #5 from Quanah Gibson-Mount quanah@openldap.org --- (In reply to Simon Pichugin from comment #4)
Hi Quanah! We can't update the machine where the crash has happened, unfortunately.
That's unfortunate, given the wide number of freely available options to use to replace distribution builds. I'd note that there have been many critical fixes to replication in the releases since 2.4.44, without which you run the real risk of divergence in the data on the provider(s) and any consumers.