Hi,
I have openldap 2.4.36 installed using mdb backend on Solaris 10. I am using this database for a while and suddenly I started getting core dumps while trying to add any data to database. Disk, memory usages seem fine. Database size is not exceeded, it is almost 1/10 full. What may be the reason for this? Please find the other information about the issue as below:
slapd version:
# ./slapd -VVV @(#) $OpenLDAP: slapd 2.4.36 (Oct 6 2013 13:56:57) $ myuser@myhost:/some/dir/openldap-2.4.36/servers/slapd
Included static overlays: syncprov Included static backends: config ldif mdb
slapd.conf for mdb backend:
database mdb directory /some/dir/openldap/var/openldap-data suffix "dc=mycomp,dc=com" rootdn "cn=Manager,dc=mycomp,dc=com" rootpw password index objectClass eq maxsize 45899345920 dbnosync checkpoint 0 10 threads 32 listener-threads 16 concurrency 1024 envflags writemap,nometasync
slapd debug log:
53e15b8d conn=1001 op=17 do_add ber_scanf fmt ({m) ber: 53e15b8d conn=1001 op=17 do_add: dn (BatchGroupName=test,uid=myuser,OperationType=create,o=cust,dc=mycomp,dc=com) ber_scanf fmt ({m{W}}) ber: ber_scanf fmt ({m{W}}) ber: ber_scanf fmt (}) ber: 53e15b8d >>> dnPrettyNormal: <BatchGroupName=test,uid=myuser,OperationType=create,o=cust,dc=mycomp,dc=com> => ldap_bv2dn(BatchGroupName=test,uid=myuser,OperationType=create,o=cust,dc=mycomp,dc=com,0) <= ldap_bv2dn(BatchGroupName=test,uid=myuser,OperationType=create,o=cust,dc=mycomp,dc=com)=0 => ldap_dn2bv(272) <= ldap_dn2bv(BatchGroupName=test,uid=myuser,OperationType=create,o=cust,dc=mycomp,dc=com)=0 => ldap_dn2bv(272) <= ldap_dn2bv(BatchGroupName=test,uid=myuser,OperationType=create,o=cust,dc=mycomp,dc=com)=0 53e15b8d <<< dnPrettyNormal: <BatchGroupName=test,uid=myuser,OperationType=create,o=cust,dc=mycomp,dc=com>, <BatchGroupName=test,uid=myuser,OperationType=create,o=cust,dc=mycomp,dc=com> 53e15b8d ==> mdb_add: BatchGroupName=test,uid=myuser,OperationType=create,o=cust,dc=mycomp,dc=com 53e15b8d oc_check_required entry (BatchGroupName=test,uid=myuser,OperationType=create,o=cust,dc=mycomp,dc=com), objectClass "RTEBatchGroup" 53e15b8d oc_check_allowed type "BatchGroupName" 53e15b8d oc_check_allowed type "objectClass" 53e15b8d oc_check_allowed type "structuralObjectClass" 53e15b8d mdb_dn2entry("BatchGroupName=test,uid=myuser,OperationType=create,o=cust,dc=mycomp,dc=com") 53e15b8d => mdb_dn2id("BatchGroupName=test,uid=myuser,OperationType=create,o=cust,dc=mycomp,dc=com") 53e15b8d <= mdb_dn2id: get failed: MDB_NOTFOUND: No matching key/data pair found (-30798) 53e15b8d => mdb_entry_decode: 53e15b8d <= mdb_entry_decode 53e15b8d => mdb_dn2id_add 0x1a28eb: "BatchGroupName=test,uid=myuser,OperationType=create,o=cust,dc=mycomp,dc=com" zsh: bus error (core dumped) ./slapd -d5
gdb backtrace of the generated core file:
# gdb slapd core GNU gdb 6.8 Copyright (C) 2008 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "sparc-sun-solaris2.10"... (no debugging symbols found) Reading symbols from /some/dir/lib/libldap_r-2.4.so.2...(no debugging symbols found)...done. Loaded symbols for /some/dir/lib/libldap_r-2.4.so.2 Reading symbols from /some/dir/lib/liblber-2.4.so.2...(no debugging symbols found)...done. Loaded symbols for /some/dir/lib/liblber-2.4.so.2 Reading symbols from /usr/lib/sparcv9/libicuuc.so.3...(no debugging symbols found)...done. Loaded symbols for /usr/lib/64/libicuuc.so.3 Reading symbols from /usr/lib/sparcv9/libicudata.so.3...(no debugging symbols found)...done. Loaded symbols for /usr/lib/64/libicudata.so.3 Reading symbols from /some/dir/lib/libsasl2.so.2... (no debugging symbols found)...done. Loaded symbols for /some/dir/lib/libsasl2.so.2 Reading symbols from /lib/sparcv9/libdl.so.1... warning: Lowest section in /lib/sparcv9/libdl.so.1 is .hash at 0000000000000120 (no debugging symbols found)...done. Loaded symbols for /lib/64/libdl.so.1 Reading symbols from /opt/csw/lib/sparcv9/libssl.so.0.9.8...(no debugging symbols found)...done. Loaded symbols for /opt/csw/lib/sparcv9/libssl.so.0.9.8 Reading symbols from /opt/csw/lib/sparcv9/libcrypto.so.0.9.8...(no debugging symbols found)...done. Loaded symbols for /opt/csw/lib/sparcv9/libcrypto.so.0.9.8 Reading symbols from /lib/sparcv9/libresolv.so.2... (no debugging symbols found)...done. Loaded symbols for /lib/64/libresolv.so.2 Reading symbols from /lib/sparcv9/libgen.so.1...(no debugging symbols found)...done. Loaded symbols for /lib/64/libgen.so.1 Reading symbols from /lib/sparcv9/libnsl.so.1...(no debugging symbols found)...done. Loaded symbols for /lib/64/libnsl.so.1 Reading symbols from /lib/sparcv9/libsocket.so.1...(no debugging symbols found)...done. Loaded symbols for /lib/64/libsocket.so.1 Reading symbols from /lib/sparcv9/librt.so.1... (no debugging symbols found)...done. Loaded symbols for /lib/64/librt.so.1 Reading symbols from /lib/sparcv9/libc.so.1...(no debugging symbols found)...done. Loaded symbols for /lib/64/libc.so.1 Reading symbols from /opt/csw/gcc4/lib/sparcv9/libgcc_s.so.1...(no debugging symbols found)...done. Loaded symbols for /opt/csw/gcc4/lib/sparcv9/libgcc_s.so.1 Reading symbols from /lib/sparcv9/libpthread.so.1... warning: Lowest section in /lib/sparcv9/libpthread.so.1 is .dynamic at 00000000000000b0 (no debugging symbols found)...done. Loaded symbols for /lib/64/libpthread.so.1 Reading symbols from /lib/sparcv9/libm.so.2... (no debugging symbols found)...done. Loaded symbols for /lib/64/libm.so.2 Reading symbols from /usr/lib/sparcv9/libCrun.so.1...(no debugging symbols found)...done. Loaded symbols for /usr/lib/64/libCrun.so.1 Reading symbols from /lib/sparcv9/libaio.so.1...(no debugging symbols found)...done. Loaded symbols for /lib/64/libaio.so.1 Reading symbols from /lib/sparcv9/libmd.so.1...(no debugging symbols found)...done. Loaded symbols for /lib/64/libmd.so.1 Reading symbols from /platform/sun4v/lib/sparcv9/libc_psr.so.1... (no debugging symbols found)...done. Loaded symbols for /platform/SUNW,Netra-T2000/lib/sparcv9/libc_psr.so.1 Reading symbols from /lib/sparcv9/ld.so.1...(no debugging symbols found)...done. Loaded symbols for /lib/sparcv9/ld.so.1 Core was generated by `./slapd -d5'. Program terminated with signal 10, Bus error. [New process 1335262 ] [New process 90078 ] [New process 155614 ] [New process 221150 ] [New process 286686 ] [New process 352222 ] [New process 417758 ] [New process 483294 ] [New process 548830 ] [New process 614366 ] [New process 679902 ] [New process 745438 ] [New process 810974 ] [New process 876510 ] [New process 942046 ] [New process 1007582 ] [New process 1073118 ] [New process 1138654 ] [New process 1204190 ] [New process 1269726 ] #0 0x0000000100159c3c in mdb_page_search_root () (gdb) bt #0 0x0000000100159c3c in mdb_page_search_root () #1 0x000000010015a2d0 in mdb_page_search () #2 0x000000010015c984 in mdb_cursor_first () #3 0x000000010015d93c in mdb_cursor_get () #4 0x000000010015204c in mdb_page_alloc () #5 0x0000000100152b6c in mdb_page_touch () #6 0x000000010015a264 in mdb_page_search () #7 0x000000010015dba0 in mdb_cursor_touch () #8 0x000000010015e314 in mdb_cursor_put () #9 0x000000010018f25c in mdb_dn2id_add () #10 0x0000000100184e94 in mdb_add () #11 0x0000000100060970 in fe_op_add () #12 0x00000001000601fc in do_add () #13 0x000000010005128c in connection_operation () #14 0x0000000100051a4c in connection_read_thread () #15 0xffffffff7f4164bc in ldap_int_thread_pool_wrapper () from /some/dir/lib/libldap_r-2.4.so.2 #16 0xffffffff7ced7374 in _lwp_start () from /lib/64/libc.so.1 #17 0xffffffff7ced7374 in _lwp_start () from /lib/64/libc.so.1 Backtrace stopped: previous frame identical to this frame (corrupt stack?)
Thanks