On Wed, Nov 21, 2012 at 1:16 PM, hyc@symas.com wrote:
More information is needed than this. Your stack traces aren't even complete. What was the workload leading up to this? Give us a sample configuration that reproduces the assert.
I came across what looks like this issue today. Let's see if I can provide more info.
I noticed one of my test machines (2.4.35) had crashed overnight. Unfortunately, I had neglected to set ulimit -c, so I had no core dump to work with. Upon restarting, however, I got the following:
slapd: ./../../../libraries/liblmdb/mdb.c:4025: mdb_page_search_root: Assertion `(((mp)->mp_pb.pb.pb_lower - ((unsigned) __builtin_offsetof (MDB_page, mp_ptrs))) >> 1) > 1' failed.
A backtrace shows that this occurred during an accesslog_purge(). The backtrace is at the end of this email.
The setup I have is a simple accesslog that records binds:
##### slapd.conf ##### ... database mdb suffix "cn=authstats" directory /apps/local/openldap-2.4.35/var/openldap-data/authstats maxsize 17179869184 dbnosync
rootdn cn=manager,dc=vt,dc=edu index default eq index entryCSN,objectClass,reqEnd,reqResult,reqStart
checkpoint 1024 10
overlay syncprov syncprov-nopresent TRUE syncprov-reloadhint TRUE ... # dc=vt,dc=edu overlay accesslog logdb "cn=authstats" logops bind logpurge 01+00:00 0+08:00 ... #####
If I comment out the logpurge line, slapd starts without incident, cn=authstats is queryable, and new entries are logged.
If I slapcat cn=authstats, remove the cn=authstats .mdb files, and slapadd everything back, upon startup the logpurge works as expected. If I move the original .mdb files back, the assert fails again.
I'm attempting to reproduce this starting with an empty accesslog. I have so far been unsuccessful. I do have the original .mdb files if they may be helpful (though they are around 1G).
One thing to note with the following backtrace is that the accesslog_purge() line number will be incorrect, as I have patched it to record connection info (like ITS#7345). None of this affects accesslog_purge(), though I can run these tests without the patch if necessary.
##### backtrace ##### Program received signal SIGABRT, Aborted. [Switching to Thread 0x7ff7f1722700 (LWP 1576)] 0x00007ffff66b28a5 in raise () from /lib64/libc.so.6 Missing separate debuginfos, use: debuginfo-install db4-4.7.25-17.el6.x86_64 glibc-2.12-1.107.el6.x86_64 keyutils-libs-1.4-4.el6.x86_64 krb5-libs-1.10.3-10.el6_4.2.x86_64 libcom_err-1.41.12-14.el6.x86_64 libselinux-2.0.94-5.3.el6.x86_64 libtool-ltdl-2.2.6-15.5.el6.x86_64 nss-softokn-freebl-3.12.9-11.el6.x86_64 openssl-1.0.0-27.el6_4.2.x86_64 zlib-1.2.3-29.el6.x86_64 #0 0x00007ffff66b28a5 in raise () from /lib64/libc.so.6 No symbol table info available. #1 0x00007ffff66b4085 in abort () from /lib64/libc.so.6 No symbol table info available. #2 0x00007ffff66aba1e in __assert_fail_base () from /lib64/libc.so.6 No symbol table info available. #3 0x00007ffff66abae0 in __assert_fail () from /lib64/libc.so.6 No symbol table info available. #4 0x0000000000511836 in mdb_page_search_root (mc=0x7ff7e01023f8, key=0x0, modify=0) at ./../../../libraries/liblmdb/mdb.c:4025 node = 0x8 i = 0 mp = 0x7ff7e03c9000 rc = 0 __PRETTY_FUNCTION__ = "mdb_page_search_root" #5 0x0000000000517839 in mdb_node_move (csrc=0x7ff7f1720750, cdst=0x7ff7e01023f8) at ./../../../libraries/liblmdb/mdb.c:5988 snum = 3 s2 = 0x7ffff31270c0 bkey = {mv_size = 4294967304, mv_data = 0x0} srcnode = 0x7ff7e03ccc18 key = {mv_size = 80, mv_data = 0x7ff7e03ccc20} data = {mv_size = 111622, mv_data = 0x7ff7e03ccc70} srcpg = 111622 mn = {mc_next = 0x7ff7f1720530, mc_orig = 0x51cb38, mc_xcursor = 0x7ff7f1720740, mc_txn = 0x7ffff31270c0, mc_dbi = 4050782896, mc_db = 0x800000008, mc_dbx = 0x7ff700000063, mc_dbflag = 0x40 <Address 0x40 out of bounds>, mc_snum = 8, mc_top = 0, mc_flags = 0, mc_pg = {0x1030, 0x0, 0x0, 0x1, 0x7000000101, 0x44ab4, 0x7ffff31270c0, 0x7ff7f1720860, 0x800000c12, 0x100000009, 0x0, 0x7ff7f17205d0, 0x51cb38, 0x7ff7f17207e0, 0x7ffff31270c0, 0xc12, 0x900000009, 0x7ff7f17208b0, 0x50ca2c, 0x300000003, 0x7ff7f17208e0, 0x11013f9a8, 0x7ff7f1720a00, 0x7ff7e01025f0, 0x0, 0x7ff7e03c0b48, 0x50cb83, 0x24188, 0x7ff7f1720ab0, 0x7ffc2a2f6000, 0xe03bfd40, 0x1}, mc_ki = {64832, 57403, 32759, 0, 1696, 61810, 32759, 0, 5899, 81, 1, 0, 9648, 57360, 32759, 0, 2736, 61810, 32759, 0, 16776, 2, 0, 0, 57296, 57403, 32759, 0, 64832, 57403, 32759, 0}} rc = 0 flags = 0 __PRETTY_FUNCTION__ = "mdb_node_move" #6 0x00000000005191b9 in mdb_rebalance (mc=0x7ff7e01023f8) at ./../../../libraries/liblmdb/mdb.c:6391 node = 0x7ff7e03c8fe8 rc = 0 minkeys = 2 ptop = 1 mn = {mc_next = 0x7ff7e03bdfd0, mc_orig = 0x7ff7e03ca010, mc_xcursor = 0x0, mc_txn = 0x7ff7e03bdfd0, mc_dbi = 4, mc_db = 0x7ff7e0102580, mc_dbx = 0x7ff7e01025b0, mc_dbflag = 0x7ff7f1720e70 "", mc_snum = 3, mc_top = 2, mc_flags = 5, mc_pg = {0x7ff7e03c6fe0, 0x7ff7e03c7ff0, 0x7ff7e03cc030, 0x7ff7e01025f0, 0x0, 0x7ff7e03cabda, 0x50cb83, 0x0, 0x100000000000, 0x44ab4, 0x7ff7e03cb020, 0x7ff7e03bdfd0, 0x7ff7e03cb020, 0x44ab4, 0x0, 0x7ff7f17208a0, 0x50cc73, 0x7ff7f1720dcc, 0x7ff7e01023f8, 0x7ffc37bf9000, 0x7ff7e03c5fd0, 0x7ff7e03ca010, 0x7ff7e03ca010, 0x7ff7e01025f0, 0xf1720b00, 0x7ff7e03cbc58, 0x50cb83, 0x3f17208b0, 0x100000024188, 0x7ffc34e0d000, 0x7ff7e03cb020, 0x7ff7e03cb884}, mc_ki = {0, 0, 12, 0, 12616, 185, 0, 0, 3090, 0, 0, 0, 2368, 61810, 0, 0, 0, 0, 0, 0, 2400, 61810, 32759, 0, 2416, 61810, 32759, 0, 0, 0, 21, 0}} __PRETTY_FUNCTION__ = "mdb_rebalance" #7 0x000000000051880d in mdb_page_merge (csrc=0x7ff7e01023f8, cdst=0x7ff7f1720a00) at ./../../../libraries/liblmdb/mdb.c:6220 rc = 0 i = 11 j = 22 srcnode = 0x7ff7e03cac8c key = {mv_size = 81, mv_data = 0x7ff7e03cac94} data = {mv_size = 0, mv_data = 0x7ff7e03cace5} nkeys = 11 __PRETTY_FUNCTION__ = "mdb_page_merge" #8 0x000000000051920f in mdb_rebalance (mc=0x7ff7e01023f8) at ./../../../libraries/liblmdb/mdb.c:6396 node = 0x7ff7e03c9c3c rc = 0 minkeys = 1 ptop = 2 mn = {mc_next = 0x7ffc2ff89ff8, mc_orig = 0x5114df, mc_xcursor = 0x0, mc_txn = 0x7ff7e03bdfd0, mc_dbi = 4, mc_db = 0x7ff7e0102580, mc_dbx = 0x7ff7e01025b0, mc_dbflag = 0x7ff7e03bfd40 "\210A\002", mc_snum = 4, mc_top = 3, mc_flags = 5, mc_pg = {0x7ff7e03c6fe0, 0x7ff7e03c7ff0, 0x7ff7e03c9000, 0x7ff7e03cb020, 0x51117c, 0x7ff7f1720c70, 0x511cf4, 0x7ff7f1720ae0, 0x200b5ca30, 0x0, 0x7ff7e01025f0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x7ff7e03bdfd0, 0x7ff700000001, 0x7ff7e03be068, 0xb5c9d0, 0x7ff7e03bfca9, 0x1, 0x7ff7e03bfd40, 0x7ff7f1720cb0, 0x0, 0x0, 0x7ff7e03bdfd0, 0x7ff700000001, 0x7ff7e03be068, 0xb5c9d0, 0x7ff7e03bfca9, 0x1}, mc_ki = {0, 1, 0, 10, 3072, 61810, 32759, 0, 44982, 57404, 32759, 0, 45072, 57404, 32759, 0, 44084, 57404, 32759, 0, 988, 0, 0, 0, 15065, 63088, 32767, 0, 3008, 61810, 32759, 0}} __PRETTY_FUNCTION__ = "mdb_rebalance" #9 0x00000000005193cb in mdb_cursor_del0 (mc=0x7ff7e01023f8, leaf=0x7ff7e03cafb6) at ./../../../libraries/liblmdb/mdb.c:6424 rc = 15 #10 0x0000000000515b10 in mdb_cursor_del (mc=0x7ff7e01023f8, flags=0) at ./../../../libraries/liblmdb/mdb.c:5305 leaf = 0x7ff7e03cafb6 rc = 0 #11 0x00000000005159d1 in mdb_cursor_del (mc=0x7ff7e0102270, flags=0) at ./../../../libraries/liblmdb/mdb.c:5278 leaf = 0x7ff7e03c6f24 rc = 0 #12 0x000000000056e5ee in mdb_dn2id_delete (op=0x7ff7f17213c0, mc=0x7ff7e0102270, id=1087351, nsubs=1) at dn2id.c:217 nid = 73 ptr = 0x7ffc37bf9f0a "" rc = 0 #13 0x000000000056d058 in mdb_delete (op=0x7ff7f17213c0, rs=0x7ff7f1721350) at delete.c:335 mdb = 0x7ffff7f2a010 pdn = {bv_len = 12, bv_val = 0x7ff7e010347f "cn=authstats"} e = 0x7ff7e0000be0 p = 0x7ff7e00009b0 manageDSAit = 0 children = 0x935970 entry = 0x9356d0 txn = 0x7ff7e03bdfd0 mc = 0x7ff7e0102270 opinfo = {moi_oe = {oe_next = {sle_next = 0x0}, oe_key = 0x7ffff7f2a010}, moi_txn = 0x7ff7e03bdfd0, moi_ref = 1, moi_flag = 0 '\000'} moi = 0x7ff7f1720e70 preread_ctrl = 0x0 ctrls = {0x0, 0x7ff7f172127e, 0x7ff7e0000908, 0x7ff7f1721281, 0x7ff7f172127e, 0x88} num_ctrls = 0 parent_is_glue = 0 parent_is_leaf = 0 __PRETTY_FUNCTION__ = "mdb_delete" #14 0x00000000004d8b1d in overlay_op_walk (op=0x7ff7f17213c0, rs=0x7ff7f1721350, which=op_delete, oi=0x9b7e60, on=0x0) at backover.c:671 func = 0x85d558 rc = 32768 #15 0x00000000004d8d34 in over_op_func (op=0x7ff7f17213c0, rs=0x7ff7f1721350, which=op_delete) at backover.c:723 oi = 0x9b7e60 on = 0x9b8040 be = 0x9b7190 db = {bd_info = 0x85d500, bd_self = 0x9b7190, be_ctrls = "\000\001\001\001\000\001\000\000\001\000\000\001\001\000\001\000\000\001", '\000' <repeats 14 times>, "\001", be_flags = 2312, be_restrictops = 0, be_requires = 0, be_ssf_set = {sss_ssf = 0, sss_transport = 0, sss_tls = 0, sss_sasl = 0, sss_update_ssf = 0, sss_update_transport = 0, sss_update_tls = 0, sss_update_sasl = 0, sss_simple_bind = 128}, be_suffix = 0x9b79e0, be_nsuffix = 0x9b7a10, be_schemadn = {bv_len = 0, bv_val = 0x0}, be_schemandn = {bv_len = 0, bv_val = 0x0}, be_rootdn = {bv_len = 23, bv_val = 0x9b7c50 "cn=manager,dc=vt,dc=edu"}, be_rootndn = { bv_len = 23, bv_val = 0x9b7ca0 "cn=manager,dc=vt,dc=edu"}, be_rootpw = {bv_len = 0, bv_val = 0x0}, be_max_deref_depth = 15, be_def_limit = {lms_t_soft = 3600, lms_t_hard = 0, lms_s_soft = 500, lms_s_hard = 0, lms_s_unchecked = -1, lms_s_pr = 0, lms_s_pr_hide = 0, lms_s_pr_total = 0}, be_limits = 0x9b7cf0, be_acl = 0x9b84c0, be_dfltaccess = ACL_READ, be_extra_anlist = 0x0, be_update_ndn = {bv_len = 0, bv_val = 0x0}, be_update_refs = 0x0, be_pending_csn_list = 0xb5f2d0, be_pcl_mutex = {__data = { __lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, be_syncinfo = 0x0, be_pb = 0x0, be_cf_ocs = 0x864500, be_private = 0x7ffff7f2a010, be_next = {stqe_next = 0x9b8760}} cb = {sc_next = 0x7ffff35e5fe0, sc_response = 0x4d7afa <over_back_response>, sc_cleanup = 0, sc_private = 0x9b7e60} sc = 0x7ff7f1721354 rc = 32768 __PRETTY_FUNCTION__ = "over_op_func" #16 0x00000000004d8eee in over_op_delete (op=0x7ff7f17213c0, rs=0x7ff7f1721350) at backover.c:780 No locals. #17 0x00007ffff33dcb46 in accesslog_purge (ctx=0x7ff7f1721aa0, arg=0xa1e4e0) at accesslog.c:707 i = 0 rtask = 0xa1e4e0 li = 0xa1f300 conn = {c_struct_state = SLAP_C_UNINITIALIZED, c_conn_state = SLAP_C_INVALID, c_conn_idx = -1, c_sd = 0, c_close_reason = 0x0, c_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __list = { __prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, c_sb = 0x0, c_starttime = 0, c_activitytime = 0, c_connid = 18446744073709551615, c_peer_domain = {bv_len = 0, bv_val = 0x5d1370 ""}, c_peer_name = {bv_len = 0, bv_val = 0x5d1370 ""}, c_listener = 0x5d92a0, c_sasl_bind_mech = { bv_len = 0, bv_val = 0x0}, c_sasl_dn = {bv_len = 0, bv_val = 0x0}, c_sasl_authz_dn = {bv_len = 0, bv_val = 0x0}, c_authz_backend = 0x0, c_authz_cookie = 0x0, c_authz = {sai_method = 0, sai_mech = { bv_len = 0, bv_val = 0x0}, sai_dn = {bv_len = 0, bv_val = 0x0}, sai_ndn = {bv_len = 0, bv_val = 0x0}, sai_ssf = 0, sai_transport_ssf = 0, sai_tls_ssf = 0, sai_sasl_ssf = 0}, c_protocol = 0, c_ops = {stqh_first = 0x0, stqh_last = 0x0}, c_pending_ops = {stqh_first = 0x0, stqh_last = 0x0}, c_write1_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, c_write1_cv = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}, c_write2_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, c_write2_cv = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}, c_currentber = 0x0, c_writers = 0, c_writing = 0 '\000', c_sasl_bind_in_progress = 0 '\000', c_writewaiter = 0 '\000', c_is_tls = 0 '\000', c_needs_tls_accept = 0 '\000', c_sasl_layers = 0 '\000', c_sasl_done = 0 '\000', c_sasl_authctx = 0x0, c_sasl_sockctx = 0x0, c_sasl_extra = 0x0, c_sasl_bindop = 0x0, c_pagedresults_state = {ps_be = 0x0, ps_size = 0, ps_count = 0, ps_cookie = 0, ps_cookieval = { bv_len = 0, bv_val = 0x0}}, c_n_ops_received = 0, c_n_ops_executing = 0, c_n_ops_pending = 0, c_n_ops_completed = 0, c_n_get = 0, c_n_read = 0, c_n_write = 0, c_extensions = 0x0, c_clientfunc = 0, c_clientarg = 0x0, c_send_ldap_result = 0x45be00 <slap_send_ldap_result>, c_send_search_entry = 0x45cb08 <slap_send_search_entry>, c_send_search_reference = 0x45ebb8 <slap_send_search_reference>, c_send_ldap_extended = 0x45c667 <slap_send_ldap_extended>, c_send_ldap_intermediate = 0x45c8e5 <slap_send_ldap_intermediate>} opbuf = {ob_op = {o_hdr = 0x7ff7f1721530, o_tag = 74, o_time = 1369246130, o_tincr = 2, o_bd = 0x7ff7f1720fe0, o_req_dn = {bv_len = 44, bv_val = 0x7ff7e0103420 "reqStart=20130520202434.000180Z,cn=authstats"}, o_req_ndn = {bv_len = 43, bv_val = 0x7ff7e0103460 "reqStart=20130520202434.00018Z,cn=authstats"}, o_request = {oq_add = {rs_modlist = 0x1, rs_e = 0xffffffffffffffff}, oq_bind = {rb_method = 1, rb_cred = { bv_len = 18446744073709551615, bv_val = 0x0}, rb_edn = { bv_len = 1, bv_val = 0x861740 "\003"}, rb_ssf = 4050785040, rb_mech = {bv_len = 27, bv_val = 0x7ff7e0000908 "\300\017r\361\367\177"}}, oq_compare = {rs_ava = 0x1}, oq_modify = {rs_mods = { rs_modlist = 0x1, rs_no_opattrs = -1 '\377'}, rs_increment = 0}, oq_modrdn = {rs_mods = {rs_modlist = 0x1, rs_no_opattrs = -1 '\377'}, rs_deleteoldrdn = 0, rs_newrdn = {bv_len = 1, bv_val = 0x861740 "\003"}, rs_nnewrdn = {bv_len = 140702884434704, bv_val = 0x1b <Address 0x1b out of bounds>}, rs_newSup = 0x7ff7e0000908, rs_nnewSup = 0x0}, oq_search = { rs_scope = 1, rs_deref = 0, rs_slimit = -1, rs_tlimit = -1, rs_limit = 0x0, rs_attrsonly = 1, rs_attrs = 0x861740, rs_filter = 0x7ff7f1721310, rs_filterstr = {bv_len = 27, bv_val = 0x7ff7e0000908 "\300\017r\361\367\177"}}, oq_abandon = {rs_msgid = 1}, oq_cancel = {rs_msgid = 1}, oq_extended = {rs_reqoid = {bv_len = 1, bv_val = 0xffffffffffffffff <Address 0xffffffffffffffff out of bounds>}, rs_flags = 0, rs_reqdata = 0x1}, oq_pwdexop = {rs_extended = { rs_reqoid = {bv_len = 1, bv_val = 0xffffffffffffffff <Address 0xffffffffffffffff out of bounds>}, rs_flags = 0, rs_reqdata = 0x1}, rs_old = {bv_len = 8787776, bv_val = 0x7ff7f1721310 "\246"}, rs_new = {bv_len = 27, bv_val = 0x7ff7e0000908 "\300\017r\361\367\177"}, rs_mods = 0x0, rs_modtail = 0x0}}, o_abandon = 0, o_cancel = 0, o_groups = 0x0, o_do_not_cache = 0 '\000', o_is_auth_check = 0 '\000', o_dont_replicate = 1 '\001', o_acl_priv = ACL_NONE, o_nocaching = 0 '\000', o_delete_glue_parent = 0 '\000', o_no_schema_check = 0 '\000', o_no_subordinate_glue = 0 '\000', o_ctrlflag = '\000' <repeats 31 times>, o_controls = 0x7ff7f1721678, o_authz = {sai_method = 0, sai_mech = {bv_len = 0, bv_val = 0x0}, sai_dn = {bv_len = 23, bv_val = 0x9b7c50 "cn=manager,dc=vt,dc=edu"}, sai_ndn = { bv_len = 23, bv_val = 0x9b7ca0 "cn=manager,dc=vt,dc=edu"}, sai_ssf = 0, sai_transport_ssf = 0, sai_tls_ssf = 0, sai_sasl_ssf = 0}, o_ber = 0x0, o_res_ber = 0x0, o_callback = 0x7ff7e0000908, o_ctrls = 0x0, o_csn = {bv_len = 40, bv_val = 0x7ff7f1721260 "20130521180706.125179Z#000000#000#000000"}, o_private = 0x0, o_extra = {slh_first = 0x7ff7f1720e70}, o_next = { stqe_next = 0x0}}, ob_hdr = {oh_opid = 0, oh_connid = 18446744073709551615, oh_conn = 0x7ff7f1721780, oh_msgid = 0, oh_protocol = 0, oh_tid = 140702884439808, oh_threadctx = 0x7ff7f1721aa0, oh_tmpmemctx = 0x7ff7e00008c0, oh_tmpmfuncs = 0x861ce0, oh_counters = 0x8b7d40, oh_log_prefix = "conn=-1 op=0", '\000' <repeats 243 times>}, ob_controls = {0x0 <repeats 32 times>}} op = 0x7ff7f17213c0 rs = {sr_type = REP_RESULT, sr_tag = 0, sr_msgid = 0, sr_err = -30798, sr_matched = 0x0, sr_text = 0x0, sr_ref = 0x0, sr_ctrls = 0x0, sr_un = {sru_search = {r_entry = 0x0, r_attr_flags = 0, r_operational_attrs = 0x0, r_attrs = 0x0, r_nentries = 0, r_v2ref = 0x0}, sru_sasl = {r_sasldata = 0x0}, sru_extended = { r_rspoid = 0x0, r_rspdata = 0x0}}, sr_flags = 0} cb = {sc_next = 0x0, sc_response = 0x7ffff33dc4f0 <log_old_lookup>, sc_cleanup = 0, sc_private = 0x7ff7f17212c0} f = {f_choice = 166, f_un = {f_un_result = -244182288, f_un_desc = 0x7ff7f17212f0, f_un_ava = 0x7ff7f17212f0, f_un_ssa = 0x7ff7f17212f0, f_un_mra = 0x7ff7f17212f0, f_un_complex = 0x7ff7f17212f0}, f_next = 0x0} ava = {aa_desc = 0x9ad600, aa_value = {bv_len = 15, bv_val = 0x7ff7f17212a0 "20130521180850Z"}} pd = {slots = 22700, used = 22689, dn = 0x7ff7e04cab30, ndn = 0x7ff7e0523600, csn = {bv_len = 40, bv_val = 0x7ff7f1721260 "20130521180706.125179Z#000000#000#000000"}} timebuf = "20130521180850Z\000\000\000\000\000\000" csnbuf = "20130521180706.125179Z#000000#000#000000", '\000' <repeats 23 times> old = 1369159730 __PRETTY_FUNCTION__ = "accesslog_purge" #18 0x0000000000595639 in ldap_int_thread_pool_wrapper (xpool=0x93a270) at tpool.c:688 pool = 0x93a270 task = 0x7ff7ec000a20 work_list = 0x93a308 ctx = {ltu_id = 140702884439808, ltu_key = {{ltk_key = 0x4b8d8b, ltk_data = 0x7ff7e00008c0, ltk_free = 0x4b8bb0 <slap_sl_mem_destroy>}, { ltk_key = 0x7ffff3127010, ltk_data = 0x7ff7e0100910, ltk_free = 0x5746dc <mdb_reader_free>}, {ltk_key = 0x525df8, ltk_data = 0x7ff7ea7fe010, ltk_free = 0x525dd5 <search_stack_free>}, {ltk_key = 0x52329d, ltk_data = 0x7ff7f0420010, ltk_free = 0x523255 <scope_chunk_free>}, {ltk_key = 0x0, ltk_data = 0x0, ltk_free = 0} <repeats 25 times>, { ltk_key = 0x0, ltk_data = 0x0, ltk_free = 0x80}, {ltk_key = 0x0, ltk_data = 0x0, ltk_free = 0}, {ltk_key = 0x0, ltk_data = 0x0, ltk_free = 0}}} kctx = 0x0 i = 32 keyslot = 672 hash = 284303008 __PRETTY_FUNCTION__ = "ldap_int_thread_pool_wrapper" #19 0x00007ffff764a851 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #20 0x00007ffff676890d in clone () from /lib64/libc.so.6 No symbol table info available. #####