Michael Ströder wrote:
Quanah Gibson-Mount wrote:
--On January 28, 2009 4:33:53 PM +0100 Michael Ströder michael@stroeder.com wrote:
(gdb) bt all No symbol "all" in current context.
Generally:
thr apply all bt
I think is what you are going for (i.e., get a full backtrace on all threads).
Now gdb backtrace of hanging test008 with HEAD and compiled with only -O2:
Thanks. Seems like this is a race condition due to the edge-triggered epoll - if the socket becomes writable before we wait for it, we won't wake up from the wait. Apparently we'll have to go back to level-triggered like before. Too bad we can't set different epoll behaviors for different events.
(Thread 15 is waiting to write again...)
Thread 18 (Thread 0x40d14b90 (LWP 26740)): #0 0xffffe430 in __kernel_vsyscall () #1 0x40579c26 in epoll_wait () from /lib/libc.so.6 #2 0x0806617f in slapd_daemon_task (ptr=0x0) at daemon.c:2291 #3 0x4024a1b5 in start_thread () from /lib/libpthread.so.0 #4 0x4057938e in clone () from /lib/libc.so.6
Thread 17 (Thread 0x41115b90 (LWP 26741)): #0 0xffffe430 in __kernel_vsyscall () #1 0x4024dc35 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0 #2 0x40030094 in ldap_pvt_thread_cond_wait (cond=0x82ba4f4, mutex=0x82ba4dc) at thr_posix.c:277 #3 0x4002f472 in ldap_int_thread_pool_wrapper (xpool=0x82ba4d8) at tpool.c:654 #4 0x4024a1b5 in start_thread () from /lib/libpthread.so.0 #5 0x4057938e in clone () from /lib/libc.so.6
Thread 16 (Thread 0x41e18b90 (LWP 27131)): #0 0xffffe430 in __kernel_vsyscall () #1 0x4024dc35 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0 #2 0x40030094 in ldap_pvt_thread_cond_wait (cond=0x82ba4f4, mutex=0x82ba4dc) at thr_posix.c:277 #3 0x4002f472 in ldap_int_thread_pool_wrapper (xpool=0x82ba4d8) at tpool.c:654 #4 0x4024a1b5 in start_thread () from /lib/libpthread.so.0 #5 0x4057938e in clone () from /lib/libc.so.6
Thread 15 (Thread 0x42b1bb90 (LWP 27136)): #0 0xffffe430 in __kernel_vsyscall () #1 0x4024dc35 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0 #2 0x40030094 in ldap_pvt_thread_cond_wait (cond=0x4073a288, mutex=0x4073a270) at thr_posix.c:277 #3 0x0807a418 in send_ldap_ber (conn=0x4073a178, ber=0x42b19de4) at result.c:217 #4 0x0807c43b in slap_send_search_entry (op=0x8381ce0, rs=0x42b1b0c8) at result.c:1246 #5 0x0810b2ba in monitor_send_children (op=0x8381ce0, rs=0x42b1b0c8, e_parent=0x835f3e4, sub=1) at search.c:115 #6 0x0810b1cc in monitor_send_children (op=0x8381ce0, rs=0x42b1b0c8, e_parent=<value optimized out>, sub=1) at search.c:123 #7 0x0810b5dd in monitor_back_search (op=0x8381ce0, rs=0x42b1b0c8) at search.c:245 #8 0x0806bb73 in fe_op_search (op=0x8381ce0, rs=0x42b1b0c8) at search.c:366 #9 0x0806c3e8 in do_search (op=0x8381ce0, rs=0x42b1b0c8) at search.c:217 #10 0x080696bb in connection_operation (ctx=0x42b1b1cc, arg_v=0x8381ce0) at connection.c:1100 #11 0x0806a117 in connection_read_thread (ctx=0x42b1b1cc, argv=0x1c) at connection.c:1226 #12 0x4002f400 in ldap_int_thread_pool_wrapper (xpool=0x82ba4d8) at tpool.c:663 #13 0x4024a1b5 in start_thread () from /lib/libpthread.so.0 #14 0x4057938e in clone () from /lib/libc.so.6
Thread 14 (Thread 0x4381eb90 (LWP 27138)): #0 0xffffe430 in __kernel_vsyscall () #1 0x4024dc35 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0 #2 0x40030094 in ldap_pvt_thread_cond_wait (cond=0x82ba4f4, mutex=0x82ba4dc) at thr_posix.c:277 #3 0x4002f472 in ldap_int_thread_pool_wrapper (xpool=0x82ba4d8) at tpool.c:654 #4 0x4024a1b5 in start_thread () from /lib/libpthread.so.0 #5 0x4057938e in clone () from /lib/libc.so.6
Thread 13 (Thread 0x43c1fb90 (LWP 27144)): #0 0xffffe430 in __kernel_vsyscall () #1 0x4024dc35 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0 #2 0x40030094 in ldap_pvt_thread_cond_wait (cond=0x82ba4f4, mutex=0x82ba4dc) at thr_posix.c:277 #3 0x4002f472 in ldap_int_thread_pool_wrapper (xpool=0x82ba4d8) at tpool.c:654 #4 0x4024a1b5 in start_thread () from /lib/libpthread.so.0 #5 0x4057938e in clone () from /lib/libc.so.6
Thread 12 (Thread 0x44020b90 (LWP 27145)): ---Type<return> to continue, or q<return> to quit--- #0 0xffffe430 in __kernel_vsyscall () #1 0x4024dc35 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0 #2 0x40030094 in ldap_pvt_thread_cond_wait (cond=0x82ba4f4, mutex=0x82ba4dc) at thr_posix.c:277 #3 0x4002f472 in ldap_int_thread_pool_wrapper (xpool=0x82ba4d8) at tpool.c:654 #4 0x4024a1b5 in start_thread () from /lib/libpthread.so.0 #5 0x4057938e in clone () from /lib/libc.so.6
Thread 11 (Thread 0x44421b90 (LWP 27146)): #0 0xffffe430 in __kernel_vsyscall () #1 0x402507b9 in __lll_lock_wait () from /lib/libpthread.so.0 #2 0x4024bcd1 in _L_lock_89 () from /lib/libpthread.so.0 #3 0x4024b5b2 in pthread_mutex_lock () from /lib/libpthread.so.0 #4 0x4002fffd in ldap_pvt_thread_mutex_lock (mutex=0x837a8c8) at thr_posix.c:296 #5 0x0815f27c in monitor_cache_lock (e=0x835f3e4) at cache.c:117 #6 0x0810b155 in monitor_send_children (op=0x83891a8, rs=0x444210c8, e_parent=<value optimized out>, sub=1) at search.c:144 #7 0x0810b1cc in monitor_send_children (op=0x83891a8, rs=0x444210c8, e_parent=<value optimized out>, sub=1) at search.c:123 #8 0x0810b5dd in monitor_back_search (op=0x83891a8, rs=0x444210c8) at search.c:245 #9 0x0806bb73 in fe_op_search (op=0x83891a8, rs=0x444210c8) at search.c:366 #10 0x0806c3e8 in do_search (op=0x83891a8, rs=0x444210c8) at search.c:217 #11 0x080696bb in connection_operation (ctx=0x444211cc, arg_v=0x83891a8) at connection.c:1100 #12 0x0806a117 in connection_read_thread (ctx=0x444211cc, argv=0x18) at connection.c:1226 #13 0x4002f400 in ldap_int_thread_pool_wrapper (xpool=0x82ba4d8) at tpool.c:663 #14 0x4024a1b5 in start_thread () from /lib/libpthread.so.0 #15 0x4057938e in clone () from /lib/libc.so.6
Thread 10 (Thread 0x45427b90 (LWP 27150)): #0 0xffffe430 in __kernel_vsyscall () #1 0x4024dc35 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0 #2 0x40030094 in ldap_pvt_thread_cond_wait (cond=0x82ba4f4, mutex=0x82ba4dc) at thr_posix.c:277 #3 0x4002f472 in ldap_int_thread_pool_wrapper (xpool=0x82ba4d8) at tpool.c:654 #4 0x4024a1b5 in start_thread () from /lib/libpthread.so.0 #5 0x4057938e in clone () from /lib/libc.so.6
Thread 9 (Thread 0x45929b90 (LWP 27154)): #0 0xffffe430 in __kernel_vsyscall () #1 0x4024dc35 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0 #2 0x40030094 in ldap_pvt_thread_cond_wait (cond=0x82ba4f4, mutex=0x82ba4dc) at thr_posix.c:277 #3 0x4002f472 in ldap_int_thread_pool_wrapper (xpool=0x82ba4d8) at tpool.c:654 #4 0x4024a1b5 in start_thread () from /lib/libpthread.so.0 #5 0x4057938e in clone () from /lib/libc.so.6
Thread 8 (Thread 0x45e2bb90 (LWP 27160)): #0 0xffffe430 in __kernel_vsyscall () #1 0x4024dc35 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0 #2 0x40030094 in ldap_pvt_thread_cond_wait (cond=0x82ba4f4, mutex=0x82ba4dc) at thr_posix.c:277 #3 0x4002f472 in ldap_int_thread_pool_wrapper (xpool=0x82ba4d8) at tpool.c:654 #4 0x4024a1b5 in start_thread () from /lib/libpthread.so.0 #5 0x4057938e in clone () from /lib/libc.so.6
Thread 7 (Thread 0x4632db90 (LWP 27163)): #0 0xffffe430 in __kernel_vsyscall () #1 0x402507b9 in __lll_lock_wait () from /lib/libpthread.so.0 #2 0x4024bcd1 in _L_lock_89 () from /lib/libpthread.so.0 #3 0x4024b5b2 in pthread_mutex_lock () from /lib/libpthread.so.0 #4 0x4002fffd in ldap_pvt_thread_mutex_lock (mutex=0x837a8c8) at thr_posix.c:296 #5 0x0815f27c in monitor_cache_lock (e=0x835f3e4) at cache.c:117 #6 0x0810b155 in monitor_send_children (op=0x839b3a0, rs=0x4632d0c8, e_parent=<value optimized out>, sub=1) at search.c:144 #7 0x0810b1cc in monitor_send_children (op=0x839b3a0, rs=0x4632d0c8, e_parent=<value optimized out>, sub=1) at search.c:123 #8 0x0810b5dd in monitor_back_search (op=0x839b3a0, rs=0x4632d0c8) at search.c:245 #9 0x0806bb73 in fe_op_search (op=0x839b3a0, rs=0x4632d0c8) at search.c:366 #10 0x0806c3e8 in do_search (op=0x839b3a0, rs=0x4632d0c8) at search.c:217 #11 0x080696bb in connection_operation (ctx=0x4632d1cc, arg_v=0x839b3a0) at connection.c:1100 #12 0x0806a117 in connection_read_thread (ctx=0x4632d1cc, argv=0x1f) at connection.c:1226 #13 0x4002f400 in ldap_int_thread_pool_wrapper (xpool=0x82ba4d8) at tpool.c:663 #14 0x4024a1b5 in start_thread () from /lib/libpthread.so.0 #15 0x4057938e in clone () from /lib/libc.so.6
Thread 6 (Thread 0x4682fb90 (LWP 27166)): #0 0xffffe430 in __kernel_vsyscall () #1 0x4024dc35 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0 #2 0x40030094 in ldap_pvt_thread_cond_wait (cond=0x82ba4f4, mutex=0x82ba4dc) at thr_posix.c:277 #3 0x4002f472 in ldap_int_thread_pool_wrapper (xpool=0x82ba4d8) at tpool.c:654 #4 0x4024a1b5 in start_thread () from /lib/libpthread.so.0 #5 0x4057938e in clone () from /lib/libc.so.6
Thread 5 (Thread 0x46d31b90 (LWP 27169)): #0 0xffffe430 in __kernel_vsyscall () #1 0x4024dc35 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0 #2 0x40030094 in ldap_pvt_thread_cond_wait (cond=0x82ba4f4, mutex=0x82ba4dc) at thr_posix.c:277 #3 0x4002f472 in ldap_int_thread_pool_wrapper (xpool=0x82ba4d8) at tpool.c:654 #4 0x4024a1b5 in start_thread () from /lib/libpthread.so.0 #5 0x4057938e in clone () from /lib/libc.so.6
Thread 4 (Thread 0x47233b90 (LWP 27170)): #0 0xffffe430 in __kernel_vsyscall () #1 0x4024dc35 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0 #2 0x40030094 in ldap_pvt_thread_cond_wait (cond=0x82ba4f4, mutex=0x82ba4dc) at thr_posix.c:277 #3 0x4002f472 in ldap_int_thread_pool_wrapper (xpool=0x82ba4d8) at tpool.c:654 #4 0x4024a1b5 in start_thread () from /lib/libpthread.so.0 #5 0x4057938e in clone () from /lib/libc.so.6
Thread 3 (Thread 0x47634b90 (LWP 27173)): #0 0xffffe430 in __kernel_vsyscall () #1 0x4024dc35 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0 #2 0x40030094 in ldap_pvt_thread_cond_wait (cond=0x82ba4f4, mutex=0x82ba4dc) at thr_posix.c:277 #3 0x4002f472 in ldap_int_thread_pool_wrapper (xpool=0x82ba4d8) at tpool.c:654 #4 0x4024a1b5 in start_thread () from /lib/libpthread.so.0 #5 0x4057938e in clone () from /lib/libc.so.6
Thread 2 (Thread 0x47a35b90 (LWP 27174)): #0 0xffffe430 in __kernel_vsyscall () #1 0x4024dc35 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0 #2 0x40030094 in ldap_pvt_thread_cond_wait (cond=0x82ba4f4, mutex=0x82ba4dc) at thr_posix.c:277 #3 0x4002f472 in ldap_int_thread_pool_wrapper (xpool=0x82ba4d8) at tpool.c:654 #4 0x4024a1b5 in start_thread () from /lib/libpthread.so.0 #5 0x4057938e in clone () from /lib/libc.so.6
Thread 1 (Thread 0x406f1930 (LWP 26725)): #0 0xffffe430 in __kernel_vsyscall () #1 0x4024a747 in pthread_join () from /lib/libpthread.so.0 #2 0x400301d4 in ldap_pvt_thread_join (thread=1087458192, thread_return=0x0) at thr_posix.c:197 #3 0x08063693 in slapd_daemon () at daemon.c:2658 #4 0x08050ee6 in main (argc=8, argv=0xbf90dc64) at main.c:948