A few weeks ago, we upgraded to OpenLDAP 2.4.19. All seemed well, for about 4 weeks, and then within the last few days (starting Jan 14), slapd has been dying on our replica servers. It doesn't seem to follow any pattern, and the system seems fine when slapd dies; it isn't out of memory, and doesn't show any load spikes. In our logs we get messages like:
Jan 14 22:18:34 slapd[7940]: ch_malloc of 14698087920909635104 bytes failed (from ldap.log) Jan 15 16:58:03 kernel: slapd[22663] general protection rip:2ac667af65cc rsp:4659d970 error:0 Jan 16 15:42:34 kernel: slapd[10272] general protection rip:2b71f6c6d9c4 rsp:45441980 error:0 Jan 16 17:20:01 kernel: slapd[7538] general protection rip:2aec51954ae0 rsp:449518d0 error:0 Jan 19 13:38:46 kernel: slapd[2821] general protection rip:2aeac3070ae0 rsp:4918f8d0 error:0 (from /var/log/messages)
from another replica server: Jan 16 20:56:15 kernel: slapd[17948] general protection rip:2aedf150d9c4 rsp:4154f980 error:0 Jan 18 01:42:46 kernel: slapd[9446] general protection rip:2ae369401ae0 rsp:454c08d0 error:0 Jan 19 13:04:29 kernel: slapd[25339] general protection rip:2b9803877ae0 rsp:4b6bc8d0 error:0
We're running on RHEL 5.4, with Heimdal 1.2.1-3, OpenSSL 0.9.8k, Cyrus-SASL 2.1.23, BDB 4.7.25 (with patches), libunwind 0.99 (for Google tcmalloc), Google tcmalloc 1.3.
1. Is there any useful information that can be obtained from these log entries, or do we simply need to change to a more verbose log level and wait for slapd to die again? 2. If we need to change our log level, what is a suggested level? Right now we're using "loglevel sync stats". Would it be wise to change the log level to -1 (any)? These are production servers, and I imagine that'd be a huge performance hit. 3. Also, we're logging asynchronously at the moment. Should we disable this while debugging?
Thanks!
--On Wednesday, January 20, 2010 8:39 AM -0600 "Bryan J. Maupin" bmaupin@uta.edu wrote:
We're running on RHEL 5.4, with Heimdal 1.2.1-3, OpenSSL 0.9.8k, Cyrus-SASL 2.1.23, BDB 4.7.25 (with patches), libunwind 0.99 (for Google tcmalloc), Google tcmalloc 1.3.
libunwind is not required for tcmalloc, you must be building it incorrectly.
- Is there any useful information that can be obtained from these log
entries, or do we simply need to change to a more verbose log level and wait for slapd to die again? 2. If we need to change our log level, what is a suggested level? Right now we're using "loglevel sync stats". Would it be wise to change the log level to -1 (any)? These are production servers, and I imagine that'd be a huge performance hit. 3. Also, we're logging asynchronously at the moment. Should we disable this while debugging?
I would suggest you
(a) Upgrade to 2.4.21 (b) Fix your tcmalloc build (c) If the problem still occurs, run slapd under gdb so you can get a backtrace of some kind.
Make sure your OpenLDAP build, etc, has debugging symbols.
--Quanah
--
Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration
Quanah Gibson-Mount quanah@zimbra.com writes:
--On Wednesday, January 20, 2010 8:39 AM -0600 "Bryan J. Maupin" bmaupin@uta.edu wrote:
We're running on RHEL 5.4, with Heimdal 1.2.1-3, OpenSSL 0.9.8k, Cyrus-SASL 2.1.23, BDB 4.7.25 (with patches), libunwind 0.99 (for Google tcmalloc), Google tcmalloc 1.3.
libunwind is not required for tcmalloc, you must be building it incorrectly.
From google-perftools/INSTALL
*** NOTE FOR 64-BIT LINUX SYSTEMS The glibc built-in stack-unwinder on 64-bit systems has some problems with the perftools libraries. ... For that reason, if you use a 64-bit system, we strongly recommend you install libunwind before trying to configure or install google perftools. ... [...]
-Dieter
--On Thursday, January 21, 2010 9:02 PM +0100 Dieter Kluenter dieter@dkluenter.de wrote:
*** NOTE FOR 64-BIT LINUX SYSTEMS The glibc built-in stack-unwinder on 64-bit systems has some problems with the perftools libraries. ... For that reason, if you use a 64-bit system, we strongly recommend you install libunwind before trying to configure or install google perftools. ... [...]
The perf tools yes. tcmalloc itself, no. You can just build tcmalloc-minimal, in fact, and skip a ton of purposeless building time.
--Quanah
--
Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration
Quanah Gibson-Mount wrote:
--On Wednesday, January 20, 2010 8:39 AM -0600 "Bryan J. Maupin" bmaupin@uta.edu wrote:
We're running on RHEL 5.4, with Heimdal 1.2.1-3, OpenSSL 0.9.8k, Cyrus-SASL 2.1.23, BDB 4.7.25 (with patches), libunwind 0.99 (for Google tcmalloc), Google tcmalloc 1.3.
libunwind is not required for tcmalloc, you must be building it incorrectly.
From the "INSTALL" file that came with Google performance tools 1.3:
"if you use a 64-bit system, we strongly recommend you install libunwind before trying to configure or install google perftools. libunwind can be found at
http://download.savannah.gnu.org/releases/libunwind/libunwind-0.99-beta.tar...."
- Is there any useful information that can be obtained from these log
entries, or do we simply need to change to a more verbose log level and wait for slapd to die again? 2. If we need to change our log level, what is a suggested level? Right now we're using "loglevel sync stats". Would it be wise to change the log level to -1 (any)? These are production servers, and I imagine that'd be a huge performance hit. 3. Also, we're logging asynchronously at the moment. Should we disable this while debugging?
I would suggest you
(a) Upgrade to 2.4.21 (b) Fix your tcmalloc build (c) If the problem still occurs, run slapd under gdb so you can get a backtrace of some kind.
Make sure your OpenLDAP build, etc, has debugging symbols.
--Quanah
--
Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc
Zimbra :: the leader in open source messaging and collaboration
--On Thursday, January 21, 2010 2:24 PM -0600 "Bryan J. Maupin" bmaupin@uta.edu wrote:
Quanah Gibson-Mount wrote:
--On Wednesday, January 20, 2010 8:39 AM -0600 "Bryan J. Maupin" bmaupin@uta.edu wrote:
We're running on RHEL 5.4, with Heimdal 1.2.1-3, OpenSSL 0.9.8k, Cyrus-SASL 2.1.23, BDB 4.7.25 (with patches), libunwind 0.99 (for Google tcmalloc), Google tcmalloc 1.3.
libunwind is not required for tcmalloc, you must be building it incorrectly.
From the "INSTALL" file that came with Google performance tools 1.3:
"if you use a 64-bit system, we strongly recommend you install libunwind before trying to configure or install google perftools. libunwind can be found at
Yeah, for the perf tools. Not for tcmalloc itself. It's also simpler to just build the minimal version as well.
--enable-minimal
--Quanah
--
Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration
Using --enable-minimal will only create libtcmalloc_minimal.so, so you're saying that's enough, and we don't need libtcmalloc.so, which is what we're linking to now?
Thanks!
Quanah Gibson-Mount escribió:
--On Thursday, January 21, 2010 2:24 PM -0600 "Bryan J. Maupin" bmaupin@uta.edu wrote:
Quanah Gibson-Mount wrote:
--On Wednesday, January 20, 2010 8:39 AM -0600 "Bryan J. Maupin" bmaupin@uta.edu wrote:
We're running on RHEL 5.4, with Heimdal 1.2.1-3, OpenSSL 0.9.8k, Cyrus-SASL 2.1.23, BDB 4.7.25 (with patches), libunwind 0.99 (for Google tcmalloc), Google tcmalloc 1.3.
libunwind is not required for tcmalloc, you must be building it incorrectly.
From the "INSTALL" file that came with Google performance tools 1.3:
"if you use a 64-bit system, we strongly recommend you install libunwind before trying to configure or install google perftools. libunwind can be found at
Yeah, for the perf tools. Not for tcmalloc itself. It's also simpler to just build the minimal version as well.
--enable-minimal
--Quanah
--
Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc
Zimbra :: the leader in open source messaging and collaboration
--On Thursday, January 21, 2010 4:20 PM -0600 "Bryan J. Maupin" bmaupin@uta.edu wrote:
Using --enable-minimal will only create libtcmalloc_minimal.so, so you're saying that's enough, and we don't need libtcmalloc.so, which is what we're linking to now?
Thanks!
That's correct. :)
--Quanah
--
Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration
Well, I went ahead and built just the minimal tcmalloc (and upgraded it to 1.5 while I was at it), since that seemed to be something that needed to be fixed anyway. I installed it to one of our replica servers, and it ran for about 3 days, and then slapd died again this morning.
So next I'm gonna upgrade to 2.4.21. I'll probably do it on just one of our replica servers for now.
1. A 2.4.21 replica should work fine with a 2.4.19 master, correct? 2. On the machine I upgrade, can I just stop slapd, upgrade, and start slapd again, or should I slapcat/slapadd?
Thanks!
Quanah Gibson-Mount escribió:
--On Wednesday, January 20, 2010 8:39 AM -0600 "Bryan J. Maupin" bmaupin@uta.edu wrote:
We're running on RHEL 5.4, with Heimdal 1.2.1-3, OpenSSL 0.9.8k, Cyrus-SASL 2.1.23, BDB 4.7.25 (with patches), libunwind 0.99 (for Google tcmalloc), Google tcmalloc 1.3.
libunwind is not required for tcmalloc, you must be building it incorrectly.
- Is there any useful information that can be obtained from these log
entries, or do we simply need to change to a more verbose log level and wait for slapd to die again? 2. If we need to change our log level, what is a suggested level? Right now we're using "loglevel sync stats". Would it be wise to change the log level to -1 (any)? These are production servers, and I imagine that'd be a huge performance hit. 3. Also, we're logging asynchronously at the moment. Should we disable this while debugging?
I would suggest you
(a) Upgrade to 2.4.21 (b) Fix your tcmalloc build (c) If the problem still occurs, run slapd under gdb so you can get a backtrace of some kind.
Make sure your OpenLDAP build, etc, has debugging symbols.
--Quanah
--
Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc
Zimbra :: the leader in open source messaging and collaboration
--On Monday, January 25, 2010 10:29 AM -0600 "Bryan J. Maupin" bmaupin@uta.edu wrote:
Well, I went ahead and built just the minimal tcmalloc (and upgraded it to 1.5 while I was at it), since that seemed to be something that needed to be fixed anyway. I installed it to one of our replica servers, and it ran for about 3 days, and then slapd died again this morning.
So next I'm gonna upgrade to 2.4.21. I'll probably do it on just one of our replica servers for now.
- A 2.4.21 replica should work fine with a 2.4.19 master, correct?
- On the machine I upgrade, can I just stop slapd, upgrade, and start
slapd again, or should I slapcat/slapadd?
As long as you don't change the underlying BDB version that slapd is linked against, then yes, going from 2.4.19 -> 2.4.21 only requires updating the binaries with a stop, upgrade, start format. ;)
--Quanah
--
Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration
Quanah Gibson-Mount escribió:
--On Wednesday, January 20, 2010 8:39 AM -0600 "Bryan J. Maupin" bmaupin@uta.edu wrote:
We're running on RHEL 5.4, with Heimdal 1.2.1-3, OpenSSL 0.9.8k, Cyrus-SASL 2.1.23, BDB 4.7.25 (with patches), libunwind 0.99 (for Google tcmalloc), Google tcmalloc 1.3.
libunwind is not required for tcmalloc, you must be building it incorrectly.
- Is there any useful information that can be obtained from these log
entries, or do we simply need to change to a more verbose log level and wait for slapd to die again? 2. If we need to change our log level, what is a suggested level? Right now we're using "loglevel sync stats". Would it be wise to change the log level to -1 (any)? These are production servers, and I imagine that'd be a huge performance hit. 3. Also, we're logging asynchronously at the moment. Should we disable this while debugging?
I would suggest you
(a) Upgrade to 2.4.21
done
(b) Fix your tcmalloc build
done
(c) If the problem still occurs, run slapd under gdb so you can get a backtrace of some kind.
problem still occurring
Make sure your OpenLDAP build, etc, has debugging symbols.
I've never done this, so just to be sure, to do this I need to pass CFLAGS="-g -O0" when running configure, then make install STRIP="" when it's time for that step, correct?
Then, I simply "gdb /path/to/slapd [options] /path/to/slapd.core"?
Lastly, do I need to change the slapd log level at all, or will all of the helpful information be in the core file?
Thanks.
--Quanah
--
Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc
Zimbra :: the leader in open source messaging and collaboration
--On Monday, February 08, 2010 11:05 AM -0600 "Bryan J. Maupin" bmaupin@uta.edu wrote:
Make sure your OpenLDAP build, etc, has debugging symbols.
I've never done this, so just to be sure, to do this I need to pass CFLAGS="-g -O0" when running configure, then make install STRIP="" when it's time for that step, correct?
Correct
Then, I simply "gdb /path/to/slapd [options] /path/to/slapd.core"?
Correct
Lastly, do I need to change the slapd log level at all, or will all of the helpful information be in the core file?
For now, loglevel is likely unimportant. The core file has all the information necessary.
--Quanah
--
Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration
Okay, ran slapd using gdb, obtained a backtrace (using instructions from http://www.openldap.org/faq/data/cache/59.html). If needed, I can also send results from "thread apply all bt"
(gdb) run -d 0 -h "ldap:/// ldaps:///" -u ldap Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x4860e940 (LWP 20454)] tcmalloc::CentralFreeList::FetchFromSpans (this=0x2b1cdb76c5e0) at src/central_freelist.cc:249 249 src/central_freelist.cc: No such file or directory. in src/central_freelist.cc Current language: auto; currently c++ (gdb) bt full #0 tcmalloc::CentralFreeList::FetchFromSpans (this=0x2b1cdb76c5e0) at src/central_freelist.cc:249 span = (tcmalloc::Span *) 0x37efaaf0 result = (void *) 0x6f433d6e632c3230 #1 0x00002b1cdb551939 in tcmalloc::CentralFreeList::FetchFromSpansSafe ( this=0x2b1cdb76c5e0) at src/central_freelist.cc:234 t = <value optimized out> #2 0x00002b1cdb5519d9 in tcmalloc::CentralFreeList::RemoveRange ( this=0x2b1cdb76c5e0, start=0x4860d840, end=0x4860d838, N=<value optimized out>) at src/central_freelist.cc:214 result = 0 head = (void *) 0x0 tail = <value optimized out> #3 0x00002b1cdb554ce4 in tcmalloc::ThreadCache::FetchFromCentralCache ( this=0x61ba168, cl=<value optimized out>, byte_size=256) at src/thread_cache.cc:149 list = (tcmalloc::ThreadCache::FreeList *) 0x61ba328 batch_size = 32 start = <value optimized out> end = <value optimized out> fetch_count = <value optimized out> #4 0x00002b1cdb5590b6 in tc_malloc (size=244) at src/thread_cache.h:340 result = <value optimized out> #5 0x00002b1cdd4fc0c2 in CRYPTO_malloc () from /opt/openssl/lib/libcrypto.so.0.9.8 No symbol table info available. #6 0x00002b1cdd55d1a3 in EVP_CipherInit_ex () from /opt/openssl/lib/libcrypto.so.0.9.8 No symbol table info available. #7 0x00002b1cdd272192 in tls1_change_cipher_state () from /opt/openssl/lib/libssl.so.0.9.8 No symbol table info available. #8 0x00002b1cdd264a91 in ssl3_accept () from /opt/openssl/lib/libssl.so.0.9.8 No symbol table info available. #9 0x00002b1cdb7cc21e in tlso_session_accept (sess=0x37287e00) at tls_o.c:376 s = (tlso_session *) 0x37287e00 #10 0x00002b1cdb7c9edc in ldap_pvt_tls_accept (sb=0x25e5c7e0, ctx_arg=0x65c3e00) at tls2.c:429 err = -2 ssl = (tls_session *) 0x37287e00 #11 0x000000000043a8c2 in connection_read (s=70, cri=0x4860dd10) at connection.c:1326 rc = 0 c = (Connection *) 0x6658080 __PRETTY_FUNCTION__ = "connection_read" #12 0x000000000043a47e in connection_read_thread (ctx=0x4860dd70, argv=0x46) at connection.c:1238 rc = 102813704 cri = {op = 0x0, func = 0, arg = 0x0, ctx = 0x4860dd70, nullop = 0} s = 70 #13 0x00002b1cdb797cc3 in ldap_int_thread_pool_wrapper (xpool=0x620d000) at tpool.c:685 pool = (struct ldap_int_thread_pool_s *) 0x620d000 task = (ldap_int_thread_task_t *) 0x9fcbd60 work_list = (ldap_int_tpool_plist_t *) 0x620d098 ctx = {ltu_id = 1214310720, ltu_key = {{ltk_key = 0x439b17, ltk_data = 0x2c728e00, ltk_free = 0x43997a <conn_counter_destroy>}, { ltk_key = 0x4a9316, ltk_data = 0x1fbd71c0, ltk_free = 0x4a9110 <slap_sl_mem_destroy>}, {ltk_key = 0x65d3800, ltk_data = 0x2c854480, ltk_free = 0x518a54 <bdb_reader_free>}, { ltk_key = 0x4ea875, ltk_data = 0x2ca5b000, ltk_free = 0x4ea855 <search_stack_free>}, {ltk_key = 0x4544a4, ltk_data = 0x65b8000, ltk_free = 0x454400 <slap_op_q_destroy>}, { ltk_key = 0x0, ltk_data = 0x31fa0000, ltk_free = 0}, {ltk_key = 0x0, ltk_data = 0x0, ltk_free = 0} <repeats 26 times>}} kctx = (ldap_int_thread_userctx_t *) 0x0 i = 32 keyslot = 81 hash = 1878162513 pool = (struct ldap_int_thread_pool_s *) 0x620d000 task = (ldap_int_thread_task_t *) 0x9fcbd60 work_list = (ldap_int_tpool_plist_t *) 0x620d098 ctx = {ltu_id = 1214310720, ltu_key = {{ltk_key = 0x439b17, ltk_data = 0x2c728e00, ltk_free = 0x43997a <conn_counter_destroy>}, { ltk_key = 0x4a9316, ltk_data = 0x1fbd71c0, ltk_free = 0x4a9110 <slap_sl_mem_destroy>}, {ltk_key = 0x65d3800, ltk_data = 0x2c854480, ltk_free = 0x518a54 <bdb_reader_free>}, { ltk_key = 0x4ea875, ltk_data = 0x2ca5b000, ltk_free = 0x4ea855 <search_stack_free>}, {ltk_key = 0x4544a4, ltk_data = 0x65b8000, ltk_free = 0x454400 <slap_op_q_destroy>}, { ltk_key = 0x0, ltk_data = 0x31fa0000, ltk_free = 0}, {ltk_key = 0x0, ltk_data = 0x0, ltk_free = 0} <repeats 26 times>}} kctx = (ldap_int_thread_userctx_t *) 0x0 i = 32 keyslot = 81 hash = 1878162513 __PRETTY_FUNCTION__ = "ldap_int_thread_pool_wrapper" #14 0x0000003e65406617 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #15 0x0000003e64cd3c2d in clone () from /lib64/libc.so.6 No symbol table info available.
Quanah Gibson-Mount escribio':
--On Monday, February 08, 2010 11:05 AM -0600 "Bryan J. Maupin" bmaupin@uta.edu wrote:
Make sure your OpenLDAP build, etc, has debugging symbols.
I've never done this, so just to be sure, to do this I need to pass CFLAGS="-g -O0" when running configure, then make install STRIP="" when it's time for that step, correct?
Correct
Then, I simply "gdb /path/to/slapd [options] /path/to/slapd.core"?
Correct
Lastly, do I need to change the slapd log level at all, or will all of the helpful information be in the core file?
For now, loglevel is likely unimportant. The core file has all the information necessary.
--Quanah
--
Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc
Zimbra :: the leader in open source messaging and collaboration
You need to submit an ITS at http://www.openldap.org/its/ and with the full output of thr apply all bt full rather than sending this to the list. ;)
--Quanah
----- "Bryan J. Maupin" bmaupin@uta.edu wrote:
Okay, ran slapd using gdb, obtained a backtrace (using instructions from http://www.openldap.org/faq/data/cache/59.html). If needed, I can also
send results from "thread apply all bt"
(gdb) run -d 0 -h "ldap:/// ldaps:///" -u ldap Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x4860e940 (LWP 20454)] tcmalloc::CentralFreeList::FetchFromSpans (this=0x2b1cdb76c5e0) at src/central_freelist.cc:249 249 src/central_freelist.cc: No such file or directory. in src/central_freelist.cc Current language: auto; currently c++ (gdb) bt full #0 tcmalloc::CentralFreeList::FetchFromSpans (this=0x2b1cdb76c5e0) at src/central_freelist.cc:249 span = (tcmalloc::Span *) 0x37efaaf0 result = (void *) 0x6f433d6e632c3230 #1 0x00002b1cdb551939 in tcmalloc::CentralFreeList::FetchFromSpansSafe ( this=0x2b1cdb76c5e0) at src/central_freelist.cc:234 t = <value optimized out> #2 0x00002b1cdb5519d9 in tcmalloc::CentralFreeList::RemoveRange ( this=0x2b1cdb76c5e0, start=0x4860d840, end=0x4860d838, N=<value optimized out>) at src/central_freelist.cc:214 result = 0 head = (void *) 0x0 tail = <value optimized out> #3 0x00002b1cdb554ce4 in tcmalloc::ThreadCache::FetchFromCentralCache ( this=0x61ba168, cl=<value optimized out>, byte_size=256) at src/thread_cache.cc:149 list = (tcmalloc::ThreadCache::FreeList *) 0x61ba328 batch_size = 32 start = <value optimized out> end = <value optimized out> fetch_count = <value optimized out> #4 0x00002b1cdb5590b6 in tc_malloc (size=244) at src/thread_cache.h:340 result = <value optimized out> #5 0x00002b1cdd4fc0c2 in CRYPTO_malloc () from /opt/openssl/lib/libcrypto.so.0.9.8 No symbol table info available. #6 0x00002b1cdd55d1a3 in EVP_CipherInit_ex () from /opt/openssl/lib/libcrypto.so.0.9.8 No symbol table info available. #7 0x00002b1cdd272192 in tls1_change_cipher_state () from /opt/openssl/lib/libssl.so.0.9.8 No symbol table info available. #8 0x00002b1cdd264a91 in ssl3_accept () from /opt/openssl/lib/libssl.so.0.9.8 No symbol table info available. #9 0x00002b1cdb7cc21e in tlso_session_accept (sess=0x37287e00) at tls_o.c:376 s = (tlso_session *) 0x37287e00 #10 0x00002b1cdb7c9edc in ldap_pvt_tls_accept (sb=0x25e5c7e0, ctx_arg=0x65c3e00) at tls2.c:429 err = -2 ssl = (tls_session *) 0x37287e00 #11 0x000000000043a8c2 in connection_read (s=70, cri=0x4860dd10) at connection.c:1326 rc = 0 c = (Connection *) 0x6658080 __PRETTY_FUNCTION__ = "connection_read" #12 0x000000000043a47e in connection_read_thread (ctx=0x4860dd70, argv=0x46) at connection.c:1238 rc = 102813704 cri = {op = 0x0, func = 0, arg = 0x0, ctx = 0x4860dd70, nullop = 0} s = 70 #13 0x00002b1cdb797cc3 in ldap_int_thread_pool_wrapper (xpool=0x620d000) at tpool.c:685 pool = (struct ldap_int_thread_pool_s *) 0x620d000 task = (ldap_int_thread_task_t *) 0x9fcbd60 work_list = (ldap_int_tpool_plist_t *) 0x620d098 ctx = {ltu_id = 1214310720, ltu_key = {{ltk_key = 0x439b17, ltk_data = 0x2c728e00, ltk_free = 0x43997a <conn_counter_destroy>}, { ltk_key = 0x4a9316, ltk_data = 0x1fbd71c0, ltk_free = 0x4a9110 <slap_sl_mem_destroy>}, {ltk_key = 0x65d3800, ltk_data = 0x2c854480, ltk_free = 0x518a54 <bdb_reader_free>}, { ltk_key = 0x4ea875, ltk_data = 0x2ca5b000, ltk_free = 0x4ea855 <search_stack_free>}, {ltk_key = 0x4544a4, ltk_data = 0x65b8000, ltk_free = 0x454400 <slap_op_q_destroy>}, { ltk_key = 0x0, ltk_data = 0x31fa0000, ltk_free = 0}, {ltk_key = 0x0, ltk_data = 0x0, ltk_free = 0} <repeats 26 times>}} kctx = (ldap_int_thread_userctx_t *) 0x0 i = 32 keyslot = 81 hash = 1878162513 pool = (struct ldap_int_thread_pool_s *) 0x620d000 task = (ldap_int_thread_task_t *) 0x9fcbd60 work_list = (ldap_int_tpool_plist_t *) 0x620d098 ctx = {ltu_id = 1214310720, ltu_key = {{ltk_key = 0x439b17, ltk_data = 0x2c728e00, ltk_free = 0x43997a <conn_counter_destroy>}, { ltk_key = 0x4a9316, ltk_data = 0x1fbd71c0, ltk_free = 0x4a9110 <slap_sl_mem_destroy>}, {ltk_key = 0x65d3800, ltk_data = 0x2c854480, ltk_free = 0x518a54 <bdb_reader_free>}, { ltk_key = 0x4ea875, ltk_data = 0x2ca5b000, ltk_free = 0x4ea855 <search_stack_free>}, {ltk_key = 0x4544a4, ltk_data = 0x65b8000, ltk_free = 0x454400 <slap_op_q_destroy>}, { ltk_key = 0x0, ltk_data = 0x31fa0000, ltk_free = 0}, {ltk_key = 0x0, ltk_data = 0x0, ltk_free = 0} <repeats 26 times>}} kctx = (ldap_int_thread_userctx_t *) 0x0 i = 32 keyslot = 81 hash = 1878162513 __PRETTY_FUNCTION__ = "ldap_int_thread_pool_wrapper" #14 0x0000003e65406617 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #15 0x0000003e64cd3c2d in clone () from /lib64/libc.so.6 No symbol table info available.
Quanah Gibson-Mount escribio':
--On Monday, February 08, 2010 11:05 AM -0600 "Bryan J. Maupin" bmaupin@uta.edu wrote:
Make sure your OpenLDAP build, etc, has debugging symbols.
I've never done this, so just to be sure, to do this I need to
pass
CFLAGS="-g -O0" when running configure, then make install STRIP=""
when
it's time for that step, correct?
Correct
Then, I simply "gdb /path/to/slapd [options] /path/to/slapd.core"?
Correct
Lastly, do I need to change the slapd log level at all, or will all
of
the helpful information be in the core file?
For now, loglevel is likely unimportant. The core file has all the
information necessary.
--Quanah
--
Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc
Zimbra :: the leader in open source messaging and collaboration
openldap-technical@openldap.org
-
Bryan J. Maupin -
Dieter Kluenter -
Quanah Gibson-Mount