6:39 a.m.
A few weeks ago, we upgraded to OpenLDAP 2.4.19. All seemed well, for
about 4 weeks, and then within the last few days (starting Jan 14),
slapd has been dying on our replica servers. It doesn't seem to follow
any pattern, and the system seems fine when slapd dies; it isn't out of
memory, and doesn't show any load spikes. In our logs we get messages like:
Jan 14 22:18:34 slapd[7940]: ch_malloc of 14698087920909635104 bytes failed
(from ldap.log)
Jan 15 16:58:03 kernel: slapd[22663] general protection
rip:2ac667af65cc rsp:4659d970 error:0
Jan 16 15:42:34 kernel: slapd[10272] general protection
rip:2b71f6c6d9c4 rsp:45441980 error:0
Jan 16 17:20:01 kernel: slapd[7538] general protection rip:2aec51954ae0
rsp:449518d0 error:0
Jan 19 13:38:46 kernel: slapd[2821] general protection rip:2aeac3070ae0
rsp:4918f8d0 error:0
(from /var/log/messages)
from another replica server:
Jan 16 20:56:15 kernel: slapd[17948] general protection
rip:2aedf150d9c4 rsp:4154f980 error:0
Jan 18 01:42:46 kernel: slapd[9446] general protection rip:2ae369401ae0
rsp:454c08d0 error:0
Jan 19 13:04:29 kernel: slapd[25339] general protection
rip:2b9803877ae0 rsp:4b6bc8d0 error:0
We're running on RHEL 5.4, with Heimdal 1.2.1-3, OpenSSL 0.9.8k,
Cyrus-SASL 2.1.23, BDB 4.7.25 (with patches), libunwind 0.99 (for Google
tcmalloc), Google tcmalloc 1.3.
1. Is there any useful information that can be obtained from these log
entries, or do we simply need to change to a more verbose log level and
wait for slapd to die again?
2. If we need to change our log level, what is a suggested level? Right
now we're using "loglevel sync stats". Would it be wise to change the
log level to -1 (any)? These are production servers, and I imagine
that'd be a huge performance hit.
3. Also, we're logging asynchronously at the moment. Should we disable
this while debugging?
Thanks!
9:44 a.m.
--On Wednesday, January 20, 2010 8:39 AM -0600 "Bryan J. Maupin"
<bmaupin(a)uta.edu> wrote:
We're running on RHEL 5.4, with Heimdal 1.2.1-3, OpenSSL 0.9.8k,
Cyrus-SASL 2.1.23, BDB 4.7.25 (with patches), libunwind 0.99 (for Google
tcmalloc), Google tcmalloc 1.3.
libunwind is not required for tcmalloc, you must be building it incorrectly.
1. Is there any useful information that can be obtained from these
log
entries, or do we simply need to change to a more verbose log level and
wait for slapd to die again?
2. If we need to change our log level, what is a suggested level? Right
now we're using "loglevel sync stats". Would it be wise to change the
log level to -1 (any)? These are production servers, and I imagine
that'd be a huge performance hit.
3. Also, we're logging asynchronously at the moment. Should we disable
this while debugging?
I would suggest you
(a) Upgrade to 2.4.21
(b) Fix your tcmalloc build
(c) If the problem still occurs, run slapd under gdb so you can get a
backtrace of some kind.
Make sure your OpenLDAP build, etc, has debugging symbols.
--Quanah
--
Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra :: the leader in open source messaging and collaboration
12:02 p.m.
Quanah Gibson-Mount <quanah(a)zimbra.com> writes:
--On Wednesday, January 20, 2010 8:39 AM -0600 "Bryan J.
Maupin"
<bmaupin(a)uta.edu> wrote:
> We're running on RHEL 5.4, with Heimdal 1.2.1-3, OpenSSL 0.9.8k,
> Cyrus-SASL 2.1.23, BDB 4.7.25 (with patches), libunwind 0.99 (for Google
> tcmalloc), Google tcmalloc 1.3.
libunwind is not required for tcmalloc, you must be building it incorrectly.
From google-perftools/INSTALL
*** NOTE FOR 64-BIT LINUX SYSTEMS
The glibc built-in stack-unwinder on 64-bit systems has some problems
with the perftools libraries. ...
For that reason, if you use a 64-bit system, we strongly recommend you
install libunwind before trying to configure or install google
perftools. ...
[...]
-Dieter
--
Dieter Klünter | Systemberatung
http://dkluenter.de
GPG Key ID:8EF7B6C6
53°37'09,95"N
10°08'02,42"E
12:24 p.m.
--On Thursday, January 21, 2010 9:02 PM +0100 Dieter Kluenter
<dieter(a)dkluenter.de> wrote:
*** NOTE FOR 64-BIT LINUX SYSTEMS
The glibc built-in stack-unwinder on 64-bit systems has some problems
with the perftools libraries. ...
For that reason, if you use a 64-bit system, we strongly recommend you
install libunwind before trying to configure or install google
perftools. ...
[...]
The perf tools yes. tcmalloc itself, no. You can just build
tcmalloc-minimal, in fact, and skip a ton of purposeless building time.
--Quanah
--
Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra :: the leader in open source messaging and collaboration
12:24 p.m.
Quanah Gibson-Mount wrote:
--On Wednesday, January 20, 2010 8:39 AM -0600 "Bryan J.
Maupin"
<bmaupin(a)uta.edu> wrote:
> We're running on RHEL 5.4, with Heimdal 1.2.1-3, OpenSSL 0.9.8k,
> Cyrus-SASL 2.1.23, BDB 4.7.25 (with patches), libunwind 0.99 (for Google
> tcmalloc), Google tcmalloc 1.3.
>
libunwind is not required for tcmalloc, you must be building it incorrectly.
From the "INSTALL" file that came with Google performance tools 1.3:
"if you use a 64-bit system, we strongly recommend you
install libunwind before trying to configure or install google
perftools. libunwind can be found at
http://download.savannah.gnu.org/releases/libunwind/libunwind-0.99-beta.t...
> 1. Is there any useful information that can be obtained from these log
> entries, or do we simply need to change to a more verbose log level and
> wait for slapd to die again?
> 2. If we need to change our log level, what is a suggested level? Right
> now we're using "loglevel sync stats". Would it be wise to change the
> log level to -1 (any)? These are production servers, and I imagine
> that'd be a huge performance hit.
> 3. Also, we're logging asynchronously at the moment. Should we disable
> this while debugging?
>
I would suggest you
(a) Upgrade to 2.4.21
(b) Fix your tcmalloc build
(c) If the problem still occurs, run slapd under gdb so you can get a
backtrace of some kind.
Make sure your OpenLDAP build, etc, has debugging symbols.
--Quanah
--
Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra :: the leader in open source messaging and collaboration
12:27 p.m.
--On Thursday, January 21, 2010 2:24 PM -0600 "Bryan J. Maupin"
<bmaupin(a)uta.edu> wrote:
Quanah Gibson-Mount wrote:
--On Wednesday, January 20, 2010 8:39 AM -0600 "Bryan J. Maupin"
<bmaupin(a)uta.edu> wrote:
We're running on RHEL 5.4, with Heimdal 1.2.1-3, OpenSSL 0.9.8k,
Cyrus-SASL 2.1.23, BDB 4.7.25 (with patches), libunwind 0.99 (for Google
tcmalloc), Google tcmalloc 1.3.
libunwind is not required for tcmalloc, you must be building it
incorrectly.
From the "INSTALL" file that came with Google performance tools 1.3:
"if you use a 64-bit system, we strongly recommend you
install libunwind before trying to configure or install google
perftools. libunwind can be found at
Yeah, for the perf tools. Not for tcmalloc itself. It's also simpler to
just build the minimal version as well.
--enable-minimal
--Quanah
--
Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra :: the leader in open source messaging and collaboration
2:20 p.m.
Using --enable-minimal will only create libtcmalloc_minimal.so, so
you're saying that's enough, and we don't need libtcmalloc.so, which is
what we're linking to now?
Thanks!
Quanah Gibson-Mount escribió:
--On Thursday, January 21, 2010 2:24 PM -0600 "Bryan J.
Maupin"
<bmaupin(a)uta.edu> wrote:
> Quanah Gibson-Mount wrote:
>
>
> --On Wednesday, January 20, 2010 8:39 AM -0600 "Bryan J. Maupin"
> <bmaupin(a)uta.edu> wrote:
>
>
>
>
> We're running on RHEL 5.4, with Heimdal 1.2.1-3, OpenSSL 0.9.8k,
> Cyrus-SASL 2.1.23, BDB 4.7.25 (with patches), libunwind 0.99 (for Google
> tcmalloc), Google tcmalloc 1.3.
>
>
>
>
> libunwind is not required for tcmalloc, you must be building it
> incorrectly.
>
>
> From the "INSTALL" file that came with Google performance tools 1.3:
>
> "if you use a 64-bit system, we strongly recommend you
> install libunwind before trying to configure or install google
> perftools. libunwind can be found at
>
Yeah, for the perf tools. Not for tcmalloc itself. It's also simpler to
just build the minimal version as well.
--enable-minimal
--Quanah
--
Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra :: the leader in open source messaging and collaboration
2:33 p.m.
--On Thursday, January 21, 2010 4:20 PM -0600 "Bryan J. Maupin"
<bmaupin(a)uta.edu> wrote:
Using --enable-minimal will only create libtcmalloc_minimal.so, so
you're
saying that's enough, and we don't need libtcmalloc.so, which is what
we're linking to now?
Thanks!
That's correct. :)
--Quanah
--
Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra :: the leader in open source messaging and collaboration
8:29 a.m.
Well, I went ahead and built just the minimal tcmalloc (and upgraded it
to 1.5 while I was at it), since that seemed to be something that needed
to be fixed anyway. I installed it to one of our replica servers, and
it ran for about 3 days, and then slapd died again this morning.
So next I'm gonna upgrade to 2.4.21. I'll probably do it on just one of
our replica servers for now.
1. A 2.4.21 replica should work fine with a 2.4.19 master, correct?
2. On the machine I upgrade, can I just stop slapd, upgrade, and start
slapd again, or should I slapcat/slapadd?
Thanks!
Quanah Gibson-Mount escribió:
--On Wednesday, January 20, 2010 8:39 AM -0600 "Bryan J.
Maupin"
<bmaupin(a)uta.edu> wrote:
> We're running on RHEL 5.4, with Heimdal 1.2.1-3, OpenSSL 0.9.8k,
> Cyrus-SASL 2.1.23, BDB 4.7.25 (with patches), libunwind 0.99 (for Google
> tcmalloc), Google tcmalloc 1.3.
>
libunwind is not required for tcmalloc, you must be building it incorrectly.
> 1. Is there any useful information that can be obtained from these log
> entries, or do we simply need to change to a more verbose log level and
> wait for slapd to die again?
> 2. If we need to change our log level, what is a suggested level? Right
> now we're using "loglevel sync stats". Would it be wise to change the
> log level to -1 (any)? These are production servers, and I imagine
> that'd be a huge performance hit.
> 3. Also, we're logging asynchronously at the moment. Should we disable
> this while debugging?
>
I would suggest you
(a) Upgrade to 2.4.21
(b) Fix your tcmalloc build
(c) If the problem still occurs, run slapd under gdb so you can get a
backtrace of some kind.
Make sure your OpenLDAP build, etc, has debugging symbols.
--Quanah
--
Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra :: the leader in open source messaging and collaboration
10:03 a.m.
--On Monday, January 25, 2010 10:29 AM -0600 "Bryan J. Maupin"
<bmaupin(a)uta.edu> wrote:
Well, I went ahead and built just the minimal tcmalloc (and upgraded
it
to 1.5 while I was at it), since that seemed to be something that needed
to be fixed anyway. I installed it to one of our replica servers, and it
ran for about 3 days, and then slapd died again this morning.
So next I'm gonna upgrade to 2.4.21. I'll probably do it on just one of
our replica servers for now.
1. A 2.4.21 replica should work fine with a 2.4.19 master, correct?
2. On the machine I upgrade, can I just stop slapd, upgrade, and start
slapd again, or should I slapcat/slapadd?
As long as you don't change the underlying BDB version that slapd is linked
against, then yes, going from 2.4.19 -> 2.4.21 only requires updating the
binaries with a stop, upgrade, start format. ;)
--Quanah
--
Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra :: the leader in open source messaging and collaboration
9:05 a.m.
Quanah Gibson-Mount escribió:
--On Wednesday, January 20, 2010 8:39 AM -0600 "Bryan J.
Maupin"
<bmaupin(a)uta.edu> wrote:
> We're running on RHEL 5.4, with Heimdal 1.2.1-3, OpenSSL 0.9.8k,
> Cyrus-SASL 2.1.23, BDB 4.7.25 (with patches), libunwind 0.99 (for Google
> tcmalloc), Google tcmalloc 1.3.
>
libunwind is not required for tcmalloc, you must be building it incorrectly.
> 1. Is there any useful information that can be obtained from these log
> entries, or do we simply need to change to a more verbose log level and
> wait for slapd to die again?
> 2. If we need to change our log level, what is a suggested level? Right
> now we're using "loglevel sync stats". Would it be wise to change the
> log level to -1 (any)? These are production servers, and I imagine
> that'd be a huge performance hit.
> 3. Also, we're logging asynchronously at the moment. Should we disable
> this while debugging?
>
I would suggest you
(a) Upgrade to 2.4.21
done
(b) Fix your tcmalloc build
done
(c) If the problem still occurs, run slapd under gdb so you can get a
backtrace of some kind.
problem still occurring
Make sure your OpenLDAP build, etc, has debugging symbols.
I've never done this, so just to be sure, to do this I need to pass
CFLAGS="-g -O0" when running configure, then make install STRIP=""
when
it's time for that step, correct?
Then, I simply "gdb /path/to/slapd [options] /path/to/slapd.core"?
Lastly, do I need to change the slapd log level at all, or will all of
the helpful information be in the core file?
Thanks.
--Quanah
--
Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra :: the leader in open source messaging and collaboration
9:11 a.m.
--On Monday, February 08, 2010 11:05 AM -0600 "Bryan J. Maupin"
<bmaupin(a)uta.edu> wrote:
Make sure your OpenLDAP build, etc, has debugging symbols.
I've never done this, so just to be sure, to do this I need to pass
CFLAGS="-g -O0" when running configure, then make install STRIP=""
when
it's time for that step, correct?
Correct
Then, I simply "gdb /path/to/slapd [options]
/path/to/slapd.core"?
Correct
Lastly, do I need to change the slapd log level at all, or will all
of
the helpful information be in the core file?
For now, loglevel is likely unimportant. The core file has all the
information necessary.
--Quanah
--
Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra :: the leader in open source messaging and collaboration
10:35 a.m.
Okay, ran slapd using gdb, obtained a backtrace (using instructions from
http://www.openldap.org/faq/data/cache/59.html). If needed, I can also
send results from "thread apply all bt"
(gdb) run -d 0 -h "ldap:/// ldaps:///" -u ldap
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x4860e940 (LWP 20454)]
tcmalloc::CentralFreeList::FetchFromSpans (this=0x2b1cdb76c5e0)
at src/central_freelist.cc:249
249 src/central_freelist.cc: No such file or directory.
in src/central_freelist.cc
Current language: auto; currently c++
(gdb) bt full
#0 tcmalloc::CentralFreeList::FetchFromSpans (this=0x2b1cdb76c5e0)
at src/central_freelist.cc:249
span = (tcmalloc::Span *) 0x37efaaf0
result = (void *) 0x6f433d6e632c3230
#1 0x00002b1cdb551939 in tcmalloc::CentralFreeList::FetchFromSpansSafe (
this=0x2b1cdb76c5e0) at src/central_freelist.cc:234
t = <value optimized out>
#2 0x00002b1cdb5519d9 in tcmalloc::CentralFreeList::RemoveRange (
this=0x2b1cdb76c5e0, start=0x4860d840, end=0x4860d838,
N=<value optimized out>) at src/central_freelist.cc:214
result = 0
head = (void *) 0x0
tail = <value optimized out>
#3 0x00002b1cdb554ce4 in tcmalloc::ThreadCache::FetchFromCentralCache (
this=0x61ba168, cl=<value optimized out>, byte_size=256)
at src/thread_cache.cc:149
list = (tcmalloc::ThreadCache::FreeList *) 0x61ba328
batch_size = 32
start = <value optimized out>
end = <value optimized out>
fetch_count = <value optimized out>
#4 0x00002b1cdb5590b6 in tc_malloc (size=244) at src/thread_cache.h:340
result = <value optimized out>
#5 0x00002b1cdd4fc0c2 in CRYPTO_malloc ()
from /opt/openssl/lib/libcrypto.so.0.9.8
No symbol table info available.
#6 0x00002b1cdd55d1a3 in EVP_CipherInit_ex ()
from /opt/openssl/lib/libcrypto.so.0.9.8
No symbol table info available.
#7 0x00002b1cdd272192 in tls1_change_cipher_state ()
from /opt/openssl/lib/libssl.so.0.9.8
No symbol table info available.
#8 0x00002b1cdd264a91 in ssl3_accept () from
/opt/openssl/lib/libssl.so.0.9.8
No symbol table info available.
#9 0x00002b1cdb7cc21e in tlso_session_accept (sess=0x37287e00) at
tls_o.c:376
s = (tlso_session *) 0x37287e00
#10 0x00002b1cdb7c9edc in ldap_pvt_tls_accept (sb=0x25e5c7e0,
ctx_arg=0x65c3e00) at tls2.c:429
err = -2
ssl = (tls_session *) 0x37287e00
#11 0x000000000043a8c2 in connection_read (s=70, cri=0x4860dd10)
at connection.c:1326
rc = 0
c = (Connection *) 0x6658080
__PRETTY_FUNCTION__ = "connection_read"
#12 0x000000000043a47e in connection_read_thread (ctx=0x4860dd70, argv=0x46)
at connection.c:1238
rc = 102813704
cri = {op = 0x0, func = 0, arg = 0x0, ctx = 0x4860dd70, nullop = 0}
s = 70
#13 0x00002b1cdb797cc3 in ldap_int_thread_pool_wrapper (xpool=0x620d000)
at tpool.c:685
pool = (struct ldap_int_thread_pool_s *) 0x620d000
task = (ldap_int_thread_task_t *) 0x9fcbd60
work_list = (ldap_int_tpool_plist_t *) 0x620d098
ctx = {ltu_id = 1214310720, ltu_key = {{ltk_key = 0x439b17,
ltk_data = 0x2c728e00, ltk_free = 0x43997a <conn_counter_destroy>}, {
ltk_key = 0x4a9316, ltk_data = 0x1fbd71c0,
ltk_free = 0x4a9110 <slap_sl_mem_destroy>}, {ltk_key = 0x65d3800,
ltk_data = 0x2c854480, ltk_free = 0x518a54 <bdb_reader_free>}, {
ltk_key = 0x4ea875, ltk_data = 0x2ca5b000,
ltk_free = 0x4ea855 <search_stack_free>}, {ltk_key = 0x4544a4,
ltk_data = 0x65b8000, ltk_free = 0x454400 <slap_op_q_destroy>}, {
ltk_key = 0x0, ltk_data = 0x31fa0000, ltk_free = 0}, {ltk_key = 0x0,
ltk_data = 0x0, ltk_free = 0} <repeats 26 times>}}
kctx = (ldap_int_thread_userctx_t *) 0x0
i = 32
keyslot = 81
hash = 1878162513
pool = (struct ldap_int_thread_pool_s *) 0x620d000
task = (ldap_int_thread_task_t *) 0x9fcbd60
work_list = (ldap_int_tpool_plist_t *) 0x620d098
ctx = {ltu_id = 1214310720, ltu_key = {{ltk_key = 0x439b17,
ltk_data = 0x2c728e00, ltk_free = 0x43997a <conn_counter_destroy>}, {
ltk_key = 0x4a9316, ltk_data = 0x1fbd71c0,
ltk_free = 0x4a9110 <slap_sl_mem_destroy>}, {ltk_key = 0x65d3800,
ltk_data = 0x2c854480, ltk_free = 0x518a54 <bdb_reader_free>}, {
ltk_key = 0x4ea875, ltk_data = 0x2ca5b000,
ltk_free = 0x4ea855 <search_stack_free>}, {ltk_key = 0x4544a4,
ltk_data = 0x65b8000, ltk_free = 0x454400 <slap_op_q_destroy>}, {
ltk_key = 0x0, ltk_data = 0x31fa0000, ltk_free = 0}, {ltk_key = 0x0,
ltk_data = 0x0, ltk_free = 0} <repeats 26 times>}}
kctx = (ldap_int_thread_userctx_t *) 0x0
i = 32
keyslot = 81
hash = 1878162513
__PRETTY_FUNCTION__ = "ldap_int_thread_pool_wrapper"
#14 0x0000003e65406617 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#15 0x0000003e64cd3c2d in clone () from /lib64/libc.so.6
No symbol table info available.
Quanah Gibson-Mount escribio':
--On Monday, February 08, 2010 11:05 AM -0600 "Bryan J.
Maupin"
<bmaupin(a)uta.edu> wrote:
> Make sure your OpenLDAP build, etc, has debugging symbols.
>
>
> I've never done this, so just to be sure, to do this I need to pass
> CFLAGS="-g -O0" when running configure, then make install
STRIP="" when
> it's time for that step, correct?
>
Correct
> Then, I simply "gdb /path/to/slapd [options] /path/to/slapd.core"?
>
Correct
> Lastly, do I need to change the slapd log level at all, or will all of
> the helpful information be in the core file?
>
For now, loglevel is likely unimportant. The core file has all the
information necessary.
--Quanah
--
Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra :: the leader in open source messaging and collaboration
5:29 p.m.
You need to submit an ITS at http://www.openldap.org/its/ and with the full output of thr
apply all bt full rather than sending this to the list. ;)
--Quanah
----- "Bryan J. Maupin" <bmaupin(a)uta.edu> wrote:
Okay, ran slapd using gdb, obtained a backtrace (using instructions
from
http://www.openldap.org/faq/data/cache/59.html). If needed, I can also
send results from "thread apply all bt"
(gdb) run -d 0 -h "ldap:/// ldaps:///" -u ldap
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x4860e940 (LWP 20454)]
tcmalloc::CentralFreeList::FetchFromSpans (this=0x2b1cdb76c5e0)
at src/central_freelist.cc:249
249 src/central_freelist.cc: No such file or directory.
in src/central_freelist.cc
Current language: auto; currently c++
(gdb) bt full
#0 tcmalloc::CentralFreeList::FetchFromSpans (this=0x2b1cdb76c5e0)
at src/central_freelist.cc:249
span = (tcmalloc::Span *) 0x37efaaf0
result = (void *) 0x6f433d6e632c3230
#1 0x00002b1cdb551939 in tcmalloc::CentralFreeList::FetchFromSpansSafe
(
this=0x2b1cdb76c5e0) at src/central_freelist.cc:234
t = <value optimized out>
#2 0x00002b1cdb5519d9 in tcmalloc::CentralFreeList::RemoveRange (
this=0x2b1cdb76c5e0, start=0x4860d840, end=0x4860d838,
N=<value optimized out>) at src/central_freelist.cc:214
result = 0
head = (void *) 0x0
tail = <value optimized out>
#3 0x00002b1cdb554ce4 in tcmalloc::ThreadCache::FetchFromCentralCache
(
this=0x61ba168, cl=<value optimized out>, byte_size=256)
at src/thread_cache.cc:149
list = (tcmalloc::ThreadCache::FreeList *) 0x61ba328
batch_size = 32
start = <value optimized out>
end = <value optimized out>
fetch_count = <value optimized out>
#4 0x00002b1cdb5590b6 in tc_malloc (size=244) at
src/thread_cache.h:340
result = <value optimized out>
#5 0x00002b1cdd4fc0c2 in CRYPTO_malloc ()
from /opt/openssl/lib/libcrypto.so.0.9.8
No symbol table info available.
#6 0x00002b1cdd55d1a3 in EVP_CipherInit_ex ()
from /opt/openssl/lib/libcrypto.so.0.9.8
No symbol table info available.
#7 0x00002b1cdd272192 in tls1_change_cipher_state ()
from /opt/openssl/lib/libssl.so.0.9.8
No symbol table info available.
#8 0x00002b1cdd264a91 in ssl3_accept () from
/opt/openssl/lib/libssl.so.0.9.8
No symbol table info available.
#9 0x00002b1cdb7cc21e in tlso_session_accept (sess=0x37287e00) at
tls_o.c:376
s = (tlso_session *) 0x37287e00
#10 0x00002b1cdb7c9edc in ldap_pvt_tls_accept (sb=0x25e5c7e0,
ctx_arg=0x65c3e00) at tls2.c:429
err = -2
ssl = (tls_session *) 0x37287e00
#11 0x000000000043a8c2 in connection_read (s=70, cri=0x4860dd10)
at connection.c:1326
rc = 0
c = (Connection *) 0x6658080
__PRETTY_FUNCTION__ = "connection_read"
#12 0x000000000043a47e in connection_read_thread (ctx=0x4860dd70,
argv=0x46)
at connection.c:1238
rc = 102813704
cri = {op = 0x0, func = 0, arg = 0x0, ctx = 0x4860dd70, nullop = 0}
s = 70
#13 0x00002b1cdb797cc3 in ldap_int_thread_pool_wrapper
(xpool=0x620d000)
at tpool.c:685
pool = (struct ldap_int_thread_pool_s *) 0x620d000
task = (ldap_int_thread_task_t *) 0x9fcbd60
work_list = (ldap_int_tpool_plist_t *) 0x620d098
ctx = {ltu_id = 1214310720, ltu_key = {{ltk_key = 0x439b17,
ltk_data = 0x2c728e00, ltk_free = 0x43997a <conn_counter_destroy>}, {
ltk_key = 0x4a9316, ltk_data = 0x1fbd71c0,
ltk_free = 0x4a9110 <slap_sl_mem_destroy>}, {ltk_key = 0x65d3800,
ltk_data = 0x2c854480, ltk_free = 0x518a54 <bdb_reader_free>}, {
ltk_key = 0x4ea875, ltk_data = 0x2ca5b000,
ltk_free = 0x4ea855 <search_stack_free>}, {ltk_key = 0x4544a4,
ltk_data = 0x65b8000, ltk_free = 0x454400 <slap_op_q_destroy>}, {
ltk_key = 0x0, ltk_data = 0x31fa0000, ltk_free = 0}, {ltk_key = 0x0,
ltk_data = 0x0, ltk_free = 0} <repeats 26 times>}}
kctx = (ldap_int_thread_userctx_t *) 0x0
i = 32
keyslot = 81
hash = 1878162513
pool = (struct ldap_int_thread_pool_s *) 0x620d000
task = (ldap_int_thread_task_t *) 0x9fcbd60
work_list = (ldap_int_tpool_plist_t *) 0x620d098
ctx = {ltu_id = 1214310720, ltu_key = {{ltk_key = 0x439b17,
ltk_data = 0x2c728e00, ltk_free = 0x43997a <conn_counter_destroy>}, {
ltk_key = 0x4a9316, ltk_data = 0x1fbd71c0,
ltk_free = 0x4a9110 <slap_sl_mem_destroy>}, {ltk_key = 0x65d3800,
ltk_data = 0x2c854480, ltk_free = 0x518a54 <bdb_reader_free>}, {
ltk_key = 0x4ea875, ltk_data = 0x2ca5b000,
ltk_free = 0x4ea855 <search_stack_free>}, {ltk_key = 0x4544a4,
ltk_data = 0x65b8000, ltk_free = 0x454400 <slap_op_q_destroy>}, {
ltk_key = 0x0, ltk_data = 0x31fa0000, ltk_free = 0}, {ltk_key = 0x0,
ltk_data = 0x0, ltk_free = 0} <repeats 26 times>}}
kctx = (ldap_int_thread_userctx_t *) 0x0
i = 32
keyslot = 81
hash = 1878162513
__PRETTY_FUNCTION__ = "ldap_int_thread_pool_wrapper"
#14 0x0000003e65406617 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#15 0x0000003e64cd3c2d in clone () from /lib64/libc.so.6
No symbol table info available.
Quanah Gibson-Mount escribio':
> --On Monday, February 08, 2010 11:05 AM -0600 "Bryan J. Maupin"
> <bmaupin(a)uta.edu> wrote:
>
>
>> Make sure your OpenLDAP build, etc, has debugging symbols.
>>
>>
>> I've never done this, so just to be sure, to do this I need to
pass
>> CFLAGS="-g -O0" when running configure, then make install
STRIP=""
when
>> it's time for that step, correct?
>>
>
> Correct
>
>
>> Then, I simply "gdb /path/to/slapd [options] /path/to/slapd.core"?
>>
>
> Correct
>
>
>> Lastly, do I need to change the slapd log level at all, or will all
of
>> the helpful information be in the core file?
>>
>
> For now, loglevel is likely unimportant. The core file has all the
> information necessary.
>
> --Quanah
>
> --
>
> Quanah Gibson-Mount
> Principal Software Engineer
> Zimbra, Inc
> --------------------
> Zimbra :: the leader in open source messaging and collaboration
>
--
Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra :: the leader in open source messaging and collaboration
4793
days inactive
4814
days old
openldap-technical@openldap.org
13 comments
3 participants
participants (3)
-
Bryan J. Maupin -
Dieter Kluenter -
Quanah Gibson-Mount