6:55 a.m.
--0-1358063323-1230648878=:64021
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable
Pierangelo,
I could run the valgrind but at this time I do not have the DB with the aro=
und 4 million subscribers. In this condition, with a small number of users,=
I already could see some "definitive" memory leaks and I believe they are =
associated with OpenLdap.
Just for a history I already compiled the openldap 2.4.11 souce code withou=
t modules and TLS since many lists make reference to some possible issues i=
n RHEL with TLS. Even so I could not have a load without the possible memor=
y leak.
So I do not believe the problems are in the RHEL libtool or glibc.
In any case the result for the valgrind is (short number of entrances at th=
is time):
=3D=3D32608=3D=3D Memcheck, a memory error detector.
=3D=3D32608=3D=3D Copyright (C) 2002-2005, and GNU GPL'd, by Julian Seward =
et al.
=3D=3D32608=3D=3D Using LibVEX rev 1575, a library for dynamic binary trans=
lation.
=3D=3D32608=3D=3D Copyright (C) 2004-2005, and GNU GPL'd, by OpenWorks LLP.
=3D=3D32608=3D=3D Using valgrind-3.1.1, a dynamic binary instrumentation fr=
amework.
=3D=3D32608=3D=3D Copyright (C) 2000-2005, and GNU GPL'd, by Julian Seward =
et al.
=3D=3D32608=3D=3D For more details, rerun with: -v
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 86 f=
rom 2)
=3D=3D32608=3D=3D malloc/free: in use at exit: 751 bytes in 32 blocks.
=3D=3D32608=3D=3D malloc/free: 29,775 allocs, 29,743 frees, 22,150,722 byte=
s allocated.
=3D=3D32608=3D=3D For counts of detected errors, rerun with: -v
=3D=3D32608=3D=3D searching for pointers to 32 not-freed blocks.
=3D=3D32608=3D=3D checked 8,191,292 bytes.
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D 16 bytes in 1 blocks are still reachable in loss record 1=
of 9
=3D=3D32608=3D=3D=A0=A0=A0 at 0x40056BF: calloc (vg_replace_malloc.c:279)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x519364: _dlerror_run (in /lib/libdl-2.3.4.s=
o)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x518D10: dlopen@@GLIBC_2.1 (in /lib/libdl-2.=
3.4.so)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x52D80D: _sasl_get_plugin (in /usr/lib/libsa=
sl2.so.2.0.19)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x52DC61: _sasl_load_plugins (in /usr/lib/lib=
sasl2.so.2.0.19)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x52B063: sasl_server_init (in /usr/lib/libsa=
sl2.so.2.0.19)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x80C82C0: slap_sasl_init (in /usr/libexec/sl=
apd)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x3FFDE2: (below main) (in /lib/tls/libc-2.3.=
4.so)
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D 22 bytes in 1 blocks are still reachable in loss record 2=
of 9
=3D=3D32608=3D=3D=A0=A0=A0 at 0x4004405: malloc (vg_replace_malloc.c:149)
=3D=3D32608=3D=3D=A0=A0=A0 by 0xB77567: (within /usr/lib/libltdl.so.3.1.0)
=3D=3D32608=3D=3D=A0=A0=A0 by 0xB780C8: (within /usr/lib/libltdl.so.3.1.0)
=3D=3D32608=3D=3D=A0=A0=A0 by 0xB7A79D: lt_dlsetsearchpath (in /usr/lib/lib=
ltdl.so.3.1.0)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x8068147: (within /usr/libexec/slapd)
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D 24 bytes in 1 blocks are still reachable in loss record 3=
of 9
=3D=3D32608=3D=3D=A0=A0=A0 at 0x400579F: realloc (vg_replace_malloc.c:306)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x8151E6F: ber_bvarray_add (in /usr/libexec/s=
lapd)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x44DD627: ???
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D 28 bytes in 2 blocks are definitely lost in loss record 4=
of 9
=3D=3D32608=3D=3D=A0=A0=A0 at 0x4004405: malloc (vg_replace_malloc.c:149)
=3D=3D32608=3D=3D=A0=A0=A0 by 0xB77567: (within /usr/lib/libltdl.so.3.1.0)
=3D=3D32608=3D=3D=A0=A0=A0 by 0xB780C8: (within /usr/lib/libltdl.so.3.1.0)
=3D=3D32608=3D=3D=A0=A0=A0 by 0xB78CDC: (within /usr/lib/libltdl.so.3.1.0)
=3D=3D32608=3D=3D=A0=A0=A0 by 0xB7972B: lt_dlopen (in /usr/lib/libltdl.so.3=
.1.0)
=3D=3D32608=3D=3D=A0=A0=A0 by 0xB7980C: lt_dlopenext (in /usr/lib/libltdl.s=
o.3.1.0)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x80CA1E3: module_load (in /usr/libexec/slapd=
)
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D 28 bytes in 1 blocks are still reachable in loss record 5=
of 9
=3D=3D32608=3D=3D=A0=A0=A0 at 0x4004405: malloc (vg_replace_malloc.c:149)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x3D81D7: _dl_map_object_deps (in /lib/ld-2.3=
.4.so)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x4E7158: dl_open_worker (in /lib/tls/libc-2.=
3.4.so)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x3D90FD: _dl_catch_error (in /lib/ld-2.3.4.s=
o)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x4E7CB7: _dl_open (in /lib/tls/libc-2.3.4.so=
)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x4E904C: do_dlopen (in /lib/tls/libc-2.3.4.s=
o)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x3D90FD: _dl_catch_error (in /lib/ld-2.3.4.s=
o)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x4E912D: __libc_dlopen_mode (in /lib/tls/lib=
c-2.3.4.so)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x561087: pthread_cancel_init (in /lib/tls/li=
bpthread-2.3.4.so)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x5611FC: _Unwind_ForcedUnwind (in /lib/tls/l=
ibpthread-2.3.4.so)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x55F020: __pthread_unwind (in /lib/tls/libpt=
hread-2.3.4.so)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x55AF8F: pthread_exit (in /lib/tls/libpthrea=
d-2.3.4.so)
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D 32 bytes in 1 blocks are still reachable in loss record 6=
of 9
=3D=3D32608=3D=3D=A0=A0=A0 at 0x4004405: malloc (vg_replace_malloc.c:149)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x4005728: realloc (vg_replace_malloc.c:306)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x810E504: rewrite_mapper_register (in /usr/l=
ibexec/slapd)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x3FFDE2: (below main) (in /lib/tls/libc-2.3.=
4.so)
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D 77 bytes in 5 blocks are still reachable in loss record 7=
of 9
=3D=3D32608=3D=3D=A0=A0=A0 at 0x4004405: malloc (vg_replace_malloc.c:149)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x8150E2E: ber_memalloc_x (in /usr/libexec/sl=
apd)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x3D6363FF: ???
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D 252 bytes in 18 blocks are definitely lost in loss record=
8 of 9
=3D=3D32608=3D=3D=A0=A0=A0 at 0x4004405: malloc (vg_replace_malloc.c:149)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x8150E2E: ber_memalloc_x (in /usr/libexec/sl=
apd)
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D 272 bytes in 2 blocks are possibly lost in loss record 9 =
of 9
=3D=3D32608=3D=3D=A0=A0=A0 at 0x40056BF: calloc (vg_replace_malloc.c:279)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x3DB71A: _dl_allocate_tls (in /lib/ld-2.3.4.=
so)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x55A91E: pthread_create@@GLIBC_2.1 (in /lib/=
tls/libpthread-2.3.4.so)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x811D4A9: ldap_pvt_thread_create (in /usr/li=
bexec/slapd)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x3FFDE2: (below main) (in /lib/tls/libc-2.3.=
4.so)
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D LEAK SUMMARY:
=3D=3D32608=3D=3D=A0=A0=A0 definitely lost: 280 bytes in 20 blocks.
=3D=3D32608=3D=3D=A0=A0=A0=A0=A0 possibly lost: 272 bytes in 2 blocks.
=3D=3D32608=3D=3D=A0=A0=A0 still reachable: 199 bytes in 10 blocks.
=3D=3D32608=3D=3D=A0=A0=A0=A0=A0=A0=A0=A0 suppressed: 0 bytes in 0 blocks.
My suspicious is this memory leak is happening at ber_memalloc_x (in /usr/l=
ibexec/slapd). Or in the part above :
=3D=3D32608=3D=3D 252 bytes in 18 blocks are definitely lost in loss record=
8 of 9
=3D=3D32608=3D=3D=A0=A0=A0 at 0x4004405: malloc (vg_replace_malloc.c:149)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x8150E2E: ber_memalloc_x (in /usr/libexec/sl=
apd)
Since right now I do not have a large number of entrances in the DB the sea=
rch is easily cached in memory and the response is very quick.
I will load again the DB but since this takes around 6-8hours I sent this i=
nitial result from valgrind since it can already give some directions. Tomo=
rrow I should send more information related with a more heavy load DB.
I would like your help to see if this can identify something related with t=
his memory leakage.
Please let me know if I can help in this meantime with something else.
Best regards,
Rodrigo.
--- On Sat, 12/20/08, Pierangelo Masarati <ando(a)sys-net.it> wrote:
From: Pierangelo Masarati <ando(a)sys-net.it>
Subject: Re: (ITS#5860) slapd memeory leak under openldap 2.4
To: rlvcosta(a)yahoo.com
Cc: openldap-its(a)openldap.org
Date: Saturday, December 20, 2008, 7:05 PM
rlvcosta(a)yahoo.com wrote:
Full_Name: Rodrigo Costa
Version: 2.4.11
OS: RHEL4 U4
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (201.82.161.220)
=20
=20
Opeldap dev team,
=20
I'm facing an issue and after many tentatives I believe this is really
a bug
that maybe could be identified and solved by openldap community. I
tried
to
include dmalloc into openldap 2.4.11 code but even I could compile
with i=
t
any
tentative to execute the code generated "Segmentation
Fault" so
I could not use
dmalloc to identify where the memory leak is happening.
Can you try with valgrind?
p.
Ing. Pierangelo Masarati
OpenLDAP Core Team
SysNet s.r.l.
via Dossi, 8 - 27100 Pavia - ITALIA
http://www.sys-net.it
-----------------------------------
Office: +39 02 23998309
Mobile: +39 333 4963172
Fax: +39 0382 476497
Email: ando(a)sys-net.it
-----------------------------------
=0A=0A=0A
--0-1358063323-1230648878=:64021
Content-Type: text/html; charset=us-ascii
<table cellspacing="0" cellpadding="0" border="0"
><tr><td valign="top" style="font:
inherit;">Pierangelo,<br><br>I could run the valgrind but at this time
I do not have the DB with the around 4 million subscribers. In this condition, with a
small number of users, I already could see some "definitive" memory leaks and I
believe they are associated with OpenLdap.<br><br>Just for a history I already
compiled the openldap 2.4.11 souce code without modules and TLS since many lists make
reference to some possible issues in RHEL with TLS. Even so I could not have a load
without the possible memory leak.<br><br>So I do not believe the problems are
in the RHEL libtool or glibc.<br><br>In any case the result for the valgrind
is (short number of entrances at this time):<br><br>==32608== Memcheck, a
memory error detector.<br>==32608== Copyright (C) 2002-2005, and GNU GPL'd, by
Julian Seward et al.<br>==32608== Using LibVEX rev 1575, a library for dynamic
binary
translation.<br>==32608== Copyright (C) 2004-2005, and GNU GPL'd, by OpenWorks
LLP.<br>==32608== Using valgrind-3.1.1, a dynamic binary instrumentation
framework.<br>==32608== Copyright (C) 2000-2005, and GNU GPL'd, by Julian Seward
et al.<br>==32608== For more details, rerun with: -v<br>==32608==
<br>==32608== <br>==32608== ERROR SUMMARY: 0 errors from 0 contexts
(suppressed: 86 from 2)<br>==32608== malloc/free: in use at exit: 751 bytes in 32
blocks.<br>==32608== malloc/free: 29,775 allocs, 29,743 frees, 22,150,722 bytes
allocated.<br>==32608== For counts of detected errors, rerun with:
-v<br>==32608== searching for pointers to 32 not-freed blocks.<br>==32608==
checked 8,191,292 bytes.<br>==32608== <br>==32608== 16 bytes in 1 blocks are
still reachable in loss record 1 of 9<br>==32608== at
0x40056BF: calloc
(vg_replace_malloc.c:279)<br>==32608== by 0x519364:
_dlerror_run (in
/lib/libdl-2.3.4.so)<br>==32608== by 0x518D10:
dlopen@@GLIBC_2.1 (in
/lib/libdl-2.3.4.so)<br>==32608== by 0x52D80D:
_sasl_get_plugin (in
/usr/lib/libsasl2.so.2.0.19)<br>==32608== by 0x52DC61:
_sasl_load_plugins (in
/usr/lib/libsasl2.so.2.0.19)<br>==32608== by 0x52B063:
sasl_server_init (in
/usr/lib/libsasl2.so.2.0.19)<br>==32608== by
0x80C82C0: slap_sasl_init (in
/usr/libexec/slapd)<br>==32608== by 0x3FFDE2: (below
main) (in /lib/tls/libc-2.3.4.so)<br>==32608== <br>==32608==
<br>==32608== 22 bytes in 1 blocks are still reachable in loss record 2 of
9<br>==32608== at 0x4004405: malloc
(vg_replace_malloc.c:149)<br>==32608== by 0xB77567:
(within /usr/lib/libltdl.so.3.1.0)<br>==32608== by
0xB780C8: (within
/usr/lib/libltdl.so.3.1.0)<br>==32608== by 0xB7A79D:
lt_dlsetsearchpath (in
/usr/lib/libltdl.so.3.1.0)<br>==32608== by 0x8068147:
(within /usr/libexec/slapd)<br>==32608== <br>==32608== <br>==32608== 24
bytes in 1 blocks are still reachable in loss record 3 of
9<br>==32608== at 0x400579F: realloc
(vg_replace_malloc.c:306)<br>==32608== by 0x8151E6F:
ber_bvarray_add (in /usr/libexec/slapd)<br>==32608==
by 0x44DD627: ???<br>==32608== <br>==32608== <br>==32608== 28 bytes in 2
blocks are definitely lost in loss record 4 of
9<br>==32608== at 0x4004405: malloc
(vg_replace_malloc.c:149)<br>==32608== by 0xB77567:
(within /usr/lib/libltdl.so.3.1.0)<br>==32608== by
0xB780C8: (within
/usr/lib/libltdl.so.3.1.0)<br>==32608== by 0xB78CDC:
(within /usr/lib/libltdl.so.3.1.0)<br>==32608== by
0xB7972B: lt_dlopen (in
/usr/lib/libltdl.so.3.1.0)<br>==32608== by 0xB7980C:
lt_dlopenext (in
/usr/lib/libltdl.so.3.1.0)<br>==32608== by 0x80CA1E3:
module_load (in /usr/libexec/slapd)<br>==32608== <br>==32608==
<br>==32608== 28 bytes in 1 blocks are still reachable in loss record 5 of
9<br>==32608== at 0x4004405: malloc
(vg_replace_malloc.c:149)<br>==32608== by 0x3D81D7:
_dl_map_object_deps (in /lib/ld-2.3.4.so)<br>==32608==
by 0x4E7158: dl_open_worker (in
/lib/tls/libc-2.3.4.so)<br>==32608== by 0x3D90FD:
_dl_catch_error (in /lib/ld-2.3.4.so)<br>==32608== by
0x4E7CB7: _dl_open (in
/lib/tls/libc-2.3.4.so)<br>==32608== by 0x4E904C:
do_dlopen (in /lib/tls/libc-2.3.4.so)<br>==32608== by
0x3D90FD: _dl_catch_error (in
/lib/ld-2.3.4.so)<br>==32608== by 0x4E912D:
__libc_dlopen_mode (in
/lib/tls/libc-2.3.4.so)<br>==32608== by 0x561087:
pthread_cancel_init (in
/lib/tls/libpthread-2.3.4.so)<br>==32608== by
0x5611FC: _Unwind_ForcedUnwind (in
/lib/tls/libpthread-2.3.4.so)<br>==32608== by
0x55F020: __pthread_unwind (in
/lib/tls/libpthread-2.3.4.so)<br>==32608== by
0x55AF8F: pthread_exit (in /lib/tls/libpthread-2.3.4.so)<br>==32608==
<br>==32608== <br>==32608== 32 bytes in 1 blocks are still reachable in loss
record 6 of 9<br>==32608== at 0x4004405: malloc
(vg_replace_malloc.c:149)<br>==32608== by 0x4005728:
realloc (vg_replace_malloc.c:306)<br>==32608== by
0x810E504: rewrite_mapper_register (in
/usr/libexec/slapd)<br>==32608== by 0x3FFDE2: (below
main) (in /lib/tls/libc-2.3.4.so)<br>==32608== <br>==32608==
<br>==32608== 77 bytes in 5 blocks are still reachable in loss record 7 of
9<br>==32608== at 0x4004405: malloc
(vg_replace_malloc.c:149)<br>==32608==
by 0x8150E2E: ber_memalloc_x (in
/usr/libexec/slapd)<br>==32608== by 0x3D6363FF:
???<br>==32608== <br>==32608== <br>==32608== 252 bytes in 18 blocks are
definitely lost in loss record 8 of 9<br>==32608== at
0x4004405: malloc
(vg_replace_malloc.c:149)<br>==32608== by 0x8150E2E:
ber_memalloc_x (in /usr/libexec/slapd)<br>==32608== <br>==32608==
<br>==32608== 272 bytes in 2 blocks are possibly lost in loss record 9 of
9<br>==32608== at 0x40056BF: calloc
(vg_replace_malloc.c:279)<br>==32608== by 0x3DB71A:
_dl_allocate_tls (in /lib/ld-2.3.4.so)<br>==32608== by
0x55A91E: pthread_create@@GLIBC_2.1 (in
/lib/tls/libpthread-2.3.4.so)<br>==32608== by
0x811D4A9: ldap_pvt_thread_create (in
/usr/libexec/slapd)<br>==32608== by 0x3FFDE2: (below
main) (in /lib/tls/libc-2.3.4.so)<br>==32608== <br>==32608== LEAK
SUMMARY:<br>==32608== definitely lost: 280 bytes in
20 blocks.<br>==32608== possibly
lost: 272 bytes in 2 blocks.<br>==32608== still
reachable: 199 bytes in 10
blocks.<br>==32608==
suppressed: 0 bytes in 0 blocks.<br><br>My suspicious is this memory leak is
happening at ber_memalloc_x (in /usr/libexec/slapd). Or in the part above
:<br><br>==32608== 252 bytes in 18 blocks are definitely lost in loss record 8
of 9<br>
==32608== at 0x4004405: malloc
(vg_replace_malloc.c:149)<br>
==32608== by 0x8150E2E: ber_memalloc_x (in
/usr/libexec/slapd)<br><br>Since right now I do not have a large number of
entrances in the DB the search is easily cached in memory and the response is very
quick.<br><br>I will load again the DB but since this takes around 6-8hours I
sent this initial result from valgrind since it can already give some directions. Tomorrow
I should send more information related with a more heavy load DB.<br><br>I
would like your help to see if this can identify something related with this memory
leakage.<br><br>Please let me know if I can help in this meantime with
something else.<br><br>Best regards,<br><br>Rodrigo.<span
style="font-family: monospace;"><br><br></span>
--- On <b>Sat, 12/20/08, Pierangelo Masarati
<i>&lt;ando(a)sys-net.it&gt;</i></b>
wrote:<br><blockquote style="border-left: 2px solid rgb(16, 16, 255);
margin-left: 5px; padding-left: 5px;">From: Pierangelo Masarati
&lt;ando(a)sys-net.it&gt;<br>Subject: Re: (ITS#5860) slapd memeory leak under
openldap 2.4<br>To: rlvcosta(a)yahoo.com<br>Cc:
openldap-its(a)openldap.org<br>Date: Saturday, December 20, 2008, 7:05
PM<br><br><pre>rlvcosta(a)yahoo.com wrote:<br>> Full_Name:
Rodrigo Costa<br>> Version: 2.4.11<br>> OS: RHEL4
U4<br>> URL: ftp://ftp.openldap.org/incoming/<br>> Submission
from: (NULL) (201.82.161.220)<br>> <br>> <br>>
Opeldap dev team,<br>> <br>> I'm facing an issue and after
many tentatives I believe this is really<br>a bug<br>> that maybe could
be identified and solved by openldap community. I tried<br>to<br>>
include dmalloc into openldap 2.4.11 code but even I could compile with
it<br>any<br>> tentative to execute
the code generated "Segmentation Fault" so<br>I could not
use<br>> dmalloc to identify where the memory leak is
happening.<br><br>Can you try with
valgrind?<br><br>p.<br><br><br>Ing. Pierangelo
Masarati<br>OpenLDAP Core Team<br><br>SysNet s.r.l.<br>via Dossi,
8 - 27100 Pavia -
ITALIA<br>http://www.sys-net.it<br>-----------------------------------<br>Office:
+39 02 23998309<br>Mobile: +39 333 4963172<br>Fax: +39 0382
476497<br>Email:
ando@sys-net.it<br>-----------------------------------<br><br></pre></blockquote></td></tr></table><br>
--0-1358063323-1230648878=:64021--
5205
days inactive
5205
days old
0 comments
1 participants
participants (1)
-
rlvcosta＠yahoo.com