ando(a)sys-net.it wrote:
> Full_Name: Pierangelo Masarati
> Version: HEAD
> OS: irrelevant
> URL: ftp://ftp.openldap.org/incoming/
> Submission from: (NULL) (82.63.140.131)
> Submitted by: ando
>
>
> After updating tests/run.in from CVS (rev 1.59) and reconfiguring, executing
> test050 with SYNCMODE=ro results in a failure. The failure is not strictly
> reproducible, but it always occurred to me, either affecting one or another
> delete that is not propagated to all participants to the MMR pool. I've raised
> the sleep time, to make sure there's no chance replication did not occur yet.
>
> I wonder if refreshOnly makes any sense for a MMR pool. If it doesn't, it
> should not be allowed. If it does, deletes should be replicated correctly.
This sounds like a dup of #5843. Note that you cannot disallow MMR for
refreshOnly, since servers must be able to refresh when starting up.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
Full_Name: Sandeep Kumbhar
Version: openldap-2.3.27-8.el5_1.3
OS: CentOS 5.2
URL:
Submission from: (NULL) (59.181.122.24)
1. I am using below schema for my LDAP server
include /etc/openldap/schema/core.schema
include /etc/openldap/schema/cosine.schema
include /etc/openldap/schema/inetorgperson.schema
include /etc/openldap/schema/nis.schema
2. I created local user on the same server and did the following to generate the
data for step 3.
# grep 'username' /etc/passwd > /etc/openldap/passwd.username
Ref: <http://fedoranews.org/mediawiki/index.php/How_to_setup_and_maintain_OpenLDA…>
3. Then I used the migration perl script for converting the local users to LDAP
users.
# /usr/share/openldap/migration/migrate_passwd.pl
/etc/openldap/passwd.username \
/etc/openldap/username.ldif
4. Now after creation of the *.ldif I used the below command to add this
database
file into the LDAP server.
# ldapadd -x -D "cn=Manager,dc=intra,dc=exlinuz,dc=com" -W -f /etc/openldap/
\
username.ldif
5. After typing the ldapadd command I gave the LDAP password I got the error
below
adding new entry "uid=sandeepk,ou=People1,dc=intra,dc=exlinuz,dc=com"
ldap_add: No such object (32)
matched DN: dc=intra,dc=exlinuz,dc=com
6. I have created the domain.ldif and root.ldif and added them successfully into
the
LDAP Server using the official Openldap documentation.
However I could not find anything adding Unix users and therefore I used the
above referenced link to generate Unix users database file for LDAP.
Please see the ldif file output by the migration script
dn: uid=samk,ou=People1,dc=intra,dc=exlinuz,dc=com
uid: samk
cn: samk
objectClass: account
objectClass: posixAccount
objectClass: top
objectClass: shadowAccount
userPassword: {crypt}$1$eBNsXmF1$L9/bK4vbjkAOCKa5DKKAE0
shadowLastChange: 14249
shadowMax: 99999
shadowWarning: 7
loginShell: /bin/bash
uidNumber: 500
gidNumber: 500
homeDirectory: /home/samk
Full_Name: Pierangelo Masarati
Version: HEAD
OS: irrelevant
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (82.63.140.131)
Submitted by: ando
Many features could benefit from being restricted to a(n extended) X.501 subtree
specification using a common interface (API, configuration). Examples are:
- ACL
- limits
- custom functionalities provided by overlays
p.
rlvcosta(a)yahoo.com wrote:
> --0-1358063323-1230648878=:64021
> Content-Type: text/plain; charset=iso-8859-1
> Content-Transfer-Encoding: quoted-printable
>
> Pierangelo,
>
> I could run the valgrind but at this time I do not have the DB with the aro=
> und 4 million subscribers. In this condition, with a small number of users,=
> I already could see some "definitive" memory leaks and I believe they are =
> associated with OpenLdap.
>
> Just for a history I already compiled the openldap 2.4.11 souce code withou=
> t modules and TLS since many lists make reference to some possible issues i=
> n RHEL with TLS. Even so I could not have a load without the possible memor=
> y leak.
>
> So I do not believe the problems are in the RHEL libtool or glibc.
>
> In any case the result for the valgrind is (short number of entrances at th=
> is time):
You probably need to use an unstripped binary in order to find out where
mallocs/frees occur. Most of the occurrences you notice are either
one-time leaks. Moreover, 2.4.13 is out, and it might have fixed some.
I suggest you further investigate using the latest code, using an
unstripped binary.
p.
Ing. Pierangelo Masarati
OpenLDAP Core Team
SysNet s.r.l.
via Dossi, 8 - 27100 Pavia - ITALIA
http://www.sys-net.it
-----------------------------------
Office: +39 02 23998309
Mobile: +39 333 4963172
Fax: +39 0382 476497
Email: ando(a)sys-net.it
-----------------------------------
--0-1358063323-1230648878=:64021
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable
Pierangelo,
I could run the valgrind but at this time I do not have the DB with the aro=
und 4 million subscribers. In this condition, with a small number of users,=
I already could see some "definitive" memory leaks and I believe they are =
associated with OpenLdap.
Just for a history I already compiled the openldap 2.4.11 souce code withou=
t modules and TLS since many lists make reference to some possible issues i=
n RHEL with TLS. Even so I could not have a load without the possible memor=
y leak.
So I do not believe the problems are in the RHEL libtool or glibc.
In any case the result for the valgrind is (short number of entrances at th=
is time):
=3D=3D32608=3D=3D Memcheck, a memory error detector.
=3D=3D32608=3D=3D Copyright (C) 2002-2005, and GNU GPL'd, by Julian Seward =
et al.
=3D=3D32608=3D=3D Using LibVEX rev 1575, a library for dynamic binary trans=
lation.
=3D=3D32608=3D=3D Copyright (C) 2004-2005, and GNU GPL'd, by OpenWorks LLP.
=3D=3D32608=3D=3D Using valgrind-3.1.1, a dynamic binary instrumentation fr=
amework.
=3D=3D32608=3D=3D Copyright (C) 2000-2005, and GNU GPL'd, by Julian Seward =
et al.
=3D=3D32608=3D=3D For more details, rerun with: -v
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 86 f=
rom 2)
=3D=3D32608=3D=3D malloc/free: in use at exit: 751 bytes in 32 blocks.
=3D=3D32608=3D=3D malloc/free: 29,775 allocs, 29,743 frees, 22,150,722 byte=
s allocated.
=3D=3D32608=3D=3D For counts of detected errors, rerun with: -v
=3D=3D32608=3D=3D searching for pointers to 32 not-freed blocks.
=3D=3D32608=3D=3D checked 8,191,292 bytes.
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D 16 bytes in 1 blocks are still reachable in loss record 1=
of 9
=3D=3D32608=3D=3D=A0=A0=A0 at 0x40056BF: calloc (vg_replace_malloc.c:279)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x519364: _dlerror_run (in /lib/libdl-2.3.4.s=
o)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x518D10: dlopen@@GLIBC_2.1 (in /lib/libdl-2.=
3.4.so)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x52D80D: _sasl_get_plugin (in /usr/lib/libsa=
sl2.so.2.0.19)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x52DC61: _sasl_load_plugins (in /usr/lib/lib=
sasl2.so.2.0.19)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x52B063: sasl_server_init (in /usr/lib/libsa=
sl2.so.2.0.19)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x80C82C0: slap_sasl_init (in /usr/libexec/sl=
apd)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x3FFDE2: (below main) (in /lib/tls/libc-2.3.=
4.so)
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D 22 bytes in 1 blocks are still reachable in loss record 2=
of 9
=3D=3D32608=3D=3D=A0=A0=A0 at 0x4004405: malloc (vg_replace_malloc.c:149)
=3D=3D32608=3D=3D=A0=A0=A0 by 0xB77567: (within /usr/lib/libltdl.so.3.1.0)
=3D=3D32608=3D=3D=A0=A0=A0 by 0xB780C8: (within /usr/lib/libltdl.so.3.1.0)
=3D=3D32608=3D=3D=A0=A0=A0 by 0xB7A79D: lt_dlsetsearchpath (in /usr/lib/lib=
ltdl.so.3.1.0)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x8068147: (within /usr/libexec/slapd)
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D 24 bytes in 1 blocks are still reachable in loss record 3=
of 9
=3D=3D32608=3D=3D=A0=A0=A0 at 0x400579F: realloc (vg_replace_malloc.c:306)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x8151E6F: ber_bvarray_add (in /usr/libexec/s=
lapd)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x44DD627: ???
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D 28 bytes in 2 blocks are definitely lost in loss record 4=
of 9
=3D=3D32608=3D=3D=A0=A0=A0 at 0x4004405: malloc (vg_replace_malloc.c:149)
=3D=3D32608=3D=3D=A0=A0=A0 by 0xB77567: (within /usr/lib/libltdl.so.3.1.0)
=3D=3D32608=3D=3D=A0=A0=A0 by 0xB780C8: (within /usr/lib/libltdl.so.3.1.0)
=3D=3D32608=3D=3D=A0=A0=A0 by 0xB78CDC: (within /usr/lib/libltdl.so.3.1.0)
=3D=3D32608=3D=3D=A0=A0=A0 by 0xB7972B: lt_dlopen (in /usr/lib/libltdl.so.3=
.1.0)
=3D=3D32608=3D=3D=A0=A0=A0 by 0xB7980C: lt_dlopenext (in /usr/lib/libltdl.s=
o.3.1.0)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x80CA1E3: module_load (in /usr/libexec/slapd=
)
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D 28 bytes in 1 blocks are still reachable in loss record 5=
of 9
=3D=3D32608=3D=3D=A0=A0=A0 at 0x4004405: malloc (vg_replace_malloc.c:149)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x3D81D7: _dl_map_object_deps (in /lib/ld-2.3=
.4.so)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x4E7158: dl_open_worker (in /lib/tls/libc-2.=
3.4.so)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x3D90FD: _dl_catch_error (in /lib/ld-2.3.4.s=
o)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x4E7CB7: _dl_open (in /lib/tls/libc-2.3.4.so=
)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x4E904C: do_dlopen (in /lib/tls/libc-2.3.4.s=
o)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x3D90FD: _dl_catch_error (in /lib/ld-2.3.4.s=
o)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x4E912D: __libc_dlopen_mode (in /lib/tls/lib=
c-2.3.4.so)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x561087: pthread_cancel_init (in /lib/tls/li=
bpthread-2.3.4.so)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x5611FC: _Unwind_ForcedUnwind (in /lib/tls/l=
ibpthread-2.3.4.so)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x55F020: __pthread_unwind (in /lib/tls/libpt=
hread-2.3.4.so)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x55AF8F: pthread_exit (in /lib/tls/libpthrea=
d-2.3.4.so)
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D 32 bytes in 1 blocks are still reachable in loss record 6=
of 9
=3D=3D32608=3D=3D=A0=A0=A0 at 0x4004405: malloc (vg_replace_malloc.c:149)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x4005728: realloc (vg_replace_malloc.c:306)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x810E504: rewrite_mapper_register (in /usr/l=
ibexec/slapd)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x3FFDE2: (below main) (in /lib/tls/libc-2.3.=
4.so)
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D 77 bytes in 5 blocks are still reachable in loss record 7=
of 9
=3D=3D32608=3D=3D=A0=A0=A0 at 0x4004405: malloc (vg_replace_malloc.c:149)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x8150E2E: ber_memalloc_x (in /usr/libexec/sl=
apd)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x3D6363FF: ???
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D 252 bytes in 18 blocks are definitely lost in loss record=
8 of 9
=3D=3D32608=3D=3D=A0=A0=A0 at 0x4004405: malloc (vg_replace_malloc.c:149)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x8150E2E: ber_memalloc_x (in /usr/libexec/sl=
apd)
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D 272 bytes in 2 blocks are possibly lost in loss record 9 =
of 9
=3D=3D32608=3D=3D=A0=A0=A0 at 0x40056BF: calloc (vg_replace_malloc.c:279)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x3DB71A: _dl_allocate_tls (in /lib/ld-2.3.4.=
so)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x55A91E: pthread_create@@GLIBC_2.1 (in /lib/=
tls/libpthread-2.3.4.so)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x811D4A9: ldap_pvt_thread_create (in /usr/li=
bexec/slapd)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x3FFDE2: (below main) (in /lib/tls/libc-2.3.=
4.so)
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D LEAK SUMMARY:
=3D=3D32608=3D=3D=A0=A0=A0 definitely lost: 280 bytes in 20 blocks.
=3D=3D32608=3D=3D=A0=A0=A0=A0=A0 possibly lost: 272 bytes in 2 blocks.
=3D=3D32608=3D=3D=A0=A0=A0 still reachable: 199 bytes in 10 blocks.
=3D=3D32608=3D=3D=A0=A0=A0=A0=A0=A0=A0=A0 suppressed: 0 bytes in 0 blocks.
My suspicious is this memory leak is happening at ber_memalloc_x (in /usr/l=
ibexec/slapd). Or in the part above :
=3D=3D32608=3D=3D 252 bytes in 18 blocks are definitely lost in loss record=
8 of 9
=3D=3D32608=3D=3D=A0=A0=A0 at 0x4004405: malloc (vg_replace_malloc.c:149)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x8150E2E: ber_memalloc_x (in /usr/libexec/sl=
apd)
Since right now I do not have a large number of entrances in the DB the sea=
rch is easily cached in memory and the response is very quick.
I will load again the DB but since this takes around 6-8hours I sent this i=
nitial result from valgrind since it can already give some directions. Tomo=
rrow I should send more information related with a more heavy load DB.
I would like your help to see if this can identify something related with t=
his memory leakage.
Please let me know if I can help in this meantime with something else.
Best regards,
Rodrigo.
--- On Sat, 12/20/08, Pierangelo Masarati <ando(a)sys-net.it> wrote:
From: Pierangelo Masarati <ando(a)sys-net.it>
Subject: Re: (ITS#5860) slapd memeory leak under openldap 2.4
To: rlvcosta(a)yahoo.com
Cc: openldap-its(a)openldap.org
Date: Saturday, December 20, 2008, 7:05 PM
rlvcosta(a)yahoo.com wrote:
> Full_Name: Rodrigo Costa
> Version: 2.4.11
> OS: RHEL4 U4
> URL: ftp://ftp.openldap.org/incoming/
> Submission from: (NULL) (201.82.161.220)
>=20
>=20
> Opeldap dev team,
>=20
> I'm facing an issue and after many tentatives I believe this is really
a bug
> that maybe could be identified and solved by openldap community. I tried
to
> include dmalloc into openldap 2.4.11 code but even I could compile with i=
t
any
> tentative to execute the code generated "Segmentation Fault" so
I could not use
> dmalloc to identify where the memory leak is happening.
Can you try with valgrind?
p.
Ing. Pierangelo Masarati
OpenLDAP Core Team
SysNet s.r.l.
via Dossi, 8 - 27100 Pavia - ITALIA
http://www.sys-net.it
-----------------------------------
Office: +39 02 23998309
Mobile: +39 333 4963172
Fax: +39 0382 476497
Email: ando(a)sys-net.it
-----------------------------------
=0A=0A=0A
--0-1358063323-1230648878=:64021
Content-Type: text/html; charset=us-ascii
<table cellspacing="0" cellpadding="0" border="0" ><tr><td valign="top" style="font: inherit;">Pierangelo,<br><br>I could run the valgrind but at this time I do not have the DB with the around 4 million subscribers. In this condition, with a small number of users, I already could see some "definitive" memory leaks and I believe they are associated with OpenLdap.<br><br>Just for a history I already compiled the openldap 2.4.11 souce code without modules and TLS since many lists make reference to some possible issues in RHEL with TLS. Even so I could not have a load without the possible memory leak.<br><br>So I do not believe the problems are in the RHEL libtool or glibc.<br><br>In any case the result for the valgrind is (short number of entrances at this time):<br><br>==32608== Memcheck, a memory error detector.<br>==32608== Copyright (C) 2002-2005, and GNU GPL'd, by Julian Seward et al.<br>==32608== Using LibVEX rev 1575, a library for dynamic binary
translation.<br>==32608== Copyright (C) 2004-2005, and GNU GPL'd, by OpenWorks LLP.<br>==32608== Using valgrind-3.1.1, a dynamic binary instrumentation framework.<br>==32608== Copyright (C) 2000-2005, and GNU GPL'd, by Julian Seward et al.<br>==32608== For more details, rerun with: -v<br>==32608== <br>==32608== <br>==32608== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 86 from 2)<br>==32608== malloc/free: in use at exit: 751 bytes in 32 blocks.<br>==32608== malloc/free: 29,775 allocs, 29,743 frees, 22,150,722 bytes allocated.<br>==32608== For counts of detected errors, rerun with: -v<br>==32608== searching for pointers to 32 not-freed blocks.<br>==32608== checked 8,191,292 bytes.<br>==32608== <br>==32608== 16 bytes in 1 blocks are still reachable in loss record 1 of 9<br>==32608== at 0x40056BF: calloc (vg_replace_malloc.c:279)<br>==32608== by 0x519364: _dlerror_run (in
/lib/libdl-2.3.4.so)<br>==32608== by 0x518D10: dlopen@@GLIBC_2.1 (in /lib/libdl-2.3.4.so)<br>==32608== by 0x52D80D: _sasl_get_plugin (in /usr/lib/libsasl2.so.2.0.19)<br>==32608== by 0x52DC61: _sasl_load_plugins (in /usr/lib/libsasl2.so.2.0.19)<br>==32608== by 0x52B063: sasl_server_init (in /usr/lib/libsasl2.so.2.0.19)<br>==32608== by 0x80C82C0: slap_sasl_init (in /usr/libexec/slapd)<br>==32608== by 0x3FFDE2: (below main) (in /lib/tls/libc-2.3.4.so)<br>==32608== <br>==32608== <br>==32608== 22 bytes in 1 blocks are still reachable in loss record 2 of 9<br>==32608== at 0x4004405: malloc (vg_replace_malloc.c:149)<br>==32608== by 0xB77567: (within /usr/lib/libltdl.so.3.1.0)<br>==32608== by 0xB780C8: (within /usr/lib/libltdl.so.3.1.0)<br>==32608== by 0xB7A79D: lt_dlsetsearchpath (in
/usr/lib/libltdl.so.3.1.0)<br>==32608== by 0x8068147: (within /usr/libexec/slapd)<br>==32608== <br>==32608== <br>==32608== 24 bytes in 1 blocks are still reachable in loss record 3 of 9<br>==32608== at 0x400579F: realloc (vg_replace_malloc.c:306)<br>==32608== by 0x8151E6F: ber_bvarray_add (in /usr/libexec/slapd)<br>==32608== by 0x44DD627: ???<br>==32608== <br>==32608== <br>==32608== 28 bytes in 2 blocks are definitely lost in loss record 4 of 9<br>==32608== at 0x4004405: malloc (vg_replace_malloc.c:149)<br>==32608== by 0xB77567: (within /usr/lib/libltdl.so.3.1.0)<br>==32608== by 0xB780C8: (within /usr/lib/libltdl.so.3.1.0)<br>==32608== by 0xB78CDC: (within /usr/lib/libltdl.so.3.1.0)<br>==32608== by 0xB7972B: lt_dlopen (in /usr/lib/libltdl.so.3.1.0)<br>==32608== by 0xB7980C:
lt_dlopenext (in /usr/lib/libltdl.so.3.1.0)<br>==32608== by 0x80CA1E3: module_load (in /usr/libexec/slapd)<br>==32608== <br>==32608== <br>==32608== 28 bytes in 1 blocks are still reachable in loss record 5 of 9<br>==32608== at 0x4004405: malloc (vg_replace_malloc.c:149)<br>==32608== by 0x3D81D7: _dl_map_object_deps (in /lib/ld-2.3.4.so)<br>==32608== by 0x4E7158: dl_open_worker (in /lib/tls/libc-2.3.4.so)<br>==32608== by 0x3D90FD: _dl_catch_error (in /lib/ld-2.3.4.so)<br>==32608== by 0x4E7CB7: _dl_open (in /lib/tls/libc-2.3.4.so)<br>==32608== by 0x4E904C: do_dlopen (in /lib/tls/libc-2.3.4.so)<br>==32608== by 0x3D90FD: _dl_catch_error (in /lib/ld-2.3.4.so)<br>==32608== by 0x4E912D: __libc_dlopen_mode (in /lib/tls/libc-2.3.4.so)<br>==32608== by 0x561087: pthread_cancel_init (in
/lib/tls/libpthread-2.3.4.so)<br>==32608== by 0x5611FC: _Unwind_ForcedUnwind (in /lib/tls/libpthread-2.3.4.so)<br>==32608== by 0x55F020: __pthread_unwind (in /lib/tls/libpthread-2.3.4.so)<br>==32608== by 0x55AF8F: pthread_exit (in /lib/tls/libpthread-2.3.4.so)<br>==32608== <br>==32608== <br>==32608== 32 bytes in 1 blocks are still reachable in loss record 6 of 9<br>==32608== at 0x4004405: malloc (vg_replace_malloc.c:149)<br>==32608== by 0x4005728: realloc (vg_replace_malloc.c:306)<br>==32608== by 0x810E504: rewrite_mapper_register (in /usr/libexec/slapd)<br>==32608== by 0x3FFDE2: (below main) (in /lib/tls/libc-2.3.4.so)<br>==32608== <br>==32608== <br>==32608== 77 bytes in 5 blocks are still reachable in loss record 7 of 9<br>==32608== at 0x4004405: malloc (vg_replace_malloc.c:149)<br>==32608==
by 0x8150E2E: ber_memalloc_x (in /usr/libexec/slapd)<br>==32608== by 0x3D6363FF: ???<br>==32608== <br>==32608== <br>==32608== 252 bytes in 18 blocks are definitely lost in loss record 8 of 9<br>==32608== at 0x4004405: malloc (vg_replace_malloc.c:149)<br>==32608== by 0x8150E2E: ber_memalloc_x (in /usr/libexec/slapd)<br>==32608== <br>==32608== <br>==32608== 272 bytes in 2 blocks are possibly lost in loss record 9 of 9<br>==32608== at 0x40056BF: calloc (vg_replace_malloc.c:279)<br>==32608== by 0x3DB71A: _dl_allocate_tls (in /lib/ld-2.3.4.so)<br>==32608== by 0x55A91E: pthread_create@@GLIBC_2.1 (in /lib/tls/libpthread-2.3.4.so)<br>==32608== by 0x811D4A9: ldap_pvt_thread_create (in /usr/libexec/slapd)<br>==32608== by 0x3FFDE2: (below main) (in /lib/tls/libc-2.3.4.so)<br>==32608== <br>==32608== LEAK
SUMMARY:<br>==32608== definitely lost: 280 bytes in 20 blocks.<br>==32608== possibly lost: 272 bytes in 2 blocks.<br>==32608== still reachable: 199 bytes in 10 blocks.<br>==32608== suppressed: 0 bytes in 0 blocks.<br><br>My suspicious is this memory leak is happening at ber_memalloc_x (in /usr/libexec/slapd). Or in the part above :<br><br>==32608== 252 bytes in 18 blocks are definitely lost in loss record 8 of 9<br>
==32608== at 0x4004405: malloc (vg_replace_malloc.c:149)<br>
==32608== by 0x8150E2E: ber_memalloc_x (in /usr/libexec/slapd)<br><br>Since right now I do not have a large number of entrances in the DB the search is easily cached in memory and the response is very quick.<br><br>I will load again the DB but since this takes around 6-8hours I sent this initial result from valgrind since it can already give some directions. Tomorrow I should send more information related with a more heavy load DB.<br><br>I would like your help to see if this can identify something related with this memory leakage.<br><br>Please let me know if I can help in this meantime with something else.<br><br>Best regards,<br><br>Rodrigo.<span style="font-family: monospace;"><br><br></span>
--- On <b>Sat, 12/20/08, Pierangelo Masarati <i><ando(a)sys-net.it></i></b> wrote:<br><blockquote style="border-left: 2px solid rgb(16, 16, 255); margin-left: 5px; padding-left: 5px;">From: Pierangelo Masarati <ando(a)sys-net.it><br>Subject: Re: (ITS#5860) slapd memeory leak under openldap 2.4<br>To: rlvcosta(a)yahoo.com<br>Cc: openldap-its(a)openldap.org<br>Date: Saturday, December 20, 2008, 7:05 PM<br><br><pre>rlvcosta(a)yahoo.com wrote:<br>> Full_Name: Rodrigo Costa<br>> Version: 2.4.11<br>> OS: RHEL4 U4<br>> URL: ftp://ftp.openldap.org/incoming/<br>> Submission from: (NULL) (201.82.161.220)<br>> <br>> <br>> Opeldap dev team,<br>> <br>> I'm facing an issue and after many tentatives I believe this is really<br>a bug<br>> that maybe could be identified and solved by openldap community. I tried<br>to<br>> include dmalloc into openldap 2.4.11 code but even I could compile with it<br>any<br>> tentative to execute
the code generated "Segmentation Fault" so<br>I could not use<br>> dmalloc to identify where the memory leak is happening.<br><br>Can you try with valgrind?<br><br>p.<br><br><br>Ing. Pierangelo Masarati<br>OpenLDAP Core Team<br><br>SysNet s.r.l.<br>via Dossi, 8 - 27100 Pavia - ITALIA<br>http://www.sys-net.it<br>-----------------------------------<br>Office: +39 02 23998309<br>Mobile: +39 333 4963172<br>Fax: +39 0382 476497<br>Email: ando(a)sys-net.it<br>-----------------------------------<br><br></pre></blockquote></td></tr></table><br>
--0-1358063323-1230648878=:64021--
Pierangelo Masarati <ando(a)sys-net.it> wrote:
> At a quick glance, you're leaking the callback containing the cloak
> response. You should remove (and free it) when called with REP_RESULT
> as sr_type, and add a slap_freeself_cb() cleanup hook. You won't notice
> the memory leak unless you build slapd with SLAP_NO_SL_MALLOC set, or
> unless your overlay operates within a slab memory intensive context.
Here is an update, I hope it addresses the concerns you raised:
ftp://ftp.openldap.org/incoming/manu-cloak-20081230.c
--
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu(a)netbsd.org
On Dec 28, 2008, at 4:52 PM, kurt(a)openldap.org wrote:
> On a technical note, use of platform macros (sun and gcc versions)
> should be avoided. Instead, autoconf should be used to detect
> conditions for need to be adapted for.
>
> On an IPR review, I suspect it would be easier to update to:
> http://www.openbsd.org/cgi-bin/cvsweb/src/lib/libc/hash/sha1.c?rev=1.21;con…
>
> then to try to identify the contributors and get their "okay".
I should note that, from a technical point of view, it might also be
better to integrate the latest from OpenBSD. This would hopefully
provide a SHA1 implementation which works on more platforms than the
current version, or the current version as modified by Klaus's patch.
-- Kurt
For Klaus's benefit.
Begin forwarded message:
> From: Kurt Zeilenga <Kurt(a)OpenLDAP.org>
> Date: December 29, 2008 11:01:08 AM PST
> To: manu(a)netbsd.org
> Cc: openldap-its(a)openldap.org
> Subject: Re: (ITS#5875) sparc64/gcc-3.3 build fix
>
>
> On Dec 28, 2008, at 10:37 PM, manu(a)netbsd.org wrote:
>
>> Howard Chu <hyc(a)symas.com> wrote:
>>
>>>> NetBSD's pkgsrc has a 5 years old OpenLDAP patch that address a
>>>> build
>>>> problem on sparc64 with gcc 3.x (x<= 3). The build failure is
>>>> caused by
>>>> a bug in gcc.
>>>>
>>>> I think it would not hurt to commit this patch in OpenLDAP CVS:
>>>> everything is guarded by macros, so it cannot cause any harm to
>>>> unaffected systems, and the workaround has been tested by NetBSD
>>>> people
>>>> for 5 years.
>>>
>>> Kurt will probably have something to say about this too - this is
>>> more than a
>>> couple-line patch, it needs to be submitted by its original
>>> author, with
>>> appropriate rights attributions etc.
>>
>> Klaus Klein <kleink(a)kleink.org> is the contributor (I added him to
>> the
>> recipients of this messages).
>>
>> Do we need anything more than an e-mail with the patch and the
>> excerpt
>> below correctly filled?
>>
>> "This patch file is derived from OpenLDAP Software. All of the
>> modifications to OpenLDAP Software represented in the following
>> patch(es) were developed by <YOUR NAME> <YOUR-EMAIL-ADDRESS>. I
>> have not
>> assigned rights and/or interest in this work to any party. "
>
> One would also need an appropriate "rights statement". See http://www.openldap.org/devel/contributing.html
> .
>
> -- Kurt
On Dec 28, 2008, at 10:37 PM, manu(a)netbsd.org wrote:
> Howard Chu <hyc(a)symas.com> wrote:
>
>>> NetBSD's pkgsrc has a 5 years old OpenLDAP patch that address a
>>> build
>>> problem on sparc64 with gcc 3.x (x<= 3). The build failure is
>>> caused by
>>> a bug in gcc.
>>>
>>> I think it would not hurt to commit this patch in OpenLDAP CVS:
>>> everything is guarded by macros, so it cannot cause any harm to
>>> unaffected systems, and the workaround has been tested by NetBSD
>>> people
>>> for 5 years.
>>
>> Kurt will probably have something to say about this too - this is
>> more than a
>> couple-line patch, it needs to be submitted by its original author,
>> with
>> appropriate rights attributions etc.
>
> Klaus Klein <kleink(a)kleink.org> is the contributor (I added him to the
> recipients of this messages).
>
> Do we need anything more than an e-mail with the patch and the excerpt
> below correctly filled?
>
> "This patch file is derived from OpenLDAP Software. All of the
> modifications to OpenLDAP Software represented in the following
> patch(es) were developed by <YOUR NAME> <YOUR-EMAIL-ADDRESS>. I have
> not
> assigned rights and/or interest in this work to any party. "
One would also need an appropriate "rights statement". See http://www.openldap.org/devel/contributing.html
.
-- Kurt
ando(a)sys-net.it wrote:
> michael(a)stroeder.com wrote:
>> ando(a)sys-net.it wrote:
>>> Hallvard B Furuseth wrote:
>>>> ando(a)sys-net.it writes:
>>>>> On a related note, if this can be considered of general usefulness, LDAP
>>>>> specs would need to be changed in order to define a finer grain of
>>>>> attribute request; something like:
>>>>>
>>>>> empty or "*" ; all user, except attrs that need to be explicitly req.
>>>>> "+" ; all operational
>>>>> <all including attrs that need to be explicitly requested>
>>>>> <...>
>>>> Would it be cleaner if slapo-cloak redefines the attributes to be
>>>> operational, or to behave as if they are? Maybe give them an
>>>> X-AS-OPERATIONAL extension? Or would that just mess up schema code,
>>>> things like attribute inheritance?
>>> I think things would mess up.
>> I'd also recommend *not* to turn user attribute types into operational
>> attribute types. This would certainly confuse schema-aware clients.
>>
>>> Moreover, I see a number of features system administrators could ask
>>> for; e.g. hide attributes only when matching a URI (base, scope,
>>> filter),
>> Well, that's something many overlays would benefit from.
>
> In fact, based on recent modifications to slapo-constraint(5) and other
> overlays, I'm considering the opportunity to introduce generic helpers
> to parse and check this type of generic limitations.
We've discussed this before - overlays ought to have a configurable range of
effect, using an X.500 Subtree Search Specification.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/