On Monday 05 January 2009 12:31:23 sandeep.kumbhar(a)silverarc.biz wrote:
> Full_Name: Sandeep Kumbhar
> Version: openldap-2.3.27-8.el5_1.3
> OS: CentOS 5.2
> URL:
> Submission from: (NULL) (59.181.122.24)
>
>
> 1. I am using below schema for my LDAP server
>
> include /etc/openldap/schema/core.schema
> include /etc/openldap/schema/cosine.schema
> include /etc/openldap/schema/inetorgperson.schema
> include /etc/openldap/schema/nis.schema
>
> 2. I created local user on the same server and did the following to
> generate the
>
> data for step 3.
>
> # grep 'username' /etc/passwd > /etc/openldap/passwd.username
>
> Ref:
> <http://fedoranews.org/mediawiki/index.php/How_to_setup_and_maintain_OpenLD
>AP_server_for_your_network#Bonus:_Exporting_LDAP_users_home_folders_with_NFS
>>
In general, I note that this documentation leaves a lot to be desired.
>
> 3. Then I used the migration perl script for converting the local users to
> LDAP users.
>
> # /usr/share/openldap/migration/migrate_passwd.pl
> /etc/openldap/passwd.username \
> /etc/openldap/username.ldif
>
> 4. Now after creation of the *.ldif I used the below command to add this
> database
> file into the LDAP server.
>
You should run migrate_base.pl before you add any other data generated by
migrationtools.
> # ldapadd -x -D "cn=Manager,dc=intra,dc=exlinuz,dc=com" -W -f
> /etc/openldap/ \
> username.ldif
>
> 5. After typing the ldapadd command I gave the LDAP password I got the
> error below
>
> adding new entry "uid=sandeepk,ou=People1,dc=intra,dc=exlinuz,dc=com"
> ldap_add: No such object (32)
> matched DN: dc=intra,dc=exlinuz,dc=com
According to the error, the deepest part of this DN that exists is
dc=intra,dc=exlinuz,dc=com, so you haven't created
ou=People1,dc=intra,dc=exlinuz,dc=com, which migrate_base.pl would have done
for you.
> 6. I have created the domain.ldif and root.ldif and added them successfully
> into the
> LDAP Server using the official Openldap documentation.
> However I could not find anything adding Unix users and therefore I used
> the
>
> above referenced link to generate Unix users database file for LDAP.
>
> Please see the ldif file output by the migration script
There is no bug here. Please consult the documentation of the tools you are
using (migrationtools) when your spoonfeeding doesn't work perfectly. The only
problem you encountered was the 'HOWTO' you used, file bugs on it instead. It
is quite evident that the author of the HOWTO has not bothered to consult the
documentation for migrationtools ... see the migration-tools.txt file shipped
with the software.
Honestly, http://www.zarb.org/~bgmilne/make_master.sh could replace about half
the HOWTO, and cover something the HOWTO doesn't (migrating groups ....).
Regards,
Buchan
On Monday 05 January 2009 12:31:23 sandeep.kumbhar(a)silverarc.biz wrote:
> Full_Name: Sandeep Kumbhar
> Version: openldap-2.3.27-8.el5_1.3
> OS: CentOS 5.2
> URL:
> Submission from: (NULL) (59.181.122.24)
>
>
> 1. I am using below schema for my LDAP server
>
> include /etc/openldap/schema/core.schema
> include /etc/openldap/schema/cosine.schema
> include /etc/openldap/schema/inetorgperson.schema
> include /etc/openldap/schema/nis.schema
>
> 2. I created local user on the same server and did the following to
> generate the
>
> data for step 3.
>
> # grep 'username' /etc/passwd > /etc/openldap/passwd.username
>
> Ref:
> <http://fedoranews.org/mediawiki/index.php/How_to_setup_and_maintain_OpenLD
>AP_server_for_your_network#Bonus:_Exporting_LDAP_users_home_folders_with_NFS
>>
In general, I note that this documentation leaves a lot to be desired.
>
> 3. Then I used the migration perl script for converting the local users to
> LDAP users.
>
> # /usr/share/openldap/migration/migrate_passwd.pl
> /etc/openldap/passwd.username \
> /etc/openldap/username.ldif
>
> 4. Now after creation of the *.ldif I used the below command to add this
> database
> file into the LDAP server.
>
You should run migrate_base.pl before you add any other data generated by
migrationtools.
> # ldapadd -x -D "cn=Manager,dc=intra,dc=exlinuz,dc=com" -W -f
> /etc/openldap/ \
> username.ldif
>
> 5. After typing the ldapadd command I gave the LDAP password I got the
> error below
>
> adding new entry "uid=sandeepk,ou=People1,dc=intra,dc=exlinuz,dc=com"
> ldap_add: No such object (32)
> matched DN: dc=intra,dc=exlinuz,dc=com
According to the error, the deepest part of this DN that exists is
dc=intra,dc=exlinuz,dc=com, so you haven't created
ou=People1,dc=intra,dc=exlinuz,dc=com, which migrate_base.pl would have done
for you.
> 6. I have created the domain.ldif and root.ldif and added them successfully
> into the
> LDAP Server using the official Openldap documentation.
> However I could not find anything adding Unix users and therefore I used
> the
>
> above referenced link to generate Unix users database file for LDAP.
>
> Please see the ldif file output by the migration script
There is no bug here. Please consult the documentation of the tools you are
using (migrationtools) when your spoonfeeding doesn't work perfectly. The only
problem you encountered was the 'HOWTO' you used, file bugs on it instead. It
is quite evident that the author of the HOWTO has not bothered to consult the
documentation for migrationtools ... see the migration-tools.txt file shipped
with the software.
Honestly, http://www.zarb.org/~bgmilne/make_master.sh could replace about half
the HOWTO, and cover something the HOWTO doesn't (migrating groups ....).
Regards,
Buchan
--=-SUZpECi7CptfRynVUs4v
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable
On Fri, 2008-12-12 at 14:23 +1100, Andrew Bartlett wrote:
> On Thu, 2008-12-11 at 23:22 +0100, Pierangelo Masarati wrote:
> > Andrew Bartlett wrote:
> > > On Thu, 2008-10-23 at 00:15 +0200, Pierangelo Masarati wrote:
> > >> A tentative implementation is in HEAD, please test. You need to:
> > >=20
> > > Thankyou very much. I downloaded CVS HEAD and tested it out (finally=
-
> > > the Samba4 side of the implementation took far longer than I expected=
).
> > >=20
> > >> - configure as --enable-deref
> > >>
> > >> - enable the "deref" overlay in slapd, with "overlay deref" (doesn't
> > >> work as global overlay yet, sorry).
> > >=20
> > > This is something Samba4 will need, as many of our links are
> > > cross-database. But fixing this for a single DB is a big help in any
> > > case.
> >=20
> > Apparently this was fixed during the overlay's shakedown, as it seems t=
o=20
> > work as expected when only instantiated as global. In fact, nothing wa=
s=20
> > preventing it from working this way by design, it only didn't work at=20
> > some point of its evolution. Please test.
>=20
> Indeed, it works well as a global. Thanks!
>=20
> My only issue remaining is the clarification over the ASN.1 encoding of
> the control.
While I'm still confused about the ASN.1, I've coded to match OpenLDAP's
current behaviour.
The deref overlay seems to be working well. Many thanks!
Andrew Bartlett
--=20
Andrew Bartlett
http://samba.org/~abartlet/
Authentication Developer, Samba Team http://samba.org
Samba Developer, Red Hat Inc.
--=-SUZpECi7CptfRynVUs4v
Content-Type: application/pgp-signature; name=signature.asc
Content-Description: This is a digitally signed message part
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
iD8DBQBJYuLrz4A8Wyi0NrsRAgALAJ0b1YEcsKRo86mc2LCZbowtSyTo1gCeM3ws
slxgoMVr/MkEZcEk881oaLE=
=6rsz
-----END PGP SIGNATURE-----
--=-SUZpECi7CptfRynVUs4v--
ando(a)sys-net.it wrote:
> Full_Name: Pierangelo Masarati
> Version: HEAD
> OS: irrelevant
> URL: ftp://ftp.openldap.org/incoming/
> Submission from: (NULL) (82.63.140.131)
> Submitted by: ando
>
>
> After updating tests/run.in from CVS (rev 1.59) and reconfiguring, executing
> test050 with SYNCMODE=ro results in a failure. The failure is not strictly
> reproducible, but it always occurred to me, either affecting one or another
> delete that is not propagated to all participants to the MMR pool. I've raised
> the sleep time, to make sure there's no chance replication did not occur yet.
>
> I wonder if refreshOnly makes any sense for a MMR pool. If it doesn't, it
> should not be allowed. If it does, deletes should be replicated correctly.
This sounds like a dup of #5843. Note that you cannot disallow MMR for
refreshOnly, since servers must be able to refresh when starting up.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
Full_Name: Sandeep Kumbhar
Version: openldap-2.3.27-8.el5_1.3
OS: CentOS 5.2
URL:
Submission from: (NULL) (59.181.122.24)
1. I am using below schema for my LDAP server
include /etc/openldap/schema/core.schema
include /etc/openldap/schema/cosine.schema
include /etc/openldap/schema/inetorgperson.schema
include /etc/openldap/schema/nis.schema
2. I created local user on the same server and did the following to generate the
data for step 3.
# grep 'username' /etc/passwd > /etc/openldap/passwd.username
Ref: <http://fedoranews.org/mediawiki/index.php/How_to_setup_and_maintain_OpenLDA…>
3. Then I used the migration perl script for converting the local users to LDAP
users.
# /usr/share/openldap/migration/migrate_passwd.pl
/etc/openldap/passwd.username \
/etc/openldap/username.ldif
4. Now after creation of the *.ldif I used the below command to add this
database
file into the LDAP server.
# ldapadd -x -D "cn=Manager,dc=intra,dc=exlinuz,dc=com" -W -f /etc/openldap/
\
username.ldif
5. After typing the ldapadd command I gave the LDAP password I got the error
below
adding new entry "uid=sandeepk,ou=People1,dc=intra,dc=exlinuz,dc=com"
ldap_add: No such object (32)
matched DN: dc=intra,dc=exlinuz,dc=com
6. I have created the domain.ldif and root.ldif and added them successfully into
the
LDAP Server using the official Openldap documentation.
However I could not find anything adding Unix users and therefore I used the
above referenced link to generate Unix users database file for LDAP.
Please see the ldif file output by the migration script
dn: uid=samk,ou=People1,dc=intra,dc=exlinuz,dc=com
uid: samk
cn: samk
objectClass: account
objectClass: posixAccount
objectClass: top
objectClass: shadowAccount
userPassword: {crypt}$1$eBNsXmF1$L9/bK4vbjkAOCKa5DKKAE0
shadowLastChange: 14249
shadowMax: 99999
shadowWarning: 7
loginShell: /bin/bash
uidNumber: 500
gidNumber: 500
homeDirectory: /home/samk
Full_Name: Pierangelo Masarati
Version: HEAD
OS: irrelevant
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (82.63.140.131)
Submitted by: ando
Many features could benefit from being restricted to a(n extended) X.501 subtree
specification using a common interface (API, configuration). Examples are:
- ACL
- limits
- custom functionalities provided by overlays
p.
rlvcosta(a)yahoo.com wrote:
> --0-1358063323-1230648878=:64021
> Content-Type: text/plain; charset=iso-8859-1
> Content-Transfer-Encoding: quoted-printable
>
> Pierangelo,
>
> I could run the valgrind but at this time I do not have the DB with the aro=
> und 4 million subscribers. In this condition, with a small number of users,=
> I already could see some "definitive" memory leaks and I believe they are =
> associated with OpenLdap.
>
> Just for a history I already compiled the openldap 2.4.11 souce code withou=
> t modules and TLS since many lists make reference to some possible issues i=
> n RHEL with TLS. Even so I could not have a load without the possible memor=
> y leak.
>
> So I do not believe the problems are in the RHEL libtool or glibc.
>
> In any case the result for the valgrind is (short number of entrances at th=
> is time):
You probably need to use an unstripped binary in order to find out where
mallocs/frees occur. Most of the occurrences you notice are either
one-time leaks. Moreover, 2.4.13 is out, and it might have fixed some.
I suggest you further investigate using the latest code, using an
unstripped binary.
p.
Ing. Pierangelo Masarati
OpenLDAP Core Team
SysNet s.r.l.
via Dossi, 8 - 27100 Pavia - ITALIA
http://www.sys-net.it
-----------------------------------
Office: +39 02 23998309
Mobile: +39 333 4963172
Fax: +39 0382 476497
Email: ando(a)sys-net.it
-----------------------------------
--0-1358063323-1230648878=:64021
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable
Pierangelo,
I could run the valgrind but at this time I do not have the DB with the aro=
und 4 million subscribers. In this condition, with a small number of users,=
I already could see some "definitive" memory leaks and I believe they are =
associated with OpenLdap.
Just for a history I already compiled the openldap 2.4.11 souce code withou=
t modules and TLS since many lists make reference to some possible issues i=
n RHEL with TLS. Even so I could not have a load without the possible memor=
y leak.
So I do not believe the problems are in the RHEL libtool or glibc.
In any case the result for the valgrind is (short number of entrances at th=
is time):
=3D=3D32608=3D=3D Memcheck, a memory error detector.
=3D=3D32608=3D=3D Copyright (C) 2002-2005, and GNU GPL'd, by Julian Seward =
et al.
=3D=3D32608=3D=3D Using LibVEX rev 1575, a library for dynamic binary trans=
lation.
=3D=3D32608=3D=3D Copyright (C) 2004-2005, and GNU GPL'd, by OpenWorks LLP.
=3D=3D32608=3D=3D Using valgrind-3.1.1, a dynamic binary instrumentation fr=
amework.
=3D=3D32608=3D=3D Copyright (C) 2000-2005, and GNU GPL'd, by Julian Seward =
et al.
=3D=3D32608=3D=3D For more details, rerun with: -v
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 86 f=
rom 2)
=3D=3D32608=3D=3D malloc/free: in use at exit: 751 bytes in 32 blocks.
=3D=3D32608=3D=3D malloc/free: 29,775 allocs, 29,743 frees, 22,150,722 byte=
s allocated.
=3D=3D32608=3D=3D For counts of detected errors, rerun with: -v
=3D=3D32608=3D=3D searching for pointers to 32 not-freed blocks.
=3D=3D32608=3D=3D checked 8,191,292 bytes.
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D 16 bytes in 1 blocks are still reachable in loss record 1=
of 9
=3D=3D32608=3D=3D=A0=A0=A0 at 0x40056BF: calloc (vg_replace_malloc.c:279)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x519364: _dlerror_run (in /lib/libdl-2.3.4.s=
o)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x518D10: dlopen@@GLIBC_2.1 (in /lib/libdl-2.=
3.4.so)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x52D80D: _sasl_get_plugin (in /usr/lib/libsa=
sl2.so.2.0.19)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x52DC61: _sasl_load_plugins (in /usr/lib/lib=
sasl2.so.2.0.19)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x52B063: sasl_server_init (in /usr/lib/libsa=
sl2.so.2.0.19)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x80C82C0: slap_sasl_init (in /usr/libexec/sl=
apd)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x3FFDE2: (below main) (in /lib/tls/libc-2.3.=
4.so)
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D 22 bytes in 1 blocks are still reachable in loss record 2=
of 9
=3D=3D32608=3D=3D=A0=A0=A0 at 0x4004405: malloc (vg_replace_malloc.c:149)
=3D=3D32608=3D=3D=A0=A0=A0 by 0xB77567: (within /usr/lib/libltdl.so.3.1.0)
=3D=3D32608=3D=3D=A0=A0=A0 by 0xB780C8: (within /usr/lib/libltdl.so.3.1.0)
=3D=3D32608=3D=3D=A0=A0=A0 by 0xB7A79D: lt_dlsetsearchpath (in /usr/lib/lib=
ltdl.so.3.1.0)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x8068147: (within /usr/libexec/slapd)
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D 24 bytes in 1 blocks are still reachable in loss record 3=
of 9
=3D=3D32608=3D=3D=A0=A0=A0 at 0x400579F: realloc (vg_replace_malloc.c:306)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x8151E6F: ber_bvarray_add (in /usr/libexec/s=
lapd)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x44DD627: ???
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D 28 bytes in 2 blocks are definitely lost in loss record 4=
of 9
=3D=3D32608=3D=3D=A0=A0=A0 at 0x4004405: malloc (vg_replace_malloc.c:149)
=3D=3D32608=3D=3D=A0=A0=A0 by 0xB77567: (within /usr/lib/libltdl.so.3.1.0)
=3D=3D32608=3D=3D=A0=A0=A0 by 0xB780C8: (within /usr/lib/libltdl.so.3.1.0)
=3D=3D32608=3D=3D=A0=A0=A0 by 0xB78CDC: (within /usr/lib/libltdl.so.3.1.0)
=3D=3D32608=3D=3D=A0=A0=A0 by 0xB7972B: lt_dlopen (in /usr/lib/libltdl.so.3=
.1.0)
=3D=3D32608=3D=3D=A0=A0=A0 by 0xB7980C: lt_dlopenext (in /usr/lib/libltdl.s=
o.3.1.0)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x80CA1E3: module_load (in /usr/libexec/slapd=
)
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D 28 bytes in 1 blocks are still reachable in loss record 5=
of 9
=3D=3D32608=3D=3D=A0=A0=A0 at 0x4004405: malloc (vg_replace_malloc.c:149)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x3D81D7: _dl_map_object_deps (in /lib/ld-2.3=
.4.so)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x4E7158: dl_open_worker (in /lib/tls/libc-2.=
3.4.so)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x3D90FD: _dl_catch_error (in /lib/ld-2.3.4.s=
o)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x4E7CB7: _dl_open (in /lib/tls/libc-2.3.4.so=
)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x4E904C: do_dlopen (in /lib/tls/libc-2.3.4.s=
o)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x3D90FD: _dl_catch_error (in /lib/ld-2.3.4.s=
o)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x4E912D: __libc_dlopen_mode (in /lib/tls/lib=
c-2.3.4.so)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x561087: pthread_cancel_init (in /lib/tls/li=
bpthread-2.3.4.so)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x5611FC: _Unwind_ForcedUnwind (in /lib/tls/l=
ibpthread-2.3.4.so)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x55F020: __pthread_unwind (in /lib/tls/libpt=
hread-2.3.4.so)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x55AF8F: pthread_exit (in /lib/tls/libpthrea=
d-2.3.4.so)
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D 32 bytes in 1 blocks are still reachable in loss record 6=
of 9
=3D=3D32608=3D=3D=A0=A0=A0 at 0x4004405: malloc (vg_replace_malloc.c:149)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x4005728: realloc (vg_replace_malloc.c:306)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x810E504: rewrite_mapper_register (in /usr/l=
ibexec/slapd)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x3FFDE2: (below main) (in /lib/tls/libc-2.3.=
4.so)
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D 77 bytes in 5 blocks are still reachable in loss record 7=
of 9
=3D=3D32608=3D=3D=A0=A0=A0 at 0x4004405: malloc (vg_replace_malloc.c:149)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x8150E2E: ber_memalloc_x (in /usr/libexec/sl=
apd)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x3D6363FF: ???
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D 252 bytes in 18 blocks are definitely lost in loss record=
8 of 9
=3D=3D32608=3D=3D=A0=A0=A0 at 0x4004405: malloc (vg_replace_malloc.c:149)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x8150E2E: ber_memalloc_x (in /usr/libexec/sl=
apd)
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D 272 bytes in 2 blocks are possibly lost in loss record 9 =
of 9
=3D=3D32608=3D=3D=A0=A0=A0 at 0x40056BF: calloc (vg_replace_malloc.c:279)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x3DB71A: _dl_allocate_tls (in /lib/ld-2.3.4.=
so)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x55A91E: pthread_create@@GLIBC_2.1 (in /lib/=
tls/libpthread-2.3.4.so)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x811D4A9: ldap_pvt_thread_create (in /usr/li=
bexec/slapd)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x3FFDE2: (below main) (in /lib/tls/libc-2.3.=
4.so)
=3D=3D32608=3D=3D=20
=3D=3D32608=3D=3D LEAK SUMMARY:
=3D=3D32608=3D=3D=A0=A0=A0 definitely lost: 280 bytes in 20 blocks.
=3D=3D32608=3D=3D=A0=A0=A0=A0=A0 possibly lost: 272 bytes in 2 blocks.
=3D=3D32608=3D=3D=A0=A0=A0 still reachable: 199 bytes in 10 blocks.
=3D=3D32608=3D=3D=A0=A0=A0=A0=A0=A0=A0=A0 suppressed: 0 bytes in 0 blocks.
My suspicious is this memory leak is happening at ber_memalloc_x (in /usr/l=
ibexec/slapd). Or in the part above :
=3D=3D32608=3D=3D 252 bytes in 18 blocks are definitely lost in loss record=
8 of 9
=3D=3D32608=3D=3D=A0=A0=A0 at 0x4004405: malloc (vg_replace_malloc.c:149)
=3D=3D32608=3D=3D=A0=A0=A0 by 0x8150E2E: ber_memalloc_x (in /usr/libexec/sl=
apd)
Since right now I do not have a large number of entrances in the DB the sea=
rch is easily cached in memory and the response is very quick.
I will load again the DB but since this takes around 6-8hours I sent this i=
nitial result from valgrind since it can already give some directions. Tomo=
rrow I should send more information related with a more heavy load DB.
I would like your help to see if this can identify something related with t=
his memory leakage.
Please let me know if I can help in this meantime with something else.
Best regards,
Rodrigo.
--- On Sat, 12/20/08, Pierangelo Masarati <ando(a)sys-net.it> wrote:
From: Pierangelo Masarati <ando(a)sys-net.it>
Subject: Re: (ITS#5860) slapd memeory leak under openldap 2.4
To: rlvcosta(a)yahoo.com
Cc: openldap-its(a)openldap.org
Date: Saturday, December 20, 2008, 7:05 PM
rlvcosta(a)yahoo.com wrote:
> Full_Name: Rodrigo Costa
> Version: 2.4.11
> OS: RHEL4 U4
> URL: ftp://ftp.openldap.org/incoming/
> Submission from: (NULL) (201.82.161.220)
>=20
>=20
> Opeldap dev team,
>=20
> I'm facing an issue and after many tentatives I believe this is really
a bug
> that maybe could be identified and solved by openldap community. I tried
to
> include dmalloc into openldap 2.4.11 code but even I could compile with i=
t
any
> tentative to execute the code generated "Segmentation Fault" so
I could not use
> dmalloc to identify where the memory leak is happening.
Can you try with valgrind?
p.
Ing. Pierangelo Masarati
OpenLDAP Core Team
SysNet s.r.l.
via Dossi, 8 - 27100 Pavia - ITALIA
http://www.sys-net.it
-----------------------------------
Office: +39 02 23998309
Mobile: +39 333 4963172
Fax: +39 0382 476497
Email: ando(a)sys-net.it
-----------------------------------
=0A=0A=0A
--0-1358063323-1230648878=:64021
Content-Type: text/html; charset=us-ascii
<table cellspacing="0" cellpadding="0" border="0" ><tr><td valign="top" style="font: inherit;">Pierangelo,<br><br>I could run the valgrind but at this time I do not have the DB with the around 4 million subscribers. In this condition, with a small number of users, I already could see some "definitive" memory leaks and I believe they are associated with OpenLdap.<br><br>Just for a history I already compiled the openldap 2.4.11 souce code without modules and TLS since many lists make reference to some possible issues in RHEL with TLS. Even so I could not have a load without the possible memory leak.<br><br>So I do not believe the problems are in the RHEL libtool or glibc.<br><br>In any case the result for the valgrind is (short number of entrances at this time):<br><br>==32608== Memcheck, a memory error detector.<br>==32608== Copyright (C) 2002-2005, and GNU GPL'd, by Julian Seward et al.<br>==32608== Using LibVEX rev 1575, a library for dynamic binary
translation.<br>==32608== Copyright (C) 2004-2005, and GNU GPL'd, by OpenWorks LLP.<br>==32608== Using valgrind-3.1.1, a dynamic binary instrumentation framework.<br>==32608== Copyright (C) 2000-2005, and GNU GPL'd, by Julian Seward et al.<br>==32608== For more details, rerun with: -v<br>==32608== <br>==32608== <br>==32608== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 86 from 2)<br>==32608== malloc/free: in use at exit: 751 bytes in 32 blocks.<br>==32608== malloc/free: 29,775 allocs, 29,743 frees, 22,150,722 bytes allocated.<br>==32608== For counts of detected errors, rerun with: -v<br>==32608== searching for pointers to 32 not-freed blocks.<br>==32608== checked 8,191,292 bytes.<br>==32608== <br>==32608== 16 bytes in 1 blocks are still reachable in loss record 1 of 9<br>==32608== at 0x40056BF: calloc (vg_replace_malloc.c:279)<br>==32608== by 0x519364: _dlerror_run (in
/lib/libdl-2.3.4.so)<br>==32608== by 0x518D10: dlopen@@GLIBC_2.1 (in /lib/libdl-2.3.4.so)<br>==32608== by 0x52D80D: _sasl_get_plugin (in /usr/lib/libsasl2.so.2.0.19)<br>==32608== by 0x52DC61: _sasl_load_plugins (in /usr/lib/libsasl2.so.2.0.19)<br>==32608== by 0x52B063: sasl_server_init (in /usr/lib/libsasl2.so.2.0.19)<br>==32608== by 0x80C82C0: slap_sasl_init (in /usr/libexec/slapd)<br>==32608== by 0x3FFDE2: (below main) (in /lib/tls/libc-2.3.4.so)<br>==32608== <br>==32608== <br>==32608== 22 bytes in 1 blocks are still reachable in loss record 2 of 9<br>==32608== at 0x4004405: malloc (vg_replace_malloc.c:149)<br>==32608== by 0xB77567: (within /usr/lib/libltdl.so.3.1.0)<br>==32608== by 0xB780C8: (within /usr/lib/libltdl.so.3.1.0)<br>==32608== by 0xB7A79D: lt_dlsetsearchpath (in
/usr/lib/libltdl.so.3.1.0)<br>==32608== by 0x8068147: (within /usr/libexec/slapd)<br>==32608== <br>==32608== <br>==32608== 24 bytes in 1 blocks are still reachable in loss record 3 of 9<br>==32608== at 0x400579F: realloc (vg_replace_malloc.c:306)<br>==32608== by 0x8151E6F: ber_bvarray_add (in /usr/libexec/slapd)<br>==32608== by 0x44DD627: ???<br>==32608== <br>==32608== <br>==32608== 28 bytes in 2 blocks are definitely lost in loss record 4 of 9<br>==32608== at 0x4004405: malloc (vg_replace_malloc.c:149)<br>==32608== by 0xB77567: (within /usr/lib/libltdl.so.3.1.0)<br>==32608== by 0xB780C8: (within /usr/lib/libltdl.so.3.1.0)<br>==32608== by 0xB78CDC: (within /usr/lib/libltdl.so.3.1.0)<br>==32608== by 0xB7972B: lt_dlopen (in /usr/lib/libltdl.so.3.1.0)<br>==32608== by 0xB7980C:
lt_dlopenext (in /usr/lib/libltdl.so.3.1.0)<br>==32608== by 0x80CA1E3: module_load (in /usr/libexec/slapd)<br>==32608== <br>==32608== <br>==32608== 28 bytes in 1 blocks are still reachable in loss record 5 of 9<br>==32608== at 0x4004405: malloc (vg_replace_malloc.c:149)<br>==32608== by 0x3D81D7: _dl_map_object_deps (in /lib/ld-2.3.4.so)<br>==32608== by 0x4E7158: dl_open_worker (in /lib/tls/libc-2.3.4.so)<br>==32608== by 0x3D90FD: _dl_catch_error (in /lib/ld-2.3.4.so)<br>==32608== by 0x4E7CB7: _dl_open (in /lib/tls/libc-2.3.4.so)<br>==32608== by 0x4E904C: do_dlopen (in /lib/tls/libc-2.3.4.so)<br>==32608== by 0x3D90FD: _dl_catch_error (in /lib/ld-2.3.4.so)<br>==32608== by 0x4E912D: __libc_dlopen_mode (in /lib/tls/libc-2.3.4.so)<br>==32608== by 0x561087: pthread_cancel_init (in
/lib/tls/libpthread-2.3.4.so)<br>==32608== by 0x5611FC: _Unwind_ForcedUnwind (in /lib/tls/libpthread-2.3.4.so)<br>==32608== by 0x55F020: __pthread_unwind (in /lib/tls/libpthread-2.3.4.so)<br>==32608== by 0x55AF8F: pthread_exit (in /lib/tls/libpthread-2.3.4.so)<br>==32608== <br>==32608== <br>==32608== 32 bytes in 1 blocks are still reachable in loss record 6 of 9<br>==32608== at 0x4004405: malloc (vg_replace_malloc.c:149)<br>==32608== by 0x4005728: realloc (vg_replace_malloc.c:306)<br>==32608== by 0x810E504: rewrite_mapper_register (in /usr/libexec/slapd)<br>==32608== by 0x3FFDE2: (below main) (in /lib/tls/libc-2.3.4.so)<br>==32608== <br>==32608== <br>==32608== 77 bytes in 5 blocks are still reachable in loss record 7 of 9<br>==32608== at 0x4004405: malloc (vg_replace_malloc.c:149)<br>==32608==
by 0x8150E2E: ber_memalloc_x (in /usr/libexec/slapd)<br>==32608== by 0x3D6363FF: ???<br>==32608== <br>==32608== <br>==32608== 252 bytes in 18 blocks are definitely lost in loss record 8 of 9<br>==32608== at 0x4004405: malloc (vg_replace_malloc.c:149)<br>==32608== by 0x8150E2E: ber_memalloc_x (in /usr/libexec/slapd)<br>==32608== <br>==32608== <br>==32608== 272 bytes in 2 blocks are possibly lost in loss record 9 of 9<br>==32608== at 0x40056BF: calloc (vg_replace_malloc.c:279)<br>==32608== by 0x3DB71A: _dl_allocate_tls (in /lib/ld-2.3.4.so)<br>==32608== by 0x55A91E: pthread_create@@GLIBC_2.1 (in /lib/tls/libpthread-2.3.4.so)<br>==32608== by 0x811D4A9: ldap_pvt_thread_create (in /usr/libexec/slapd)<br>==32608== by 0x3FFDE2: (below main) (in /lib/tls/libc-2.3.4.so)<br>==32608== <br>==32608== LEAK
SUMMARY:<br>==32608== definitely lost: 280 bytes in 20 blocks.<br>==32608== possibly lost: 272 bytes in 2 blocks.<br>==32608== still reachable: 199 bytes in 10 blocks.<br>==32608== suppressed: 0 bytes in 0 blocks.<br><br>My suspicious is this memory leak is happening at ber_memalloc_x (in /usr/libexec/slapd). Or in the part above :<br><br>==32608== 252 bytes in 18 blocks are definitely lost in loss record 8 of 9<br>
==32608== at 0x4004405: malloc (vg_replace_malloc.c:149)<br>
==32608== by 0x8150E2E: ber_memalloc_x (in /usr/libexec/slapd)<br><br>Since right now I do not have a large number of entrances in the DB the search is easily cached in memory and the response is very quick.<br><br>I will load again the DB but since this takes around 6-8hours I sent this initial result from valgrind since it can already give some directions. Tomorrow I should send more information related with a more heavy load DB.<br><br>I would like your help to see if this can identify something related with this memory leakage.<br><br>Please let me know if I can help in this meantime with something else.<br><br>Best regards,<br><br>Rodrigo.<span style="font-family: monospace;"><br><br></span>
--- On <b>Sat, 12/20/08, Pierangelo Masarati <i><ando(a)sys-net.it></i></b> wrote:<br><blockquote style="border-left: 2px solid rgb(16, 16, 255); margin-left: 5px; padding-left: 5px;">From: Pierangelo Masarati <ando(a)sys-net.it><br>Subject: Re: (ITS#5860) slapd memeory leak under openldap 2.4<br>To: rlvcosta(a)yahoo.com<br>Cc: openldap-its(a)openldap.org<br>Date: Saturday, December 20, 2008, 7:05 PM<br><br><pre>rlvcosta(a)yahoo.com wrote:<br>> Full_Name: Rodrigo Costa<br>> Version: 2.4.11<br>> OS: RHEL4 U4<br>> URL: ftp://ftp.openldap.org/incoming/<br>> Submission from: (NULL) (201.82.161.220)<br>> <br>> <br>> Opeldap dev team,<br>> <br>> I'm facing an issue and after many tentatives I believe this is really<br>a bug<br>> that maybe could be identified and solved by openldap community. I tried<br>to<br>> include dmalloc into openldap 2.4.11 code but even I could compile with it<br>any<br>> tentative to execute
the code generated "Segmentation Fault" so<br>I could not use<br>> dmalloc to identify where the memory leak is happening.<br><br>Can you try with valgrind?<br><br>p.<br><br><br>Ing. Pierangelo Masarati<br>OpenLDAP Core Team<br><br>SysNet s.r.l.<br>via Dossi, 8 - 27100 Pavia - ITALIA<br>http://www.sys-net.it<br>-----------------------------------<br>Office: +39 02 23998309<br>Mobile: +39 333 4963172<br>Fax: +39 0382 476497<br>Email: ando(a)sys-net.it<br>-----------------------------------<br><br></pre></blockquote></td></tr></table><br>
--0-1358063323-1230648878=:64021--
Pierangelo Masarati <ando(a)sys-net.it> wrote:
> At a quick glance, you're leaking the callback containing the cloak
> response. You should remove (and free it) when called with REP_RESULT
> as sr_type, and add a slap_freeself_cb() cleanup hook. You won't notice
> the memory leak unless you build slapd with SLAP_NO_SL_MALLOC set, or
> unless your overlay operates within a slab memory intensive context.
Here is an update, I hope it addresses the concerns you raised:
ftp://ftp.openldap.org/incoming/manu-cloak-20081230.c
--
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu(a)netbsd.org
On Dec 28, 2008, at 4:52 PM, kurt(a)openldap.org wrote:
> On a technical note, use of platform macros (sun and gcc versions)
> should be avoided. Instead, autoconf should be used to detect
> conditions for need to be adapted for.
>
> On an IPR review, I suspect it would be easier to update to:
> http://www.openbsd.org/cgi-bin/cvsweb/src/lib/libc/hash/sha1.c?rev=1.21;con…
>
> then to try to identify the contributors and get their "okay".
I should note that, from a technical point of view, it might also be
better to integrate the latest from OpenBSD. This would hopefully
provide a SHA1 implementation which works on more platforms than the
current version, or the current version as modified by Klaus's patch.
-- Kurt