openldap-bugs

openldap-bugs@openldap.org

1 participants
20963 discussions

Re: (ITS#7415) Add MALLOC_CHECK_ and MALLOC_PERTURB_ libc env to the test suite for detecting heap corruption
by gitter.spiros＠gmail.com 12 Oct '12

12 Oct '12

2012/10/11 Kurt Zeilenga <Kurt(a)openldap.org>: > I note that these environment variables are specific to Linux, and then only certain Linux systems. If we add these, then we should also add similar environment variables for FreeBSD malloc(3), MacOS/X malloc(3), and various alternative malloc(3) libraries. And then we might get into cases where variables for one set of environmental variables conflict with another. As these can be set by the developer as desired in their development environment, there's little reason to get into setting all the malloc(3) environmental variables that might be useful... and we shouldn't play favors and only set variables for Linux. Sorry for the delay, problems to the my $DAYWORK Ok. First of all, elibc (debian/ubuntu recently) and glibc cover probably 95% of the Linux distributions, and they are compatible, in particular for the MALLOC_CHECK http://lwn.net/Articles/333755/ - i have also checked the source. A considerable part of the remaining operating systems are like BSD - FreeBSD for example - and MacOSX, FreeBSD based. In fact, these latter two have different malloc library and it may be necessary to activate different controls for different systems (see the first reference) http://permalink.gmane.org/gmane.network.openvswitch.devel/14838 http://www.unix.com/man-page/freebsd/5/malloc.conf/ But the benefits of automating these tests is to do a better AUTOQA for the buildsystem. It is also not true, as experience shows, that all developers know everything. Beeing expert in LDAP and C programming, for example, does not mean necessarily be an expert in everything: this can certainly be true for the OPENLDAP developers but it is not always the case. Here some examples: http://lists.fedoraproject.org/pipermail/devel/2010-May/135683.html (this check had found many bug that the upstream developer haven't catched) or this, from my mantainer, http://lists.gnupg.org/pipermail/gcrypt-devel/2010-June/001605.html I saw that you have closed the case(WANTFIX i think), however, I have to tell you better my opinion. If, as can happen, you decide that my reasons make sense, i can reroll my patch for Linux, macos and freebsd. Let me know. Thanks anyway for your reply > > -- Kurt >

1 0

Re: (ITS#7414) rwm: ldapmodify, slapd segmentation fault
by jvcelak＠redhat.com 11 Oct '12

11 Oct '12

> > The slapd daemon will crash, here is the full backtrace: > > Thanks for the detailed report. Fixed now in master. > Thank you for the very fast resolution. Jan

1 0

Re: (ITS#7415) Add MALLOC_CHECK_ and MALLOC_PERTURB_ libc env to the test suite for detecting heap corruption
by kurt＠OpenLDAP.org 11 Oct '12

11 Oct '12

On Oct 11, 2012, at 1:44 PM, h.b.furuseth(a)usit.uio.no wrote: > We can describe these and other variables in tests/README. There's lots of malloc variables one could describe. Better off to simply say that one might want to read the documentation for whatever malloc they are using, and use, if they want, any environmental variables they might want to use. This avoids duplicating detail better left to the malloc documentation. > > Yeah I know, people don't read documentation. The testing > announcements could mention the file though. Wrong audience, me thinks, especially if those announcements are posted to openldap-techncial. We don't need openldap users reporting malloc use issues in non-OpenLDAP programs. Use of such variables should be limited to developers. -- Kurt > >

1 0

(ITS#7416) entryCSN index errors when changing olcDbEnvFlags with back-mdb
by quanah＠OpenLDAP.org 11 Oct '12

11 Oct '12

Full_Name: Quanah Gibson-Mount Version: 2.4.33 OS: Linux 2.6 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (74.196.25.250) I found that if I modify the olcDbEnvFlags on a back-mdb database to have: olcDbEnvFlags: writemap olcDbEnvFlags: nometasync Then modifications in the primary database will throw the following error until slapd is restarted: Oct 11 14:38:01 zre-ldap002 slapd[22940]: conn=1001 fd=11 ACCEPT from PATH=/opt/zimbra/openldap-2.4.33.2z/var/run/ldapi (PATH=/opt/zimbra/openldap-2.4.33.2z/var/run/ldapi) Oct 11 14:38:01 zre-ldap002 slapd[22940]: conn=1001 op=0 BIND dn="cn=config" method=128 Oct 11 14:38:01 zre-ldap002 slapd[22940]: conn=1001 op=0 BIND dn="cn=config" mech=SIMPLE ssf=0 Oct 11 14:38:01 zre-ldap002 slapd[22940]: conn=1001 op=0 RESULT tag=97 err=0 text= Oct 11 14:38:01 zre-ldap002 slapd[22940]: conn=1001 op=1 SRCH base="cn=accesslog" scope=0 deref=2 filter="(objectClass=*)" Oct 11 14:38:01 zre-ldap002 slapd[22940]: conn=1001 op=1 SRCH attr=1.1 Oct 11 14:38:01 zre-ldap002 slapd[22940]: conn=1001 op=1 SEARCH RESULT tag=101 err=32 nentries=0 text= Oct 11 14:38:01 zre-ldap002 slapd[22940]: conn=1001 op=2 SRCH base="olcDatabase={2}mdb,cn=config" scope=0 deref=2 filter="(objectClass=*)" Oct 11 14:38:01 zre-ldap002 slapd[22940]: conn=1001 op=2 SRCH attr=olcDbEnvFlags Oct 11 14:38:01 zre-ldap002 slapd[22940]: conn=1001 op=2 SEARCH RESULT tag=101 err=0 nentries=1 text= Oct 11 14:38:01 zre-ldap002 slapd[22940]: conn=1001 op=3 MOD dn="olcDatabase={2}mdb,cn=config" Oct 11 14:38:01 zre-ldap002 slapd[22940]: conn=1001 op=3 MOD attr=olcDbEnvFlags Oct 11 14:38:01 zre-ldap002 slapd[22940]: slap_queue_csn: queing 0x7f88c17fc3f0 20121011213801.832254Z#000000#000#000000 Oct 11 14:38:01 zre-ldap002 slapd[22940]: conn=1001 op=3 RESULT tag=103 err=0 text= Oct 11 14:38:01 zre-ldap002 slapd[22940]: slap_graduate_commit_csn: removing 0x41d4690 20121011213801.832254Z#000000#000#000000 Oct 11 14:38:01 zre-ldap002 slapd[22940]: conn=1001 op=4 MOD dn="uid=zmpostfix,cn=appaccts,cn=zimbra" Oct 11 14:38:01 zre-ldap002 slapd[22940]: conn=1001 op=4 MOD attr=zimbraId Oct 11 14:38:01 zre-ldap002 slapd[22940]: slap_queue_csn: queing 0x7f88c17fd1a0 20121011213801.864697Z#000000#000#000000 Oct 11 14:38:01 zre-ldap002 slapd[22940]: conn=1001 op=4: attribute "entryCSN" index delete failure Oct 11 14:38:01 zre-ldap002 slapd[22940]: conn=1001 op=4 RESULT tag=103 err=80 text= Oct 11 14:38:01 zre-ldap002 slapd[22940]: slap_graduate_commit_csn: removing 0x41d4690 20121011213801.864697Z#000000#000#000000 Oct 11 14:38:01 zre-ldap002 slapd[22940]: conn=1001 fd=11 closed (connection lost) Oct 11 14:38:06 zre-ldap002 slapd[22940]: conn=1004 fd=14 closed (connection lost) Oct 11 14:38:09 zre-ldap002 slapd[22940]: conn=1005 fd=11 ACCEPT from IP=10.137.242.52:47811 (IP=10.137.242.52:389) Oct 11 14:38:09 zre-ldap002 slapd[22940]: conn=1005 op=0 EXT oid=1.3.6.1.4.1.1466.20037 Oct 11 14:38:09 zre-ldap002 slapd[22940]: conn=1005 op=0 STARTTLS Oct 11 14:38:09 zre-ldap002 slapd[22940]: conn=1005 op=0 RESULT oid= err=0 text= Oct 11 14:38:09 zre-ldap002 slapd[22940]: conn=1005 fd=11 TLS established tls_ssf=256 ssf=256 Oct 11 14:38:09 zre-ldap002 slapd[22940]: conn=1005 op=1 BIND dn="uid=zimbra,cn=admins,cn=zimbra" method=128 Oct 11 14:38:09 zre-ldap002 slapd[22940]: conn=1005 op=1 BIND dn="uid=zimbra,cn=admins,cn=zimbra" mech=SIMPLE ssf=0 Oct 11 14:38:09 zre-ldap002 slapd[22940]: conn=1005 op=1 RESULT tag=97 err=0 text= Oct 11 14:38:09 zre-ldap002 slapd[22940]: conn=1005 op=2 SRCH base="cn=zre-ldap002.eng.vmware.com,cn=servers,cn=zimbra" scope=0 deref=0 filter="(objectClass=*)" Oct 11 14:38:09 zre-ldap002 slapd[22940]: conn=1005 op=2 SEARCH RESULT tag=101 err=0 nentries=1 text= Oct 11 14:38:09 zre-ldap002 slapd[22940]: conn=1005 op=3 SRCH base="cn=config,cn=zimbra" scope=0 deref=0 filter="(objectClass=*)" Oct 11 14:38:09 zre-ldap002 slapd[22940]: conn=1005 op=3 SEARCH RESULT tag=101 err=0 nentries=1 text= Oct 11 14:38:10 zre-ldap002 slapd[22940]: conn=1005 op=4 MOD dn="cn=zre-ldap002.eng.vmware.com,cn=servers,cn=zimbra" Oct 11 14:38:10 zre-ldap002 slapd[22940]: conn=1005 op=4 MOD attr=zimbraMtaSmtpdMilters Oct 11 14:38:10 zre-ldap002 slapd[22940]: slap_queue_csn: queing 0x7f88c17fd1a0 20121011213810.017749Z#000000#000#000000 Oct 11 14:38:10 zre-ldap002 slapd[22940]: conn=1005 op=4: attribute "entryCSN" index delete failure Oct 11 14:38:10 zre-ldap002 slapd[22940]: conn=1005 op=4 RESULT tag=103 err=80 text= Oct 11 14:38:10 zre-ldap002 slapd[22940]: slap_graduate_commit_csn: removing 0x41d5110 20121011213810.017749Z#000000#000#000000 Oct 11 14:38:10 zre-ldap002 slapd[22940]: conn=1005 fd=11 closed (connection lost)

1 0

Re: (ITS#7415) Add MALLOC_CHECK_ and MALLOC_PERTURB_ libc env to the test suite for detecting heap corruption
by h.b.furuseth＠usit.uio.no 11 Oct '12

11 Oct '12

We can describe these and other variables in tests/README. Yeah I know, people don't read documentation. The testing announcements could mention the file though.

1 0

Re: (ITS#7415) Add MALLOC_CHECK_ and MALLOC_PERTURB_ libc env to the test suite for detecting heap corruption
by masarati＠aero.polimi.it 11 Oct '12

11 Oct '12

> It is true in general, but it is also a matter of a personal taste. If > the openldap project use a buildboot that set in the environment these > variables , that patch is useless. This is also partially true if the > openldap developer use to set these variables in their personal > environment, because other contributor could not do the same. Other > project use a similar patch , for example popt 1.17 devel and git 1.8 > in master, i am the author so i know . what is more openldap now are > not using automake and the little test environment that automake could > use, so i have some difficulty to understeand what is this alternative > "test environment" . Perhaps i have missed something ? I meant the environment of the shell tests are run from. IMHO, I'd consider setting those macros a developer's conscious choice rather than a normal user's act. Moreover, they're specific to glibc, while users might want to use different malloc implementations, or build on platforms where glibc is not available, so the fact they're not honored might not be obvious. My 2c, of course. p. -- Pierangelo Masarati Associate Professor Dipartimento di Ingegneria Aerospaziale Politecnico di Milano

1 0

Re: (ITS#7415) Add MALLOC_CHECK_ and MALLOC_PERTURB_ libc env to the test suite for detecting heap corruption
by Kurt＠OpenLDAP.org 11 Oct '12

11 Oct '12

And, I should add, setting them up generally (not just for OpenLDAP command line tools), means they catch non-OpenLDAP bugs... and that leads to non-OpenLDAP issues being reported to the OpenLDAP Project. We should avoid this for when 'make test' is simply used by deployers as a quick sanity tools. Developers, on the other hand, hopefully know what they are doing when they set such environmental variables... and they know how to manage this locally. Best to avoid adding such to the public test system. -- Kurt

1 0

Re: (ITS#7415) Add MALLOC_CHECK_ and MALLOC_PERTURB_ libc env to the test suite for detecting heap corruption
by Kurt＠OpenLDAP.org 11 Oct '12

11 Oct '12

I note that these environment variables are specific to Linux, and then only certain Linux systems. If we add these, then we should also add similar environment variables for FreeBSD malloc(3), MacOS/X malloc(3), and various alternative malloc(3) libraries. And then we might get into cases where variables for one set of environmental variables conflict with another. As these can be set by the developer as desired in their development environment, there's little reason to get into setting all the malloc(3) environmental variables that might be useful... and we shouldn't play favors and only set variables for Linux. -- Kurt

1 0

Re: (ITS#7415) Add MALLOC_CHECK_ and MALLOC_PERTURB_ libc env to the test suite for detecting heap corruption
by gitter.spiros＠gmail.com 11 Oct '12

11 Oct '12

It is true in general, but it is also a matter of a personal taste. If the openldap project use a buildboot that set in the environment these variables , that patch is useless. This is also partially true if the openldap developer use to set these variables in their personal environment, because other contributor could not do the same. Other project use a similar patch , for example popt 1.17 devel and git 1.8 in master, i am the author so i know . what is more openldap now are not using automake and the little test environment that automake could use, so i have some difficulty to understeand what is this alternative "test environment" . Perhaps i have missed something ? 2012/10/11, Pierangelo Masarati <masarati(a)aero.polimi.it>: > These vars can be set directly from the test environment, no need to > modify the test scripts. > > p. > >> Full_Name: Elia Pinto >> Version: 2.4 master >> OS: Linux FC12 >> URL: ftp://ftp.openldap.org/incoming/ >> Submission from: (NULL) (2605:4400:1:781:216:3eff:fe31:f4d4) >> >> >>>From a8ff21429c29f1d2b6ef8f58ec84b7a9036cea73 Mon Sep 17 00:00:00 2001 >> From: Elia Pinto <gitter.spiros(a)gmail.com> >> Date: Thu, 11 Oct 2012 17:49:06 +0200 >> Subject: [PATCH] Add MALLOC_CHECK_ and MALLOC_PERTURB_ libc env to the >> test >> suite for detecting heap corruption >> >> Recent versions of Linux libc (later than 5.4.23) and glibc (2.x) >> include a malloc() implementation which is tunable via environment >> variables. When MALLOC_CHECK_ is set, a special (less efficient) >> implementation is used which is designed to be tolerant against >> simple errors, such as double calls of free() with the same argument, >> or overruns of a single byte (off-by-one bugs). When MALLOC_CHECK_ >> is set to 3, a diagnostic message is printed on stderr >> and the program is aborted. >> >> Setting the MALLOC_PERTURB_ environment variable causes the malloc >> functions in libc to return memory which has been wiped and clear >> memory when it is returned. >> Of course this does not affect calloc which always does clear the memory. >> >> The reason for this exercise is, of course, to find code which uses >> memory returned by malloc without initializing it and code which uses >> code after it is freed. valgrind can do this but it's costly to run. >> The MALLOC_PERTURB_ exchanges the ability to detect problems in 100% >> of the cases with speed. >> >> The byte value used to initialize values returned by malloc is the byte >> value of the environment value. The value used to clear memory is the >> bitwise inverse. Setting MALLOC_PERTURB_ to zero disables the feature. >> >> This technique can find hard to detect bugs. >> It is therefore suggested to always use this flag (at least temporarily) >> when testing out code or a new distribution. >> >> Signed-off-by: Elia Pinto <gitter.spiros(a)gmail.com> >> --- >> tests/run.in | 5 +++++ >> 1 file changed, 5 insertions(+) >> >> diff --git a/tests/run.in b/tests/run.in >> index 5e6178b..fe25d0c 100644 >> --- a/tests/run.in >> +++ b/tests/run.in >> @@ -241,6 +241,11 @@ fi >> >> # disable LDAP initialization >> LDAPNOINIT=true; export LDAPNOINIT >> +# Add libc malloc_check and MALLOC_PERTURB test >> +MALLOC_CHECK_=3 >> +export MALLOC_CHECK_ >> +MALLOC_PERTURB_="$( expr $ $$ % 255 $ + 1)" >> +export MALLOC_PERTURB_ >> >> echo "Running ${SCRIPT} for ${BACKEND}..." >> while [ $COUNTER -le $LOOP ]; do >> -- >> 1.7.11.rc1 >> >> >> >> >> >> > > > -- > Pierangelo Masarati > Associate Professor > Dipartimento di Ingegneria Aerospaziale > Politecnico di Milano > > -- Inviato dal mio dispositivo mobile

1 0

Re: (ITS#7415) Add MALLOC_CHECK_ and MALLOC_PERTURB_ libc env to the test suite for detecting heap corruption
by masarati＠aero.polimi.it 11 Oct '12

11 Oct '12

These vars can be set directly from the test environment, no need to modify the test scripts. p. > Full_Name: Elia Pinto > Version: 2.4 master > OS: Linux FC12 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (2605:4400:1:781:216:3eff:fe31:f4d4) > > >>From a8ff21429c29f1d2b6ef8f58ec84b7a9036cea73 Mon Sep 17 00:00:00 2001 > From: Elia Pinto <gitter.spiros(a)gmail.com> > Date: Thu, 11 Oct 2012 17:49:06 +0200 > Subject: [PATCH] Add MALLOC_CHECK_ and MALLOC_PERTURB_ libc env to the > test > suite for detecting heap corruption > > Recent versions of Linux libc (later than 5.4.23) and glibc (2.x) > include a malloc() implementation which is tunable via environment > variables. When MALLOC_CHECK_ is set, a special (less efficient) > implementation is used which is designed to be tolerant against > simple errors, such as double calls of free() with the same argument, > or overruns of a single byte (off-by-one bugs). When MALLOC_CHECK_ > is set to 3, a diagnostic message is printed on stderr > and the program is aborted. > > Setting the MALLOC_PERTURB_ environment variable causes the malloc > functions in libc to return memory which has been wiped and clear > memory when it is returned. > Of course this does not affect calloc which always does clear the memory. > > The reason for this exercise is, of course, to find code which uses > memory returned by malloc without initializing it and code which uses > code after it is freed. valgrind can do this but it's costly to run. > The MALLOC_PERTURB_ exchanges the ability to detect problems in 100% > of the cases with speed. > > The byte value used to initialize values returned by malloc is the byte > value of the environment value. The value used to clear memory is the > bitwise inverse. Setting MALLOC_PERTURB_ to zero disables the feature. > > This technique can find hard to detect bugs. > It is therefore suggested to always use this flag (at least temporarily) > when testing out code or a new distribution. > > Signed-off-by: Elia Pinto <gitter.spiros(a)gmail.com> > --- > tests/run.in | 5 +++++ > 1 file changed, 5 insertions(+) > > diff --git a/tests/run.in b/tests/run.in > index 5e6178b..fe25d0c 100644 > --- a/tests/run.in > +++ b/tests/run.in > @@ -241,6 +241,11 @@ fi > > # disable LDAP initialization > LDAPNOINIT=true; export LDAPNOINIT > +# Add libc malloc_check and MALLOC_PERTURB test > +MALLOC_CHECK_=3 > +export MALLOC_CHECK_ > +MALLOC_PERTURB_="$( expr $ $$ % 255 $ + 1)" > +export MALLOC_PERTURB_ > > echo "Running ${SCRIPT} for ${BACKEND}..." > while [ $COUNTER -le $LOOP ]; do > -- > 1.7.11.rc1 > > > > > > -- Pierangelo Masarati Associate Professor Dipartimento di Ingegneria Aerospaziale Politecnico di Milano

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs