openldap-bugs October 2009

openldap-bugs@openldap.org

34 participants
223 discussions

Re: (ITS#6334) hang during ldapmodify
by quanah＠zimbra.com 16 Oct '09

16 Oct '09

--On Friday, October 16, 2009 1:00 PM +0000 mkd(a)cs.brown.edu wrote: > - we are running back_bdb > - syncprov > - GSSAPI authentication > > I'm happy to provide any additional details or debugging information, > just let me know what you need. What is the output from db_stat (For your BDB version) -c in your database where you are making modifications? Have you run out of locks/lockers/lock objects? --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

Re: (ITS#6335) syncprov modtarget deadlock
by h.b.furuseth＠usit.uio.no 16 Oct '09

16 Oct '09

rein(a)OpenLDAP.org writes: > If this is actually true, and if so why the lock and/or > destroy calls didn't fail beats me... If you mean mutex locks, maybe they did. OpenLDAP code generally doesn't check ldap_pvt_thread_*() for error return:-( You can turn on assert(success) inside these calls with ./configure CPPFLAGS='-DLDAP_THREAD_DEBUG=2 -D_GNU_SOURCE' make -s LDAP_THREAD_DEBUG=nocount nice make test _GNU_SOURCE is for Linux. But there might be some assert crashes during shutdown after failure, in particular in slap tools. -- Hallvard

1 0

Re: (ITS#6330) slapd memory leak
by quanah＠zimbra.com 16 Oct '09

16 Oct '09

--On Friday, October 16, 2009 9:24 AM -0700 Bill MacAllister <whm(a)stanford.edu> wrote: > > > --On Friday, October 16, 2009 06:58:44 AM -0700 Quanah Gibson-Mount > <quanah(a)zimbra.com> wrote: > >> --On Friday, October 16, 2009 10:39 AM +0000 whm(a)stanford.edu wrote: >> >> >>> And in researching the problem I see that this is a well trodden path. >>> I found a particularly useful thread from 2006 entitled "better malloc >>> strageties" in the OpenLDAP archives. As I result I packaged up >>> libhoard, installed it, and re-ran the valgrind test. The short test >>> showed no memory leaks. I will perform more extensive tests, but this >>> appears to have fixed the problem. Thanks for your work, then and >>> now. >> >> You were running slapd without hoard or tcmalloc???? That was a >> basic requirement I put into place for Stanford ages ago. >> >> --Quanah > > The desire was to run as close to stock debian was we could. The > libhoard package was dropped from debian with the comment "I no longer > use this software hence I orphan it." I took that to mean that it was > not maintained any more which was wrong. The upstream source appears to > be actively maintained. > > I was curious if you had an opinion about tcmalloc versus hoard? I switched to tcmalloc due to licensing issues. You can read Howard's presentation on the differences in how they behave (at some point in the past, tcmalloc has had a lot of development since then) at: <http://www.openldap.org/pub/hyc/scale2007.pdf> --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

Re: (ITS#6330) slapd memory leak
by whm＠stanford.edu 16 Oct '09

16 Oct '09

--On Friday, October 16, 2009 06:58:44 AM -0700 Quanah Gibson-Mount <quanah(a)zimbra.com> wrote: > --On Friday, October 16, 2009 10:39 AM +0000 whm(a)stanford.edu wrote: > > >> And in researching the problem I see that this is a well trodden path. >> I found a particularly useful thread from 2006 entitled "better malloc >> strageties" in the OpenLDAP archives. As I result I packaged up >> libhoard, installed it, and re-ran the valgrind test. The short test >> showed no memory leaks. I will perform more extensive tests, but this >> appears to have fixed the problem. Thanks for your work, then and >> now. > > You were running slapd without hoard or tcmalloc???? That was a > basic requirement I put into place for Stanford ages ago. > > --Quanah The desire was to run as close to stock debian was we could. The libhoard package was dropped from debian with the comment "I no longer use this software hence I orphan it." I took that to mean that it was not maintained any more which was wrong. The upstream source appears to be actively maintained. I was curious if you had an opinion about tcmalloc versus hoard? Bill -- Bill MacAllister, System Software Programmer Unix Systems Group, Stanford University

1 0

(ITS#6335) syncprov modtarget deadlock
by rein＠OpenLDAP.org 16 Oct '09

16 Oct '09

Full_Name: Rein Tollevik Version: CVS HEAD OS: linux URL: Submission from: (NULL) (2a01:600:0:1:21c:23ff:feab:61cd) Submitted by: rein A fix for a syncprov modtarget deadlock is coming. I have only seen this happen once, during shutdown of one of the servers in test058, but I'm uncertain as to whether the shutdown condition was coincidental or not. It is probably the same deadlock as reported by Aaron Richton during 2.4.19 testing. It looks as if syncprov was hanging waiting for a lock no-one held, possibly as a result of another thread removing the modtarget before or while it tried to acquire the modtarget mutex. If this is actually true, and if so why the lock and/or destroy calls didn't fail beats me... Anyhow, with this patch I have had test058 running without hanging for more than a week now. Rein Tollevik Basefarm AS

1 0

Re: (ITS#6330) slapd memory leak
by quanah＠zimbra.com 16 Oct '09

16 Oct '09

--On Friday, October 16, 2009 10:39 AM +0000 whm(a)stanford.edu wrote: > And in researching the problem I see that this is a well trodden path. > I found a particularly useful thread from 2006 entitled "better malloc > strageties" in the OpenLDAP archives. As I result I packaged up > libhoard, installed it, and re-ran the valgrind test. The short test > showed no memory leaks. I will perform more extensive tests, but this > appears to have fixed the problem. Thanks for your work, then and > now. You were running slapd without hoard or tcmalloc???? That was a basic requirement I put into place for Stanford ages ago. --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

(ITS#6334) hang during ldapmodify
by mkd＠cs.brown.edu 16 Oct '09

16 Oct '09

Full_Name: Mark Dieterich Version: 2.4.19 OS: debian linux URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (128.148.33.203) Hi all, I'm not sure if this is a bug or we are doing something wrong. For quite some time now, we have been using ldap to provide group information for our linux/unix desktop machines. I believer there are multiple methods of providing group access, this is the format we are using: # foobar, group, cs.brown.edu dn: cn=foobar,ou=group,dc=cs,dc=brown,dc=edu objectClass: posixGroup objectClass: top cn: ugrad gidNumber: 1200 memberUid: t1 memberUid: t2 memberUid: t3 memberUid: t4 memberUid: t5 . . . Up until recently, this had been working great. We are now experiencing hangs whenever we try to update the records with one particular group. I think the hangs are occur when we try to feed too much data to ldapmodify at a time. For instance, if I have the above group and try to apply the following ldif file: dn: cn=foobar,ou=group,dc=cs,dc=brown,dc=edu changetype: modify replace: memberUid memberUid: t1 memberUid: t2 memberUid: t3 memberUid: t4 memberUid: t5 memberUid: t6 memberUid: t7 memberUid: t8 memberUid: t9 . . . memberUid: t2477 memberUid: t2478 memberUid: t2479 The update simply hangs. Tossing a "-d 65535" shows that the process is sitting in loop spitting out messages similar to: ldap_int_select ldap_result ld 0x1e72bf0 msgid 5 wait4msg ld 0x1e72bf0 msgid 5 (timeout 100000 usec) wait4msg continue ld 0x1e72bf0 msgid 5 all 1 ** ld 0x1e72bf0 Connections: * host: ldapmaster.cs.brown.edu port: 6360 (default) refcnt: 2 status: Connected last used: Thu Oct 15 10:27:45 2009 ** ld 0x1e72bf0 Outstanding Requests: * msgid 5, origid 5, status InProgress outstanding referrals 0, parent count 0 ld 0x1e72bf0 request count 1 (abandoned 0) ** ld 0x1e72bf0 Response Queue: Empty ld 0x1e72bf0 response count 0 ldap_chkResponseList ld 0x1e72bf0 msgid 5 all 1 ldap_chkResponseList returns ld 0x1e72bf0 NULL ldap_int_select Hopefully, this means something more to someone on this list than it does to me;) Interestingly, if I drop the last entry and change the ldif file to be: dn: cn=foobar,ou=group,dc=cs,dc=brown,dc=edu changetype: modify replace: memberUid memberUid: t1 memberUid: t2 memberUid: t3 memberUid: t4 memberUid: t5 memberUid: t6 memberUid: t7 memberUid: t8 memberUid: t9 . . . memberUid: t2476 memberUid: t2477 memberUid: t2478 Then the update proceeds. However, if I change the last entry to read: memberUid: t2478000 Then it hangs again. Whereas with the last entry of memberUid: t247800 it updates just fine. This smells of some buffer filling or something else similar. While we originally discovered this using debian's packaged version of openldap, I confirmed yesterday that the problem still exists in the most recent stable release (2.4.19). Other details which may be pertinent: - we are running back_bdb - syncprov - GSSAPI authentication I'm happy to provide any additional details or debugging information, just let me know what you need. Thanks! Mark

1 0

Re: (ITS#6330) slapd memory leak
by h.b.furuseth＠usit.uio.no 16 Oct '09

16 Oct '09

hyc(a)symas.com writes: > Merely increasing --num-callers isn't going to be sufficient to track this. > Since valgrind only reports leaks after a program exits, The statement VALGRIND_DO_LEAK_CHECK; will check for leaks at that point in the code. The source file must #include <stdio.h> #include <valgrind/memcheck.h> Then run the program under valgrind. Regarding Cyrus SASL, I often ./configure --without-cyrus-sasl when testing OpenLDAP, to shut valgrind up about SASL. And I build Berkeley DB with ../dist/configure --enable-umrw trying to shut it up about DB. -- Hallvard

1 0

Re: (ITS#6330) slapd memory leak
by whm＠stanford.edu 16 Oct '09

16 Oct '09

--On Friday, October 16, 2009 07:16:23 AM +0000 Bill MacAllister <whm(a)stanford.edu> wrote: > > > --On Friday, October 16, 2009 12:02:59 AM -0700 Howard Chu <hyc(a)symas.com> wrote: > >> whm(a)stanford.edu wrote: >>> Here is the valgrind output. The complete log is at >>> >>> http://www.stanford.edu/~whm/files/slapd/valgrind-memcheck.txt >> >> All of this indicates memory leaks and errors in the SASL library, >> not any bug in OpenLDAP. > > That is what I suspected when I looked at the output, but I wanted > confirmation from someone that is more familar with valgrind output > and the code than me. Of course, until we resolve the issue it > effectively disables OpenLDAP 2.4 for us. And in researching the problem I see that this is a well trodden path. I found a particularly useful thread from 2006 entitled "better malloc strageties" in the OpenLDAP archives. As I result I packaged up libhoard, installed it, and re-ran the valgrind test. The short test showed no memory leaks. I will perform more extensive tests, but this appears to have fixed the problem. Thanks for your work, then and now. Bill -- Bill MacAllister, System Software Programmer Unix Systems Group, Stanford University

1 0

Re: (ITS#6330) slapd memory leak
by hyc＠symas.com 16 Oct '09

16 Oct '09

quanah(a)zimbra.com wrote: > --On Friday, October 16, 2009 7:57 AM +0000 masarati(a)aero.polimi.it wrote: > >> hyc(a)symas.com wrote: >>> whm(a)stanford.edu wrote: >>>> Here is the valgrind output. The complete log is at >>>> >>>> http://www.stanford.edu/~whm/files/slapd/valgrind-memcheck.txt >>> >>> All of this indicates memory leaks and errors in the SASL library, not >>> any bug in OpenLDAP. >> >> That was exactly my suspect when you originally stated that the leak >> only appeared after wrapping dynlist requests within a SASL bind/unbind. >> The leak could still be related to an improper use of SASL library >> calls by OpenLDAP, although unlikely. However, that valgrind output >> does not help as leaks related to malloc are too hidden into SASL; you >> should add --num-callers=<more than 12, the default; enough to see >> something related to OpenLDAP's code>. > > > Given that the memory issue is not seen with OpenLDAP 2.3 + dyngroup, I'd > still have to agree there's something wrong with what OpenLDAP 2.4 + > dynlist is doing. Merely increasing --num-callers isn't going to be sufficient to track this. Since valgrind only reports leaks after a program exits, and the Cyrus SASL library unloads the offending plugin before it exits, most of the stack trace will be missing any useful symbols. You'll have to run a modified libsasl2.so that omits its dlclose() calls so that the libgssapiv2 plugin will remain loaded, so that its symbols will still be accessible when slapd exits. Also, since the trace only shows leaks triggered from sasl_client_step and sasl_server_step, which are only executed during SASL Bind processing, you ought to be able to duplicate these leaks regardless of dynlist or any other overlays, since the leaks are occurring before any of these overlays have even started to run. Of course, if you're unable to reproduce these leaks with dynlist out of the picture, then I guess the finger points into dynlist again... -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

← Newer
1
...
9
10
11
12
13
14
15
...
23
Older →

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs October 2009