openldap-bugs January 2009

openldap-bugs@openldap.org

50 participants
201 discussions

Re: (ITS#5894) test039 fails: ldap_back_conn_delete: Assertion
by quanah＠zimbra.com 23 Jan '09

23 Jan '09

--On Wednesday, January 21, 2009 2:35 PM +0100 Michael Ströder <michael(a)stroeder.com> wrote: > quanah(a)zimbra.com wrote: >> --On Saturday, January 17, 2009 3:25 PM +0000 michael(a)stroeder.com wrote: >> >>> Howard Chu wrote: >>>> michael(a)stroeder.com wrote: >>>>> Full_Name: Michael Str?der >>>>> Version: HEAD >>>>> OS: Linux >>>>> URL: ftp://ftp.openldap.org/incoming/ >>>>> Submission from: (NULL) (84.163.100.169) >>>>> >>>>> >>>>> lt-slapd: bind.c:157: ldap_back_conn_delete: Assertion >>>>> `!(*(&((lc))->lc_lcflags) >>>>> & (((0x00000020U))))' failed. >>>> I'm not seeing that here. Ran into a couple other problems though, >>>> fixed now in HEAD. Please re-test... >>> Now test008 hangs... >> >> Multiple runs of test008 work for me, using BDB 4.7.25 with all 3 >> patches, and HEAD CVS as of 1/20/2009 @ 12:00 PST. >> >> Did you fully rebuild OpenLDAP after the latest commits? What version >> of BDB are you using? Is it fully patched? What backend was it using >> when it hung? Did you get a backtrace of where it hung? > > I'm using libdb-4_5-4.5.20-94.17 from openSUSE 11.1 which is AFAIK fully > patched. > > I think this is related to modifications in connection.c because trouble > started after HEAD contained these changes. You still didn't answer most of these questions, or what kernel you're using. I ask about the full rebuild of OpenLDAP because the changes in the connection code changed the size of the connection struct, and I know it will fail without that rebuild. A backtrace of the hang would be useful for diagnosis, since so far neither Howard nor I can reproduce this. I'd also like to know the Linux kernel version, since some of the changes affect epoll (linux 2.6) specifically. I'm running a 2.6.18 kernel+patches currently. Also, what flags are you passing to gcc while building? -g? -O2? etc. I.e., are you putting optimizations into place. Thanks, Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

Re: (ITS#5860) slapd memeory leak under openldap 2.4
by hyc＠symas.com 22 Jan '09

22 Jan '09

rlvcosta(a)yahoo.com wrote: > Quanah, > > I just used tcmalloc but I still with the same behavior. slapd doesn't > have limit for the cache and then it cannot run large databases. As a side comment, we have already documented that about 1 million entries is the practical limit for slapd on a 32 bit machine. Since 64 bit machines are so common today, making slapd perform well on a 32 bit machine is no longer a priority. Also, by design, tcmalloc doesn't release freed memory back to the OS, since it will most likely be needed again. It's normal for the slapd process size to never shrink. One thing you can check is to add a "database monitor" to your slapd configuration; one of the entries under cn=monitor will report the number of items in the back-bdb cache. Please provide this information here. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#5860) slapd memeory leak under openldap 2.4
by rlvcosta＠yahoo.com 22 Jan '09

22 Jan '09

Quanah, I just used tcmalloc but I still with the same behavior. slapd doesn't have limit for the cache and then it cannot run large databases. My DB entrance is not large. One example is : dn: uid=1219843773896\2B554184XXXXXX,ou=REPOSITORY,ou=CONTENT,o=domain,c=country creationdate: 200808271329Z localelist: en,fr weblogin: +554184XXXXXX expirationdate: 200810161329Z profileid: 0 mail: +554184XXXXXX/type=PLMN@<domain> uid: 1219843773896+554184XXXXXX objectClass: commonuser objectClass: user accountStatus: 1 webpassword: 0000 cn: +554184XXXXXX optionvalues:: AAEAAAcBAgEAAQEBAQEBAAAAAQEBAAEBAQEAAQAAAAAAAAMAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA== optionstates:: AAAAAAABAwAAAAAAAAABAAAAAQIAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA== isintercepted: FALSE dn: pnnumber=\2B554184XXXXXX,uid=1219843773896\2B554184XXXXXX,ou=REPOSITORY,ou =CONTENT,o=domain,c=country subpnid: 0 pntype: 2 pncaps: 7 objectClass: phoneinfo pnnumber: +554184XXXXXX dn: submxid=0,uid=1219843773896\2B554184XXXXXX,ou=REPOSITORY,ou=CONTENT,o=domain,c=country secretcode: 0000 fraudaccessnb: 0 mailMessageStore: ./Maildir maillogin: +554184XXXXXX/type=PLMN@<domain> objectClass: commonmailbox objectClass: courierinfo submxid: 0 mailQuota: 1048576 mailpassword: <secret> homeDirectory: /var/mms/mail/0/00/0000000000/0 See above I have 3 DN's. In 64 bits systems with more memory and small dn's this could be hidden but with the time all physical memory will be used. So, I made an ldapsearch as (searching a single DN): time ldapsearch -LLL -x -D "cn=admin,ou=CONTENT,o=domain,c=country" -w secret -b "ou=CONTENT,o=domain,c=country" -H ldap://10.142.15.42:389 'pnnumber=*'| egrep -e '^pnnumber*' |wc -l 1000003 real 6m56.321s user 0m25.617s sys 0m8.392s So just one of the 3 DNs this DB has. Since the filesize is : 1.9G dn2id.bdb 3.1G id2entry.bdb 29M maillogin.bdb 3.3M objectClass.bdb 28M pnnumber.bdb 1.3M submxid.bdb 84M uid.bdb I have part of size load into cache(only for pnnnumber). But slapd doesn't respect cache limits and load all into memory. It also never release it. The entrances are not large and caches in the order of 1000 entrances should not make slapd process stay with almost 1GB of memory. Also once it occupies this memory it never release anymore. I use lsof and the tcmalloc is being used. Regards, Rodrigo. --- On Thu, 1/22/09, Quanah Gibson-Mount <quanah(a)zimbra.com> wrote: > From: Quanah Gibson-Mount <quanah(a)zimbra.com> > Subject: Re: (ITS#5860) slapd memeory leak under openldap 2.4 > To: rlvcosta(a)yahoo.com, openldap-its(a)openldap.org > Date: Thursday, January 22, 2009, 4:04 PM > --On Thursday, January 22, 2009 10:01 AM -0800 Rodrigo Costa > <rlvcosta(a)yahoo.com> wrote: > > > Quanah, > > > > Just a correction about the man, it was incorrect > since my man was > > getting from cache(and so from 2.3 version). Now it is > ok. > > > > I would like a little more directions about how can be > compiled openldap > > 2.4.13 with tcmalloc. Just need to include a lib path? > Some configure > > parameter in openldap? > > I use tcmalloc myself, so basically you go and build > google-perf-tools, and then you preload the library before > starting slapd. I have a little startup script for slapd: > > #!/bin/bash > ulimit -n 32768 > ulimit -c unlimited > export LD_PRELOAD=/opt/zimbra/tcmalloc/lib/libtcmalloc.so > exec /opt/zimbra/openldap/sbin/slapd "$@" > > > --Quanah > > > -- > > Quanah Gibson-Mount > Principal Software Engineer > Zimbra, Inc > -------------------- > Zimbra :: the leader in open source messaging and > collaboration

1 0

Re: (ITS#5860) slapd memeory leak under openldap 2.4
by quanah＠zimbra.com 22 Jan '09

22 Jan '09

--On Thursday, January 22, 2009 10:01 AM -0800 Rodrigo Costa <rlvcosta(a)yahoo.com> wrote: > Quanah, > > Just a correction about the man, it was incorrect since my man was > getting from cache(and so from 2.3 version). Now it is ok. > > I would like a little more directions about how can be compiled openldap > 2.4.13 with tcmalloc. Just need to include a lib path? Some configure > parameter in openldap? I use tcmalloc myself, so basically you go and build google-perf-tools, and then you preload the library before starting slapd. I have a little startup script for slapd: #!/bin/bash ulimit -n 32768 ulimit -c unlimited export LD_PRELOAD=/opt/zimbra/tcmalloc/lib/libtcmalloc.so exec /opt/zimbra/openldap/sbin/slapd "$@" --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

Re: (ITS#5860) slapd memeory leak under openldap 2.4
by rlvcosta＠yahoo.com 22 Jan '09

22 Jan '09

Quanah, Just a correction about the man, it was incorrect since my man was getting from cache(and so from 2.3 version). Now it is ok. I would like a little more directions about how can be compiled openldap 2.4.13 with tcmalloc. Just need to include a lib path? Some configure parameter in openldap? Regards, Rodrigo. --- On Thu, 1/22/09, Quanah Gibson-Mount <quanah(a)zimbra.com> wrote: > From: Quanah Gibson-Mount <quanah(a)zimbra.com> > Subject: Re: (ITS#5860) slapd memeory leak under openldap 2.4 > To: rlvcosta(a)yahoo.com, openldap-its(a)openldap.org > Date: Thursday, January 22, 2009, 2:53 PM > --On Thursday, January 22, 2009 4:46 PM +0000 > rlvcosta(a)yahoo.com wrote: > > > [root@brtldp12 yum.repos.d]# ll > /usr/man/man8/slapd.8.gz > > -rw-r--r-- 1 root root 4392 Jan 22 03:28 > /usr/man/man8/slapd.8.gz > > > > [root@brtldp12 yum.repos.d]# ll > /usr/man/man5/slapd-bdb.5.gz > > -rw-r--r-- 1 root root 4089 Jan 22 03:28 > /usr/man/man5/slapd-bdb.5.gz > > > > I was expecting to find information about dncachesize > in slapd-bdb but > > there isn't nothing there. > > [build@build11 man5]$ grep dncachesize * > slapd-bdb.5:.BI dncachesize \ <integer> > slapd-hdb.5:.BI dncachesize \ <integer> > > It's documented for me... > > .BI dncachesize \ <integer> > Specify the maximum number of DNs in the in-memory DN > cache. The > default is twice the \fBcachesize\fP. Ideally this > cache should be > large enough to contain the DNs of every entry in the > database. > > You can also find it in the online manual pages: > > <http://www.openldap.org/software/man.cgi?query=slapd-bdb&apropos=0&sektion=…> > > > > > In any case slapd never respects memory limitation and > consumes it > > without boundaries. It also never releases any memory. > > > > Then after sometime the process cannot allocate more > memory and crashes. > > As Howard noted, you need to use an alternative memory > allocator than > glibc. Investigate horde or tcmalloc. > > --Quanah > > > -- > > Quanah Gibson-Mount > Principal Software Engineer > Zimbra, Inc > -------------------- > Zimbra :: the leader in open source messaging and > collaboration

1 0

Re: (ITS#5900) Problem to populate LDAP 2.4.12 with an Ldiff file on SUNBLADE
by hyc＠symas.com 22 Jan '09

22 Jan '09

fethi.tekitek(a)alcatel-lucent.com wrote: > Full_Name: Fathy Tekitek > Version: 2.4.12 > OS: Solaris 10 (rev 8) > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (62.23.212.109) > > > Hi, > I try to populate the openldap 2.4.12 running with BerkeleyDB 4.7.25 using an > ldiff file > The first two entry are populate but It remains blocked > This problem is reproducible on SunBlade 100 servers but not in T5000, > SunBlade2500 and Ultra45 where it works OK. Make sure you have applied the patches to BDB 4.7.25 from the Oracle web site. This ITS will be closed. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#5860) slapd memeory leak under openldap 2.4
by hyc＠symas.com 22 Jan '09

22 Jan '09

rlvcosta(a)yahoo.com wrote: > Howard, > > I have the always the same DB_CONFIG file as below : > > set_cachesize 0 52428800 1 > > set_flags DB_LOG_AUTOREMOVE > > set_lg_regionmax 1048576 > set_lg_max 10485760 > set_lg_bsize 2097152 > > set_tmp_dir /var/tmp > > I just removed the comment lines so it fits better on screen. > > About versions I have : > > [root@brtldp12 yum.repos.d]# /usr/libexec/slapd -V > @(#) $OpenLDAP: slapd 2.4.13 (Jan 22 2009 02:28:12) $ > root@brtldp12:/root/openldap-2.4.13/servers/slapd > > [root@brtldp12 yum.repos.d]# db_verify -V > Berkeley DB 4.7.25: (May 15, 2008) > > About the man pages I have : > > [root@brtldp12 yum.repos.d]# ll /usr/man/man8/slapd.8.gz > -rw-r--r-- 1 root root 4392 Jan 22 03:28 /usr/man/man8/slapd.8.gz > > [root@brtldp12 yum.repos.d]# ll /usr/man/man5/slapd-bdb.5.gz > -rw-r--r-- 1 root root 4089 Jan 22 03:28 /usr/man/man5/slapd-bdb.5.gz > > I was expecting to find information about dncachesize in slapd-bdb but there isn't nothing there. According to CVS the documentation was added in August 2007. It's been online for quite a long time as well. http://www.openldap.org/software/man.cgi?query=slapd-bdb&sektion=5&apropos=… Another point is that the cache configurations are based on a specific number of items, and not on the size of each item in bytes. If you have particularly large entries or long DNs then you'll have greater memory use as well. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#5860) slapd memeory leak under openldap 2.4
by quanah＠zimbra.com 22 Jan '09

22 Jan '09

--On Thursday, January 22, 2009 4:46 PM +0000 rlvcosta(a)yahoo.com wrote: > [root@brtldp12 yum.repos.d]# ll /usr/man/man8/slapd.8.gz > -rw-r--r-- 1 root root 4392 Jan 22 03:28 /usr/man/man8/slapd.8.gz > > [root@brtldp12 yum.repos.d]# ll /usr/man/man5/slapd-bdb.5.gz > -rw-r--r-- 1 root root 4089 Jan 22 03:28 /usr/man/man5/slapd-bdb.5.gz > > I was expecting to find information about dncachesize in slapd-bdb but > there isn't nothing there. [build@build11 man5]$ grep dncachesize * slapd-bdb.5:.BI dncachesize \ <integer> slapd-hdb.5:.BI dncachesize \ <integer> It's documented for me... .BI dncachesize \ <integer> Specify the maximum number of DNs in the in-memory DN cache. The default is twice the \fBcachesize\fP. Ideally this cache should be large enough to contain the DNs of every entry in the database. You can also find it in the online manual pages: <http://www.openldap.org/software/man.cgi?query=slapd-bdb&apropos=0&sektion=…> > > In any case slapd never respects memory limitation and consumes it > without boundaries. It also never releases any memory. > > Then after sometime the process cannot allocate more memory and crashes. As Howard noted, you need to use an alternative memory allocator than glibc. Investigate horde or tcmalloc. --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

Re: (ITS#5860) slapd memeory leak under openldap 2.4
by rlvcosta＠yahoo.com 22 Jan '09

22 Jan '09

Howard, I have the always the same DB_CONFIG file as below : set_cachesize 0 52428800 1 set_flags DB_LOG_AUTOREMOVE set_lg_regionmax 1048576 set_lg_max 10485760 set_lg_bsize 2097152 set_tmp_dir /var/tmp I just removed the comment lines so it fits better on screen. About versions I have : [root@brtldp12 yum.repos.d]# /usr/libexec/slapd -V @(#) $OpenLDAP: slapd 2.4.13 (Jan 22 2009 02:28:12) $ root@brtldp12:/root/openldap-2.4.13/servers/slapd [root@brtldp12 yum.repos.d]# db_verify -V Berkeley DB 4.7.25: (May 15, 2008) About the man pages I have : [root@brtldp12 yum.repos.d]# ll /usr/man/man8/slapd.8.gz -rw-r--r-- 1 root root 4392 Jan 22 03:28 /usr/man/man8/slapd.8.gz [root@brtldp12 yum.repos.d]# ll /usr/man/man5/slapd-bdb.5.gz -rw-r--r-- 1 root root 4089 Jan 22 03:28 /usr/man/man5/slapd-bdb.5.gz I was expecting to find information about dncachesize in slapd-bdb but there isn't nothing there. Where at config.c I have : "( OLcfgDbAt:1.5 NAME 'olcDbDirtyRead' " "DESC 'Allow reads of uncommitted data' " "SYNTAX OMsBoolean SINGLE-VALUE )", NULL, NULL }, { "dncachesize", "size", 2, 2, 0, ARG_UINT|ARG_OFFSET, (void *)offsetof(struct bdb_info, bi_cache.c_eimax), "( OLcfgDbAt:1.12 NAME 'olcDbDNcacheSize' " "DESC 'DN cache size' " "SYNTAX OMsInteger SINGLE-VALUE )", NULL, NULL } So the dncachesize is supposed to be in the code. In any case slapd never respects memory limitation and consumes it without boundaries. It also never releases any memory. Then after sometime the process cannot allocate more memory and crashes. Best Regards, Rodrigo. --- On Thu, 1/22/09, Howard Chu <hyc(a)symas.com> wrote: > From: Howard Chu <hyc(a)symas.com> > Subject: Re: (ITS#5860) slapd memeory leak under openldap 2.4 > To: rlvcosta(a)yahoo.com > Cc: openldap-its(a)openldap.org > Date: Thursday, January 22, 2009, 1:35 PM > rlvcosta(a)yahoo.com wrote: > > > See that I have 2 DBs and in each DB area in > slapd.conf, read without error by slapd as seen above, I > have the cachesize, idlcachesize and dncachesize specified > with small values just to monitor the memory usage by slapd. > > > > Unfortunately even with this configuration I still > having slapd consuming memory without release it. So if a > new entrance is queried by some LDAP client, slapd will > consume more memory. Since my DB dn's are bigger than > 3GB then soon or later the slapd will crash. > > > > See my DB information : > > > > 7.2G dn2id.bdb > > 12G id2entry.bdb > > 110M maillogin.bdb > > 3.3M objectClass.bdb > > 108M pnnumber.bdb > > 1.3M submxid.bdb > > 323M uid.bdb > > > > See my dn2id is much bigger than yours and maybe you > just did not saw this > issue happenig since your dn can be loaded in your memory > and you in a 64 bits > environment can use more than 3GB per process. > > > > But in any case looks like there is a problem in slapd > that do not respect > the cache limitations imposed. > > > > In this way we cannot use slapd for large databases. > Please see my > configuration above and let me know if a made a mistake(I > hope so) since > dncachesize is not documented at man pages. > > You're mistaken, dncachesize is in the 2.4 manpages. > You seem to not be > reading the documentation that came with the software > release you're using. > > > Just as example, with the DB I loaded for tests with 1 > million entrances, with the cache sizes definitions as in > the slapd.conf described above and making a ldapsearch on > all entrances from the CONTENT DB, in the end I have : > > > > PID USER PR NI VIRT RES SHR S %CPU %MEM > TIME+ COMMAND > > 3189 ldap 15 0 842m 724m 67m S 99 6.1 > 7:00.48 slapd > > > > See the slapd process already consumed 842m and I just > read 1 million entrances, even dncachesize is defined to be > 1000. > > > > I do not see anyway to control the memory usage by > slapd. > > > > Is this "disrespect" to slapd.conf > directives a possible problem? > > Looks like slapd is behaving normally so far, though > it's hard to tell without > also seeing your DB_CONFIG settings. Test again with your > full database and > report the results. > > -- > -- Howard Chu > CTO, Symas Corp. http://www.symas.com > Director, Highland Sun http://highlandsun.com/hyc/ > Chief Architect, OpenLDAP > http://www.openldap.org/project/

1 0

Re: (ITS#5860) slapd memeory leak under openldap 2.4
by hyc＠symas.com 22 Jan '09

22 Jan '09

hyc(a)symas.com wrote: > rlvcosta(a)yahoo.com wrote: > >> See that I have 2 DBs and in each DB area in slapd.conf, read without error by slapd as seen above, I have the cachesize, idlcachesize and dncachesize specified with small values just to monitor the memory usage by slapd. >> >> Unfortunately even with this configuration I still having slapd consuming memory without release it. So if a new entrance is queried by some LDAP client, slapd will consume more memory. Since my DB dn's are bigger than 3GB then soon or later the slapd will crash. >> >> See my DB information : >> >> 7.2G dn2id.bdb >> 12G id2entry.bdb >> 110M maillogin.bdb >> 3.3M objectClass.bdb >> 108M pnnumber.bdb >> 1.3M submxid.bdb >> 323M uid.bdb > You're mistaken, dncachesize is in the 2.4 manpages. You seem to not be > reading the documentation that came with the software release you're using. > >> Just as example, with the DB I loaded for tests with 1 million entrances, with the cache sizes definitions as in the slapd.conf described above and making a ldapsearch on all entrances from the CONTENT DB, in the end I have : >> >> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND >> 3189 ldap 15 0 842m 724m 67m S 99 6.1 7:00.48 slapd >> >> See the slapd process already consumed 842m and I just read 1 million entrances, even dncachesize is defined to be 1000. >> >> I do not see anyway to control the memory usage by slapd. >> >> Is this "disrespect" to slapd.conf directives a possible problem? > > Looks like slapd is behaving normally so far, though it's hard to tell without > also seeing your DB_CONFIG settings. Test again with your full database and > report the results. > I'll note that this is a pathological use case for glibc malloc. With a cache of only 1000 entries, searching 1 million entries will result in several million malloc/free calls which will fragment the heap, and that will cause results like what you're seeing. You'll have better luck with hoard or tcmalloc. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

← Newer
1
...
8
9
10
11
12
13
14
...
21
Older →

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs January 2009