openldap-bugs September 2010

openldap-bugs@openldap.org

26 participants
109 discussions

Re: (ITS#6660) paged result searches fail to deallocate memory until slapd shutdown
by masarati＠aero.polimi.it 29 Sep '10

29 Sep '10

> --On Wednesday, September 29, 2010 12:38 AM +0000 quanah(a)zimbra.com wrote: > >> It appears to be a problem with the entry cache, which is set to 25,000: > > Running with a fix from Howard, the entry cache behaves correctly. > However, slapd still grows at the same rate. > > If I limit to only 10 paged results searches, slapd grows at a rate of > 300MB Virtual and 300MB Resident for every set of 10 paged results > searches > I do concurrently, up until I run slapd out of memory. There's something > very wrong with paged results searches. Could it be configuration-specific? I tested with a plain configuration resulting from test003; maybe some player in the middle, say, is causing entries to be duplicated and leaked, or read-locks on originals are not released correctly? Can you post a configuration that shows the issue? p.

1 0

Re: (ITS#6660) paged result searches fail to deallocate memory until slapd shutdown
by quanah＠zimbra.com 29 Sep '10

29 Sep '10

--On Wednesday, September 29, 2010 12:38 AM +0000 quanah(a)zimbra.com wrote: > It appears to be a problem with the entry cache, which is set to 25,000: Running with a fix from Howard, the entry cache behaves correctly. However, slapd still grows at the same rate. If I limit to only 10 paged results searches, slapd grows at a rate of 300MB Virtual and 300MB Resident for every set of 10 paged results searches I do concurrently, up until I run slapd out of memory. There's something very wrong with paged results searches. --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

ITS#6661
by masarati＠aero.polimi.it 29 Sep '10

29 Sep '10

Please try back-ndb/bind.cpp 1.5->1.6 from HEAD's CVS. Thanks for the report. p.

1 0

(ITS#6661) rootpw is not verified with slapd.conf
by gtzanetis＠pylones.gr 29 Sep '10

29 Sep '10

Full_Name: George Tzanetis Version: 2.4.23 stable OS: Red Hat Enterprise 5.5 URL: Submission from: (NULL) (62.169.213.126) I have built openldap 2.4.23 with the back-ndb in 4 machines. I created the slapd.conf as follows: pidfile /usr/local/openldap/var/run/slapd.pid argsfile /usr/local/openldap/var/run/slapd.args ####################################################################### # NDB database definitions ####################################################################### #NDB database defintions database ndb suffix "dc=example,dc=gr" rootdn "cn=root,dc=example,dc=gr" rootpw secret dbconnect 192.168.6.11 dbhost 192.168.6.12 dbport 3306 dbname openldap dbuser ldapUser dbpass "1234" dbconnections 3 dbsocket /tmp/mysql.sock attrblob description index uid ####################################################################### # Monitor Database definitions ####################################################################### database monitor loglevel 5 My problem is that I can authenticate to the ldap with any password for the cn=root,dc=example,dc=gr (rootdn) user, as long as I specify a password. To make it clearer, all the following ldapsearches work: ldapsearch -h 192.168.6.10 -b 'dc=example,dc=gr' -w secret1 -D "cn=root,dc=example,dc=gr" ldapsearch -h 192.168.6.10 -b 'dc=example,dc=gr' -w secret -D "cn=root,dc=example,dc=gr" ldapsearch -h 192.168.6.10 -b 'dc=example,dc=gr' -w sec -D "cn=root,dc=example,dc=gr" ldapsearch -h 192.168.6.10 -b 'dc=example,dc=gr' -w " " -D "cn=root,dc=example,dc=gr" If I do not specify a password, (i.e. -w flag is omitted) I get the message: ldap_bind: Server is unwilling to perform (53) additional info: unauthenticated bind (DN with no password) disallowed In addition if I don input the correct rootdn user, I get the message: ldap_bind: Invalid credentials (49). This behavior exists in all instances of openldap with ndb as back-end. I did some more testing, and I built openldap with the bdb and ndb backends. The issue appears only to the suffix that is stored in the ndb back-end and not to the bdb back-end, so there must be something wrong with the bind operation of the slapd-ndb. Finally, I would like to state that with the slapd-ndb, all the ldapsearches / modifications / deletions are performed correctly, even if the rootpw password is wrong.

1 0

Re: (ITS#6659) logfile directive not logging any log
by masarati＠aero.polimi.it 29 Sep '10

29 Sep '10

hayashis(a)indiana.edu wrote: > --001636d34cf42a20380491547f9e > Content-Type: text/plain; charset=ISO-8859-1 > > Hi. I will use the correct mailing list. > > The place that I found the *wrong* information for logfile directive > (according to Google) was this doc >> http://www.zytrax.com/books/ldap/ch6/#logfile > > Here, it states, > > OpenLDAP logs via syslogd (using LOCAL4) in all cases (see loglevel for > configuration information on streaming syslogd LDAP messages to a separate > file). In addition the logfile directive may be used to create a separate > file containing just LDAP log information. > > Based on this information, I have *perceived* the behavior of slapd as a > BUG.. my apologies. Your opinion is valuable, just like that of anyone. The code is perfectible. However, to discuss opinions and improvements of code, a technical discussion forum is more appropriate that a bug tracking system. I believe that slapd already has provisions to address your needs, and pointed you to what I think are the appropriate sources of information. Changing the way software works (and has been working for decades) requires a real need and consensus. Fixing bugs is a completely different business. p.

1 0

Re: (ITS#6660) paged result searches fail to deallocate memory until slapd shutdown
by quanah＠zimbra.com 28 Sep '10

28 Sep '10

--On Tuesday, September 28, 2010 11:10 PM +0000 quanah(a)zimbra.com wrote: > --On Tuesday, September 28, 2010 8:04 PM +0200 masarati(a)aero.polimi.it > wrote: > >> Each paged results control request mallocs a small amount of memory on >> the slab which is not explicitly freed. It should not be an issue, >> since it occurs so early in the life of an operation that the slab >> should never be exhausted. In any case, I've committed a fix. Can you >> test and report, please? Thanks, p. > > Hi Pierangelo, > > I'm testing a build with your patch now. In the meantime, you can see a > callgraph generated from tcmalloc's heap profiling tool at > <http://www.stanford.edu/~quanah/pprof.pdf>. I baselined the graph to > start after the first initial searches have been to load the cache, etc. > So you can see that slapd grew by 14.3GB once I started doing the paged > results searches. It appears to be a problem with the entry cache, which is set to 25,000: olcDatabase={2}hdb.ldif:olcDbCacheSize: 25000 yet: olmBDBEntryCache: 209759 from cn=monitor. --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

Re: (ITS#6660) paged result searches fail to deallocate memory until slapd shutdown
by quanah＠zimbra.com 28 Sep '10

28 Sep '10

--On Tuesday, September 28, 2010 8:04 PM +0200 masarati(a)aero.polimi.it wrote: > Each paged results control request mallocs a small amount of memory on the > slab which is not explicitly freed. It should not be an issue, since it > occurs so early in the life of an operation that the slab should never be > exhausted. In any case, I've committed a fix. Can you test and report, > please? Thanks, p. Hi Pierangelo, I'm testing a build with your patch now. In the meantime, you can see a callgraph generated from tcmalloc's heap profiling tool at <http://www.stanford.edu/~quanah/pprof.pdf>. I baselined the graph to start after the first initial searches have been to load the cache, etc. So you can see that slapd grew by 14.3GB once I started doing the paged results searches. --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

Re: (ITS#6660) paged result searches fail to deallocate memory until slapd shutdown
by masarati＠aero.polimi.it 28 Sep '10

28 Sep '10

Each paged results control request mallocs a small amount of memory on the slab which is not explicitly freed. It should not be an issue, since it occurs so early in the life of an operation that the slab should never be exhausted. In any case, I've committed a fix. Can you test and report, please? Thanks, p.

1 0

(ITS#6660) paged result searches fail to deallocate memory until slapd shutdown
by quanah＠zimbra.com 28 Sep '10

28 Sep '10

Full_Name: Quanah Gibson-Mount Version: 2.4.23 OS: Linux 2.6 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (75.111.45.108) When using paged result searches with large databases, it quickly becomes apparent that slapd is failing to deallocate memory allocated during those searches once they complete. The example database I am using has 6,326,513 entries, and is some 8.5GB on disk (id2entry.bdb + dn2id.bdb). What has been found is that over time when using multiple paged results searches, slapd's memory continually grows until it eventually consumes all memory on the system. Once slapd is stopped, the allocated memory is freed. This behavior can be seen in a more pronounced fashion by running multiple paged results searches and searches of the full database simultaneously. For example, I start slapd, and preload all the caches by doing 4 full searches of the database. At this point, top reports (V/R/S): 12.2G/10G/8.4G I then start 10 ldap searches dumping the full database (using root dn), and 10 paged result searches that only return some 2.6 million entries. After these all complete, I run this sequence again. Instead of slapd's memory remaining stable (i.e., re-using all the memory it had previously allocated), it instead continues to grow. At the end of just 2 of these runs, we end up at: 22.5G/20G/8.4G Or a 10.3GB growth in virtual memory and 10GB growth in resident memory. As soon as slapd is halted, all of this memory is freed (Making it impossible to track this via valgrind). I'm currently using the heap profiler shipped with tcmalloc to gather further information and will follow up when that is complete. --Quanah

1 0

Re: (ITS#6659) logfile directive not logging any log
by quanah＠zimbra.com 28 Sep '10

28 Sep '10

--On Tuesday, September 28, 2010 12:40 PM -0400 Soichi Hayashi <hayashis(a)indiana.edu> wrote: > Hi. I will use the correct mailing list. > > The place that I found the *wrong* information for logfile directive > (according to Google) was this doc >> http://www.zytrax.com/books/ldap/ch6/#logfile This is why we always suggest you read the project supplied documentation. Zytrax is known to have numerous issues with what they have written. --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs September 2010