openldap-bugs

openldap-bugs@openldap.org

1 participants
20961 discussions

Re: (ITS#6660) paged result searches fail to deallocate memory until slapd shutdown
by quanah＠zimbra.com 29 Sep '10

29 Sep '10

--On Tuesday, September 28, 2010 8:04 PM +0200 masarati(a)aero.polimi.it wrote: > Each paged results control request mallocs a small amount of memory on the > slab which is not explicitly freed. It should not be an issue, since it > occurs so early in the life of an operation that the slab should never be > exhausted. In any case, I've committed a fix. Can you test and report, > please? Thanks, p. Hi Pierangelo, I'm testing a build with your patch now. In the meantime, you can see a callgraph generated from tcmalloc's heap profiling tool at <http://www.stanford.edu/~quanah/pprof.pdf>. I baselined the graph to start after the first initial searches have been to load the cache, etc. So you can see that slapd grew by 14.3GB once I started doing the paged results searches. --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

Re: (ITS#6660) paged result searches fail to deallocate memory until slapd shutdown
by masarati＠aero.polimi.it 28 Sep '10

28 Sep '10

Each paged results control request mallocs a small amount of memory on the slab which is not explicitly freed. It should not be an issue, since it occurs so early in the life of an operation that the slab should never be exhausted. In any case, I've committed a fix. Can you test and report, please? Thanks, p.

1 0

(ITS#6660) paged result searches fail to deallocate memory until slapd shutdown
by quanah＠zimbra.com 28 Sep '10

28 Sep '10

Full_Name: Quanah Gibson-Mount Version: 2.4.23 OS: Linux 2.6 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (75.111.45.108) When using paged result searches with large databases, it quickly becomes apparent that slapd is failing to deallocate memory allocated during those searches once they complete. The example database I am using has 6,326,513 entries, and is some 8.5GB on disk (id2entry.bdb + dn2id.bdb). What has been found is that over time when using multiple paged results searches, slapd's memory continually grows until it eventually consumes all memory on the system. Once slapd is stopped, the allocated memory is freed. This behavior can be seen in a more pronounced fashion by running multiple paged results searches and searches of the full database simultaneously. For example, I start slapd, and preload all the caches by doing 4 full searches of the database. At this point, top reports (V/R/S): 12.2G/10G/8.4G I then start 10 ldap searches dumping the full database (using root dn), and 10 paged result searches that only return some 2.6 million entries. After these all complete, I run this sequence again. Instead of slapd's memory remaining stable (i.e., re-using all the memory it had previously allocated), it instead continues to grow. At the end of just 2 of these runs, we end up at: 22.5G/20G/8.4G Or a 10.3GB growth in virtual memory and 10GB growth in resident memory. As soon as slapd is halted, all of this memory is freed (Making it impossible to track this via valgrind). I'm currently using the heap profiler shipped with tcmalloc to gather further information and will follow up when that is complete. --Quanah

1 0

Re: (ITS#6659) logfile directive not logging any log
by quanah＠zimbra.com 28 Sep '10

28 Sep '10

--On Tuesday, September 28, 2010 12:40 PM -0400 Soichi Hayashi <hayashis(a)indiana.edu> wrote: > Hi. I will use the correct mailing list. > > The place that I found the *wrong* information for logfile directive > (according to Google) was this doc >> http://www.zytrax.com/books/ldap/ch6/#logfile This is why we always suggest you read the project supplied documentation. Zytrax is known to have numerous issues with what they have written. --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

Re: (ITS#6659) logfile directive not logging any log
by hayashis＠indiana.edu 28 Sep '10

28 Sep '10

--001636d34cf42a20380491547f9e Content-Type: text/plain; charset=ISO-8859-1 Hi. I will use the correct mailing list. The place that I found the *wrong* information for logfile directive (according to Google) was this doc > http://www.zytrax.com/books/ldap/ch6/#logfile Here, it states, OpenLDAP logs via syslogd (using LOCAL4) in all cases (see loglevel for configuration information on streaming syslogd LDAP messages to a separate file). In addition the logfile directive may be used to create a separate file containing just LDAP log information. Based on this information, I have *perceived* the behavior of slapd as a BUG.. my apologies. On Tue, Sep 28, 2010 at 12:16 PM, Quanah Gibson-Mount <quanah(a)zimbra.com> wrote: > --On Tuesday, September 28, 2010 2:07 PM +0000 hayashis(a)indiana.edu wrote: > >> Thanks. >> >>> Please direct further sofware usage questions to the most appropriate >>> mai= >> >> ling lists >> >> Sorry to keep bugging, but I believe this *is* the most appropriate >> mailing list. > > You are wrong. This list is about bugs in the software. It has already > been noted there is no bug here. Please use the > openldap-technical(a)openldap.org if you have further questions. > > --Quanah > > -- > > Quanah Gibson-Mount > Principal Software Engineer > Zimbra, Inc > -------------------- > Zimbra :: the leader in open source messaging and collaboration > --001636d34cf42a20380491547f9e Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hi. I will use the correct mailing list. The place that I found the = *wrong* information for logfile directive (according to Google) was this do= c > <a href=3D"http://www.zytrax.com/books/ldap/ch6/#logfile">http://= www.zytrax.com/books/ldap/ch6/#logfile</a> Here, it states, OpenLDAP logs via syslogd (using LOCAL4) in al= l cases (see loglevel for configuration information on streaming syslogd LD= AP messages to a separate file). In addition the logfile directive may be u= sed to create a separate file containing just LDAP log information. <div>Based on this information, I have *perceived* the behavior of slap= d as a BUG.. my apologies.</div><div>=A0</div><div>On Tue, Sep 28, 2010 at = 12:16 PM, Quanah Gibson-Mount <<a href=3D"mailto:quanah@zimbra.com">quan= ah(a)zimbra.com</a>> wrote: > --On Tuesday, September 28, 2010 2:07 PM +0000 <a href=3D"mailto:hayas= his(a)indiana.edu">hayashis(a)indiana.edu</a> wrote: > >> Thanks= . >> >>> Please direct further sofware usage questions= to the most appropriate >>> mai=3D >> >> ling lists >> >&= gt; Sorry to keep bugging, but I believe this *is* the most appropriate = >> mailing list. > > You are wrong. =A0This list is about= bugs in the software. =A0It has already > been noted there is no bug here. =A0Please use the > <a href=3D"= mailto:openldap-technical@openldap.org">openldap-technical(a)openldap.org</a>= if you have further questions. > > --Quanah > > --= > > Quanah Gibson-Mount > Principal Software Engineer &gt= ; Zimbra, Inc > -------------------- > Zimbra :: =A0the leader = in open source messaging and collaboration > </div> --001636d34cf42a20380491547f9e--

1 0

Re: (ITS#6659) logfile directive not logging any log
by quanah＠zimbra.com 28 Sep '10

28 Sep '10

--On Tuesday, September 28, 2010 2:07 PM +0000 hayashis(a)indiana.edu wrote: > Thanks. > >> Please direct further sofware usage questions to the most appropriate >> mai= > ling lists > > Sorry to keep bugging, but I believe this *is* the most appropriate > mailing list. You are wrong. This list is about bugs in the software. It has already been noted there is no bug here. Please use the openldap-technical(a)openldap.org if you have further questions. --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

Re: (ITS#6659) logfile directive not logging any log
by masarati＠aero.polimi.it 28 Sep '10

28 Sep '10

> Thanks. > >> Please direct further sofware usage questions to the most appropriate >> mailing lists > > Sorry to keep bugging, but I believe this *is* the most appropriate > mailing list. > > Having to create a different user just so that I can log to a > different files doesn't seem like an optimal solution. Clearly, this > is a lack of necessary feature for slapd where user can not override > the location of the log file. Also, I've re-read the documentation > many times but it seems to suggest that I am supposed to be able to > forward the content of the log (sent to syslog) to a separate file. > Perhaps the documentation needs to be updated? "user" here means "facility" in syslog's nomenclature (don't know why it was called like that originally in slapd). So you don't need to create different users, you need to run slapd with "-l LOCAL0", "-l LOCAL1" etc. to be able to discriminate logs as "local0", "local1" and so in syslog.conf. The documentation of "logfile" I'm reading says logfile <filename> Specify a file for recording debug log messages. By default these messages only go to stderr and are not recorded anywhere else. Specifying a logfile copies messages to both stderr and the logfile. so it explicitly mentions "stderr" and never mentions "syslog". So it clearly refers to logging that goes to "stderr", which is controlled by the "-d" switch, which has nothing to do with syslog as well. I don't think it needs any amendment or clarification, but feel free to suggest any improvement (possibly in the form of a specific ITS with a patch to current documentation sources). p.

1 0

Re: (ITS#6659) logfile directive not logging any log
by hayashis＠indiana.edu 28 Sep '10

28 Sep '10

Thanks. > Please direct further sofware usage questions to the most appropriate mai= ling lists Sorry to keep bugging, but I believe this *is* the most appropriate mailing list. Having to create a different user just so that I can log to a different files doesn't seem like an optimal solution. Clearly, this is a lack of necessary feature for slapd where user can not override the location of the log file. Also, I've re-read the documentation many times but it seems to suggest that I am supposed to be able to forward the content of the log (sent to syslog) to a separate file. Perhaps the documentation needs to be updated? Thanks, Soichi On Tue, Sep 28, 2010 at 9:33 AM, <masarati(a)aero.polimi.it> wrote: > >> I am running multiple BDII servers (listening on different port), and >> I need them to log to different location. If I understand correctly, >> "logfile" directive only works to capture stderr messages (not stats, >> or other logs currently sent to syslog). Since I can't tell syslog to >> log to a different file based on which instance of LDAP server, logs >> from both server goes to a single file in /var/log. >> >> Is there anyway that I can redirect logs from each LDAP server to a >> different locations? > > You can use '-l' to have different instances of slapd log as different > syslog users. =A0See slapd(8) and syslog(8) for further details. > > Please direct further sofware usage questions to the most appropriate > mailing lists (openldap-technical for OpenLDAP software; Syslog-specific > lists for syslog(8) detaild usage). =A0This ITS will be closed. > > p. > >

1 0

Re: (ITS#6659) logfile directive not logging any log
by masarati＠aero.polimi.it 28 Sep '10

28 Sep '10

> I am running multiple BDII servers (listening on different port), and > I need them to log to different location. If I understand correctly, > "logfile" directive only works to capture stderr messages (not stats, > or other logs currently sent to syslog). Since I can't tell syslog to > log to a different file based on which instance of LDAP server, logs > from both server goes to a single file in /var/log. > > Is there anyway that I can redirect logs from each LDAP server to a > different locations? You can use '-l' to have different instances of slapd log as different syslog users. See slapd(8) and syslog(8) for further details. Please direct further sofware usage questions to the most appropriate mailing lists (openldap-technical for OpenLDAP software; Syslog-specific lists for syslog(8) detaild usage). This ITS will be closed. p.

1 0

Re: (ITS#6659) logfile directive not logging any log
by hayashis＠indiana.edu 28 Sep '10

28 Sep '10

Hi. Thank you for your reply! I am running multiple BDII servers (listening on different port), and I need them to log to different location. If I understand correctly, "logfile" directive only works to capture stderr messages (not stats, or other logs currently sent to syslog). Since I can't tell syslog to log to a different file based on which instance of LDAP server, logs from both server goes to a single file in /var/log. Is there anyway that I can redirect logs from each LDAP server to a different locations? On Tue, Sep 28, 2010 at 8:26 AM, <masarati(a)aero.polimi.it> wrote: >> Full_Name: Soichi Hayashi >> Version: 2.3.43-12.el5_5.2 >> OS: RHEL5 >> URL: ftp://ftp.openldap.org/incoming/ >> Submission from: (NULL) (99.50.225.74) >> >> >> I have following in my slapd.conf >> >> logfile /var/log/ldap.log >> >> When I start slapd, it creates the file specified but the filesize remai= ns >> 0 >> although I see a lot of log sent to syslog. >> >> It looks like the logging through "logfile" directive is not working. > > According to the documentation, this directive collects messages that are > sent to stderr when "-d" is used, so it is not related to syslog. =A0It i= s > equivalent to > > $ slapd -d <level> >>/var/log/ldap.log 2>&1 > > I've checked recent code and it works as expected. =A0Unless this is what > you're expecting, and it does not happen, this ITS will be closed. > > p. > >

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs