openldap-bugs August 2015

openldap-bugs@openldap.org

24 participants
69 discussions

(ITS#8229) ldapsearch: report failed search results to stderr even when ldif == 0
by ryan＠openldap.org 28 Aug '15

28 Aug '15

Full_Name: Ryan Tandy Version: master OS: Debian URL: Submission from: (NULL) (24.68.37.4) Submitted by: ryan Forwarded from a Debian bug report: <https://bugs.debian.org/474021> ldapsearch's error reporting currently does this: $ ldapsearch -x -b dc=nonexistent > /dev/null $ ldapsearch -x -b dc=nonexistent -L > /dev/null No such object (32) $ ldapsearch -x -b dc=nonexistent -LL > /dev/null No such object (32) $ ldapsearch -x -b dc=nonexistent -LLL > /dev/null No such object (32) The submitter would like it to report the failure to stderr in the default case as well. He suggested this change: - if( !ldif ) { - printf( "result: %d %s\n", err, ldap_err2string(err) ); - } else if ( err != LDAP_SUCCESS ) { - fprintf( stderr, "%s (%d)\n", ldap_err2string(err), err ); + if ( err != LDAP_SUCCESS %2%7{ + fprintf( stderr, "Search failed: %s (%d)\n", ldap_err2string(err), err ); + } + + if ( !ldif ) { + printf( "result: %d %s\n", err, ldap_err2string(err) ); } I would probably just flip the cases, and keep the "else" (i.e. not duplicate the output): - if( !ldif ) { - printf( _("result: %d %s\n"), err, ldap_err2string(err) ); - - } else if ( err != LDAP_SUCCESS ) { + if ( err != LDAP_SUCCESS ) { fprintf( stderr, "%s (%d)\n", ldap_r2ststring(err), err ); + } else if( !ldif ) { + printf( _("result: %d %s\n"), err, ldap_err2string(err) ); } Does either of these changes sound appropriate? The submitter also suggested sending the LDAP_SYNC Cancelled message to stderr: if ( cancel_msgid != -1 && cancel_msgid == ldap_msgid( msg ) ) { printf(_("Cancelled \n")); printf(_("cancel_msgid = %d\n"), cancel_msgid); I don't know that area very well, but at a glance it seems like stdout is probably more appropriate (so no change needed). Thanks.

1 0

Re: (ITS#8209) Broken MDB_CP_COMPACT threading
by h.b.furuseth＠usit.uio.no 28 Aug '15

28 Aug '15

More: MDB_CP_COMPACT would drop a new, empty main DB. (I.e. the main DB would lose its flags.) Compacting can verify that the written metapage claims enough DB pages. But not the opposite - it should not fail if it claims too many pages since that would mean we can't compact a DB which has suffered page leaks. We could detect that and re-write the metas at the end, if the user requests it and the file is seekable.

1 0

(ITS#8228) avoid clobbering a valid pid file
by ryan＠openldap.org 28 Aug '15

28 Aug '15

Full_Name: Ryan Tandy Version: master OS: Debian URL: Submission from: (NULL) (24.68.37.4) Submitted by: ryan Forwarding a feature request from <https://bugs.debian.org/375067>: It would be nice if slapd would notice an existing pid file pointing at a live process, and refuse to overwrite it. Starting slapd twice with the same pidfile is certainly user error, but it seems like it should be preventable.

1 0

RE: (ITS#8222) memory administration in ldap_parse_result()
by Lawrence.Newman＠alcatel-lucent.com 27 Aug '15

27 Aug '15

Providing an update: Not surprisingly, the actual memory leak was in our a= pplication software (error message allocated by ldap_parse_result() not fre= ed in unusual case). But our debugging/analysis tools continued to report = memory allocated but not freed even after the problem was fixed, and I beli= eve that was due to the issue I reported with how internal (library) memory= linked to the LDAP structure is allocated/freed by ldap_parse_result() (al= located while processing a response, but not freed until the next time the = connection associated with that LDAP structure is used). Larry Newman IHN 9H-401, x90882 lawrence.newman(a)alcatel-lucent.com -----Original Message----- From: openldap-its(a)OpenLDAP.org [mailto:openldap-its@OpenLDAP.org]=20 Sent: Tuesday, August 25, 2015 10:15 AM To: Newman, Lawrence H (Lawrence) Subject: Re: (ITS#8222) memory administration in ldap_parse_result() *** THIS IS AN AUTOMATICALLY GENERATED REPLY *** Thanks for your report to the OpenLDAP Issue Tracking System. Your report = has been assigned the tracking number ITS#8222. One of our support engineers will look at your report in due course. Note that this may take some time because our support engineers are volunte= ers. They only work on OpenLDAP when they have spare time. If you need to provide additional information in regards to your issue repo= rt, you may do so by replying to this message. Note that any mail sent to = openldap-its(a)openldap.org with (ITS#8222) in the subject will automatically= be attached to the issue report. mailto:openldap-its@openldap.org?subject=3D(ITS#8222) You may follow the progress of this report by loading the following URL in = a web browser: http://www.OpenLDAP.org/its/index.cgi?findid=3D8222 Please remember to retain your issue tracking number (ITS#8222) on any furt= her messages you send to us regarding this report. If you don't then you'l= l just waste our time and yours because we won't be able to properly track = the report. Please note that the Issue Tracking System is not intended to be used to se= ek help in the proper use of OpenLDAP Software. Such requests will be closed. OpenLDAP Software is user supported. http://www.OpenLDAP.org/support/ -------------- Copyright 1998-2007 The OpenLDAP Foundation, All Rights Reserved.

1 0

Re: Re: (ITS#8223) Replication error when slave is down and insert on master are made
by quanah＠zimbra.com 27 Aug '15

27 Aug '15

--On Thursday, August 27, 2015 11:56 AM +0200 Frank Offermanns <Frank.Offermanns(a)caseris.de> wrote: > Hello, > > I have additional information. The problem is the entryCSN. > User 1, which got replicated has entryCSN: > 20150827083333.751721Z#000000#001#000000 > user 2, which got NOT replicated has entryCSN: > 20150827083333.119585Z#000000#001#000000, which seems to be in the past > user 3, replicated, 20150827083334.847604Z#000000#001#000000 > user 4, not replicated 20150827083334.228113Z#000000#001#000000, which > seems to be in the past > user 5, replicated 20150827083335.588191Z#000000#001#000000 > user 6, replicated 20150827083335.946835Z#000000#001#000000 > user 7 not replicated 20150827083335.282686Z#000000#001#000000, which > seems to be in the past > > So in previous posts I got the answer, that microseconds exact time > synchronization is only important with master master and concurrent > writes. But now it also seems to be important with delta-synrepl. > > So I suppose this ITS can be closed, since it is not a bug, but worked as > designed? > So how to get a good enough time synchronization with windows (since the > normal time synchronization of windows seems to be not enough)? > Of course, if you close the ITS I can ask this question again in the > technical list. Ah, yeah, all replication (delta-sync or regular syncrepl) absolutely requires tight time sync. --Quanah -- Quanah Gibson-Mount Platform Architect Zimbra, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

(ITS#8227) syncprov should use more threads
by hyc＠openldap.org 27 Aug '15

27 Aug '15

Full_Name: Howard Chu Version: 2.4 OS: URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (70.87.222.79) Submitted by: hyc Currently a single background thread and output queue is used by syncprov to send persist messages to all consumers. A single slow consumer can block messages from being sent to other consumers. syncprov should be enhanced to use a separate queue and thread per consumer.

1 0

(ITS#8226) long searches are bad for back-mdb
by hyc＠openldap.org 27 Aug '15

27 Aug '15

Full_Name: Howard Chu Version: 2.4 OS: URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (70.87.222.79) Submitted by: hyc Long-lived read transactions prevent old pages from being reused in LMDB. We've seen a situation triggered by the syncprov overlay that aggravates this problem - a consumer connects and performs a syncrepl refresh. Its sync cookie is too old for the syncprov sessionlog to be used, so syncprov must generate a presentlist of all of the entryUUIDs in the database. This is a very long search (in this case, around 1.5 million objects) and writes are continuing while this happens. Also, there are multiple consumers repeating this at staggered times (due to the situation described in ITS#8225 they all tend to reconnect at closely situated times). Because of these long read transactions the incoming writes are forced to use new database pages, and so the DB file size grows drastically during these refreshes. This specific case for syncprov can be mitigated by using a much larger syncprov sessionlog (and in the future, by enhancing syncprov to use accesslog), but there are many other instances when clients may legitimately issue searches that span the entire database. The fix from ITS#7904 should be extended to release and reacquire the read transaction every N entries to make large searches friendlier to write traffic. The default value of N probably should not be zero, but could be a value (e.g. 1000, with a default sizelimit of 500) that won't affect the majority of normal search requests.

1 0

Re: (ITS#8225) writetimeout should ignore consumers
by hyc＠symas.com 27 Aug '15

27 Aug '15

hyc(a)openldap.org wrote: > The idletimeout already ignores consumer connections; writetimeout should as > well. Sites can use keepalive to handle truly hung consumers, as opposed to just > overloaded consumers. It's not clear that simply "ignoring" is the correct approach - the fact that the writetimeout triggers means slapd was unable to write any more on the socket in question. As such, the provider would need to stop attempting to send any more syncrepl messages to the consumer until the connection freed up. This would just mean more data would queue up in syncprov's output queues while waiting for the socket buffers to drain. If it takes a very long time for the consumers to finally catch up (looks like on the order of hours here) then slapd's memory consumption would increase quite a lot for that time. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#8225) writetimeout should ignore consumers
by hyc＠openldap.org 27 Aug '15

27 Aug '15

Full_Name: Howard Chu Version: 2.4 OS: URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (70.87.222.79) Submitted by: hyc We found a case where a site was using regular syncrepl and replicating changes to very large groups (hundreds of thousands of members, tens of megabytes total entry size) and the socket send queues on the provider were getting filled because the consumers weren't reading or processing fast enough. At some point the writetimeout on the provider kicked in and these syncrepl connections got closed, forcing the consumers to reconnect and start the whole process all over again. This sequence of events basically meant the consumers never made any progress, because the refresh on next connect attempt had to send such a large volume of data all over again. syslogs showed consumers having their connection closed multiple times within short (5 minute) spans of time. The idletimeout already ignores consumer connections; writetimeout should as well. Sites can use keepalive to handle truly hung consumers, as opposed to just overloaded consumers.

1 0

Re: Re: (ITS#8223) Replication error when slave is down and insert on master are made
by Frank.Offermanns＠caseris.de 27 Aug '15

27 Aug '15

Dies ist eine mehrteilige Nachricht im MIME-Format. --=_alternative 003127EEC1257EAE_= Content-Type: text/plain; charset="US-ASCII" Hello, I have additional information. The problem is the entryCSN. User 1, which got replicated has entryCSN: 20150827083333.751721Z#000000#001#000000 user 2, which got NOT replicated has entryCSN: 20150827083333.119585Z#000000#001#000000, which seems to be in the past user 3, replicated, 20150827083334.847604Z#000000#001#000000 user 4, not replicated 20150827083334.228113Z#000000#001#000000, which seems to be in the past user 5, replicated 20150827083335.588191Z#000000#001#000000 user 6, replicated 20150827083335.946835Z#000000#001#000000 user 7 not replicated 20150827083335.282686Z#000000#001#000000, which seems to be in the past So in previous posts I got the answer, that microseconds exact time synchronization is only important with master master and concurrent writes. But now it also seems to be important with delta-synrepl. So I suppose this ITS can be closed, since it is not a bug, but worked as designed? So how to get a good enough time synchronization with windows (since the normal time synchronization of windows seems to be not enough)? Of course, if you close the ITS I can ask this question again in the technical list. Regards, Frank --=_alternative 003127EEC1257EAE_= Content-Type: text/html; charset="US-ASCII" Hello, I have additional information. The problem is the entryCSN. User 1, which got replicated has entryCSN: 20150827083333.751721Z#000000#001#000000 user 2, which got NOT replicated has entryCSN: 20150827083333.119585Z#000000#001#000000, which seems to be in the past user 3, replicated, 20150827083334.847604Z#000000#001#000000 user 4, not replicated 20150827083334.228113Z#000000#001#000000, which seems to be in the past user 5, replicated 20150827083335.588191Z#000000#001#000000 user 6, replicated 20150827083335.946835Z#000000#001#000000 user 7 not replicated 20150827083335.282686Z#000000#001#000000, which seems to be in the past So in previous posts I got the answer, that microseconds exact time synchronization is only important with master master and concurrent writes. But now it also seems to be important with delta-synrepl. So I suppose this ITS can be closed, since it is not a bug, but worked as designed? So how to get a good enough time synchronization with windows (since the normal time synchronization of windows seems to be not enough)? Of course, if you close the ITS I can ask this question again in the technical list. Regards, Frank --=_alternative 003127EEC1257EAE_=--

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs August 2015