(ITS#8229) ldapsearch: report failed search results to stderr even when ldif == 0
by ryan＠openldap.org
Full_Name: Ryan Tandy
Version: master
OS: Debian
URL:
Submission from: (NULL) (24.68.37.4)
Submitted by: ryan
Forwarded from a Debian bug report: <https://bugs.debian.org/474021>
ldapsearch's error reporting currently does this:
$ ldapsearch -x -b dc=nonexistent > /dev/null
$ ldapsearch -x -b dc=nonexistent -L > /dev/null
No such object (32)
$ ldapsearch -x -b dc=nonexistent -LL > /dev/null
No such object (32)
$ ldapsearch -x -b dc=nonexistent -LLL > /dev/null
No such object (32)
The submitter would like it to report the failure to stderr in the default case
as well. He suggested this change:
- if( !ldif ) {
- printf( "result: %d %s\n", err, ldap_err2string(err) );
- } else if ( err != LDAP_SUCCESS ) {
- fprintf( stderr, "%s (%d)\n", ldap_err2string(err), err );
+ if ( err != LDAP_SUCCESS %2%7{
+ fprintf( stderr, "Search failed: %s (%d)\n", ldap_err2string(err), err );
+ }
+
+ if ( !ldif ) {
+ printf( "result: %d %s\n", err, ldap_err2string(err) );
}
I would probably just flip the cases, and keep the "else" (i.e. not duplicate
the output):
- if( !ldif ) {
- printf( _("result: %d %s\n"), err, ldap_err2string(err) );
-
- } else if ( err != LDAP_SUCCESS ) {
+ if ( err != LDAP_SUCCESS ) {
fprintf( stderr, "%s (%d)\n", ldap_r2ststring(err), err );
+ } else if( !ldif ) {
+ printf( _("result: %d %s\n"), err, ldap_err2string(err) );
}
Does either of these changes sound appropriate?
The submitter also suggested sending the LDAP_SYNC Cancelled message to stderr:
if ( cancel_msgid != -1 &&
cancel_msgid == ldap_msgid( msg
) ) {
printf(_("Cancelled \n"));
printf(_("cancel_msgid = %d\n"),
cancel_msgid);
I don't know that area very well, but at a glance it seems like stdout is
probably more appropriate (so no change needed).
Thanks.
8 years, 1 month
Re: (ITS#8209) Broken MDB_CP_COMPACT threading
by h.b.furuseth＠usit.uio.no
More:
MDB_CP_COMPACT would drop a new, empty main DB.
(I.e. the main DB would lose its flags.)
Compacting can verify that the written metapage claims
enough DB pages. But not the opposite - it should not
fail if it claims too many pages since that would mean
we can't compact a DB which has suffered page leaks.
We could detect that and re-write the metas at the end,
if the user requests it and the file is seekable.
8 years, 1 month
(ITS#8228) avoid clobbering a valid pid file
by ryan＠openldap.org
Full_Name: Ryan Tandy
Version: master
OS: Debian
URL:
Submission from: (NULL) (24.68.37.4)
Submitted by: ryan
Forwarding a feature request from <https://bugs.debian.org/375067>:
It would be nice if slapd would notice an existing pid file pointing at a live
process, and refuse to overwrite it.
Starting slapd twice with the same pidfile is certainly user error, but it seems
like it should be preventable.
8 years, 1 month
RE: (ITS#8222) memory administration in ldap_parse_result()
by Lawrence.Newman＠alcatel-lucent.com
Providing an update: Not surprisingly, the actual memory leak was in our a=
pplication software (error message allocated by ldap_parse_result() not fre=
ed in unusual case). But our debugging/analysis tools continued to report =
memory allocated but not freed even after the problem was fixed, and I beli=
eve that was due to the issue I reported with how internal (library) memory=
linked to the LDAP structure is allocated/freed by ldap_parse_result() (al=
located while processing a response, but not freed until the next time the =
connection associated with that LDAP structure is used).
Larry Newman
IHN 9H-401, x90882
lawrence.newman(a)alcatel-lucent.com
-----Original Message-----
From: openldap-its(a)OpenLDAP.org [mailto:openldap-its@OpenLDAP.org]=20
Sent: Tuesday, August 25, 2015 10:15 AM
To: Newman, Lawrence H (Lawrence)
Subject: Re: (ITS#8222) memory administration in ldap_parse_result()
*** THIS IS AN AUTOMATICALLY GENERATED REPLY ***
Thanks for your report to the OpenLDAP Issue Tracking System. Your report =
has been assigned the tracking number ITS#8222.
One of our support engineers will look at your report in due course.
Note that this may take some time because our support engineers are volunte=
ers. They only work on OpenLDAP when they have spare time.
If you need to provide additional information in regards to your issue repo=
rt, you may do so by replying to this message. Note that any mail sent to =
openldap-its(a)openldap.org with (ITS#8222) in the subject will automatically=
be attached to the issue report.
mailto:openldap-its@openldap.org?subject=3D(ITS#8222)
You may follow the progress of this report by loading the following URL in =
a web browser:
http://www.OpenLDAP.org/its/index.cgi?findid=3D8222
Please remember to retain your issue tracking number (ITS#8222) on any furt=
her messages you send to us regarding this report. If you don't then you'l=
l just waste our time and yours because we won't be able to properly track =
the report.
Please note that the Issue Tracking System is not intended to be used to se=
ek help in the proper use of OpenLDAP Software.
Such requests will be closed.
OpenLDAP Software is user supported.
http://www.OpenLDAP.org/support/
--------------
Copyright 1998-2007 The OpenLDAP Foundation, All Rights Reserved.
8 years, 1 month
Re: Re: (ITS#8223) Replication error when slave is down and insert on master are made
by quanah＠zimbra.com
--On Thursday, August 27, 2015 11:56 AM +0200 Frank Offermanns
<Frank.Offermanns(a)caseris.de> wrote:
> Hello,
>
> I have additional information. The problem is the entryCSN.
> User 1, which got replicated has entryCSN:
> 20150827083333.751721Z#000000#001#000000
> user 2, which got NOT replicated has entryCSN:
> 20150827083333.119585Z#000000#001#000000, which seems to be in the past
> user 3, replicated, 20150827083334.847604Z#000000#001#000000
> user 4, not replicated 20150827083334.228113Z#000000#001#000000, which
> seems to be in the past
> user 5, replicated 20150827083335.588191Z#000000#001#000000
> user 6, replicated 20150827083335.946835Z#000000#001#000000
> user 7 not replicated 20150827083335.282686Z#000000#001#000000, which
> seems to be in the past
>
> So in previous posts I got the answer, that microseconds exact time
> synchronization is only important with master master and concurrent
> writes. But now it also seems to be important with delta-synrepl.
>
> So I suppose this ITS can be closed, since it is not a bug, but worked as
> designed?
> So how to get a good enough time synchronization with windows (since the
> normal time synchronization of windows seems to be not enough)?
> Of course, if you close the ITS I can ask this question again in the
> technical list.
Ah, yeah, all replication (delta-sync or regular syncrepl) absolutely
requires tight time sync.
--Quanah
--
Quanah Gibson-Mount
Platform Architect
Zimbra, Inc.
--------------------
Zimbra :: the leader in open source messaging and collaboration
8 years, 1 month
(ITS#8227) syncprov should use more threads
by hyc＠openldap.org
Full_Name: Howard Chu
Version: 2.4
OS:
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (70.87.222.79)
Submitted by: hyc
Currently a single background thread and output queue is used by syncprov to
send persist messages to all consumers. A single slow consumer can block
messages from being sent to other consumers. syncprov should be enhanced to use
a separate queue and thread per consumer.
8 years, 1 month
(ITS#8226) long searches are bad for back-mdb
by hyc＠openldap.org
Full_Name: Howard Chu
Version: 2.4
OS:
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (70.87.222.79)
Submitted by: hyc
Long-lived read transactions prevent old pages from being reused in LMDB. We've
seen a situation triggered by the syncprov overlay that aggravates this problem
- a consumer connects and performs a syncrepl refresh. Its sync cookie is too
old for the syncprov sessionlog to be used, so syncprov must generate a
presentlist of all of the entryUUIDs in the database. This is a very long search
(in this case, around 1.5 million objects) and writes are continuing while this
happens. Also, there are multiple consumers repeating this at staggered times
(due to the situation described in ITS#8225 they all tend to reconnect at
closely situated times).
Because of these long read transactions the incoming writes are forced to use
new database pages, and so the DB file size grows drastically during these
refreshes.
This specific case for syncprov can be mitigated by using a much larger syncprov
sessionlog (and in the future, by enhancing syncprov to use accesslog), but
there are many other instances when clients may legitimately issue searches that
span the entire database. The fix from ITS#7904 should be extended to release
and reacquire the read transaction every N entries to make large searches
friendlier to write traffic.
The default value of N probably should not be zero, but could be a value (e.g.
1000, with a default sizelimit of 500) that won't affect the majority of normal
search requests.
8 years, 1 month
Re: (ITS#8225) writetimeout should ignore consumers
by hyc＠symas.com
hyc(a)openldap.org wrote:
> The idletimeout already ignores consumer connections; writetimeout should as
> well. Sites can use keepalive to handle truly hung consumers, as opposed to just
> overloaded consumers.
It's not clear that simply "ignoring" is the correct approach - the fact that
the writetimeout triggers means slapd was unable to write any more on the
socket in question. As such, the provider would need to stop attempting to
send any more syncrepl messages to the consumer until the connection freed up.
This would just mean more data would queue up in syncprov's output queues
while waiting for the socket buffers to drain. If it takes a very long time
for the consumers to finally catch up (looks like on the order of hours here)
then slapd's memory consumption would increase quite a lot for that time.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
8 years, 1 month
(ITS#8225) writetimeout should ignore consumers
by hyc＠openldap.org
Full_Name: Howard Chu
Version: 2.4
OS:
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (70.87.222.79)
Submitted by: hyc
We found a case where a site was using regular syncrepl and replicating changes
to very large groups (hundreds of thousands of members, tens of megabytes total
entry size) and the socket send queues on the provider were getting filled
because the consumers weren't reading or processing fast enough. At some point
the writetimeout on the provider kicked in and these syncrepl connections got
closed, forcing the consumers to reconnect and start the whole process all over
again. This sequence of events basically meant the consumers never made any
progress, because the refresh on next connect attempt had to send such a large
volume of data all over again. syslogs showed consumers having their connection
closed multiple times within short (5 minute) spans of time.
The idletimeout already ignores consumer connections; writetimeout should as
well. Sites can use keepalive to handle truly hung consumers, as opposed to just
overloaded consumers.
8 years, 1 month
Re: Re: (ITS#8223) Replication error when slave is down and insert on master are made
by Frank.Offermanns＠caseris.de
Dies ist eine mehrteilige Nachricht im MIME-Format.
--=_alternative 003127EEC1257EAE_=
Content-Type: text/plain; charset="US-ASCII"
Hello,
I have additional information. The problem is the entryCSN.
User 1, which got replicated has entryCSN:
20150827083333.751721Z#000000#001#000000
user 2, which got NOT replicated has entryCSN:
20150827083333.119585Z#000000#001#000000, which seems to be in the past
user 3, replicated, 20150827083334.847604Z#000000#001#000000
user 4, not replicated 20150827083334.228113Z#000000#001#000000, which
seems to be in the past
user 5, replicated 20150827083335.588191Z#000000#001#000000
user 6, replicated 20150827083335.946835Z#000000#001#000000
user 7 not replicated 20150827083335.282686Z#000000#001#000000, which
seems to be in the past
So in previous posts I got the answer, that microseconds exact time
synchronization is only important with master master and concurrent
writes. But now it also seems to be important with delta-synrepl.
So I suppose this ITS can be closed, since it is not a bug, but worked as
designed?
So how to get a good enough time synchronization with windows (since the
normal time synchronization of windows seems to be not enough)?
Of course, if you close the ITS I can ask this question again in the
technical list.
Regards,
Frank
--=_alternative 003127EEC1257EAE_=
Content-Type: text/html; charset="US-ASCII"
<font size=3>Hello,<br>
<br>
I have additional information. The problem is the entryCSN.<br>
User 1, which got replicated has entryCSN: 20150827083333.751721Z#000000#001#000000<br>
user 2, which got NOT replicated has entryCSN: 20150827083333.119585Z#000000#001#000000,
which seems to be in the past<br>
user 3, replicated, 20150827083334.847604Z#000000#001#000000<br>
user 4, not replicated 20150827083334.228113Z#000000#001#000000, which
seems to be in the past<br>
user 5, replicated 20150827083335.588191Z#000000#001#000000<br>
user 6, replicated 20150827083335.946835Z#000000#001#000000<br>
user 7 not replicated 20150827083335.282686Z#000000#001#000000, which seems
to be in the past<br>
<br>
So in previous posts I got the answer, that microseconds exact time synchronization
is only important with master master and concurrent writes. But now it
also seems to be important with delta-synrepl.<br>
<br>
So I suppose this ITS can be closed, since it is not a bug, but worked
as designed?<br>
So how to get a good enough time synchronization with windows (since the
normal time synchronization of windows seems to be not enough)?<br>
Of course, if you close the ITS I can ask this question again in the technical
list. </font>
<br>
<br><font size=3>Regards,</font>
<br><font size=3>Frank</font>
--=_alternative 003127EEC1257EAE_=--
8 years, 1 month
