https://bugs.openldap.org/show_bug.cgi?id=8958
--- Comment #27 from Ondřej Kuzník <ondra(a)mistotebe.net> ---
On Tue, Aug 03, 2021 at 12:42:01PM +0000, openldap-its(a)openldap.org wrote:
> I don't think we should be changing anything else about how tpool
> handles pauses. We should just be fixing this specific case of the
> indexer being a slow task, by implementing checkpointing into the
> indexer. I.e., when it detects a pause request it should save its
> current progress and pause itself. If it gets resumed it can pick up
> where it left off, or if a config change affects it it can abort or
> or start over. A checkpointing mechanism is needed anyway, for the
> case of a (clean) shutdown while the indexer is running.
I'll put a suggestion here that we discussed previously: to support this
checkpointing for pauses/shutdowns, the indexer could be writing to a
"scratch" database (whatever that means for each backend) along with
resume data and move them into place when finished. You mentioned that
for liblmdb, this would need support for a database to be renamed.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=8958
--- Comment #26 from Howard Chu <hyc(a)openldap.org> ---
(In reply to Howard Chu from comment #25)
> (In reply to Hallvard Furuseth from comment #17)
>
> > Duuh, right. I got stuck looking for what's special about the
> > indexing task and couldn't find it:-( I need to make it special.
> >
> > So, let tasks declare their expected speed until finish or
> > between pausechecks. At FAST=1 (default) and SLOW=0.
> > A pause only stops tasks with speed < ltp_pause.
> > In thread_pool_pause(), replace the WANT_PAUSE stage with
> >
> > while (++ltp_pause <= max speed) {
> > wait until no more tasks with speed < ltp_pause;
> > }
> >
> > Then fast tasks should breeze past slow ones when preparing
> > to pause. Until all threads have slow tasks, anyway.
>
> I don't understand how this solves anything. If a slow indexing
> task is currently running, and a fast config mod comes in, it's
> still the case that the config change could pull the DB out from
> under the indexer task. So there's nothing safe about letting the
> fast task progress while the slow task is still running.
I don't think we should be changing anything else about how tpool
handles pauses. We should just be fixing this specific case of the
indexer being a slow task, by implementing checkpointing into the
indexer. I.e., when it detects a pause request it should save its
current progress and pause itself. If it gets resumed it can pick up
where it left off, or if a config change affects it it can abort or
or start over. A checkpointing mechanism is needed anyway, for the
case of a (clean) shutdown while the indexer is running.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=8958
--- Comment #25 from Howard Chu <hyc(a)openldap.org> ---
(In reply to Hallvard Furuseth from comment #17)
> Duuh, right. I got stuck looking for what's special about the
> indexing task and couldn't find it:-( I need to make it special.
>
> So, let tasks declare their expected speed until finish or
> between pausechecks. At FAST=1 (default) and SLOW=0.
> A pause only stops tasks with speed < ltp_pause.
> In thread_pool_pause(), replace the WANT_PAUSE stage with
>
> while (++ltp_pause <= max speed) {
> wait until no more tasks with speed < ltp_pause;
> }
>
> Then fast tasks should breeze past slow ones when preparing
> to pause. Until all threads have slow tasks, anyway.
I don't understand how this solves anything. If a slow indexing
task is currently running, and a fast config mod comes in, it's
still the case that the config change could pull the DB out from
under the indexer task. So there's nothing safe about letting the
fast task progress while the slow task is still running.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=8958
--- Comment #24 from Howard Chu <hyc(a)openldap.org> ---
(In reply to Quanah Gibson-Mount from comment #23)
> Created attachment 799 [details]
> proposed fix
In patch #5
+ ldap_pvt_thread_pool_setspeed( &connection_pool, ctx, 0 );
Shouldn't the minimum speed be 1, not 0?
Since you have
+enum { NOT_PAUSED = 0, WANT_PAUSE = LDAP_PVT_THREAD_POOL_SPEED_MAX+1, PAUSED
};
A speed of 0 would mean no pause at all, wouldn't it?
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=8649
Howard Chu <hyc(a)openldap.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Resolution|--- |TEST
Status|UNCONFIRMED |RESOLVED
--- Comment #4 from Howard Chu <hyc(a)openldap.org> ---
fixed in master
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=9618
Issue ID: 9618
Summary: Sync'ing is not happening in MDB_WRITEMAP mode on
Windows with mdb.master3
Product: LMDB
Version: 0.9.29
Hardware: All
OS: Windows
Status: UNCONFIRMED
Severity: normal
Priority: ---
Component: liblmdb
Assignee: bugs(a)openldap.org
Reporter: kriszyp(a)gmail.com
Target Milestone: ---
Created attachment 831
--> https://bugs.openldap.org/attachment.cgi?id=831&action=edit
Revert back to using standard FlushViewOfFile/FlushFileBuffers to sync data
with WRITEMAP mode on Windows
With ITS#9017, Windows sync'ing is performed by using the write-through (and
overlapped) writes, as it performs much better than
FlushViewOfFile/FlushFileBuffers. However, this functionality has regressed on
mdb.master3 because performing the writes is dependent on tracking and writing
the list of dirty pages, which is not tracked in WRITEMAP mode. This means that
on mdb.master3, with MDB_WRITEMAP, sync'ing is not really happening on Windows.
I think it is probably preferable to not try to maintain an exception for dirty
page tracking in WRITEMAP mode just for Windows. The attached patch reverts
back to using the standard map/file sync procedure
(FlushViewOfFile/FlushFileBuffers) to sync data when in WRITEMAP on Windows.
This still maintains the (better performing) write-through based sync technique
for standard, non-WRITEMAP mode on Windows.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=9603
Issue ID: 9603
Summary: tcp wrappers no longer works with IPv6
Product: OpenLDAP
Version: 2.5.5
Hardware: All
OS: All
Status: UNCONFIRMED
Severity: normal
Priority: ---
Component: slapd
Assignee: bugs(a)openldap.org
Reporter: toby(a)inf.ed.ac.uk
Target Milestone: ---
Hi,
OpenLDAP 2.5.5
Testing on Scientific Linux 7.9
It looks like slapd no longer passes an IPv6 address correctly to tcp wrappers.
Steps to Reproduce: connect to 2.5.5 slapd over IPv6 with tcp-wrappers enabled.
Actual Results: access is always denied
Expected Results: access permitted for IP addresses in /etc/hosts.allow
Additional information:
The problem seems to be that an IPv6 address is passed to hosts_ctl (~line 2324
in servers/slapd/daemon.c) as "[ipv6-address]" (with enclosing '[' and ']') -
rather than "ipv6-address", e.g. the logged output is:
warning: can't resolve hostname ([2001:630:3c1:212:a2d3:c1ff:fe25:eec3]): Name
or service not known
fd=19 DENIED from unknown ([2001:630:3c1:212:a2d3:c1ff:fe25:eec3]
... when compared to a 2.4.59 slapd:
fd=19 DENIED from unknown (2001:630:3c1:212:a2d3:c1ff:fe25:eec3)
IPv4 connections work OK.
Thanks
Toby Blake
School of Informatics
University of Edinburgh
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=9602
Issue ID: 9602
Summary: Remove the build warnings in tls_o.c
Product: OpenLDAP
Version: 2.5.5
Hardware: All
OS: All
Status: UNCONFIRMED
Severity: normal
Priority: ---
Component: libraries
Assignee: bugs(a)openldap.org
Reporter: Bin.Lan(a)windriver.com
Target Milestone: ---
Created attachment 829
--> https://bugs.openldap.org/attachment.cgi?id=829&action=edit
the issue fix patch
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=9601
Issue ID: 9601
Summary: Remove the build warnings in tpool.c
Product: OpenLDAP
Version: 2.5.5
Hardware: All
OS: All
Status: UNCONFIRMED
Severity: normal
Priority: ---
Component: libraries
Assignee: bugs(a)openldap.org
Reporter: Bin.Lan(a)windriver.com
Target Milestone: ---
Created attachment 828
--> https://bugs.openldap.org/attachment.cgi?id=828&action=edit
the issue patch
tpool.c:721:12: warning: 8 enumeration values not handled in switch:
'LDAP_PVT_THREAD_POOL_PARAM_UNKNOWN', 'LDAP_PVT_THREAD_POOL_PARAM_MAX',
'LDAP_PVT_THREAD_POOL_PARAM_MAX_PENDING'... [-Wswitch]
switch(param) {
^
1 warning generated.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=9578
Issue ID: 9578
Summary: Buffer overflow at libraries/libldap/ldif.c:907
(ldif_read_record)
Product: OpenLDAP
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Severity: normal
Priority: ---
Component: libraries
Assignee: bugs(a)openldap.org
Reporter: grapvar(a)gmail.com
Target Milestone: ---
Created attachment 827
--> https://bugs.openldap.org/attachment.cgi?id=827&action=edit
fix
libraries/libldap/ldif.c:829
> /* Squash \r\n to \n */
> if ( len > 1 && line[len-2] == '\r' ) {
> len--;
> line[len-1] = '\n';
> }
may cause buffer overflow at
libraries/libldap/ldif.c:907
> strcpy( *bufp + lcur, line );
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=9614
Issue ID: 9614
Summary: Admin guide does not document olcMultiVal
Product: OpenLDAP
Version: 2.5.5
Hardware: All
OS: All
Status: UNCONFIRMED
Severity: normal
Priority: ---
Component: documentation
Assignee: bugs(a)openldap.org
Reporter: quanah(a)openldap.org
Target Milestone: ---
The admin guide is missing documentation on the new olcMultiVal attribute
despite it being noted in the admin guide 2.5 upgrade section. Need to add a
detailed section on it.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=9613
Issue ID: 9613
Summary: olcIdlExp is not documented in the admin guide
Product: OpenLDAP
Version: 2.5.5
Hardware: All
OS: All
Status: UNCONFIRMED
Severity: normal
Priority: ---
Component: documentation
Assignee: bugs(a)openldap.org
Reporter: quanah(a)openldap.org
Target Milestone: ---
The admin guide does not document idlexp but does recommend configuring it
before upgrading to 2.5. We need to add this to the admin guide with a
detailed explanation of what it does.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=9591
Issue ID: 9591
Summary: Solaris builds broken due to map file
Product: OpenLDAP
Version: 2.5.5
Hardware: All
OS: All
Status: UNCONFIRMED
Severity: normal
Priority: ---
Component: build
Assignee: bugs(a)openldap.org
Reporter: quanah(a)openldap.org
Target Milestone: ---
Solaris 11.4 fails to build because it uses a different option for the library
symbol versioning map file. We need to detect this and provide the correct
option (more at
https://blogs.oracle.com/solaris/regex_and_glob_for_mapfiles-v2)
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=6138
Quanah Gibson-Mount <quanah(a)openldap.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
See Also| |https://bugs.openldap.org/s
| |how_bug.cgi?id=9220
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=6138
--- Comment #9 from Howard Chu <hyc(a)openldap.org> ---
(In reply to Hallvard Furuseth from comment #2)
> back-ldap:extended.c also does "suppress response, it has been sent",
> but does it by returning and setting rs->sr_err = SLAPD_ABANDON. Might
> break assumptions somewhere that SLAPD_ABANDON implies o_abandon was
> set. And I guess the hack fails if the operation gets cancelled.
Now fixed to always let the frontend handle responses.
But this is still awkward, and will need to be revisited again for ITS#9220.
> ========================================================================
>
> I think these are the Operation states related to Cancel and Abandon:
>
> op->o_abandon is set for these - could extend to multiple values:
> A) Operation Abandoned/Cancelled by client.
> B) Operation implicitly abandoned by client. (Bind or lost connection)
> C) Operation abandoned by server. (It wants to close the connection)
> D) Suppress response - a duplicate of the operation will proceed. (syncprov)
> E) Suppress response - final send_ldap_response() was done. (retcode overlay)
>
> rs->sr_err == SLAPD_ABANDON if:
> F) The backend obeyed o_abandon. (Cancel op, if any, will succeed)
> G=E) Suppress response - final send_ldap_response() was done. (back-ldap)
D, E, and G will no longer occur.
> op->o_cancel packs these states/values:
> H) The o_abandon is due to a Cancel.
> I) Cancel operation wants a result, cancelled op must set it and wait.
> J) Result is available to the Cancel operation.
> K) Result. (LDAP result code, or SLAP_CANCEL_ACK for success)
> L) Cancel operation has fetched result, cancelled operation can proceed.
>
> States that fit in none of the above, or poorly so:
>
> M) Operation must not be waited for, e.g. by Cancel.
> Operation is itself waiting for others, e.g. cn=config update.
>
> N) Operation invisible to Abandon/Cancel/internal abandon.
> msgID reusable due to result sent to client. Also case D (syncprov)?
>
> Fix by removing the op from op->o_conn->c_ops? Or does that just
> move the problem around? Would need to do something to o_conn to
> prevent connection_close() from doing connection_destroy().
>
> O) Operation result has been committed, do not abandon. ITS#6059.
>
> But o_abandon can be set while trying to commit, unless this flag is
> set before trying - in which case we can't abandon an operation which
> is failing to commit, which may be when it's most relevant.
>
> Could reset o_abandon, if anyone can keep straight the consequences.
> Or replace the 'if ( op->o_abandon )' tests with some macro call.
> Still, interactions with other states could be a problem.
>
> About the o_abandon values above:
>
> B can be treated like A, I think.
> C differs in that Cancel/Abandon(operation) should not say "already
> abandoned" since the client doen't know about the abandon.
> Could be solved with a vague error message.
>
> D-E differ in that o_abandon gets set even though the backends'
> cancel/abandon handlers were not called. Unsure of the effects of that.
>
> D Syncprov duplicating a Persistent Search operation.
> Handled similar to a server-initiated abandon? Except if the
> operation cannot be "invisible to Abandon/Cancel" above it must remain
> possible to Abandon/Cancel it.
>
> E Suppress response - response has been sent:
> Set when exiting slap_send_ldap_result() & co?
>
> Handled similar to a server-initiated abandon?
> At the time slap_send_ldap_result() is called again, the operation
> may have set up things which need to be cleaned up in the normal way.
> Yet it has already gone through that function once, doing callbacks
> etc. Must "final response" code be prepared to be called twice?
>
> Beyond that, the main problem would be code which transitions to one
> state to another, it needs to handle the other cases.
>
> --
> Hallvard
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=8890
Quanah Gibson-Mount <quanah(a)openldap.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Summary|Adapt timet to 64-bit long |adapt to 64-bit time_t on
| |systems with 32-bit long
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=6138
Howard Chu <hyc(a)openldap.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|UNCONFIRMED |IN_PROGRESS
Ever confirmed|0 |1
--- Comment #7 from Howard Chu <hyc(a)openldap.org> ---
(In reply to Hallvard Furuseth from comment #0)
> Full_Name: Hallvard B Furuseth
> Version: HEAD
> OS:
> URL:
> Submission from: (NULL) (129.240.6.233)
> Submitted by: hallvard
>
>
> Might make this a tracking issue for Cancel/Abandon problems.
> And/or copy some to separate ITSes, but they all seem interconnected:
>
>
> Cancel(abandoned operation)-requests are not rejected. Thus slapd
> sets o_cancel and turns an active Abandon into a Cancel. Presumably
> that can confuse Cancel/Abandon handlers, like that in Syncprov.
>
> Similarly, Abandon(abandoned/cancelled operation) is not ignored.
> connection_abandon() re-abandons abandoned/cancelled operations too.
Both of the above checks are now uncommented / enabled.
> However Syncprov:RefreshAndPersist abuses op->o_abandon: It sets it to
> mean "Suppress the response. A copy of this operation will send it."
> So if Cancel(op with o_abandon!=0) is fixed to respond protocolError
> "already abandoned", presumably that breaks Cancel(RefreshAndPersist).
>
> I'm not touching it with a flagpole - I don't know syncprov. Help?
syncprov has been changed to use the new SLAPD_ASYNCOP returncode so
it no longer needs to muck with this.
> Overlay retcode does something similar - sends a response and then sets
> o_abandon.
I can't see any reason why this is needed. It's been there since the initial
commit but I'm removing it. Any return code != SLAP_CB_CONTINUE already
prevents the frontend from sending its own response so there's no reason
to set this here.
> Cancel/Abandon can in any case fail by targeting the wrong operation
> though: A connection can have multiple messages with the same IDs when
> the response is sent and the client reuses the message ID, before the
> old operation in slapd can clean up and finish.
>
> syncprov_op_abandon() identifies messages by (connid, msgid). Can
> multiple messages with the same ID break that, or more importantly,
> break what gets sent/written with syncrepl?
Sounds like bad practice in clients, but should still be safe. The messages are
queued in order, so only the first matching message will be abandoned.
Presumably the first incoming abandon request can only refer to the first
queued operation of that ID.
> Anyway, the current code needs an o_abandon value which means "suppress
> response". Or maybe "abandoned, except as far as future Abandon and
> Cancel operations are concerned". Syncprov and Retcode need to handle
> the various possible orderings of Cancel/Abandon vs Suppress, including
> when they forward o_abandon from one operation to another. I haven't
> looked too closely at that code either.
Mooted by use of SLAPD_ASYNCOP returncode.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=5555
Quanah Gibson-Mount <quanah(a)openldap.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Resolution|FIXED |TEST
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=5555
Quanah Gibson-Mount <quanah(a)openldap.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Resolution|--- |FIXED
Status|IN_PROGRESS |RESOLVED
--- Comment #18 from Quanah Gibson-Mount <quanah(a)openldap.org> ---
Commits:
• 73e7034d
by andrew.findlay(a)skills-1st.co.uk at 2021-07-26T22:02:55+00:00
ITS#5555 - Correct documentation for SASL AuthzTo Check
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=8890
--- Comment #14 from tg(a)debian.org <tg(a)debian.org> ---
The bug is mistitled.
It should be “adapt to 64-bit time_t on systems with 32-bit long”, which has
become a fairly standard setup in the last 15 years and will become even more
so in the next… less than 17 years, I suppose.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=8742
--- Comment #3 from Quanah Gibson-Mount <quanah(a)openldap.org> ---
Commits:
• cb5ba678
by Quanah Gibson-Mount at 2021-07-26T16:36:26+00:00
ITS#8742 - More for bringing slapd.conf.5 and slapd-config.5 in sync
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=9593
Issue ID: 9593
Summary: excessive null check in set_chase()
Product: OpenLDAP
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Severity: normal
Priority: ---
Component: slapd
Assignee: bugs(a)openldap.org
Reporter: grapvar(a)gmail.com
Target Milestone: ---
Looks like slap_set_join( cp, nset, '|', vals ) can't return null, so return
value at line 411 should be treated like one at line 402:
servers/slapd/sets.c:399, set_chase():
for ( i = 0; !BER_BVISNULL( &set[ i ] ); i++ ) {
vals = gatherer( cp, &set[ i ], desc );
if ( vals != NULL ) {
/*402: */ nset = slap_set_join( cp, nset, '|', vals );
}
}
ber_bvarray_free_x( set, cp->set_op->o_tmpmemctx );
if ( closure ) {
for ( i = 0; !BER_BVISNULL( &nset[ i ] ); i++ ) {
vals = gatherer( cp, &nset[ i ], desc );
if ( vals != NULL ) {
/*411: */ nset = slap_set_join( cp, nset, '|', vals );
if ( nset == NULL ) {
break;
}
}
}
}
diff --git a/servers/slapd/sets.c b/servers/slapd/sets.c
index fc7b72c8b..17a6ec2c1 100644
--- a/servers/slapd/sets.c
+++ b/servers/slapd/sets.c
@@ -409,9 +409,6 @@ set_chase( SLAP_SET_GATHER gatherer,
vals = gatherer( cp, &nset[ i ], desc );
if ( vals != NULL ) {
nset = slap_set_join( cp, nset, '|', vals );
- if ( nset == NULL ) {
- break;
- }
If I'm wrong, return value at line 402 must be checked for null.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=8491
Quanah Gibson-Mount <quanah(a)openldap.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Target Milestone|2.6.0 |2.7.0
--
You are receiving this mail because:
You are on the CC list for the issue.