https://bugs.openldap.org/show_bug.cgi?id=9397
Issue ID: 9397
Summary: LMDB: A second process opening a file with
MDB_WRITEMAP can cause the first to SIGBUS
Product: LMDB
Version: 0.9.26
Hardware: All
OS: All
Status: UNCONFIRMED
Severity: normal
Priority: ---
Component: liblmdb
Assignee: bugs(a)openldap.org
Reporter: github(a)nicwatson.org
Target Milestone: ---
Created attachment 780
--> https://bugs.openldap.org/attachment.cgi?id=780&action=edit
Full reproduction of SIGBUS MDB_WRITEMAP issue (works on Linux only)
The fundamental problem is that a ftruncate() on Linux that makes a file
smaller will cause accesses past the new end of the file to SIGBUS (see the
mmap man page).
The sequence that causes a SIGBUS involves two processes.
1. The first process opens a new LMDB file with MDB_WRITEMAP.
2. The second process opens the same LMDB file with MDB_WRITEMAP and with an
explicit map_size smaller than the first process's map size.
* This causes an ftruncate that makes the underlying file *smaller*.
3. (Optional) The second process closes the environment and exits.
4. The first process opens a write transaction and writes a bunch of data.
5. The first process commits the transaction. This causes a memory read from
the mapped memory that's now past the end of the file. On Linux, this triggers
a SIGBUS.
Attached is code that fully reproduces the problem on Linux.
The most straightforward solution is to allow ftruncate to *reduce* the file
size if it is the only reader. Another possibility is check the file size and
ftruncate if necessary every time a write transaction is opened. A third
possibility is to catch the SIGBUS signal.
Repro note: I used clone() to create the subprocess to most straightforwardly
demonstrate that the problem is not due to inherited file descriptors. The
problem still manifests when the processes are completely independent.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=9207
Bug ID: 9207
Summary: Remove Moznss compatibility layer
Product: OpenLDAP
Version: 2.5
Hardware: All
OS: All
Status: UNCONFIRMED
Severity: normal
Priority: ---
Component: libraries
Assignee: bugs(a)openldap.org
Reporter: quanah(a)openldap.org
Target Milestone: ---
For the 2.5 release, remove the MozNSS compatibility layer.
--
You are receiving this mail because:
You are on the CC list for the bug.
https://bugs.openldap.org/show_bug.cgi?id=10396
Issue ID: 10396
Summary: LMDB: another issue with sorted duplicate DBs and
cursor delete
Product: LMDB
Version: 0.9.19
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: liblmdb
Assignee: bugs(a)openldap.org
Reporter: hyc(a)openldap.org
Target Milestone: ---
In mdb_cursor_del0, in the second "Adjust other cursors" section (after
mdb_rebalance was called), in these lines
if (m3->mc_xcursor && !(m3->mc_flags & C_EOF)) {
MDB_node *node = NODEPTR(m3->mc_pg[m3->mc_top],
m3->mc_ki[m3->mc_top]);
The node being referenced should be using mc->mc_top, not m3->mc_top. Most of
the time these will be identical, but sometimes due to rebalancing, the mc
cursor may be popped below its stack top when calling here.
This bug was introduced for ITS#8406 in commit
37081325f7356587c5e6ce4c1f36c3b303fa718c on 2016-04-18.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10406
Issue ID: 10406
Summary: Write regression since 0.9.19
Product: LMDB
Version: 0.9.33
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: liblmdb
Assignee: bugs(a)openldap.org
Reporter: ben.alex(a)acegi.com.au
Target Milestone: ---
I maintain LmdbJava, a Java wrapper for LMDB. We have an extensive benchmark
suite covering read, cursor, and write pathways, with isolated testing of LMDB
versions from 0.9.17 through 0.9.33.
Results: https://lmdb-benchmark.lmdbjava.org/
==============================================
SUMMARY
==============================================
Read performance: 0.9.33 consistently outperforms 0.9.17 across all read
benchmarks.
Write performance: Consistent regression since 0.9.19:
----------------------------------
Tag ms/op vs 0.9.17
----------------------------------
LMDB_0.9.17 75.790 baseline
LMDB_0.9.18 74.909 -1.2%
LMDB_0.9.19 81.404 +7.4%
LMDB_0.9.20 83.459 +10.1%
LMDB_0.9.21 83.237 +9.8%
LMDB_0.9.22 83.027 +9.5%
LMDB_0.9.23 82.870 +9.3%
LMDB_0.9.24 82.952 +9.4%
LMDB_0.9.27 82.989 +9.5%
LMDB_0.9.28 83.025 +9.5%
LMDB_0.9.29 83.063 +9.6%
LMDB_0.9.30 83.237 +9.8%
LMDB_0.9.31 88.419 +16.7%
LMDB_0.9.33 83.304 +9.9%
Excluding 0.9.31 as an outlier, write performance degraded by approximately 9%
from 0.9.20 and has remained at that level through 0.9.33.
==============================================
BENCHMARK DETAILS
==============================================
The write benchmark sequentially inserts 1,000,000 entries (100-byte values,
4-byte integer keys) into a database opened with MDB_INTEGERKEY, using a single
cursor with MDB_APPEND flag. The benchmark includes 2 warmup iterations and 3
measurement iterations of 120 seconds each.
==============================================
ANALYSIS
==============================================
Reviewing the commit history, the regressions appear linked to correctness
fixes:
- 0.9.19: ITS#8406 (xcursor fixup after cursor_del)
- 0.9.20: ITS#8557 (cursor_last and C_EOF handling)
Both issues added cursor state management overhead in write-heavy code paths.
==============================================
REQUEST
==============================================
Could you please reproduce this regression using a native C benchmark to
confirm it's not JNI-specific? If confirmed, this may be a necessary
performance/correctness tradeoff given that most LMDB workloads are read-heavy.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10266
Issue ID: 10266
Summary: Adopt broader RFC4511 NoD interpretation on lloadd's
client side
Product: OpenLDAP
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: lloadd
Assignee: bugs(a)openldap.org
Reporter: ondra(a)mistotebe.net
Target Milestone: ---
Server side, lloadd has long implemented a broad interpretation of NoD
unsolicited response handling: when the message is issued, no new requests are
accepted on the session however the client and server are both free to keep the
session open if there are any operations that have not resolved yet. The server
is still expected to close the connection as soon as no operations are still
pending.
This seems to interoperate with known clients. Those that want to will close
the session immediately, unaware of this possibility, those that also want to
interpret RFC 4511 this way can choose to wait for existing operations to
resolve.
This ticket is to track the lloadd's implementation of the client side of this
- when receiving a NoD message, we don't close the connection
immediately+unconditionally either but are willing to wait.
Related functionality:
- if connection was a bind connection processing a multi-stage SASL bind, the
bind should fail if/when the client attempts to progress it
- clients assigned to this connection through coherence at least 'connection'
are also marked closing
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10229
Issue ID: 10229
Summary: ldap_result, when invoked with MSG_RECEIVED and a
timeout value set to 0 (polling), does not return all
available messages until it is called again
Product: OpenLDAP
Version: 2.6.8
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: libraries
Assignee: bugs(a)openldap.org
Reporter: nivanova(a)symas.com
Target Milestone: ---
The issue is noticeable when ldap_result is used by the proxy back-ends. It has
not affected back-meta behavior, because when a first call is unsuccessful, it
retries with a small timeout. back-asyncmeta will also usually call it twice on
the same connection from different threads, although this is not a desired
behavior.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10250
Issue ID: 10250
Summary: syncrepl_diff_entry assumes attributes come in the
same order
Product: OpenLDAP
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: slapd
Assignee: bugs(a)openldap.org
Reporter: ondra(a)mistotebe.net
Target Milestone: ---
When trying to diff an entry, syncrepl_diff_entry explicitly assumes attribute
come in the same order. That's not always the case and could cause it to report
a spurious rewrite of the attribute.
Normally this is ok, unless the rewrite itself (not) occurring has other
side-effects, when it could cause issues. (e.g. a DB with memberof
inconsistencies being mysteriously repaired in some scenarios, which is how it
was found).
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=9888
Issue ID: 9888
Summary: When using cn=config replication, schema updates can
corrupt the index database(s)
Product: OpenLDAP
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: slapd
Assignee: bugs(a)openldap.org
Reporter: quanah(a)openldap.org
Target Milestone: ---
Today I pushed a schema update out to the config node that holds schema that is
replicated to the providers and consumers. Post schema update, 2/11 servers
crashed in the mdb online indexing function. I fixed this by slapcat the db
and slapadd the db. This is important because it was later revealed that on
the 9/11 servers that did not crash or have their database reloaded, ldapsearch
would return the wrong attribute names for some attribute:value pairs in the
database, which caused mayhem in downstream systems and caused replication
issues between the nodes. The 2 nodes that were reloaded immediately after the
schema change had the only "good" copies of the database left.
To give an example, say an entry was something like:
dn: uid=joe,ou=people,dc=example,dc=com
uid: joe
sn: smith
cn: joe smith
givenName: joe
After the change, the broken servers could return something like:
dn: uid=joe,ou=people,dc=example,dc=com
uid: joe
posixGroup: smith
cn: joe smith
givenName joe
It's not clear how deeply this bug ran in the database. It for sure affected 2
attributes used by the person objectClass. Both of the "replacement"
attributes were not valid attributes for the person objectClasses in use.
Maybe related to the changes in ITS#9858?
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10308
Issue ID: 10308
Summary: Implement cn=monitor for back-asyncmeta
Product: OpenLDAP
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: backends
Assignee: bugs(a)openldap.org
Reporter: nivanova(a)symas.com
Target Milestone: ---
Currently back-asyncmeta has no cn=monitor capabilities. It will be useful to
implement some, specifically to monitor targets and target connection states.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10473
Issue ID: 10473
Summary: syncprov leaks response controls
Product: OpenLDAP
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: overlays
Assignee: bugs(a)openldap.org
Reporter: ondra(a)mistotebe.net
Target Milestone: ---
syncprov_state_ctrl and syncprov_done_ctrl add controls to the response, but
noone cares to free them afterwards assuming they are allocated from the slab.
If there are many messages sent in a single run, this takes up fresh memory
that's leaked eventually.
See its8800 regression test for an example where this happens.
--
You are receiving this mail because:
You are on the CC list for the issue.