https://bugs.openldap.org/show_bug.cgi?id=10108
Issue ID: 10108
Summary: "mdb_dump -a" does not dump the main database
Product: LMDB
Version: 0.9.29
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: liblmdb
Assignee: bugs(a)openldap.org
Reporter: tuukka.pensala(a)gmail.com
Target Milestone: ---
In mdb_dump.c we have these instructions:
/* -a: dump main DB and all subDBs
* -s: dump only the named subDB
* -n: use NOSUBDIR flag on env_open
* -p: use printable characters
* -f: write to file instead of stdout
* -V: print version and exit
* (default) dump only the main DB
*/
However, contrary to the description, the option -a does not dump the main DB.
With argument -a "dumpit(..)" is called for the named databases, but not for
the unnamed one.
With the current behavior, if the data store contains subDBs and has user-added
data in the main DB, there seems to be no way to dump all of it at once using
mdb_dump.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=9360
Issue ID: 9360
Summary: MDB_BAD_TXN: Transaction must abort, has a child, or
is invalid
Product: LMDB
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Severity: normal
Priority: ---
Component: liblmdb
Assignee: bugs(a)openldap.org
Reporter: spam(a)markandruth.co.uk
Target Milestone: ---
I have 2 python scripts writing to a database (lmdb 0.9.26, py-lmdb 0.98) and
5-10 lua processes (with lightningmdb module which uses lmdb 0.9.22) which are
long-running serving queries from the database.
The database seems fine, not corrupted, and the python writes still working all
the time. But periodically (perhaps 10-20% of the time), in a way I am unable
to reliably reproduce, when the lua starts up every time a query is issued txn
dbi_open returns "MDB_BAD_TXN: Transaction must abort, has a child, or is
invalid". A direct restart of the processes does not fix this issue, however
stopping lua+python and then starting again after a 5-20s wait usually fixes
the issue. This has been reproduced over multiple servers but I'm at a loss as
to how to debug this any further?
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=9208
Bug ID: 9208
Summary: LMDB feature request: variant of mdb_env_copy{,fd2}
that takes transaction as parameter
Product: LMDB
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Severity: normal
Priority: ---
Component: liblmdb
Assignee: bugs(a)openldap.org
Reporter: github(a)nicwatson.org
Target Milestone: ---
The mdb_env_copy* functions create a read transaction themselves to run the
backup on. New variants of these functions (one for mdb_env_copy2 and one for
mdb_env_copyfd2) would have a transaction parameter. This transaction would be
used instead of creating a new transaction.
Application code could use these new functions to synchronize consistent live
backups across multiple LMDB instances (potentially across multiple hosts).
--
You are receiving this mail because:
You are on the CC list for the bug.
https://bugs.openldap.org/show_bug.cgi?id=10034
Issue ID: 10034
Summary: Assertion 'i < NUMKEYS(mp)' failed in
mdb_page_search_root()"
Product: LMDB
Version: 0.9.23
Hardware: Other
OS: Linux
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: liblmdb
Assignee: bugs(a)openldap.org
Reporter: 763280032(a)qq.com
Target Milestone: ---
We found that when lmdb is opened after OS startup and data is written to it,
lmdb will trigger abort probabilistically(Restart the OS 600 times will trigger
once);
We want to know what situation triggers this issue(Assertion 'i < NUMKEYS(mp)'
failed in mdb_page_search_root()); we want to know if there is a problem with
our usage;
Please Help Us
(gdb) x/8s 0x8baee988
0x8baee988: "8\373Բ\b\371Բmdb.c:5542: Assertion 'i < NUMKEYS(mp)' failed in
mdb_page_search_root()"
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=9475
Issue ID: 9475
Summary: Add support for MAP_POPULATE
Product: LMDB
Version: unspecified
Hardware: All
OS: Linux
Status: UNCONFIRMED
Severity: normal
Priority: ---
Component: liblmdb
Assignee: bugs(a)openldap.org
Reporter: aa531811820(a)gmail.com
Target Milestone: ---
In some case (such as cloud computing platforms), the reading speed of large
files is very fast while the small files is very slow, and we have enough
memory, so we hope to prefetch the entire LMDB file into the memory during MMAP
through the MAP_POPULATE flag . According to our test, this is faster than
using readahead flag. Here are some test data:
# mmap with no readahead
read one sample: 0.2s
total time: 4800s
# mmap with readahead
read one sample: 0.0001s~0.03s
total time: 95.86s
# mmap with MAP_POPULATE
db init: 20s
read one sample: 0.0001s
total time: 78s
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=9397
Issue ID: 9397
Summary: LMDB: A second process opening a file with
MDB_WRITEMAP can cause the first to SIGBUS
Product: LMDB
Version: 0.9.26
Hardware: All
OS: All
Status: UNCONFIRMED
Severity: normal
Priority: ---
Component: liblmdb
Assignee: bugs(a)openldap.org
Reporter: github(a)nicwatson.org
Target Milestone: ---
Created attachment 780
--> https://bugs.openldap.org/attachment.cgi?id=780&action=edit
Full reproduction of SIGBUS MDB_WRITEMAP issue (works on Linux only)
The fundamental problem is that a ftruncate() on Linux that makes a file
smaller will cause accesses past the new end of the file to SIGBUS (see the
mmap man page).
The sequence that causes a SIGBUS involves two processes.
1. The first process opens a new LMDB file with MDB_WRITEMAP.
2. The second process opens the same LMDB file with MDB_WRITEMAP and with an
explicit map_size smaller than the first process's map size.
* This causes an ftruncate that makes the underlying file *smaller*.
3. (Optional) The second process closes the environment and exits.
4. The first process opens a write transaction and writes a bunch of data.
5. The first process commits the transaction. This causes a memory read from
the mapped memory that's now past the end of the file. On Linux, this triggers
a SIGBUS.
Attached is code that fully reproduces the problem on Linux.
The most straightforward solution is to allow ftruncate to *reduce* the file
size if it is the only reader. Another possibility is check the file size and
ftruncate if necessary every time a write transaction is opened. A third
possibility is to catch the SIGBUS signal.
Repro note: I used clone() to create the subprocess to most straightforwardly
demonstrate that the problem is not due to inherited file descriptors. The
problem still manifests when the processes are completely independent.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=9207
Bug ID: 9207
Summary: Remove Moznss compatibility layer
Product: OpenLDAP
Version: 2.5
Hardware: All
OS: All
Status: UNCONFIRMED
Severity: normal
Priority: ---
Component: libraries
Assignee: bugs(a)openldap.org
Reporter: quanah(a)openldap.org
Target Milestone: ---
For the 2.5 release, remove the MozNSS compatibility layer.
--
You are receiving this mail because:
You are on the CC list for the bug.
https://bugs.openldap.org/show_bug.cgi?id=10396
Issue ID: 10396
Summary: LMDB: another issue with sorted duplicate DBs and
cursor delete
Product: LMDB
Version: 0.9.19
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: liblmdb
Assignee: bugs(a)openldap.org
Reporter: hyc(a)openldap.org
Target Milestone: ---
In mdb_cursor_del0, in the second "Adjust other cursors" section (after
mdb_rebalance was called), in these lines
if (m3->mc_xcursor && !(m3->mc_flags & C_EOF)) {
MDB_node *node = NODEPTR(m3->mc_pg[m3->mc_top],
m3->mc_ki[m3->mc_top]);
The node being referenced should be using mc->mc_top, not m3->mc_top. Most of
the time these will be identical, but sometimes due to rebalancing, the mc
cursor may be popped below its stack top when calling here.
This bug was introduced for ITS#8406 in commit
37081325f7356587c5e6ce4c1f36c3b303fa718c on 2016-04-18.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10406
Issue ID: 10406
Summary: Write regression since 0.9.19
Product: LMDB
Version: 0.9.33
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: liblmdb
Assignee: bugs(a)openldap.org
Reporter: ben.alex(a)acegi.com.au
Target Milestone: ---
I maintain LmdbJava, a Java wrapper for LMDB. We have an extensive benchmark
suite covering read, cursor, and write pathways, with isolated testing of LMDB
versions from 0.9.17 through 0.9.33.
Results: https://lmdb-benchmark.lmdbjava.org/
==============================================
SUMMARY
==============================================
Read performance: 0.9.33 consistently outperforms 0.9.17 across all read
benchmarks.
Write performance: Consistent regression since 0.9.19:
----------------------------------
Tag ms/op vs 0.9.17
----------------------------------
LMDB_0.9.17 75.790 baseline
LMDB_0.9.18 74.909 -1.2%
LMDB_0.9.19 81.404 +7.4%
LMDB_0.9.20 83.459 +10.1%
LMDB_0.9.21 83.237 +9.8%
LMDB_0.9.22 83.027 +9.5%
LMDB_0.9.23 82.870 +9.3%
LMDB_0.9.24 82.952 +9.4%
LMDB_0.9.27 82.989 +9.5%
LMDB_0.9.28 83.025 +9.5%
LMDB_0.9.29 83.063 +9.6%
LMDB_0.9.30 83.237 +9.8%
LMDB_0.9.31 88.419 +16.7%
LMDB_0.9.33 83.304 +9.9%
Excluding 0.9.31 as an outlier, write performance degraded by approximately 9%
from 0.9.20 and has remained at that level through 0.9.33.
==============================================
BENCHMARK DETAILS
==============================================
The write benchmark sequentially inserts 1,000,000 entries (100-byte values,
4-byte integer keys) into a database opened with MDB_INTEGERKEY, using a single
cursor with MDB_APPEND flag. The benchmark includes 2 warmup iterations and 3
measurement iterations of 120 seconds each.
==============================================
ANALYSIS
==============================================
Reviewing the commit history, the regressions appear linked to correctness
fixes:
- 0.9.19: ITS#8406 (xcursor fixup after cursor_del)
- 0.9.20: ITS#8557 (cursor_last and C_EOF handling)
Both issues added cursor state management overhead in write-heavy code paths.
==============================================
REQUEST
==============================================
Could you please reproduce this regression using a native C benchmark to
confirm it's not JNI-specific? If confirmed, this may be a necessary
performance/correctness tradeoff given that most LMDB workloads are read-heavy.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10266
Issue ID: 10266
Summary: Adopt broader RFC4511 NoD interpretation on lloadd's
client side
Product: OpenLDAP
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: lloadd
Assignee: bugs(a)openldap.org
Reporter: ondra(a)mistotebe.net
Target Milestone: ---
Server side, lloadd has long implemented a broad interpretation of NoD
unsolicited response handling: when the message is issued, no new requests are
accepted on the session however the client and server are both free to keep the
session open if there are any operations that have not resolved yet. The server
is still expected to close the connection as soon as no operations are still
pending.
This seems to interoperate with known clients. Those that want to will close
the session immediately, unaware of this possibility, those that also want to
interpret RFC 4511 this way can choose to wait for existing operations to
resolve.
This ticket is to track the lloadd's implementation of the client side of this
- when receiving a NoD message, we don't close the connection
immediately+unconditionally either but are willing to wait.
Related functionality:
- if connection was a bind connection processing a multi-stage SASL bind, the
bind should fail if/when the client attempts to progress it
- clients assigned to this connection through coherence at least 'connection'
are also marked closing
--
You are receiving this mail because:
You are on the CC list for the issue.