https://bugs.openldap.org/show_bug.cgi?id=8689
Howard Chu <hyc(a)openldap.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Resolution|--- |DUPLICATE
Status|UNCONFIRMED |RESOLVED
--- Comment #1 from Howard Chu <hyc(a)openldap.org> ---
Reproduced in 2.4.45 - 2.4.47, but not present in 2.4.48 or later.
Fixed by ITS#8964
*** This issue has been marked as a duplicate of issue 8964 ***
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=8964
Howard Chu <hyc(a)openldap.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |quanah(a)openldap.org
--- Comment #5 from Howard Chu <hyc(a)openldap.org> ---
*** Issue 8689 has been marked as a duplicate of this issue. ***
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=8307
Howard Chu <hyc(a)openldap.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Resolution|--- |TEST
Status|UNCONFIRMED |RESOLVED
--- Comment #9 from Howard Chu <hyc(a)openldap.org> ---
Fixed in master
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=5941
Quanah Gibson-Mount <quanah(a)openldap.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Assignee|bugs(a)openldap.org |ondra(a)mistotebe.net
Keywords| |OL_2_5_REQ
Target Milestone|2.6.0 |2.5.1
--- Comment #12 from Quanah Gibson-Mount <quanah(a)openldap.org> ---
Ondrej to investigate further
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=7468
Quanah Gibson-Mount <quanah(a)openldap.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
See Also| |https://bugs.openldap.org/s
| |how_bug.cgi?id=5941
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=5941
Quanah Gibson-Mount <quanah(a)openldap.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
See Also| |https://bugs.openldap.org/s
| |how_bug.cgi?id=7468
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=7439
Quanah Gibson-Mount <quanah(a)openldap.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Target Milestone|--- |2.5.1
Keywords| |OL_2_5_REQ
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=8689
Quanah Gibson-Mount <quanah(a)openldap.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Assignee|bugs(a)openldap.org |hyc(a)openldap.org
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=8617
Quanah Gibson-Mount <quanah(a)openldap.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Keywords|OL_2_5_REQ |
--- Comment #4 from Quanah Gibson-Mount <quanah(a)openldap.org> ---
Invalid configuration, rwm should not be combined with back-meta.
Future work: Abort slapd if such an invalid configuration is encountered.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=8541
Quanah Gibson-Mount <quanah(a)openldap.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Assignee|bugs(a)openldap.org |quanah(a)openldap.org
--- Comment #4 from Quanah Gibson-Mount <quanah(a)openldap.org> ---
likely fixed by recent code changes, will confirm.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=8307
Quanah Gibson-Mount <quanah(a)openldap.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Assignee|bugs(a)openldap.org |hyc(a)openldap.org
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=8950
Quanah Gibson-Mount <quanah(a)openldap.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Assignee|ondra(a)mistotebe.net |hyc(a)openldap.org
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=8759
Quanah Gibson-Mount <quanah(a)openldap.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|RESOLVED |VERIFIED
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=8759
Quanah Gibson-Mount <quanah(a)openldap.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Resolution|--- |WONTFIX
Status|UNCONFIRMED |RESOLVED
Target Milestone|2.5.1 |---
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=8677
Quanah Gibson-Mount <quanah(a)openldap.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Keywords|OL_2_5_REQ |
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=8668
Quanah Gibson-Mount <quanah(a)openldap.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Keywords|OL_2_5_REQ |
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=8143
Quanah Gibson-Mount <quanah(a)openldap.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Keywords|OL_2_5_REQ |
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=8064
Quanah Gibson-Mount <quanah(a)openldap.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Assignee|bugs(a)openldap.org |ondra(a)mistotebe.net
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=6171
Quanah Gibson-Mount <quanah(a)openldap.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|UNCONFIRMED |RESOLVED
Resolution|--- |SUSPENDED
Target Milestone|2.5.1 |---
--- Comment #2 from Quanah Gibson-Mount <quanah(a)openldap.org> ---
If someone eventually hits an issue where they need K&R support, we can fix the
header files. No one has run into this as a significant issue in 20 years.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=9434
Issue ID: 9434
Summary: Abysmal write performance with certain data patterns
Product: LMDB
Version: 0.9.24
Hardware: x86_64
OS: Linux
Status: UNCONFIRMED
Severity: normal
Priority: ---
Component: liblmdb
Assignee: bugs(a)openldap.org
Reporter: tina(a)tina.pm
Target Milestone: ---
Created attachment 784
--> https://bugs.openldap.org/attachment.cgi?id=784&action=edit
Monitoring graph of disk usage
Hi,
I have recently written a project for a customer which relies heavily on LMDB,
in which performance is critical. Sadly, after completing the project I started
having all kinds of problems when the DB started to grow. This has gotten so
bad the project release had to be postponed, and I have been asked to rewrite
the DB layer using a different engine, unless I can find some solution quickly.
I have so far found 4 serious issues, which I suspect are related either to the
size of the database or to the patterns of the data:
* Writing a value in some of the subdatabases has become increasingly slower,
and commits are taking way too long to complete. This is running on a powerful
computer with SSDs, and the 95% percentile of commits is at around 400ms. The
single-writer limitation meant that I have run out of optimisations to try.
* For some reason I cannot understand, the disk usage has grown to over 2x the
size of the actual data stored, and the free space does not seem to be
reclaimed. The file takes up 348 GB, while the used pages amount to only 162
GB.
* A couple of days ago it had a sudden spike in disk usage (not correlated to
increases in actual data stored, or even to the last pageno user) that filled
the disk in a couple of hours. You can see this in the attached captures of the
monitoring graphs which show actual disk usage (bottom) and counts of pages as
reported by LMDB (top). The bottom graph is total disk usage, although the
partition is almost exclusively the database, but ignore the few dips in size
which are from removing other stuff.
* Running `mdb_dump` for backups takes up to 7 hours for the database; restores
are totally useless: I tried to re-create the database after the weird space
spike and had to stopped after 24h when not even 30% of the data ad been
restored! This alone is a deal-breaker, as we have no usable way to backup and
restore the database.
For context, this is the mdb_stat output with descriptions of each subdatabase.
I have no explanation for the ridiculous amount of free pages, and even running
mdb_stat takes a few seconds:
Environment Info
Map address: (nil)
Map size: 397166026752
Page size: 4096
Max pages: 96964362
Number of pages used: 90991042
Last transaction ID: 14647267
Max readers: 126
Number of readers used: 4
Freelist Status
Tree depth: 3
Branch pages: 26
Leaf pages: 5168
Overflow pages: 74319
Entries: 111981
Free pages: 36352392
Status of Main DB
Tree depth: 1
Branch pages: 0
Leaf pages: 1
Overflow pages: 0
Entries: 8
Status of audit_log
Tree depth: 4
Branch pages: 309
Leaf pages: 69154
Overflow pages: 6082343
Entries: 2061655
* Audit log: MDB_INTEGERKEY, big values (12kb av). Append only, few reads.
Status of audit_idx
Tree depth: 4
Branch pages: 261
Leaf pages: 27310
Overflow pages: 0
Entries: 2006963
* Audit index 1: 40 byte keys, 8 byte values. Append only, it has less records
as I disabled it yesterday due to its impact on performance.
Status of time_idx
Tree depth: 3
Branch pages: 22
Leaf pages: 4611
Overflow pages: 0
Entries: 2061655
* Audit index 2: MDB_INTEGERKEY, MDB_DUPSORT, MDB_DUPFIXED; 40 byte values.
Append only.
Status of item_db
Tree depth: 4
Branch pages: 132
Leaf pages: 10040
Overflow pages: 0
Entries: 186291
* Main data store: 40 byte keys, small values (220b avg). Lots of reads and new
records, very few deletes and no updates.
Status of user_state_db
Tree depth: 5
Branch pages: 83283
Leaf pages: 9289578
Overflow pages: 32
Entries: 207894432
* User state: 20-40 byte keys, small values (180b avg), *many* entries. Lots
and reads and updates.
Status of item_users_idx
Tree depth: 4
Branch pages: 203
Leaf pages: 16532
Overflow pages: 0
Entries: 1035586217
* User / data matrix index: MDB_DUPSORT; 40 byte keys, 20-40 byte values,
*really big*. Lots of writes, very few deletes and no updates.
Status of user_log
Tree depth: 5
Branch pages: 361275
Leaf pages: 26570347
Overflow pages: 0
Entries: 1035586217
* User log: 30-50 byte keys, small values (100b avg), 1e9 records. Append only,
very few reads. I had to stop the restore operation while this was being
recreated, because after 24h only 50% the entries had been restored. Thanks to
monitoring, I measured this maxing out at 7000 entries per second; the other
databases showed way slower rates than this!
Any help would be really appreciated!
Thanks. Tina.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=9430
Issue ID: 9430
Summary: mdb_put() with MDB_RESERVE | MDB_NOOVERWRITE sets
'data' to invalid memory
Product: LMDB
Version: unspecified
Hardware: x86_64
OS: Linux
Status: UNCONFIRMED
Severity: normal
Priority: ---
Component: liblmdb
Assignee: bugs(a)openldap.org
Reporter: kenta(a)lithdew.net
Target Milestone: ---
Running on NixOS (x86_64-linux).
The following sequence of methods was called with an MDB_env and MDB_txn
created with default flags, with the MDB_txn open on MAIN_DBI.
The following code is pseudocode - should the code not be clear enough, happy
to provide the original test case which was written in Zig
(https://ziglang.org).
// Put KV pair ('hello', 'test').
var k: MDB_val = .{ .mv_size = "hello".len, .mv_data = &"hello" };
var v: MDB_val = .{ .mv_size = "test".len, .mv_data = &"test" };
mdb_put(txn, MAIN_DBI, &k, &v, 0);
// Attempt to put KV pair ('hello', 'world!') with MDB_RESERVE |
MDB_NOOVERWRITE, and assert that it fails.
var k2: MDB_val = .{ .mv_size = "hello".len, .mv_data = &"hello" };
var v2: MDB_val = .{ .mv_size = "world!".len, .mv_data = null };
assert(mdb_put(txn, MAIN_DBI, &k2, &v2, MDB_RESERVE | MDB_NOOVERWRITE) ==
MDB_KEYEXIST);
// Assertion fails.
assert(v2 == v);
After these steps, it is expected that v2's 'mv_data' points to 'test' with a
'mv_size' of 4 (the previously, successfully-written KV pair).
What actually happened was that v2's 'mv_size' was correctly set to 4, though
'mv_data' points to garbage memory.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=9429
Issue ID: 9429
Summary: LMDB mdb_del and mdb_cursor_del cause MDB_MAP_FULL on
Apple M1
Product: LMDB
Version: 0.9.26
Hardware: Other
OS: Mac OS
Status: UNCONFIRMED
Severity: normal
Priority: ---
Component: liblmdb
Assignee: bugs(a)openldap.org
Reporter: simonleier(a)gmail.com
Target Milestone: ---
I discovered that deleting data from the database causes MDB_MAP_FULL on my
MacBook Pro with M1 chip. The same code works fine on an Intel Mac.
I think there is no special code required to reproduce this. Open a db, put
something, try to delete it and LMDB crashes.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=8372
Quanah Gibson-Mount <quanah(a)openldap.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
See Also| |https://bugs.openldap.org/s
| |how_bug.cgi?id=9426
--- Comment #5 from Quanah Gibson-Mount <quanah(a)openldap.org> ---
May be duplicate of Issue#9426
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=9397
Issue ID: 9397
Summary: LMDB: A second process opening a file with
MDB_WRITEMAP can cause the first to SIGBUS
Product: LMDB
Version: 0.9.26
Hardware: All
OS: All
Status: UNCONFIRMED
Severity: normal
Priority: ---
Component: liblmdb
Assignee: bugs(a)openldap.org
Reporter: github(a)nicwatson.org
Target Milestone: ---
Created attachment 780
--> https://bugs.openldap.org/attachment.cgi?id=780&action=edit
Full reproduction of SIGBUS MDB_WRITEMAP issue (works on Linux only)
The fundamental problem is that a ftruncate() on Linux that makes a file
smaller will cause accesses past the new end of the file to SIGBUS (see the
mmap man page).
The sequence that causes a SIGBUS involves two processes.
1. The first process opens a new LMDB file with MDB_WRITEMAP.
2. The second process opens the same LMDB file with MDB_WRITEMAP and with an
explicit map_size smaller than the first process's map size.
* This causes an ftruncate that makes the underlying file *smaller*.
3. (Optional) The second process closes the environment and exits.
4. The first process opens a write transaction and writes a bunch of data.
5. The first process commits the transaction. This causes a memory read from
the mapped memory that's now past the end of the file. On Linux, this triggers
a SIGBUS.
Attached is code that fully reproduces the problem on Linux.
The most straightforward solution is to allow ftruncate to *reduce* the file
size if it is the only reader. Another possibility is check the file size and
ftruncate if necessary every time a write transaction is opened. A third
possibility is to catch the SIGBUS signal.
Repro note: I used clone() to create the subprocess to most straightforwardly
demonstrate that the problem is not due to inherited file descriptors. The
problem still manifests when the processes are completely independent.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=9418
Issue ID: 9418
Summary: slapd-bconfig: potential nullptr deref
Product: OpenLDAP
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Severity: normal
Priority: ---
Component: slapd
Assignee: bugs(a)openldap.org
Reporter: danix800(a)gmail.com
Target Milestone: ---
Reported by clang static analyzer in servers/slapd/bconfig.c:6429
'ce->ce_parent'
is potentially nullptr dereference based on the above test of 'ce->ce_parent'
against null before 'access_allowed' is called, so it's better to check again.
See MR!208 for a potential fix.
--
You are receiving this mail because:
You are on the CC list for the issue.