https://bugs.openldap.org/show_bug.cgi?id=9640
Issue ID: 9640
Summary: ACL privilege for MOD_INCREMENT
Product: OpenLDAP
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Severity: normal
Priority: ---
Component: slapd
Assignee: bugs(a)openldap.org
Reporter: michael(a)stroeder.com
Target Milestone: ---
I'm using LDAP write operations with MOD_INCREMENT with pre-read-control for
uidNumber/gidNumber generation.
I'd like to limit write access to an Integer attribute "nextID" to
MOD_INCREMENT, ideally even restricting the de-/increment value.
(Uniqueness is achieved with slapo-unique anyway but still I'd like to avoid
users messing with this attribute).
IMHO the ideal solution would be a new privilege "i".
Example for limiting write access to increment by one and grant read access for
using read control:
access to
attrs=nextID
val=1
by group=... =ri
Example for decrementing by two without read:
access to
attrs=nextID
val=-2
by group=... =i
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=9909
Issue ID: 9909
Summary: slap* tools leak cn=config entries on shutdown
Product: OpenLDAP
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: slapd
Assignee: bugs(a)openldap.org
Reporter: ondra(a)mistotebe.net
Target Milestone: ---
slap* tools set up their in-memory cn=config structures but cfb->cb_root is
never released on shutdown.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10026
Issue ID: 10026
Summary: Refresh handling can skip entries (si_dirty not
managed properly)
Product: OpenLDAP
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: overlays
Assignee: bugs(a)openldap.org
Reporter: ondra(a)mistotebe.net
Target Milestone: ---
Take MPR plain syncrepl with 3+ providers.
When a provider's own syncrepl session transitions to persist and a it starts a
new parallel session towards another host, that session always has to start as
a refresh. If that refresh serves entries to us, our handling of si_dirty is
not consistent:
- if the existing persist session serves some of these entries to us, we can
"forget" to pass the others to a newly connected consumer
- same if the refresh is abandoned and we start refreshing from a different
provider that might be behind what we were being served (again our consumers
could suffer)
- if we restart, si_dirty is forgotten and our consumers suffer even worse
We might need to be told (at the beginning of the refresh?) what the end state
we're going for is, so we can keep si_dirty on until then. And somehow persist
that knowledge in the DB...
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10108
Issue ID: 10108
Summary: "mdb_dump -a" does not dump the main database
Product: LMDB
Version: 0.9.29
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: liblmdb
Assignee: bugs(a)openldap.org
Reporter: tuukka.pensala(a)gmail.com
Target Milestone: ---
In mdb_dump.c we have these instructions:
/* -a: dump main DB and all subDBs
* -s: dump only the named subDB
* -n: use NOSUBDIR flag on env_open
* -p: use printable characters
* -f: write to file instead of stdout
* -V: print version and exit
* (default) dump only the main DB
*/
However, contrary to the description, the option -a does not dump the main DB.
With argument -a "dumpit(..)" is called for the named databases, but not for
the unnamed one.
With the current behavior, if the data store contains subDBs and has user-added
data in the main DB, there seems to be no way to dump all of it at once using
mdb_dump.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=9388
Issue ID: 9388
Summary: mdb_stat for DupSort DBI shows incorrect data
Product: LMDB
Version: 0.9.26
Hardware: All
OS: All
Status: UNCONFIRMED
Severity: normal
Priority: ---
Component: liblmdb
Assignee: bugs(a)openldap.org
Reporter: AskAlexSharov(a)gmail.com
Target Milestone: ---
It doesn't include pages pages used for values.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10054
Issue ID: 10054
Summary: Value size limited to 2,147,479,552 bytes
Product: LMDB
Version: unspecified
Hardware: x86_64
OS: Linux
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: liblmdb
Assignee: bugs(a)openldap.org
Reporter: louis(a)meilisearch.com
Target Milestone: ---
Hello,
According to the documentation[0], a database that is not using `MDB_DUPSORT`
can store values up to `0xffffffff` bytes (around 4GB).
In practice, under Linux, the actual limit is `0x7ffff000` though (2^31 - 4096,
so around 2GB).
This is due to the write loop in `mdb_page_flush`. The `wsize` value
determining how many bytes will be written can be as big as
`4096*dp->mp_pages`[1], and the number of overflow pages grows with the size of
the value put inside the DB.
The `wsize` is not split in smaller chunks in the case where there are many
overflow pages to write, and as a result the call to `pwrite`[2] does not
perform a full write, but only a "short" write of 2147479552 bytes (the maximum
allowed on a call to `pwrite` on Linux[3]).
This would be OK if the short write condition was handled by looping and
performing another `pwrite` with the rest of the data, but instead `EIO` is
returned[4].
There seems to be a related, but different issue on macOS when trying to
`pwrite` more the 2^31 bytes, that was already reported[5].
This issue was reported to me by a Meilisearch user because it causes their
database indexing to fail[6]. I had to investigate a bit because their setup
was peculiar (high number of documents in their database) and the `EIO` error
code is not very descriptive of the underlying issue.
I join a C reproducer of the issue that attempts to add a 2147479553 bytes
value to the DB and fails with `EIO` (decreasing `nb_items` to a smaller value
such as `2107479552` does succeed)[7].
Thank you for making LMDB!
Louis Dureuil.
[0]:
https://github.com/LMDB/lmdb/blob/mdb.master/libraries/liblmdb/lmdb.h#LL284…
[1]:
https://github.com/LMDB/lmdb/blob/mdb.master/libraries/liblmdb/mdb.c#LL3770…
[2]: https://github.com/LMDB/lmdb/blob/mdb.master/libraries/liblmdb/mdb.c#L3820
[3]:
https://stackoverflow.com/questions/70368651/why-cant-linux-write-more-than…
[4]: https://github.com/LMDB/lmdb/blob/mdb.master/libraries/liblmdb/mdb.c#L3840
[5]: https://bugs.openldap.org/show_bug.cgi?id=9736
[6]: https://github.com/meilisearch/meilisearch/issues/3654
[7]: https://github.com/dureuill/lmdb_3654/tree/main
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=9223
Bug ID: 9223
Summary: Add support for incremental backup
Product: LMDB
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Severity: normal
Priority: ---
Component: liblmdb
Assignee: bugs(a)openldap.org
Reporter: quanah(a)openldap.org
Target Milestone: ---
For LMDB 1.0, add support for incremental backups
--
You are receiving this mail because:
You are on the CC list for the bug.
https://bugs.openldap.org/show_bug.cgi?id=9619
Issue ID: 9619
Summary: mdb_env_copy2 with MDB_CP_COMPACT in mdb.master3
produces corrupt mdb file
Product: LMDB
Version: 0.9.29
Hardware: All
OS: Windows
Status: UNCONFIRMED
Severity: normal
Priority: ---
Component: liblmdb
Assignee: bugs(a)openldap.org
Reporter: kriszyp(a)gmail.com
Target Milestone: ---
When copying an LMDB database with mdb_env_copy2 with the MDB_CP_COMPACT with
mdb.master3, the resulting mdb file seems to be corrupt and when using it in
LMDB, I get segmentation faults. Copying without the compacting flag seems to
work fine. I apologize, I know this is not a very good issue report, as I
haven't had a chance to actually narrow this down to a more
reproducible/isolated case, or look for how to patch. I thought I would report
in case there are any ideas on what could cause this. The segmentation faults
always seem to be memory write faults (as opposed to try fault on trying to
read). Or perhaps the current backup/copying functionality is eventually going
to be replaced by incremental backup/copying anyway
(https://twitter.com/hyc_symas/status/1315651814096875520). I'll try to update
this if I get a chance to investigate more, but otherwise feel free to
ignore/consider low-priority since the work around is easy.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=9496
Issue ID: 9496
Summary: Some writes missing from database
Product: LMDB
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Severity: normal
Priority: ---
Component: liblmdb
Assignee: bugs(a)openldap.org
Reporter: igfoo(a)github.com
Target Milestone: ---
With the attached test program, some of my database writes appear not to
actually be written to the database. For example, a run may look like this:
$ ./run.sh
All done.
All finished
1802 test.txt
foo_200 is missing
bar_200 is missing
foo_404 is missing
bar_404 is missing
foo_407 is missing
bar_407 is missing
The script that I am using to run the program is below. This is using
mdb.master 52bc29ee2efccf09c650598635cd42a50b6ecffe on Linux, with an ext4
filesystem.
Is this an LMDB bug, or is there a bug in my code?
Thanks
Ian
#!/bin/sh
set -e
if ! [ -d lmdb ]
then
rm -rf lmdb
git clone https://github.com/LMDB/lmdb.git
INSTALL_DIR="`pwd`/inst"
cd lmdb/libraries/liblmdb
make install prefix="$INSTALL_DIR"
cd ../../..
fi
gcc -Wall -Werror -Iinst/include loop.c inst/lib/liblmdb.a -o loop -pthread
rm -f test.db test.db-lock
./loop
echo "All finished"
mdb_dump -np test.db > test.txt
wc -l test.txt
for i in `seq 100 999`
do
if ! grep -q "foo_$i" test.txt
then
echo "foo_$i is missing"
fi
if ! grep -q "bar_$i" test.txt
then
echo "bar_$i is missing"
fi
done
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10024
Issue ID: 10024
Summary: MDB_PREVSNAPSHOT broken
Product: LMDB
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: liblmdb
Assignee: bugs(a)openldap.org
Reporter: markus(a)objectbox.io
Target Milestone: ---
It seems that the patch #9496 had a negative side effect on MDB_PREVSNAPSHOT.
In certain cases, when opening the DB using MDB_PREVSNAPSHOT, the previous (2nd
latest) commit is not selected. Instead, reads show that the latest commit was
selected voiding the effect of MDB_PREVSNAPSHOT.
I observed this in our test cases a while back. Today, I was finally able to
reproduce it and debug into it.
When creating the transaction to read the data, I debugged into mdb_txn_renew0.
Here, ti (MDB_txninfo; env->me_txns) was non-NULL. However, ti->mti_txnid was 0
(!) and thus txn->mt_txnid was set to 0. That's the reason for always selecting
the first (index 0) meta page inside mdb_txn_renew0:
meta = env->me_metas[txn->mt_txnid & 1];
This line occurs twice (once for read txn and once for write txn; it affects
both txn types).
Thus, the chances of MDB_PREVSNAPSHOT selecting the correct meta page is 50-50.
It's only correct if the first meta page (index 0) is the older one.
I believe that this is related to #9496 because the patch, that was provided
there, removed the initialization of "env->me_txns->mti_txnid" in
mdb_env_open2. This would explain why txn->mt_txnid inside mdb_txn_renew0 was
set to 0.
I can confirm that adding back the following two lines back in fixes
MDB_PREVSNAPSHOT:
if (env->me_txns)
env->me_txns->mti_txnid = meta.mm_txnid;
The said patch including the removal of these two lines was applied in the
commit(s) "ITS#9496 fix mdb_env_open bug from #8704" (Howard Chu on 09.04.21).
I hope this information is useful to find a suitable fix. Please let me know if
you have questions. Also, I'd be happy to help confirming a potential fix with
our test suite.
--
You are receiving this mail because:
You are on the CC list for the issue.