https://bugs.openldap.org/show_bug.cgi?id=10024
Issue ID: 10024 Summary: MDB_PREVSNAPSHOT broken Product: LMDB Version: unspecified Hardware: All OS: All Status: UNCONFIRMED Keywords: needs_review Severity: normal Priority: --- Component: liblmdb Assignee: bugs@openldap.org Reporter: markus@objectbox.io Target Milestone: ---
It seems that the patch #9496 had a negative side effect on MDB_PREVSNAPSHOT. In certain cases, when opening the DB using MDB_PREVSNAPSHOT, the previous (2nd latest) commit is not selected. Instead, reads show that the latest commit was selected voiding the effect of MDB_PREVSNAPSHOT.
I observed this in our test cases a while back. Today, I was finally able to reproduce it and debug into it.
When creating the transaction to read the data, I debugged into mdb_txn_renew0. Here, ti (MDB_txninfo; env->me_txns) was non-NULL. However, ti->mti_txnid was 0 (!) and thus txn->mt_txnid was set to 0. That's the reason for always selecting the first (index 0) meta page inside mdb_txn_renew0:
meta = env->me_metas[txn->mt_txnid & 1];
This line occurs twice (once for read txn and once for write txn; it affects both txn types).
Thus, the chances of MDB_PREVSNAPSHOT selecting the correct meta page is 50-50. It's only correct if the first meta page (index 0) is the older one.
I believe that this is related to #9496 because the patch, that was provided there, removed the initialization of "env->me_txns->mti_txnid" in mdb_env_open2. This would explain why txn->mt_txnid inside mdb_txn_renew0 was set to 0.
I can confirm that adding back the following two lines back in fixes MDB_PREVSNAPSHOT:
if (env->me_txns) env->me_txns->mti_txnid = meta.mm_txnid;
The said patch including the removal of these two lines was applied in the commit(s) "ITS#9496 fix mdb_env_open bug from #8704" (Howard Chu on 09.04.21).
I hope this information is useful to find a suitable fix. Please let me know if you have questions. Also, I'd be happy to help confirming a potential fix with our test suite.
https://bugs.openldap.org/show_bug.cgi?id=10024
--- Comment #1 from Howard Chu hyc@openldap.org --- I can confirm that there's now a bug in env_open2 in mdb.master3, it's calling read_header with an uninit'd meta struct.
https://github.com/LMDB/lmdb/blob/mdb.master3/libraries/liblmdb/mdb.c#L5295
Will work on this soon.
https://bugs.openldap.org/show_bug.cgi?id=10024
--- Comment #2 from Markus markus@objectbox.io --- (In reply to Howard Chu from comment #1)
I can confirm that there's now a bug in env_open2 in mdb.master3, it's calling read_header with an uninit'd meta struct.
I remember that this actually looked fine from debugging because mdb_env_read_header() initializes the given meta in the first iteration (check the if for "off == 0").
The issue I am describing is another one. I had pinpointing to "env->me_txns->mti_txnid" specifically not being initialized in mdb_env_open2 anymore (nor elsewhere). I have laid out the details in the initial description.
From a quick look mdb.master3 seems to share the relevant code sections, but I had debugged mdb.master (lastest version, however the relevant sections stayed the same since the mentioned commit from 09.04.21).
https://bugs.openldap.org/show_bug.cgi?id=10024
Quanah Gibson-Mount quanah@openldap.org changed:
What |Removed |Added ---------------------------------------------------------------------------- Keywords|needs_review | Target Milestone|--- |1.0.0
https://bugs.openldap.org/show_bug.cgi?id=10024
--- Comment #3 from kero renault.cle@gmail.com --- Created attachment 1042 --> https://bugs.openldap.org/attachment.cgi?id=1042&action=edit A test file that shows that the prev snapshot is just ignored
Hello Howard and Markus,
We plan to use the MDB_PREVSNAPSHOT feature in Meilisearch to roll back multiple environments if one of the updates goes wrong.
Each environment corresponds to a Meilisearch index, and, unfortunately, we cannot keep the environment open for all of them simultaneously as it reaches the OS limit of allocated memory mapped [1]. Even if we could, there could be one (or more) failing commits that we need to roll back.
I would like to know if we can help with this. If you want us to fix it, could you guide us a bit?
I linked a test file to show that the MDB_PREVSNAPSHOST is ignored and the previous values cannot be retrieved. The following output shows the problem:
wtxn0 id=1 0 after write toto contained: titi1 expecting titi1 wtxn1 id=2 1 before write toto contained: titi1 expecting titi1 1 after write toto contained: titi2 expecting titi2 wtxn2 with PREVSNAPSHOT id=1 2 toto contained: titi2, expecting titi1 wtxn3 id=3 3 toto contained: titi2
Have a nice day, Howard, Thank you again for the work and this great library!
[1]: https://www.meilisearch.com/blog/dynamic-virtual-address-management