pierre.chambart@ocamlpro.com wrote:
Full_Name: Pierre Chambart Version: OS: URL: Submission from: (NULL) (185.194.234.133)
We have been trying LMDB compiled with the MDB_VL32 flag and encountered some MDB_CORRUPTED error that didn't occur without MDB_VL32. We could produce a minimal reproduction case: https://raw.githubusercontent.com/chambart/lmdb/vl32_readonly_fix/libraries/... This is basically writing a key, reading it, writing something else then reading it again.
After some investigation, this is how I understand what is happening:
- The chunk being mapped for the first read contains 3 pages instead of the 16
pages that would be mapped without readonly.
- The second write extend the size of the base to 5 pages
- Finally the last read retrieve the mapped chunk from the first read, which has
size 3, but assumes that it has size 5, hence reading some random value in memory past the end of the pages.
It can be verified by asserting that id3.mcnt is larger than rem at the end of mdb_rpage_get, around this line:
ok: p = (MDB_page *)((char *)id3.mptr + rem * env->me_psize);
This does not seem to be the case.
I made a dirty fix that seems to work. It's simply to remove those lines:
/* don't map past last written page in read-only envs */ if ((env->me_flags & MDB_RDONLY) && pgno + MDB_RPAGE_CHUNK-1 >txn->mt_last_pgno) id3.mcnt = txn->mt_last_pgno + 1 - pgno; else
This seems to be some optimization that avoids mapping the 16 pages of the chunk. But given that those pages are initialized, there doesn't seem to be much harm in mapping a few more pages in that relatively rare situation.
Please check git history before assuming the code is an optimization.
These lines prevent a crash on Windows.