On 19/09/2018 17:35, Howard Chu wrote:
pierre.chambart@ocamlpro.com wrote:
Full_Name: Pierre Chambart Version:=20 OS:=20 URL:=20 Submission from: (NULL) (185.194.234.133)
We have been trying LMDB compiled with the MDB_VL32 flag and encounter=
ed some
MDB_CORRUPTED error that didn't occur without MDB_VL32. We could produce a minimal reproduction case: https://raw.githubusercontent.com/chambart/lmdb/vl32_readonly_fix/libr=
aries/liblmdb/vl32_failure.c
This is basically writing a key, reading it, writing something else th=
en reading
it again.
After some investigation, this is how I understand what is happening:
- The chunk being mapped for the first read contains 3 pages instead o=
f the 16
pages that would be mapped without readonly.
- The second write extend the size of the base to 5 pages
- Finally the last read retrieve the mapped chunk from the first read,=
which has
size 3, but assumes that it has size 5, hence reading some random valu=
e in
memory past the end of the pages.
It can be verified by asserting that id3.mcnt is larger than rem at th=
e end of
mdb_rpage_get, around this line:
ok: p =3D (MDB_page *)((char *)id3.mptr + rem * env->me_psize);
This does not seem to be the case.
I made a dirty fix that seems to work. It's simply to remove those lin=
es:
/* don't map past last written page in read-only envs */ if ((env->me_flags & MDB_RDONLY) && pgno + MDB_RPAGE_CHUNK-1 >=
txn->mt_last_pgno) id3.mcnt =3D txn->mt_last_pgno + 1 - pgno; else
This seems to be some optimization that avoids mapping the 16 pages of=
the
chunk. But given that those pages are initialized, there doesn't seem to be much harm in mapping a few more =
pages in
that relatively rare situation.
Please check git history before assuming the code is an optimization.
These lines prevent a crash on Windows.
Sorry, about that.
Also I'm not suggesting that this is a proper fix, rather that this is so= me clue to understand the problem. I'm happy producing a patch if you indicate what kind of fix you would like. --=20 Pierre