Howard Chu writes:
Fixing this will either require adding a bunch of ugly code, or changing the on-disk format again. Opinions? (...) Adjusting this would require only a few minor tweaks to the code, but would require a full dump/reload of existing databases.
Then I hope for the format change. But can this be macroized without said ugliness, so existing users can compile with e.g. -DMDB_DATA_VERSION=1 until they're prepared to dump/reload?
Still, 64K pages sounds like it can get quite wasteful. I can clean up my patch to support datafile pagesize != OS pagesize. That would be a user option, not the default: Smaller datafile pagesize means more I/O for writes. (The OS operates on pages, so to write part of an OS page it must first read the full page.)
A problem here is that liblmdb lacks some checks for data sizes. We can add those, but older liblmdb can break if it commits to a datafile written by newer liblmdb and non-default sizes. IIRC it gets a bit hairy to support this for ldmb data format 2 but support V1 sizes safely, so I'm either talking V2 only or just telling users to not do this when also using an older liblmdb.