Martin Lucina wrote:
Hi,
I've just spent about two days chasing down a bug in my application using LMDB, where if I ran the code on an ARM (and ARM only) platform, data returned from mdb_get() would appear to be corrupted.
The symtomps were quite bizarre, I'm including them here in case someone else sees something similar; you can skip to the conclusion if you're in a hurry:
a) data written by an application process using mdb_put() appeared correct and consistent b) the same data read by a different application process using mdb_cursor_get() would appear consistent when examined with a live GDB on the target c) however, when actually running the application code, even under GDB, some data would mysteriously be corrupted d) single-stepping through one case of corruption lead me to a single LDR instruction which appeared to load bogus data from memory. GDB showed the same data (at the exact same memory address) as correct, i.e. not corrupted. e) the problem only occured on ARM and was always reproducible.
Much experimenting and googling later I found [1], the upshot of which is that on some ARM platforms unaligned word accesses to data using LDR produce *bogus* data. No exception, nothing, just silent data corruption.
My question is: Given that I'm storing C structs directly in LMDB, I need to get at least word-aligned pointers back from LMDB in order to be able to access the data safely.
I've not found anything in the documentation or the source about being able to tweak alignment of key and data values; I *think* that all I need is an option where LMDB would guarantee a minimum word (4 byte in this case) alignment of data.
How hard would this be to implement? Would you consider such a feature?
LMDB guarantees 2-byte alignment of keys, and data is generally stored contiguously with keys. If you want 4-byte alignment, it's your responsibility to pad your keys to 4-byte boundaries. If you pad appropriately, your alignment will be preserved.
The only workaround I can think of is explicitly copying all data I get back from LMDB into aligned struct pointers; for obvious performance reasons I don't want to do that.
Martin
[1] http://stackoverflow.com/questions/21488501/strange-pointer-arithmetic See the second answer, and the link to the ARM documentation for the LDR instruction.