I've successfully ported a particular piece of software from multiple plain
files in disk to LMDB. There were multiple reasons to do so (i.e.,
performance and maintenance).
The program is making individual calls to a storage API, that in turn
creates files or reads them (through a cache). There is no concept of
transaction. For performance and lifetime reasons, the original code avoids
to perform a fsync every time it writes to a file by queuing renames (for
atomic writes) and deletes, which in turn are executed by a background
thread after 2 seconds idling (or a max of 15 sec after first write/delete).
Replacing such a monster "files + cache" with LMDB was a breeze. I kept the
API intact, and to deal with the price of fsyncs, I thought of opening with
MDB_NOSYNC, and sync after 2 seconds of inactivity.
I must mention that the software runs on an OpenWRT based board, with a
MIPS (big endian 32-bit), using UBIFS as storage (with OverlayFS sitting on
top of it). Flash memory is rather slow, peaking at ~350 KiB/sec on
sequential writing. Kernel version is 3.18.29.
This works perfect, except for power failures. With a USB relay board, I'm
cutting the power to the CPU during the sync period. After restart, the
test program I'm running with LMDB crashes with a sigbus error. Opening the
environment seems to work, but the program crashes as soon as it tries to
write to it. The file seems "ok" (no visible garbage or missing parts) when
viewed with hexdump (but is a 5 Mb file).
Any suggestions on where to start looking for issues?
I wanted to congratulate Howard Chu for his work with LMDB, and the
OpenLDAP team for a wonderful work during the years. Keep it up!