Currently I am creating support for using LMDB as a new storage backend for
one of our products.
At the moment I am testing import bulk data into lmdb using transactions
that span a single record of 10MB. The total db size afterwards is 5GB. I
also tested with records of 1MB.
I noticed a very odd thing: when using the MDB_WRITEMAP option, memory
usage grows very quickly and linear with the amount of data stored into the
database. (memory usage ends up a bit higher than 5GB). when not using
MDB_WRITEMAP, however, memory usage stays very low. Does anyone have a
suggestion what might be wrong and what causes such different behaviour
with and without using the memorymap option?
- there are no long lived reads so that should not be the problem.
- I am using the newest git