On 12. nov. 2015 00:38, Shlomi Vaknin wrote:
When I lsof my process, I am seeing that each mdb file is appearing twice (...) I know that after mmaping a file, it is not needed to be kept open, and it seems it is in lmdb.
By default the mmap is read-only, LDMB uses file operations for updates. And it keeps an extra file descriptor with the O_DSYNC or O_SYNC flag for writing the metapage, to avoid a sync() system call.
Unless you use MDB_WRITEMAP: Then it modifies the map directly and omits the sync descriptor, but it still needs a descriptor if the user calls mdb_env_set_mapsize() and on Windows for mdb_env_sync().
I know this might simply be an artifact and might not actually be a contributing reason for my swap ins/outs, but I wanted to hear what do you think about it?
If you've just written much of the database without MDB_WRITEMAP and without sync'ing (i.e. not yet committed or you use MDB_NOSYNC), then you'll have the new data cached for the filesystem and old data in the map. But hopefully you didn't do that with half a gigabyte at the same time as you worry about too high VM usage.