selama@gmail.com wrote:
I am working on developing a new document-oriented (XML+JSON) database, using LMDB as an engine, and I have two questions.
- So far, it have been working really smoothly for me. But my one customer so far for the DB is really concerned about running LMDB in a virtual environment such as Docker, when performing reads and writes. Especially when mounting volumes. Their concern is because of the following caveat:
"Do not use LMDB databases on remote filesystems, even between processes on the same host. This breaks flock() on some OSes, possibly memory map sync, and certainly sync between programs on different hosts" I think it shouldn't be an issue with Docker, but I want to be certain.
That caveat applies to actual remote filesystems like NFS or SMB.
If you're just using a local filesystem as persistent storage it should be fine. But, LMDB records process IDs in its reader table, and containers all number their processes starting from 1. So if you have multiple containers accessing the same DB at once, you're likely to get PID collisions and break reader consistency.
- We have one server that updates the database, and another server with a read-only copy of the same database. Our plan was to simply copy the mdb files from the update machine to the read only machine, but we noticed that if we copy the file immediately after writing, the copy may end up being corrupted. My solution was to suspend all writing and wait few minutes before writing, to make sure everything back from memory, and I'm also using the "sync" command (not sure if it does anything here). It seem to be working, but I wonder if there is a more robust way of doing that? And also, is it safe to overwrite to the read-only server while it performs read transactions to the current file (or maybe rename it and copy to a new file with the same name)?
That's what the mdb_copy command is for...
And no, it is not safe to overwrite the file while any process is reading it.