Jens Alfke wrote:
I'm evaluating LMDB, and one feature we consider important is file-level encryption. The memory-mapped, zero-copy nature of LMDB would seem to be incompatible with encryption, but there is a tantalizing post from Howard Chu two years ago:
• To: "OpenLDAP-devel@openldap.org mailto:OpenLDAP-devel@openldap.org" OpenLDAP-devel@openldap.org • Subject: LMDB encryption support • From: Howard Chu <hyc@symas.com mailto:hyc@symas.com> • Date: Thu, 10 Aug 2017 16:55:57 +0100
I've recently added support for page-level encryption to LMDB 1.x using user-supplied callbacks:
( https://www.openldap.org/lists/openldap-devel/201708/msg00002.html%C2%A0)
However, the API shown in the email is not present in mainline LMDB (0.9.24), nor can I find it in a branch in the Git repository. Was this an abandoned experiment? Or is it only available with a commercial license (as is SQLite's encryption extension?)
It's in a private branch, for closed-source licensees. But we'll be opening it up soon.
(Whether or not this feature still exists, I'm curious about how it was implemented. It seems like it would either require some kind of kernel-level support for hooking into the VM pager to rewrite pages after they're faulted in, or else not using memory-mapping at all and going with a page-cache a la SQLite.)
Yes it requires LMDB to manage cached pages. I'm not too fond of that, but there was no other way.
Thanks,
—Jens