I have read a few threads on lmdb alignment but I am still not clear on what kind of padding should be done.
Use case: I want to store aligned SIMD vectors in the value and operate on them directly with SIMD instructions without copying.
As I understand it, lmdb guarantees 2-byte alignment. However, it can also move nodes around for tree rebalancing or page reclaiming. Thus, merely aligning the key and value might not be enough.
Right now, based on my reading of the code, I'm assuming that the data layout in a leaf page is like this:
[page_header][mp_ptrs]...[node_header][key][value][node_header][key][value]
Basically, nodes are stored contiguously in memory with `node_header` and `key` always aligned to 2-byte. New nodes are allocated from the bottom of the page and offsets are stored/allocated into MDB_page.mp_ptrs from the top. Is this correct?
If so, to ensure that `value` has a certain alignment, even when accounting for nodes being moved around, one would also have to ensure that `sizeof(node_header) + sizeof(key) + sizeof(value)` is a multiple of the alignment. Is this reasoning correct?
For the overflow page, is the structure similar? Just that one single page header is used for a number of contiguous pages.
bach@bullno1.com wrote:
I have read a few threads on lmdb alignment but I am still not clear on what kind of padding should be done.
Use case: I want to store aligned SIMD vectors in the value and operate on them directly with SIMD instructions without copying.
As I understand it, lmdb guarantees 2-byte alignment. However, it can also move nodes around for tree rebalancing or page reclaiming. Thus, merely aligning the key and value might not be enough.
Right now, based on my reading of the code, I'm assuming that the data layout in a leaf page is like this:
[page_header][mp_ptrs]...[node_header][key][value][node_header][key][value]
Basically, nodes are stored contiguously in memory with `node_header` and `key` always aligned to 2-byte. New nodes are allocated from the bottom of the page and offsets are stored/allocated into MDB_page.mp_ptrs from the top. Is this correct?
No.
If so, to ensure that `value` has a certain alignment, even when accounting for nodes being moved around, one would also have to ensure that `sizeof(node_header) + sizeof(key) + sizeof(value)` is a multiple of the alignment. Is this reasoning correct?
No.
For the overflow page, is the structure similar? Just that one single page header is used for a number of contiguous pages.
Howard Chu wrote:
bach@bullno1.com wrote:
I have read a few threads on lmdb alignment but I am still not clear on what kind of padding should be done.
Use case: I want to store aligned SIMD vectors in the value and operate on them directly with SIMD instructions without copying.
As I understand it, lmdb guarantees 2-byte alignment. However, it can also move nodes around for tree rebalancing or page reclaiming. Thus, merely aligning the key and value might not be enough.
LMDB's own headers are aligned to 16 byte boundaries. If your keys+values are also aligned to 16 byte boundaries then nodes shuffling around will have no effect on their alignment.
openldap-technical@openldap.org
-
bach＠bullno1.com -
Howard Chu