Yucheng Low wrote:

Hi,

I am considering using LMDB to store a dynamic graph structure, and it will be
very helpful to support "append to value" having the same semantics as
"append" in Kyoto Cabinet.
http://fallabs.com/kyotocabinet/api/classkyotocabinet_1_1BasicDB.html#a23e776e5bd1e3c5caa0f62edffb87a54

For instance, if I were to store an adjacency list representation of a graph
in LMDB, I would use vertex ID as the key, and the value is a vector of 64-bit
IDs. To insert an edge into the graph will simply require appending 8 bytes to
the end of the vector. Re-reading and re-writing the entire value will be
somewhat excessive (and will actually change the asymptotic runtime of the
insertion).

This is also one of those situations where a batch append might be useful. For
instance, if I have an edge list on file which is too large to retain in
memory. To convert it to an adjacency list in LMDB will require batch-append.
Batch-writes are insufficient since that will require me to cache at least,
large parts of the adjacency list in memory, and the original edge list could
be ordered arbitrarily.

The actual graph representation I am thinking of is somewhat more intelligent
than a straight adjacency list, but similar concepts hold.

Based solely on the description here, I would use MDB_DUPFIXED and MDB_APPENDDUP or MDB_MULTIPLE. You could later use MDB_GET_MULTIPLE to read the values and reconstruct the vector.

--
  -- Howard Chu
  CTO, Symas Corp.           http://www.symas.com
  Director, Highland Sun     http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/