First, I'm having trouble finding resources to answer a question like this myself, so please forgive me if I've missed something.

I'm considering using LMDB (versus LevelDB) for a project I'm working on where I'll be receiving a high volume (hundreds per second) of high priority requests (over HTTP) and issuing multiple (<10) database queries per request.

I'll also have a separate process receiving updates for the data and writing to the database.  This will happen often (several times a minute, perhaps), but the priority is much lower than the read requests.

LMDB appealed to me because of the read performance and that I could have one processing reading data from LMDB and another process writing data updates to LMDB.

For proof of concept, I hacked up the following (I'll use pseudocode since I used the Go bindings for my actual programs, and hopefully my question is sufficiently abstract not to matter):

Process 1, the writer, simply writes a random integer (from 0 to 1000) to a defined set of keys:

env = NewEnv()

env.Open("/tmp/foo", 0, 0664)

txn = env.BeginTxn(nil, 0)

dbi = txn.DBIOpen(nil, 0)

txn.Commit()

txn = env.BeginTxn(nil, 0)

n_entries = 5

for i = 0; i < n_entries; i++ {

    key = sprintf("Key-%d", i)

    val = sprintf("Val-%d", rand.Int(1000))

    txn.Put(dbi, key, val, 0)

}

txn.Commit()

env.DBIClose(dbi)

env.Close()

Process 2, the reader, simply loops forever and does random access reads on the data from process 1 (I won't benefit from a cursor for my actual problem), and prints out that data occasionally:

env = NewEnv()

env.Open("/tmp/foo", 0, 0664)

while {

    txn = BeginTxn(nil, 0)

    dbi = txn.DBIOpen(nil, 0)

    txn.Commit()

    for i = 0; i < n_entries; i++ {

        key = sprintf("Key-%d", i)

        val = txn.Get(dbi, key)

        print("%s: %s", key, value)

    }

    env.DBIClose(dbi)

    sleep(5)

}

So my high level question is: What am I doing wrong?  This seems to work OK, but a lot of it was guesswork, so I'm sure I'm doing some silly things.

For example, first I put the BeginTxn() and DBIOpen() calls in process 2 outside of the while loop, but when I did that, I never saw the updates values upon running process 1 simultaneously.  In my real-world application, it seems like adding these calls to every request (to be sure the data being read is up-to-date) could be an unnecessary performance penalty.

I was suspect there are flags that I can/should be using, but I'm not sure.

Thanks for any input.

        Brian