Vladimír Čunát wrote:
Hello. (I sent the same message 24h ago before subscribing, but it hasn't arrived so far. I hope you won't see a duplicate later.)
We have a process that needs to analyze the contents of a whole LMDB. So far the approach was to open a read-only transaction and use a cursor to iterate over the whole contents in order. This long transaction apparently causes other concurrent process(es) to get MDB_MAP_FULL from mdb_put() even though that LMDB has plenty free pages at that moment.
That would be a big problem for us, but fortunately we don't need the analysis to be atomic and splitting that iterating into smaller transactions seems to avoid the problem. That were just experiments without understanding why exactly it happens (though I get that long transactions are generally problematic).
Could this behavior be considered a bug in LMDB?
It's a documented feature.
Is there a way of either distinguishing this MDB_MAP_FULL event from really full LMDB or avoiding it with confidence? (We only experimented with the number of keys accessed in one transaction.)
Maybe in LMDB 1.x we'll have a way to minimize the problem. For now, this is how it works.
By the way, thanks a lot for LMDB!
--Vladimir https://knot-resolver.cz