On Thu, Aug 20, 2020 at 07:42:06PM +0200, Vladimír Čunát wrote:

(Pages touched in a RO transaction can't be freed until it aborts - that
was always clear.  For us the overall number of pages don't seem
possible to have ran out, assuming the MDB_stat counts reflected their
usage before all those operations began.)

I don't remember where to point you so in my own words...

You do understand that no pages reachable by an open read transaction
can ever be reclaimed. At the moment, AFAIK a stronger claim can be
made about LMDB:

For an open read transaction #n, it avoids reclaiming any pages freed by
transactions #n upwards. Keeping a read transaction open indefinitely,
at some point you are going to run out of pages to reclaim having to
reach for fresh ones. Later, you might run out of those too. This is
what you're seeing.

No... the numbers do NOT match that notion at all.  Now I had time to get more confidence by running proper tests as follows.

Let's have a large LMDB and two processes accessing it.  One process opens the DB, iterates over whole contents in a single RO transaction (which always takes only a fraction of second), closes the DB (to be sure), waits a split-second and then repeats it whole.

The other process is continuously inserting into the DB in very small RW transactions.  The speed of this is 1-2% DB size per second.  There are no deletions and in this case even replacements should be rare (as DB keys are generated randomly).  Around 50% usage this gets MDB_MAP_FULL; at that point the process stops and the DB shows state posted below.

All size measurements are occupied pages, as described earlier in the thread.  As stated before, shortening the RO transactions delays MDB_MAP_FULL until >99% usage... and that's our workaround for now (with some magical constant).

Thanks
--Vladimir

$ mdb_stat -ef .; and ls -l
Environment Info
  Map address: (nil)
  Map size: 671088640
  Page size: 4096
  Max pages: 163840
  Number of pages used: 163824
  Last transaction ID: 115523
  Max readers: 126
  Number of readers used: 0
Freelist Status
  Tree depth: 2
  Branch pages: 1
  Leaf pages: 203
  Overflow pages: 0
  Entries: 3321
  Free pages: 83500
Status of Main DB
  Tree depth: 5
  Branch pages: 4162
  Leaf pages: 75956
  Overflow pages: 0
  Entries: 627631