On 24.10.19 03:49, Howard Chu wrote:

You're free to define MDB_TRPAGE_MAX to a larger value. It just means you increase the chance of overrunning the 2GB available address space. There's no magic, you can't fit every 64bit database workload into only a 32bit address space. When your transactions are too large, the normal thing to do is commit more often so they don't grow so large.

I think you meant MDB_TRPAGE_SIZE? At least that seemed to work, while MDB_TRPAGE_MAX ended up in another MDB_TXN_FULL (tl[0].mid < MDB_TRPAGE_SIZE check failed).

Doubling MDB_TRPAGE_SIZE also doubled the threshold of the object count where it starts failing: using 8192 it was able to remove 4M entries while the limit with 4096 was 2M entries.

MDB_TRPAGE_SIZE is only used to malloc txn->mt_rpages (and some checks), as far as I can tell.

To make a reasonable decision here, could you please confirm:

1. txn->mt_rpages is RAM only and has no impact on mmaped sections (and/or file format).

2. RAM consumption is defined by number of transactions only: per transaction MDB_TRPAGE_SIZE bytes is allocated. Other than that it does not increase memory consumption.

3. There is no other disadvantage other than higher memory consumption per transaction as written in the previous point.

In that case, I think I'll go with 16K (or even 32K). Does not seem too much for an transaction, given that it quadruples the amount of data allowed to be processed. 32 bit devices are not servers, so I do not expect a high number of concurrent transactions anyway.

Thanks!

Markus