On 24.10.19 03:49, Howard Chu wrote:
You're free to define MDB_TRPAGE_MAX to a larger value. It just
you increase the chance of overrunning the 2GB available address space.
There's no magic, you can't fit every 64bit database workload into only
a 32bit address space. When your transactions are too large, the normal
thing to do is commit more often so they don't grow so large.
I think you meant MDB_TRPAGE_SIZE? At least that seemed to work, while
MDB_TRPAGE_MAX ended up in another MDB_TXN_FULL (tl.mid <
MDB_TRPAGE_SIZE check failed).
Doubling MDB_TRPAGE_SIZE also doubled the threshold of the object count
where it starts failing: using 8192 it was able to remove 4M entries
while the limit with 4096 was 2M entries.
MDB_TRPAGE_SIZE is only used to malloc txn->mt_rpages (and some checks),
as far as I can tell.
To make a reasonable decision here, could you please confirm:
1. txn->mt_rpages is RAM only and has no impact on mmaped sections
(and/or file format).
2. RAM consumption is defined by number of transactions only: per
transaction MDB_TRPAGE_SIZE bytes is allocated. Other than that it does
not increase memory consumption.
3. There is no other disadvantage other than higher memory consumption
per transaction as written in the previous point.
In that case, I think I'll go with 16K (or even 32K). Does not seem too
much for an transaction, given that it quadruples the amount of data
allowed to be processed. 32 bit devices are not servers, so I do not
expect a high number of concurrent transactions anyway.