https://bugs.openldap.org/show_bug.cgi?id=10072
Issue ID: 10072 Summary: Querying for transaction memory use Product: LMDB Version: unspecified Hardware: All OS: All Status: UNCONFIRMED Keywords: needs_review Severity: normal Priority: --- Component: liblmdb Assignee: bugs@openldap.org Reporter: tagwerk19.openldap@innerjoin.org Target Milestone: ---
It would be useful to have a way to ask liblmdb how much "dirty memory" a transaction is using, up to that point.
The goal is to be able to bundle writes into fewer transactions, making best use of available memory and reducing total disc writes.
It is possible for an application to count the number of writes but this has only a loose relation to the *actual* number of dirty pages flagged. The target would be to write data up until the dirty memory gets to a threshold and then commit.
I see that there's a:
txn->mt_dirty_room
and wonder if
MDB_IDL_UM_MAX - txn->mt_dirty_room
would give a count of that an "aware" application could make use of (an exact count? or maybe a usable and sufficient indication?)
The need for this has been sharpened with the use of systemd service files that cap the memory allowed for a process, for example:
MemoryHigh=512M
If the process reaches this limit and continues to write data, the system uses swap.
See:
https://invent.kde.org/frameworks/baloo/-/merge_requests/148
Thank you for providing and supporting LMDB