Ulrich Windl wrote:
>>> "Vitaly Zuevsky"
<vitaly.zuevsky(a)gmail.com> schrieb am 24.05.2021 um 22:06 in
>> Vitaly Zuevsky wrote:
>>> MAP_LOCK flag to mmap() on LMDB readers should populate page cache
>>> from the file being mapped and preclude those pages from eviction. I
>>> was wondering if that was ever tested, especially with frequent writes
>>> to the underlying file. And is there a standard way (a compile option)
>>> to include that flag without modifying source code directly? I see
>>> quite a few major faults with perf stat, and I am looking if they could be
>>> I am new to LMDB and I guess this discussion may have taken place
>>> before, if you could point me out to the right place..
>> No. In general it's a bad idea to interfere with the OS's paging
>> particular, on a machine with plenty of RAM, pageouts aren't an issue
>> On a system where memory is tight, page faults will be unavoidable, so the
>> question is what other memory are you going to lose if you force the LMDB
>> pages to stay locked in?
> Memory pressure isn't my problem. I see 2-15 major faults a second in the
> event loop of the reader thread, which can effectively stall the loop for
> anything between 1-400ms (I measure iowait jiffies of the pid). I understand
> the faults result from the writer thread, actively modifying the file mmaped
> by the reader. I assume writes to the file invalidate corresponding pages of
> the reader, and those invalid pages only get updated (via major faults) on
> first access by the reader. If not locking, perhaps madvise/willneed
> combination could help, what you think?
I don't know the internals of LMDB, but writing to a file that has mapped region
seems to be a poor idea.
Why not write the region?
On any modern OS with unified buffer cache, there is no difference.
(OpenBSD is notable in its lack of a unified buffer cache, can't think
of any other such deficient OSs these days.)
AFAIR HP-UX really didn't like that.
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/