Full_Name: Kristopher William Zyp
Version: LMDB 0.9.70
OS: Windows Server 2012 R2
Submission from: (NULL) (184.108.40.206)
We have experienced significant performance issues in the form of strange pauses
or freezing of any processes that are using our LMDB databases. These pauses are
periodic, often every few minutes, and typically completely freeze a process for
about 5 seconds, sometimes less, sometimes as much as a minute. They often occur
when the process is completely idle (we have a sleep timer to monitor for them),
but we also see a lot of unusually large pauses on commits too. We have never
been able to reproduce any of these issues with our local machines running
Windows 10, only on the virtual Azure Windows servers.
After weeks of investigation, we finally realized that all of these strange
performance issues seemed to be due to the use of NTDLL for memory mapping. I
swapped out the NtCreateSection/NtMapViewOfSection calls for standard Win32
CreateFileMapping/MapViewOfFile calls to create the memory map, and suddenly
these issues went away, and performance dramatically improved. Making this
switch does mean you lose the progressive file allocation functionality of
NtCreateSection, but this is a small price to pay for the vast performance
benefit of using standard file mapping.
Anyway, my request is to simply to provide an option to use CreateFileMapping
API instead of NtCreateSection. I recognize that the NTDLL based APIs may be
more convenient for many users, but an option to use the standard APIs, even if
it means the entire map is pre-allocated in the file size, is well worth it for
the gains. I also recognize that Windows isn't the first choice of LMDB, but
performance with these standard APIs is very good.
I'd be happy to work on putting together a branch/pull request to address this.
However, your repository is a little different than typical github repositories,
so I wanted to know if that was doable, and if there was a proper way to submit
a PR, or if it needs to be submitted through this ticket system as a patch.
Show replies by date