Apologies if this arrives twice, I sent it once two days ago before subscribing to the list and it hasn't appeared yet so I'm resending after subscribing.
Memory mapping files on Windows requires the file size to match the size of the memory mapped region however in the current implementation this requires creating a file the full size of the environment which may be much larger than the actual amount of data in the database.
This patch creates the database as a sparse file on Windows. Windows Explorer will report the file size as the size of the entire memory mapped region but the size on disk will be proportional to the size of committed data in the database. This can be checked by checking the "size on disk" property of the file.
Let me know if this is posted incorrectly, otherwise feel free to modify the patch in whatever way is necessary for approval.
Colin LeMahieu wrote:
Apologies if this arrives twice, I sent it once two days ago before subscribing to the list and it hasn't appeared yet so I'm resending after subscribing.
Thanks, but as noted on http://www.openldap.org/devel/contributing.html patches should be submitted to the ITS.
Memory mapping files on Windows requires the file size to match the size of the memory mapped region however in the current implementation this requires creating a file the full size of the environment which may be much larger than the actual amount of data in the database.
This patch creates the database as a sparse file on Windows. Windows Explorer will report the file size as the size of the entire memory mapped region but the size on disk will be proportional to the size of committed data in the database. This can be checked by checking the "size on disk" property of the file.
Let me know if this is posted incorrectly, otherwise feel free to modify the patch in whatever way is necessary for approval.
Howard Chu wrote:
Colin LeMahieu wrote:
Apologies if this arrives twice, I sent it once two days ago before subscribing to the list and it hasn't appeared yet so I'm resending after subscribing.
Thanks, but as noted on http://www.openldap.org/devel/contributing.html patches should be submitted to the ITS.
Initial feedback from some folks testing this patch are that there's a significant drop in performance. Most likely this is because Windows NTFS allocates sparse file space in 64KB chunks, while pages are still 4KB, and so filling empty space with data involves a number of RMW iterations.
Did you compare before/after performance in your own testing?
Memory mapping files on Windows requires the file size to match the size of the memory mapped region however in the current implementation this requires creating a file the full size of the environment which may be much larger than the actual amount of data in the database.
This patch creates the database as a sparse file on Windows. Windows Explorer will report the file size as the size of the entire memory mapped region but the size on disk will be proportional to the size of committed data in the database. This can be checked by checking the "size on disk" property of the file.
Let me know if this is posted incorrectly, otherwise feel free to modify the patch in whatever way is necessary for approval.
-
Colin LeMahieu -
Howard Chu