Re: Antw: Re: LMDB physical file size
Ulrich Windl wrote:
>>> Hallvard Breien Furuseth <h.b.furuseth(a)usit.uio.no>
schrieb am 09.11.2015 um
23:00 in Nachricht <56411792.8080205(a)usit.uio.no>:
> On 09/11/15 18:47, Christian Sell wrote:
>> To create the environment, we are using a mapsize of 1 GiB and the flags
>> MDB_NOSUBDIR | MDB_NOLOCK. Under Linux, this results in one file with a size
>> that seems to correspond to the size of the data actually stored. However,
> under
>> Windows, the file size is the same as the mapsize, namely 1 GiB.
>> (...) The same issue surfaced under Linux (...) with the MDB_WRITEMAP option
>
> That's the logical size, which can be bigger than the physical
> size. In lmdb's case, the end of the file doesn't use any disk
> space. On filesystems which support this, anyway. Most do.
> So, nevermind mdb_copy - there is no problem to fix.
>
> On Unix, 'du <file>' shows disk usage. Don't know about Windows.
>
> When you want to copy the file anyway, you should use mdb_copy
> rather than plain filecopy. And MDB_COMPACT does shrink the file
> somewhat since it drops pages which LMDB has freed and not yet
> reused, but that's another matter. The DB would grow later
> anyway, LDMB does need pages it can write to.
I wonder as SSD become more and more common: Should LMDB have a way to
signal to
the operating system that some parts of the file are no longer in
use? So the OS->filesystem->blockdevice could actually reclaim the space.
No.
Pages deleted in one transaction will be reused in a subsequent transaction.
There's no benefit to telling the OS to deallocate it since it will just need
to be reallocated again shortly after. It will kill both performance overall,
issuing extraneous filesystem ops, and kill the SSD itself, issuing extraneous
metadata updates to the device, causing it to wear out faster.
LMDB manages pages the way it does *because that is the optimal way to do so*.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/