https://bugs.openldap.org/show_bug.cgi?id=9735
--- Comment #8 from jtgrassie jtg@xtrabass.com ---
- growth(1): increasing the LMDB data file size within the limit of
pre-configured (and static) Mapsize
This is not part of the discussion, LMDB already does this. The question is about how hard to look for free space once you've grown to the bounds.
- growth(2): increasing the Mapsize
This is an option when you get a MDB_MAP_FULL from a write tx. As mentioned, using `mdb_env_set_mapsize`.
Do you suggest that when the free space in given env is not found (e.g. due to free space fragmentation), it's desirable to grow(2) ? I guess the code of LMDB internally never does this, but it's of course possible to implement in the application some logic like:
- try to commit a RW txn
- if MDB_MAP_FULL is encountered, grow(2) and re-try
That's exactly the pattern I described, and is used by various applications.
Could you discuss why wouldn't one configure the Mapsize to _that_ very maximal limit already at the beginning
There are many reasons why one wouldn't set the initial size to that of *all* available disk space.
From my point of view, the situation looks quite simple:
- set the Mapsize to the highest possible limit
- the DB will probably never reach it, enjoying the performance-size-trade-off
when searching for available free chunk
- when the fragmentation goes so bad that it actually reaches the limit, it's
always better to sacrifice the performance than failing to write data (with no possibility of any growth)
The database can be compacted *outside* of the main application (as I mentioned, performing an `mdb_copy -c ...`) which can be done as part of a maintenance window for example, therefore never sacrificing the performance.
It all comes down to your specific application requirements, hardware and use-case. Which is exactly *why* I suggested your patch be enabled via a flag (an option) on `mdb_env_open`. I can appreciate some may prefer a "try harder" approach to look for free space (accepting potentially slow writes as a consequence), but I think you also need to appreciate others may not want this (and thus solve fragmentation differently).