coderwangzhen@gmail.com wrote:

Hi,

We hit a production failure with LMDB and would like to propose a small feature. Feedback on the approach (and whether this is the right place for it) would be much appreciated.

Sounds like a reasonable addition. You should open a ticket in the ITS as an enhancement request. Please be sure to read the CONTRIBUTING guidelines first.

Is there any reason not to implement this on Windows as well?

Problem

On x86-64 hosts with 5-level paging (LA57) enabled, the kernel still defaults to a 47-bit (128 TiB) user virtual-address window: mmap() will not be placed above 2^47 unless the caller passes an explicit hint address ≥ 2^47. LMDB always calls mmap(NULL, ...) in mdb_env_map(), so a process that maps several large environments can exhaust the 128 TiB window and get ENOMEM — even though the same workload fits comfortably in the 56-bit address space. There is no sysctl/prctl/rlimit that lifts this default; it is opt-in per mmap call. See: https://www.kernel.org/doc/html/v5.4/x86/x86_64/5level-paging.html

Proposal

Add a per-environment, non-persistent, advisory mmap address hint:

int mdb_env_set_mapaddr(MDB_env *env, void *addr);

• Default (never called, or addr == NULL): unchanged behavior — mmap(NULL, ...) as today.
• Set to e.g. (void *)(1ULL << 47): passed as the mmap hint, so the kernel is allowed to place the map in the full 56-bit address space.

This is deliberately distinct from MDB_FIXEDMAP, which persists the address into data.mdb and is a multi-process fixed-address scheme. The requested hint is advisory: the kernel may ignore it, and the environment still opens normally.

Patch (against lib/; lib1/ is structurally identical)

--- a/lib/lmdb.h +++ b/lib/lmdb.h @@ -845,6 +845,20 @@ int mdb_env_set_mapsize(MDB_env *env, size_t size);

•   /** @brief Set a preferred address for the memory map (hint).
  

•    *
  

•    * Passes a non-binding hint to mmap() for where to place the data
  

•    * file mapping. Unlike #MDB_FIXEDMAP this hint is not persisted in the
  

•    * data file, and the kernel is free to ignore it (e.g. if the range is
  

•    * already in use); the environment still opens normally. Intended for
  

•    * applications that need the mapping in a specific region, e.g. above
  

•    * the 47-bit boundary to use 5-level paging. The hint is only used on
  

•    * POSIX systems; on Windows it is ignored.
  

•    * @param[in] env An environment handle returned by #mdb_env_create()
  

•    * @param[in] addr Preferred base address, or NULL to let the kernel choose
  

•    * @return A non-zero error value on failure and 0 on success.
  

•    */
  

+int mdb_env_set_mapaddr(MDB_env *env, void *addr);

      /** @brief Set the maximum number of threads/reader slots for the environment.

--- a/lib/mdb.c +++ b/lib/mdb.c @@ -1326,6 +1326,7 @@ struct MDB_env { char *me_map; /**< the memory map of the data file */

•   void            *me_mapaddr;    /**< preferred base address for me_map (hint) */
    MDB_txninfo     *me_txns;               /**< the memory map of the lock file or NULL */
  

@@ -4152,7 +4153,7 @@ mdb_env_map(MDB_env *env, void *addr)

•   env->me_map = mmap(addr, env->me_mapsize, prot, mmap_flags,
  

•   env->me_map = mmap(addr ? addr : env->me_mapaddr, env->me_mapsize, prot, mmap_flags,
            env->me_fd, 0);
  

@@ -4197,6 +4198,14 @@ mdb_env_set_mapsize(MDB_env *env, size_t size) return MDB_SUCCESS; }

+int ESECT +mdb_env_set_mapaddr(MDB_env *env, void *addr) +{

•   if (env->me_map)
  

•           return EINVAL;
  

•   env->me_mapaddr = addr;
  

•   return MDB_SUCCESS;
  

+}

int ESECT mdb_env_set_maxdbs(MDB_env *env, MDB_dbi dbs)

Notes

• The hint is only consulted when addr is NULL (the non-MDB_FIXEDMAP path), so the existing if (addr && env->me_map != addr) return EBUSY; check needs no change: addr stays NULL in the hint case, and MDB_FIXEDMAP semantics are untouched.
• The setter is stored on the env but only read in the POSIX branch; on Windows it is effectively ignored. Happy to gate it behind #ifndef _WIN32 if you prefer.
• Happy to iterate on naming (e.g. mdb_env_set_maphint) and to send a proper git format-patch once the approach is agreed.

Thanks, Zhen Wang