https://bugs.openldap.org/show_bug.cgi?id=10656

Issue ID: 10656 Summary: No mmap address hint to map above 47-bit (128TiB) window Product: LMDB Version: unspecified Hardware: All OS: All Status: UNCONFIRMED Keywords: needs_review Severity: normal Priority: --- Component: liblmdb Assignee: bugs@openldap.org Reporter: coder-wangzhen@qq.com Target Milestone: ---

Created attachment 1242 --> https://bugs.openldap.org/attachment.cgi?id=1242&action=edit Patch: add mdb_env_set_maphint() (git format-patch)

Description:

Overview: On x86-64 hosts with 5-level paging (LA57) enabled, the kernel still defaults to a 47-bit (128 TiB) user virtual-address window: mmap() is not placed above 2^47 unless the caller passes an explicit hint address >= 2^47. LMDB always calls mmap(NULL, ...) in mdb_env_map(), so a process mapping several large environments can exhaust the 128 TiB window and get ENOMEM, even though the workload fits in the 56-bit address space. There is no sysctl/prctl/rlimit that lifts this default; it is opt-in per mmap call. See https://www.kernel.org/doc/html/v5.4/x86/x86_64/5level-paging.html

Proposed: Add a per-environment, non-persistent, advisory mmap address hint:

int mdb_env_set_maphint(MDB_env *env, void *addr);

- Default (never called, or addr == NULL): unchanged, mmap(NULL, ...). - Set to e.g. (void *)(1ULL << 47): passed as the mmap hint so the kernel can place the map in the full 56-bit address space.

Distinct from MDB_FIXEDMAP, which persists the address into data.mdb and is a multi-process fixed-address scheme. The hint is advisory: the kernel may ignore it and the environment still opens normally.

Expected: With mdb_env_set_maphint(env, (void *)(1ULL << 47)) set before mdb_env_open(), a single process can map environments above 2^47 and its total VSZ can exceed 128 TiB.

Windows: Windows x64 uses 48-bit virtual addresses (128 TiB user space) and does not support 5-level paging, so the motivating use case (mapping above 2^47) is Linux-only. The patch nonetheless implements the Windows path (NtMapViewOfSection, with fallback to OS-chosen placement when the requested address is occupied) for API consistency. Given Windows cannot map above 128 TiB anyway, is the Windows portion still wanted, or should the change be POSIX-only?

Additional Information: Patch attached (git format-patch against mdb.master3). Discussed on openldap-devel:

https://lists.openldap.org/hyperkitty/list/openldap-devel@openldap.org/threa...