https://bugs.openldap.org/show_bug.cgi?id=10454
Issue ID: 10454 Summary: O_DSYNC is busted on macos Product: LMDB Version: unspecified Hardware: All OS: Mac OS Status: UNCONFIRMED Keywords: needs_review Severity: normal Priority: --- Component: liblmdb Assignee: bugs@openldap.org Reporter: pyry.kovanen@gmail.com Target Milestone: ---
LMDB relies on O_DSYNC for writing the meta page, unfortunately it doesn't work on macos. Previous discovery by the tigerbeetle guys: https://github.com/tigerbeetle/viewstamped-replication-made-famous#leaderboa..., some more context at https://x.com/jorandirkgreef/status/1532314169604726784.
I discovered this during benchmarking and was wondering why lmdb writes were twice as fast as macos as on linux.
https://bugs.openldap.org/show_bug.cgi?id=10454
Quanah Gibson-Mount quanah@openldap.org changed:
What |Removed |Added ---------------------------------------------------------------------------- Target Milestone|--- |0.9.36 Assignee|bugs@openldap.org |hyc@openldap.org Keywords|needs_review |
https://bugs.openldap.org/show_bug.cgi?id=10454
Howard Chu hyc@openldap.org changed:
What |Removed |Added ---------------------------------------------------------------------------- Resolution|--- |TEST See Also| |https://bugs.openldap.org/s | |how_bug.cgi?id=10296 Status|UNCONFIRMED |RESOLVED
--- Comment #1 from Howard Chu hyc@openldap.org --- Fixed in git
https://bugs.openldap.org/show_bug.cgi?id=10454
--- Comment #2 from kero renault.cle@gmail.com --- Hello everyone,
It seems this patch broke LMDB on macOS [1], specifically on the GitHub CIs (macOS 15.7.4 24G517). When using the MDB_WRITEMAP option to open an environment [2], it returns an MDB_BADTXN.
Two changes were made to the mdb_env_write_meta function to fix sync issues, and both use FDATASYNC with me->me_mfd. I tried fixing it by calling fcntl on env->me_fd rather than env->me_mfd. I'm not sure it makes sense, but it definitely made my test pass [3]. However, it's probably not synced to disk correctly.
[1]: https://github.com/meilisearch/heed/actions/runs/24962365156/job/73091106305... [2]: https://github.com/meilisearch/heed/blob/d4e3cfde9a843d8b04869d8bd432183dee1... [3]: https://github.com/meilisearch/heed/actions/runs/24992384687/job/73180864502...
https://bugs.openldap.org/show_bug.cgi?id=10454
--- Comment #3 from kero renault.cle@gmail.com --- Created attachment 1146 --> https://bugs.openldap.org/attachment.cgi?id=1146&action=edit correct-fdatasync-macos
It seems to me that env->me_mfd is never opened when MDB_WRITEMAP is used, which is the source of the problem.
if (!(flags & (MDB_RDONLY|MDB_WRITEMAP))) { rc = mdb_fopen(env, &fname, MDB_O_META, mode, &env->me_mfd);
I propose a patch to fix the call to fcntl with F_FULLFSYNC on macOS when the MDB_WRITEMAP option is used, using the correct env->me_fd rather than env->me_mfd.
https://bugs.openldap.org/show_bug.cgi?id=10454
--- Comment #4 from Howard Chu hyc@openldap.org --- Thanks, patch added to git.
https://bugs.openldap.org/show_bug.cgi?id=10454
--- Comment #5 from Quanah Gibson-Mount quanah@openldap.org --- mdb.RE/0.9:
• 568f5e98 by Howard Chu at 2026-04-15T19:48:13+01:00 ITS#10454 lmdb: can't rely on O_DSYNC on MacOS
mdb.master:
• 568f5e98 by Howard Chu at 2026-04-15T19:48:13+01:00 ITS#10454 lmdb: can't rely on O_DSYNC on MacOS
mdb.master3:
• bd9cd548 by Howard Chu at 2026-04-15T19:47:58+01:00 ITS#10454 lmdb: can't rely on O_DSYNC on MacOS
https://bugs.openldap.org/show_bug.cgi?id=10454
--- Comment #6 from Quanah Gibson-Mount quanah@openldap.org --- mdb.RE/0.9:
• 93872c35 by Kerollmops at 2026-04-27T15:18:55+01:00 ITS#10454 LMDB: fix prev commit for Mac OSX + WRITEMAP
mdb.master:
• b99dd9a5 by Kerollmops at 2026-04-27T15:19:09+01:00 ITS#10454 LMDB: fix prev commit for Mac OSX + WRITEMAP
mdb.master3:
• 4e8a093a by Kerollmops at 2026-04-27T15:18:24+01:00 ITS#10454 LMDB: fix prev commit for Mac OSX + WRITEMAP
-
openldap-its＠openldap.org