https://bugs.openldap.org/show_bug.cgi?id=10066
Issue ID: 10066 Summary: fsync -> fcntl(F_FULLFSYNC) on Apple platforms? Product: LMDB Version: unspecified Hardware: All OS: Mac OS Status: UNCONFIRMED Keywords: needs_review Severity: normal Priority: --- Component: liblmdb Assignee: bugs@openldap.org Reporter: christophersauer@pacbell.net Target Milestone: ---
Hi, Howard,
I was thinking of adopting LMDB for a cross-platform project, but when quickly browsing the code, didn't see specializations for Apple platform' weakened fsync -> fcntl(F_FULLFSYNC). (Doc quick link, if useful: https://developer.apple.com/library/archive/documentation/System/Conceptual/...)
Should I be concerned? I think it's very likely that you're way ahead of me, but I just wanted to check in. (And couldn't otherwise find anything online about this.)
Thanks so much for all you do! Chris
https://bugs.openldap.org/show_bug.cgi?id=10066
--- Comment #1 from Howard Chu hyc@openldap.org --- Yeah, have seen this before but didn't bother with it. feel free to submit a patch for this.
https://bugs.openldap.org/show_bug.cgi?id=10066
--- Comment #2 from christophersauer@pacbell.net --- I did a bunch more reading, and the plot thickens a bit. It sounds like SQLite added it long ago at Apple's request but then Apple decided to not use it themselves, because there's such a heavy performance impact. (Source: Hipp's comment here: https://bugzilla.mozilla.org/show_bug.cgi?id=431863) Further, there's no way to turn it on in Apple-bundled SQLite (https://bonsaidb.io/blog/acid-on-apple/). mdbx does have it on by default, with an option to disable.
I think it's murky enough that we probably don't do it, following Apple's actions rather than their docs? But I should instead be drawing on your much greater database expertise. Is it indeed true what people say: that most drives lie about fsync anyway, that this stuff doesn't matter that much, and that people mostly just prefer the speed? Seems like a strange world where no one actually gets the guarantees they talk about...
https://bugs.openldap.org/show_bug.cgi?id=10066
--- Comment #3 from Howard Chu hyc@openldap.org --- Yes, the perf cost is extreme. We found the same in our own testing.
Is it indeed true what people say: that most drives lie about fsync anyway, that this stuff doesn't matter that much, and that people mostly just prefer the speed?
Yes, most drives lie, and most people prefer the speed.
The need for a safe sync seems to be greatest on Windows, because the OS itself is prone to crashing, which makes ensuring that OS buffers get persisted to disk a higher priority. True hardware failures will hit all OSs equally but they're quite rare as it is. Another point in Apple's favor is that most of their devices these days are mobile, with built in batteries, so OS crashes due to sudden power outages are pretty unheard of too.
On my own Linux laptops I usually run with all fsyncs disabled, and the OS cache writeback set to 10 minutes. There's just no need for frequent flushing; the most common failure is application-level crashes, which have zero impact on LMDB as it is.
https://bugs.openldap.org/show_bug.cgi?id=10066
christophersauer@pacbell.net changed:
What |Removed |Added ---------------------------------------------------------------------------- Status|UNCONFIRMED |RESOLVED Resolution|--- |WONTFIX
--- Comment #4 from christophersauer@pacbell.net --- Thanks so much for being great, Howard. I'll close this down, then. I really appreciate what you've built and all it and you have taught me.
https://bugs.openldap.org/show_bug.cgi?id=10066
Quanah Gibson-Mount quanah@openldap.org changed:
What |Removed |Added ---------------------------------------------------------------------------- Status|RESOLVED |VERIFIED Keywords|needs_review |
-
openldap-its＠openldap.org