https://bugs.openldap.org/show_bug.cgi?id=9736
Issue ID: 9736 Summary: pwrite bug in OSX breaking LMDB promise about the maximum value size Product: LMDB Version: unspecified Hardware: All OS: Mac OS Status: UNCONFIRMED Keywords: needs_review Severity: normal Priority: --- Component: liblmdb Assignee: bugs@openldap.org Reporter: renault.cle@gmail.com Target Milestone: ---
Hi,
I was working with LMDB and found an issue when trying to write a value of approximately 3.3GiB in the database, I dive into the LMDB source code of the mdb_put method using the lldb debugger and found out that it was not related to an issue in LMDB itself but rather a bug in the pwrite function of the Mac OS libc implementation.
The pwrite function is given four parameters, the file descriptor, the buffer, the count of bytes to write from the buffer and, the offset of where to write it in the file. On Mac OS the count of bytes is a size_t that must be a 64bits unsigned integer but when you call pwrite with a number bigger or equal to 2^31 it returns an error 22 (invalid argument). LMDB was returning a 22 error from the mdb_put call and not an EINVAL because the error was cause by an internal issue and not something catchable by LMDB.
I am not sure about what we can do, can we implement this single pwrite [1] as multiple pwrite with counts smaller than 2^31 in a loop, just for Mac OS? Like for Windows where we do specific things for this operating system too?
I also found this issue on the RocksDB repository [2] about a similar problem they have with pwrite and write on Mac OS it seems. I understand that this is not a real promise that LMDB is specifying but rather an "in theory" rule [3].
Thank you for your time, kero
[1]: https://github.com/LMDB/lmdb/blob/01b1b7dc204abdf3849536979205dc9e3a0e3ece/l... [2]: https://github.com/facebook/rocksdb/issues/5169 [3]: http://www.lmdb.tech/doc/group__mdb.html#structMDB__val