kriszyp@gmail.com wrote:

However, in trying to test the mdb.master branch code with our application/server (we were previously using the 0.9 branch), there was a regression causing it to crash pretty much any time we attempted to write to a db that was larger than 2GB. After some considerable testing and investigation (which is why it took a while to get this finished), it seems the cause is the use of the off_t type for file pointer/positions, which appears to be a 32-bit signed integer compiling on Windows. This overflows for dbs over 2GB and causes references that crash the process. Replacing all the off_t types with size_t (unsigned 64-bit), fixed this issue. I am not sure if this is the right replacement type. I think this could also be addressed with compiler option for changing the file offset type size, but that seems like a hazardous hoop to jump through. But the third commit, replacing the off_t with size_t definitely fixed the problem in our application.

There should be an off64_t type instead. But it looks to me like only the env->me_size field could ever overflow, and using size_t for that should be fine. The other uses are for the meta page, which is always either offset 0 or offset 4096 (with 4KB pagesize), and the lockfile, which is always smaller than 2GB.

https://github.com/LMDB/lmdb/commit/45cf4b6ede38565cfab1c40d0d77961a0cb22b9e...