I have noticed that reading the database (iterating through with a cursor) the 2nd time tends to be much faster than the 1st time (sometimes the speedup happens after the 3rd or 4th time). I presumed this was because the memory mapping happened not immediately.
That's the background of my 2. question. What was I missing?
Sam Dave wrote:
Hi,
1. Is it possible to disable memory mapping in LMDB?
No, that is fundamental to LMDB's design. The name is "Lightning Memory-Mapped Database", it means what it says.
2. If I create a brand-new database and write data into it (but haven't read it yet) can I be assured it won't be memory-mapped yet?
The question makes no sense.
The purpose of these question is to get more consistent/reproducible time benchmarks for some stuff I'm doing.
There are two ways to get consistent timings - use a test DB that fits entirely in RAM,
or use a test DB that is several times larger than RAM.
For an example of the latter, see http://www.lmdb.tech/bench/hyperdex/
When testing LMDB itself these tests tend to be not very interesting; they only show you
that LMDB runs as fast as your storage device has IOPS. When comparing against other DBs
these tests show how badly other DB designs fare under heavy memory pressure...
(As you know the drive vs. memory-mapped
performance varies heavily.)
Memory mapping has nothing to do with variance. Physical I/O determines that, and it would
be the same variance even if you were using standard read/write calls (although it would be
slower overall).
Thanks,
Sam
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/