openldap-commit2devel@OpenLDAP.org wrote:
- Log -----------------------------------------------------------------
commit 14fb1f59c7e119a100952890f947377d7e95f135 Author: Howard Chu hyc@symas.com Date: Sat Jul 21 06:10:02 2012 -0700
Append tweaks, page_split fixes Append mode now does no key comparisons, input must be in sorted order. page_split was not updating cursor parents correctly.
MDB_APPEND mode is meant to be used for bulk loading of pre-sorted keys. (e.g., if someone wanted to write an mdb_load utility and the corresponding mdb_dump program, you would use this mode for the loader.) New key/data pairs are simply appended to the last page of the database, with no key comparisons at all. (That's the new behavior.) When a page is filled, instead of splitting the page into two new pages (with half the keys in each) it just allocates a new page and starts appending to it. (That's the same behavior as before.) If the parent page fills, it is split (into halves) as normal. The savings from avoiding the memcpy's associated with the split is. The savings from avoiding the key comparisons is another 45% on top of that.
In the context of the microbenchmarks posted earlier, with this new code our batched sequential write speed goes up from 1,157,407 entries/sec to 2,083,333 entries/sec, nearly 3 times faster than the closest competitor, LevelDB (at ~745,000 entries/sec). It also helps slapadd -q. It's around 11-1/2 times faster than BerkeleyDB.
We should not be hearing comments about slow backup/restore procedures any more...
Summary of changes: libraries/libmdb/mdb.c | 115 +++++++++++++++++++++++++++++++++++++++--------- libraries/libmdb/mdb.h | 24 ++++++++++- 2 files changed, 117 insertions(+), 22 deletions(-)