Hello, we are using lmbd in our product as database backend to save big key value ( 16K ) . First at all, lmbd is the best database in performance comparaison ( read / write) that we have tested.
My question : # Databases data_dir:/data/cache/data lmdb version:LMDB 0.9.16: (August 14, 2015) db page size:4096 b-tree depath:4 branch pages:527 leaf pages:29287 overflow oages:6659875 data items:1331990
We have a lot of leaf pages, this is due to our big value size. When the database is empty, write and read are very fast, but when we delete all keys or a big part of keys, write and read (new keys) becomes very poor in performance ( / 100 ) . Is it a normal due to our big values ? The only solution we have found is to destroy database file rather than deleting keys
Regards, Nicolas Prochazka
nicolas prochazka wrote:
Hello, we are using lmbd in our product as database backend to save big key value ( 16K ) . First at all, lmbd is the best database in performance comparaison ( read / write) that we have tested.
My question : # Databases data_dir:/data/cache/data lmdb version:LMDB 0.9.16: (August 14, 2015) db page size:4096 b-tree depath:4 branch pages:527 leaf pages:29287 overflow oages:6659875 data items:1331990
We have a lot of leaf pages, this is due to our big value size. When the database is empty, write and read are very fast, but when we delete all keys or a big part of keys, write and read (new keys) becomes very poor in performance ( / 100 ) . Is it a normal due to our big values ? The only solution we have found is to destroy database file rather than deleting keys
Yeah, the page reclaiming algorithm is somewhat simpleminded and large records can aggravate a fragmentation problem in the DB. Try upgrading to 0.9.18 first, it may help a bit.
We'll be experimenting with better reclaiming for LMDB 1.0.
thanks for this explanation Regards, Nicolas Prochazka
2016-03-11 13:10 GMT+01:00 Howard Chu hyc@symas.com:
nicolas prochazka wrote:
Hello, we are using lmbd in our product as database backend to save big key value ( 16K ) . First at all, lmbd is the best database in performance comparaison ( read / write) that we have tested.
My question : # Databases data_dir:/data/cache/data lmdb version:LMDB 0.9.16: (August 14, 2015) db page size:4096 b-tree depath:4 branch pages:527 leaf pages:29287 overflow oages:6659875 data items:1331990
We have a lot of leaf pages, this is due to our big value size. When the database is empty, write and read are very fast, but when we delete all keys or a big part of keys, write and read (new keys) becomes very poor in performance ( / 100 ) . Is it a normal due to our big values ? The only solution we have found is to destroy database file rather than deleting keys
Yeah, the page reclaiming algorithm is somewhat simpleminded and large records can aggravate a fragmentation problem in the DB. Try upgrading to 0.9.18 first, it may help a bit.
We'll be experimenting with better reclaiming for LMDB 1.0.
-- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
openldap-technical@openldap.org
-
Howard Chu -
nicolas prochazka