Christian Sell <christian(a)gsvitec.com&gt; hat am 26. November 2015 um 01:28 geschrieben: I forgot to mention: I am using MDB_RESERVE to avoid extra memcpy. Could it be that this is the cause for the extreme database bloat? > Hello, > > something wrong with the question below? > > I am trying to use LMDB to store large (huge) amounts of binary data which, > for > the reason of limiting memory footprint, are split into chunks. Each chunk > ist > stored under a separate key, made up of [collectionId, chunkId], so that I > can > later iterate the chunks using a LMDB cursor. Chunk size is configurable. > > During my tests, I encountered a strange scenario where, after inserting > some > 2000 chunks consisting of 512KB each, the database size had grown to a > value > that was roughly 135 times the calculated size of the data. I ran the stat > utility over the db and saw that there were > 12000 overflow pages vs. > approx. > 2000 data pages. When I reduced the chunk size to 4060 bytes, the number of > overflow pages went down to 1000, and the database size went down to the > expected number (I experimented with different sizes, this was the best > result). > I did not find any documentation to explain this behaviour, or how to deal > with > it. Of course it makes me worry about database bloat and the consequences. > Can > anyone shed light on this? > > thanks, > Christian > Christian Sell