On Fri, Oct 17, 2014 at 6:54 PM, Howard Chu <hyc(a)symas.com> wrote:
Luke Kenneth Casson Leighton wrote:
> howard, et al, hi,
> i notice that LMDB stores data in an "append only" manner (from the
> Q&A). i was thinking of recommending that in python the buffers be
> handed around with *direct* and persistent pointers to the data within
> the LMDB shm/memory-map.
> the reason that i believe this will be successful is because the
> circumstances are rather unusual: once data is entered (added) it is
> *never* overwritten, updated, changed, or modified. there simply is
> no need [in this application]. so there will only ever be insertion
> (once and only once) or controlled and monitored deletion (by age of
> under these circumstances is it perfectly reasonable to pass around
> the pointers to the buffers returned from value look-ups
> *indefinitely*? if so that would be awesome.
Any time a new value is inserted into an existing page, that page's address
will change due to COW. Any time an old page gets sufficiently old that no
other txn points to it, it becomes eligible for reuse - thus completely
unrelated data could get onto it. So unless you're keeping very close tabs
on your application's key+data sizes and know how full a page got, you can't
rely on particular addresses staying valid.
ok. that makes sense. a pity, though, because it would be reaally
cool to refer to data by buffer address forever. COW means it's
beyond tiblmdb's control, but hey, i tried :)