On 30. mai 2016 18:08, Juerg Bircher wrote:

If I got it right it is all about protecting against invalid DBIs which can happen for example if someone closes database "A" which was assigned DBI "5" and later the database "B" is opened and the free slot 5 is then assigned to "B". So we could be misled using DBI "5" having in mind database "A".

So why don't we extend the MDB_DBI by adding the sequence in the lower or higher bits. (...)

Ugh. I didn't expect all my words to miss that badly. Yes, you caught one issue. There are more. The central point is what I wrote here:

...

Multi-threaded use of structures, like the ones behind DBIs, must be synchronized. But DBIs and mdb_dbi_open() are not mutex-protected, nor are read-only txns (after the per-thread setup). So it's up to the user. And LMDB mostly can't catch users at less than txn-sized threading bugs - like the DBI workarounds people keep trying.

I shouldn't really be the one to answer this, Howard might answer differently if he didn't just fetch a flamethrower. But anyway:

It's about keeping a clean model of what is going on, which helps understanding and correctly coding both LMDB itself and user programs. When you don't work against the model, anyway.

It's about keeping LMDB's internal data structures consistent. LMDB must know what the heck is going on when it manages and uses DBIs. Things like ensuring that two threads do not run mdb_dbi_open() at the same time, overwriting each others' work and creating a mess.

It's about LMDB being able to know it delivers correct results, at least if the user doesn't work too hard at thwarting it.

This usually means serializing various parts of the code using locks or something similar. But for the sake of speed and simplicity, LMDB has no relevant locks. It relies on the user to lock and has some code to catch some locking errors. One key word is "tries" - without locks or other expensive code, it can't always catch them even when it does try. So DBI errors are not "this DBI is broken, so drop/refresh it" but "your program is buggy, so fix it".

And it's about living with the design choices and tradeoffs which LMDB has made, e.g. not slowing down read-only txns for anything like locking or extensive error checking.

So anyway, this ITS is about changes to LMDB which will not happen. Instead it reminded me to add some checks for user errors with DBIs.