On 27/05/16 15:11, "Hallvard Breien Furuseth" h.b.furuseth@usit.uio.no wrote:
This works as intended. Each transaction is atomic, and shall see neither DBIs nor data which were committed after the transaction began. Cross-txn mdb_dbi_open() is in any case hairy to get right, we screwed it up several times before arriving at the now-obvious semantics.
It is fine that newly opened database handles shall not be seen by other transactions as long the transaction runs.. But once the transaction which opened the handle is committed it should be valid for any other transaction. This would greatly simplify lazy database opening of a multithreaded application.
The suggested patch does exactly implement that behaviour. I believe it does not break anything. Isolation must be guaranteed for the data but not necessarily for the database handles.
An application can open and cache its DBIs in a separate startup transaction, then commit it and start new transactions.
Yes this would be a simple approach to get around that problem. But I do not like it as with each opened database the overhead increases especially if not using my suggested improvements in ITS#8430.
See my measurements with the improved lmdb and the original lmdb.
The test always performs 1000000 iterations on one database while having 1 database open in the first run and having all databases open in the second run.
lmdb improved (renew, reset)
[1000000] iterations (renew, cursor_open, cursor_close, reset) on 1 database with [1 of 10] open databases in [0.31975] seconds [1000000] iterations (renew, cursor_open, cursor_close, reset) on 1 database with [10 of 10] open databases in [0.20350] seconds
[1000000] iterations (renew, cursor_open, cursor_close, reset) on 1 database with [1 of 100] open databases in [0.25845] seconds [1000000] iterations (renew, cursor_open, cursor_close, reset) on 1 database with [100 of 100] open databases in [0.29663] seconds
[1000000] iterations (renew, cursor_open, cursor_close, reset) on 1 database with [1 of 1000] open databases in [0.28590] seconds [1000000] iterations (renew, cursor_open, cursor_close, reset) on 1 database with [1000 of 1000] open databases in [0.42897] seconds
[1000000] iterations (renew, cursor_open, cursor_close, reset) on 1 database with [1 of 10000] open databases in [0.30004] seconds [1000000] iterations (renew, cursor_open, cursor_close, reset) on 1 database with [10000 of 10000] open databases in [1.68870] seconds
lmdb improved (begin, commit)
[1000000] iterations (begin, cursor_open, cursor_close, abort) on 1 database with [1 of 10] open databases in [0.36538] seconds [1000000] iterations (begin, cursor_open, cursor_close, abort) on 1 database with [10 of 10] open databases in [0.35923] seconds
[1000000] iterations (begin, cursor_open, cursor_close, abort) on 1 database with [1 of 100] open databases in [0.34858] seconds [1000000] iterations (begin, cursor_open, cursor_close, abort) on 1 database with [100 of 100] open databases in [0.39294] seconds
[1000000] iterations (begin, cursor_open, cursor_close, abort) on 1 database with [1 of 1000] open databases in [0.40222] seconds [1000000] iterations (begin, cursor_open, cursor_close, abort) on 1 database with [1000 of 1000] open databases in [0.54752] seconds
[1000000] iterations (begin, cursor_open, cursor_close, abort) on 1 database with [1 of 10000] open databases in [0.78595] seconds [1000000] iterations (begin, cursor_open, cursor_close, abort) on 1 database with [10000 of 10000] open databases in [2.32414] seconds
lmdb original (renew, reset)
[1000000] iterations (renew, cursor_open, cursor_close, reset) on 1 database with [1 of 10] open databases in [0.18597] seconds [1000000] iterations (renew, cursor_open, cursor_close, reset) on 1 database with [10 of 10] open databases in [0.21572] seconds
[1000000] iterations (renew, cursor_open, cursor_close, reset) on 1 database with [1 of 100] open databases in [0.24173] seconds [1000000] iterations (renew, cursor_open, cursor_close, reset) on 1 database with [100 of 100] open databases in [0.46497] seconds
[1000000] iterations (renew, cursor_open, cursor_close, reset) on 1 database with [1 of 1000] open databases in [0.27127] seconds [1000000] iterations (renew, cursor_open, cursor_close, reset) on 1 database with [1000 of 1000] open databases in [4.18579] seconds
[1000000] iterations (renew, cursor_open, cursor_close, reset) on 1 database with [1 of 10000] open databases in [0.30128] seconds [1000000] iterations (renew, cursor_open, cursor_close, reset) on 1 database with [10000 of 10000] open databases in [49.35048] seconds
lmdb original (begin, commit)
[1000000] iterations (begin, cursor_open, cursor_close, abort) on 1 database with [1 of 10] open databases in [0.45135] seconds [1000000] iterations (begin, cursor_open, cursor_close, abort) on 1 database with [10 of 10] open databases in [0.38990] seconds
[1000000] iterations (begin, cursor_open, cursor_close, abort) on 1 database with [1 of 100] open databases in [0.47890] seconds [1000000] iterations (begin, cursor_open, cursor_close, abort) on 1 database with [100 of 100] open databases in [0.69917] seconds
[1000000] iterations (begin, cursor_open, cursor_close, abort) on 1 database with [1 of 1000] open databases in [1.84908] seconds [1000000] iterations (begin, cursor_open, cursor_close, abort) on 1 database with [1000 of 1000] open databases in [5.88098] seconds
[1000000] iterations (begin, cursor_open, cursor_close, abort) on 1 database with [1 of 10000] open databases in [22.12491] seconds [1000000] iterations (begin, cursor_open, cursor_close, abort) on 1 database with [10000 of 10000] open databases in [74.53854] seconds
-- Hallvard
Regards Juerg