Dear list,
I am new to lmdb and my apologies if this is on something obvious.
I encountered the MDB_BAD_DBI error in a simple single process, multi-threaded program using lmdb. The scenario is :
* One thread doing a relatively long write transaction mdb_txn_begin -> mdb_dbi_open on several database -> (write) -> mdb_txn_commit
* Another thread (or other threads) of the same process doing short read-only transactions mdb_txn_begin -> mdb_dbi_open on some of the database above -> (read) -> mdb_txn_commit
As per the advice in the document I do not close the database directly. Also the environment created / opened is shared by all threads, being in the same process.
All is fine when these threads execute separately, but if the read transaction thread(s) open the database and commit within the time span of the write transaction in another thread, the above error occurs upon attempt to commit the write transaction.
I have done a search on the MDB_BAD_DBI error and noticed the commit below.
https://gitorious.org/mdb/mdb/commit/0401f2deed75a83d2de790b8a1313e1792e5a04...
Upon a brief look into this and the source of lmdb, my understanding is that the error is due to the increment of lmdb's internal sequence numbers of the database kept by the environment (by the read thread when the read transaction commits). When the write thread subsequently tries to commit, the database sequence number kept privately by the write transaction (copied from the environment when the transaction began) does not match that of the environment, causing the error.
To avoid this in a single process multi-threaded environment, it seems that no read transactions could commit on databases while they are involved in a write transaction, i.e. I need to complete all read transactions before a write transaction on the same database could begin. Alternatively, to have concurrent read transactions with a write transaction, the read transactions have to take place in a separate process (such that they would not share the same environment and caught by the above error).
Grateful if anyone could kindly enlighten me on whether the understanding is correct, or if I am doing something contrary to the how lmdb should be used.
Thank you very much.
- H Law
You're abusing mdb_dbi_open(). Its doc says:
"This function must not be called from multiple concurrent transactions in the same process. A transaction that uses this function must finish (either commit or abort) before any other transaction in the process may use this function."
That text could have stood out a bit better in the doc:-(
The error is simple for lmdb to try to catch, though without locking it cannot guarantee to catch it. Try this patch: http://folk.uio.no/hbf/OpenLDAP/mdb.serial-open.diff
(Howard, this is branch "mdb/serial-open" in my UiO repo.) Not sure if it should do more than this. E.g. attack mdb_drop too. Maybe set MDB_TXN_ERROR since a program with broken threading can break the DB.
Thank you!
It appears that mdb_dbi_open() must be called before any read / write access to a database in a transaction, and all read / write have to be done in a transaction? If this is the case, is it correct that due to the concurrency restriction above, there is essentially no concurrent access (including read and write) to the same database in a multi-threaded process, since a database is available to one transaction in a time?
2015-03-30 19:52 GMT+08:00 Hallvard Breien Furuseth < h.b.furuseth@usit.uio.no>:
You're abusing mdb_dbi_open(). Its doc says:
"This function must not be called from multiple concurrent transactions in the same process. A transaction that uses this function must finish (either commit or abort) before any other transaction in the process may use this function."
That text could have stood out a bit better in the doc:-(
The error is simple for lmdb to try to catch, though without locking it cannot guarantee to catch it. Try this patch: http://folk.uio.no/hbf/OpenLDAP/mdb.serial-open.diff
(Howard, this is branch "mdb/serial-open" in my UiO repo.) Not sure if it should do more than this. E.g. attack mdb_drop too. Maybe set MDB_TXN_ERROR since a program with broken threading can break the DB.
-- Hallvard
hlaw wrote:
Thank you!
It appears that mdb_dbi_open() must be called before any read / write access to a database in a transaction,
True.
and all read / write have to be done in a transaction?
True.
If this is the case, is it correct that due to the concurrency restriction above, there is essentially no concurrent access (including read and write) to the same database in a multi-threaded process,
False.
since a database is available to one transaction in a time?
False. Reread the mdb_dbi_open doc. http://symas.com/mdb/doc/group__mdb.html#gac08cad5b096925642ca359a6d6f0562a
2015-03-30 19:52 GMT+08:00 Hallvard Breien Furuseth <h.b.furuseth@usit.uio.no mailto:h.b.furuseth@usit.uio.no>:
You're abusing mdb_dbi_open(). Its doc says: "This function must not be called from multiple concurrent transactions in the same process. A transaction that uses this function must finish (either commit or abort) before any other transaction in the process may use this function." That text could have stood out a bit better in the doc:-( The error is simple for lmdb to try to catch, though without locking it cannot guarantee to catch it. Try this patch: <http://folk.uio.no/hbf/__OpenLDAP/mdb.serial-open.diff <http://folk.uio.no/hbf/OpenLDAP/mdb.serial-open.diff>> (Howard, this is branch "mdb/serial-open" in my UiO repo.) Not sure if it should do more than this. E.g. attack mdb_drop too. Maybe set MDB_TXN_ERROR since a program with broken threading can break the DB. -- Hallvard
Thanks. I also have a search in the previous discussions and the following is probably the solution to my question.
https://www.openldap.org/lists/openldap-technical/201409/msg00077.html
Quoting from the above -
Before starting any other threads: Create the environment. Open a transaction. Open all DBI handles the app will need. Commit the transaction.
May I confirm my understanding that, after the above steps, each thread created for accessing the database in the same process should :
1. Open a transaction in the same environment 2. Directly use the persisted DBI handles (saved before creating the threads) for read / write without calling mdb_dbi_open 3. Commit / abort the transaction,
regardless of whether the DBI handle is used for read / write?
Also, in respect of the initial transaction for opening the DBI handles, is a read-only transaction sufficient even if the database it opened would be written to in subsequent transactions, since the initial transaction itself would not be writing to the database?
Thanks a lot.
- H Law
2015-04-03 2:54 GMT+08:00 Howard Chu hyc@symas.com:
hlaw wrote:
Thank you!
It appears that mdb_dbi_open() must be called before any read / write access to a database in a transaction,
True.
and all read / write have to be done in a transaction?
True.
If this is the case, is it correct that due to the concurrency restriction above, there is essentially no concurrent access (including read and write) to the same database in a multi-threaded process,
False.
since a database is available to one transaction in a time?
False. Reread the mdb_dbi_open doc.
http://symas.com/mdb/doc/group__mdb.html#gac08cad5b096925642ca359a6d6f0562a
2015-03-30 19:52 GMT+08:00 Hallvard Breien Furuseth <h.b.furuseth@usit.uio.no mailto:h.b.furuseth@usit.uio.no>:
You're abusing mdb_dbi_open(). Its doc says: "This function must not be called from multiple concurrent transactions in the same process. A transaction that uses this function must finish (either commit or abort) before any other transaction in the process may use this function." That text could have stood out a bit better in the doc:-( The error is simple for lmdb to try to catch, though without locking it cannot guarantee to catch it. Try this patch: <http://folk.uio.no/hbf/__OpenLDAP/mdb.serial-open.diff <http://folk.uio.no/hbf/OpenLDAP/mdb.serial-open.diff>> (Howard, this is branch "mdb/serial-open" in my UiO repo.) Not sure if it should do more than this. E.g. attack mdb_drop too. Maybe set MDB_TXN_ERROR since a program with broken threading can break the DB. -- Hallvard-- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
hlaw wrote:
Thanks. I also have a search in the previous discussions and the following is probably the solution to my question.
https://www.openldap.org/lists/openldap-technical/201409/msg00077.html
Quoting from the above -
Before starting any other threads: Create the environment. Open a transaction. Open all DBI handles the app will need. Commit the transaction.
May I confirm my understanding that, after the above steps, each thread created for accessing the database in the same process should :
- Open a transaction in the same environment
- Directly use the persisted DBI handles (saved before creating the
threads) for read / write without calling mdb_dbi_open 3. Commit / abort the transaction,
regardless of whether the DBI handle is used for read / write?
Also, in respect of the initial transaction for opening the DBI handles, is a read-only transaction sufficient even if the database it opened would be written to in subsequent transactions, since the initial transaction itself would not be writing to the database?
That is what "After that use the DBI handles freely among any transactions/threads" means.
Thanks a lot.
- H Law
2015-04-03 2:54 GMT+08:00 Howard Chu <hyc@symas.com mailto:hyc@symas.com>:
hlaw wrote:
Thank you!
It appears that mdb_dbi_open() must be called before any read / write access to a database in a transaction,
True.
and all read / write have to be done in a transaction?
True.
If this is the case, is it correct that due to the concurrency restriction above, there is essentially no concurrent access (including read and write) to the same database in a multi-threaded process,
False.
since a database is available to one transaction in a time?
False. Reread the mdb_dbi_open doc.
http://symas.com/mdb/doc/group__mdb.html#gac08cad5b096925642ca359a6d6f0562a
2015-03-30 19:52 GMT+08:00 Hallvard Breien Furuseth <h.b.furuseth@usit.uio.no mailto:h.b.furuseth@usit.uio.no
<mailto:h.b.furuseth@usit.uio.no mailto:h.b.furuseth@usit.uio.no>>:
You're abusing mdb_dbi_open(). Its doc says: "This function must not be called from multiple concurrent transactions in the same process. A transaction that uses this function must finish (either commit or abort) before any other transaction in the process may use this function." That text could have stood out a bit better in the doc:-( The error is simple for lmdb to try to catch, though without locking it cannot guarantee to catch it. Try this patch: <http://folk.uio.no/hbf/__OpenLDAP/mdb.serial-open.diff <http://folk.uio.no/hbf/OpenLDAP/mdb.serial-open.diff>> (Howard, this is branch "mdb/serial-open" in my UiO repo.) Not sure if it should do more than this. E.g. attack mdb_drop too. Maybe set MDB_TXN_ERROR since a program with broken threading can break the DB. -- Hallvard-- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
openldap-technical@openldap.org
-
Hallvard Breien Furuseth -
hlaw -
HLaw
-
Howard Chu