Hello,
I like to implement an autogrow functionality for LMDB.
The documentation for mdb_env_set_mapsize says that no transactions should be active when using this function.
When looking at the code I can see that the function only checks if there is an active WRITE transaction and in this case it returns an error.
Is it possible to reuse existing READ transactions or even associated cursors after mdb_env_set_mapsize has been called?
Thank you and best regards,
Ken
Ken Wenzel wrote:
Hello,
I like to implement an autogrow functionality for LMDB.
The documentation for mdb_env_set_mapsize says that no transactions should be active when using this function.
When looking at the code I can see that the function only checks if there is an active WRITE transaction and in this case it returns an error.
Is it possible to reuse existing READ transactions or even associated cursors after mdb_env_set_mapsize has been called?
Yes, you can reuse the existing structures as long as you have done txn_reset / txn_renew / cursor_renew. Naturally you must do txn_reset before the resize, otherwise the read txn is still active.
Thank you for the quick response. But this means that also READ transactions must be closed/reset before using mdb_env_set_mapsize? It is not enough to lock any readers while calling mdb_env_set_mapsize? Would it be hard to support resyncing read transactions and cursors after a map resize directly in the LMDB library?
Thank you and best regards, Ken
-----Ursprüngliche Nachricht----- Von: Howard Chu hyc@symas.com Gesendet: Montag, 31. Januar 2022 17:42 An: Ken Wenzel kenwenzel@gmx.net; openldap-technical@openldap.org Betreff: Re: [LMDB] mdb_env_set_mapsize and read transactions
Ken Wenzel wrote:
Hello,
I like to implement an autogrow functionality for LMDB.
The documentation for mdb_env_set_mapsize says that no transactions should
be active when using this function.
When looking at the code I can see that the function only checks if there
is an active WRITE transaction and in this case it returns an error.
Is it possible to reuse existing READ transactions or even associated
cursors after mdb_env_set_mapsize has been called?
Yes, you can reuse the existing structures as long as you have done txn_reset / txn_renew / cursor_renew. Naturally you must do txn_reset before the resize, otherwise the read txn is still active.
-- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Ken Wenzel wrote:
Thank you for the quick response. But this means that also READ transactions must be closed/reset before using mdb_env_set_mapsize?
The documentation is quite explicit. No active transactions.
It is not enough to lock any readers while calling mdb_env_set_mapsize? Would it be hard to support resyncing read transactions and cursors after a map resize directly in the LMDB library?
Not a goal for LMDB. The documented way to use LMDB is to set the size once at env creation time.
Thank you and best regards, Ken
-----Ursprüngliche Nachricht----- Von: Howard Chu hyc@symas.com Gesendet: Montag, 31. Januar 2022 17:42 An: Ken Wenzel kenwenzel@gmx.net; openldap-technical@openldap.org Betreff: Re: [LMDB] mdb_env_set_mapsize and read transactions
Ken Wenzel wrote:
Hello,
I like to implement an autogrow functionality for LMDB.
The documentation for mdb_env_set_mapsize says that no transactions should
be active when using this function.
When looking at the code I can see that the function only checks if there
is an active WRITE transaction and in this case it returns an error.
Is it possible to reuse existing READ transactions or even associated
cursors after mdb_env_set_mapsize has been called?
Yes, you can reuse the existing structures as long as you have done txn_reset / txn_renew / cursor_renew. Naturally you must do txn_reset before the resize, otherwise the read txn is still active.
-- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
"Ken Wenzel" kenwenzel@gmx.net schrieb am 31.01.2022 um 08:16 in Nachricht
002001d81672$866fdb30$934f9190$@gmx.net:
Hello,
I like to implement an autogrow functionality for LMDB.
The documentation for mdb_env_set_mapsize says that no transactions should be active when using this function.
When looking at the code I can see that the function only checks if there is an active WRITE transaction and in this case it returns an error.
Probably a classic example of underspecification: The developers reserve the right to change details later. Obviously when mdb_ebv_set_mapsize does not relocate any blocks all tranbsactions should be able to continue, but as it seems, blocks may be reallocated during or after mdb_ebv_set_mapsize.
Regards, Ulrich
Is it possible to reuse existing READ transactions or even associated cursors after mdb_env_set_mapsize has been called?
Thank you and best regards,
Ken
openldap-technical@openldap.org
-
Howard Chu -
Ken Wenzel -
Ulrich Windl