https://bugs.openldap.org/show_bug.cgi?id=9524
Issue ID: 9524 Summary: Removing last entry in database triggers MDB_PROBLEM Product: LMDB Version: unspecified Hardware: All OS: All Status: UNCONFIRMED Severity: normal Priority: --- Component: liblmdb Assignee: bugs@openldap.org Reporter: kriszyp@gmail.com Target Milestone: ---
With a fresh database, if you have have an open read transaction, and you create a new entry in a write transaction and commit it, and then create a new transaction and delete that entry, when you commit, it will return an MDB_PROBLEM from approximately line 4408 from the mt_loose_count/dirty_list check. This seems to occur on mdb.master3, but not mdb.master. Here is a minimal example/test-case of how to reproduce:
MDB_env* env; mdb_env_create(&env); int rc, flags = 0; mdb_env_open(env, "test", flags, 0664); MDB_txn* readonly_txn; mdb_txn_begin(env, nullptr, MDB_RDONLY, &readonly_txn); MDB_txn* txn; MDB_dbi dbi; mdb_txn_begin(env, nullptr, 0, &txn); mdb_dbi_open(txn, nullptr, MDB_CREATE, &dbi); MDB_val val; val.mv_data = (void*) "test"; val.mv_size = 4; mdb_put(txn, dbi, &val, &val, 0); mdb_txn_commit(txn); mdb_txn_begin(env, nullptr, 0, &txn); mdb_del(txn, dbi, &val, nullptr); rc = mdb_txn_commit(txn); // this returns MDB_PROBLEM
(let me know if this should be submitted differently)
https://bugs.openldap.org/show_bug.cgi?id=9524
kriszyp@gmail.com kriszyp@gmail.com changed:
What |Removed |Added ---------------------------------------------------------------------------- Summary|Removing last entry in |Removing last entry in |database triggers |database can trigger |MDB_PROBLEM |MDB_PROBLEM
https://bugs.openldap.org/show_bug.cgi?id=9524
--- Comment #1 from kriszyp@gmail.com kriszyp@gmail.com --- Created attachment 824 --> https://bugs.openldap.org/attachment.cgi?id=824&action=edit Fix to tolerate missing loose count that is greater than dirty list
This seems to fix the issue. It no longer triggers the MDB_PROBLEM error in the case of removing the last entry. I assume the hazard of this is missing a case where page leaks occur, but I have tried repeated removal of the last entry in a database and it does not seem to cause any continual page leakage (maybe there is a single page that gets leaked on the first time).
https://bugs.openldap.org/show_bug.cgi?id=9524
--- Comment #2 from kriszyp@gmail.com kriszyp@gmail.com --- For reference, I think this is also related to ITS#9376 and the work that was done to assert that the transaction state was correct.
https://bugs.openldap.org/show_bug.cgi?id=9524
Howard Chu hyc@openldap.org changed:
What |Removed |Added ---------------------------------------------------------------------------- CC| |info@parlepeuple.fr
--- Comment #3 from Howard Chu hyc@openldap.org --- *** Issue 9911 has been marked as a duplicate of this issue. ***
https://bugs.openldap.org/show_bug.cgi?id=9524
Howard Chu hyc@openldap.org changed:
What |Removed |Added ---------------------------------------------------------------------------- Ever confirmed|0 |1 Status|UNCONFIRMED |CONFIRMED
--- Comment #4 from Howard Chu hyc@openldap.org --- Stepping thru, there is an actual bug here that your patch is masking. Page #3 is loose and therefore included in the freelist, but also gets flushed as a dirty page on the dirty list. Since it's a freelist page it should not be getting flushed in the dirty list.
https://bugs.openldap.org/show_bug.cgi?id=9524
--- Comment #5 from NikoPLP info@parlepeuple.fr --- (In reply to Howard Chu from comment #4)
Stepping thru, there is an actual bug here that your patch is masking.
Is the bug you mention Howard, related to this commit ? https://git.openldap.org/openldap/openldap/-/commit/12ee1a2d7104616ccb812db5...
https://bugs.openldap.org/show_bug.cgi?id=9524
--- Comment #6 from Howard Chu hyc@openldap.org --- Seems likely, yes.
https://bugs.openldap.org/show_bug.cgi?id=9524
--- Comment #7 from Howard Chu hyc@openldap.org --- Actually no, and reverting that commit doesn't improve anything.
Here's a run with mdb_debug enabled. It shows that the loose page (#3) actually doesn't get flushed in the dirty list, so all of that is actually OK. On a previous run I saw the page being committed, which is a bit strange.
gdb its9524 GNU gdb (GDB) 12.0.50.20220202-git Copyright (C) 2022 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-pc-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: https://www.gnu.org/software/gdb/bugs/. Find the GDB manual and other documentation resources online at: http://www.gnu.org/software/gdb/documentation/.
For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from its9524... (gdb) br main Breakpoint 1 at 0x2995: file its9524.c, line 5. (gdb) run Starting program: /home/hyc/OD/mdb/libraries/liblmdb/its9524 [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Breakpoint 1, main () at its9524.c:5 5 main() { (gdb) set mdb_debug=1 (gdb) c Continuing. mdb_env_open2:5274 new mdbenv mdb_env_init_meta:4563 writing new meta page mdb_env_open2:5364 opened database version 2, pagesize 4096 mdb_env_open2:5366 using meta page 0 mdb_env_open2:5367 depth: 0 mdb_env_open2:5368 entries: 0 mdb_env_open2:5369 branch pages: 0 mdb_env_open2:5370 leaf pages: 0 mdb_env_open2:5371 overflow pages: 0 mdb_env_open2:5372 root: 18446744073709551615 mdb_env_open:6083 opened dbenv 0x55555557f2a0 mdb_txn_begin:3494 begin txn 0r 0x555555580630 on mdbenv 0x55555557f2a0, root page 18446744073709551615 mdb_txn_begin:3494 begin txn 1w 0x555555580500 on mdbenv 0x55555557f2a0, root page 18446744073709551615 mdb_cursor_put:8276 ==> put db 1 key [74657374], size 4, data size 4 mdb_cursor_put:8337 allocating new root leaf page mdb_page_new:8871 allocated new mpage 2, page size 4096 mdb_cursor_push:6538 pushing page 2 on db 1 cursor 0x7fffffffe0d0 mdb_cursor_put:8359 inserting key at index 0 mdb_node_add:8976 add to leaf page 2 index 0, data size 4 key size 4 [74657374] mdb_txn_commit:4362 committing txn 1 0x555555580500 on mdbenv 0x55555557f2a0, root page 2 mdb_page_search:7289 tree is empty mdb_page_flush:4123 committing page 2 mdb_env_write_meta:4632 writing meta page 1 for root page 2 mdb_txn_end:3561 committed txn 1w 0x555555580500 on mdbenv 0x55555557f2a0, root page 2 mdb_txn_begin:3494 begin txn 2w 0x555555580500 on mdbenv 0x55555557f2a0, root page 2 mdb_del0:10271 ====> delete db 1 key [74657374] mdb_page_search:7315 db 1 root page 2 has flags 0x2 mdb_page_search_root:7199 found leaf page 2 for key [74657374] mdb_node_search:6424 searching 1 keys in leaf page 2 mdb_node_search:6469 found leaf index 0 [74657374], rc = 0 mdb_cursor_set:7909 ==> cursor placed on key [74657374] mdb_page_touch:2933 touched db 1 page 2 -> 3 mdb_node_del:9101 delete node 0 on leaf page 3 mdb_rebalance:9969 rebalancing leaf page 3 (has 0 keys, 0.0% full) mdb_rebalance:9988 tree is completely empty mdb_page_loose:2322 loosen db 1 page 3 mdb_txn_commit:4362 committing txn 2 0x555555580500 on mdbenv 0x55555557f2a0, root page 18446744073709551615 mdb_page_search:7289 tree is empty mdb_cursor_put:8276 ==> put db 0 key [0200000000000000], size 8, data size 24 mdb_cursor_put:8337 allocating new root leaf page mdb_page_new:8871 allocated new mpage 4, page size 4096 mdb_cursor_push:6538 pushing page 4 on db 0 cursor 0x7fffffffe080 mdb_cursor_put:8359 inserting key at index 0 mdb_node_add:8976 add to leaf page 4 index 0, data size 24 key size 8 [0200000000000000] mdb_freelist_save:3807 IDL write txn 2 root 4 num 2 mdb_freelist_save:3810 IDL 2 mdb_freelist_save:3810 IDL 3 mdb_page_search:7315 db 0 root page 4 has flags 0x2 mdb_page_search_root:7199 found leaf page 4 for key [null] mdb_cursor_next:7586 cursor_next: top page is 4 in cursor 0x7fffffffde90 mdb_cursor_next:7594 =====> move to next sibling page mdb_page_flush:4123 committing page 4 mdb_txn_end:3561 abort txn 2w 0x555555580500 on mdbenv 0x55555557f2a0, root page 18446744073709551615 [Inferior 1 (process 199768) exited normally] (gdb)
https://bugs.openldap.org/show_bug.cgi?id=9524
Howard Chu hyc@openldap.org changed:
What |Removed |Added ---------------------------------------------------------------------------- Status|CONFIRMED |RESOLVED Resolution|--- |FIXED
--- Comment #8 from Howard Chu hyc@openldap.org --- Ah never mind, the extra commit I saw was from a second run on the same test DB.
Fixed now in git mdb.master3 97e7e9ac795f77a1443cac2c534037dd7feb5dc5
https://bugs.openldap.org/show_bug.cgi?id=9524
--- Comment #9 from NikoPLP info@parlepeuple.fr --- Thank you very much. I confirm that this fix solves my issue.
https://bugs.openldap.org/show_bug.cgi?id=9524
Quanah Gibson-Mount quanah@openldap.org changed:
What |Removed |Added ---------------------------------------------------------------------------- Status|RESOLVED |VERIFIED
-
openldap-its＠openldap.org