Hello,
I've tried stress-testing LMDB in our use case and I've discovered something I consider a bug.
In our use case we need to keep inserting to the DB until it's full, flush it and repeat. We've designed the insert function to fail if there are not enough headroom pages for safe delete / DB drop. During the tests the DB works correctly for some amount of repeats, but then it suddenly refuses to insert anything. It seems LMDB doesn't clear or reuse all of its pages. I would like to get anyone's opinion on this.
I have attached the source file containing the test. The DB size is set to 20MB, the inserted values have by default 25kB. In the first run the DB accepts 365 inserts, which decreases over time and stabilizes at 280. After 18 repeats the DB does not accept any inserts at all: ok 1 - pass #1 fillup run (365 inserts) ok 2 - pass #2 fillup run (300 inserts) ok 3 - pass #3 fillup run (286 inserts) ok 4 - pass #4 fillup run (283 inserts) ok 5 - pass #5 fillup run (281 inserts) ok 6 - pass #6 fillup run (280 inserts) ok 7 - pass #7 fillup run (280 inserts) ok 8 - pass #8 fillup run (280 inserts) ok 9 - pass #9 fillup run (280 inserts) ok 10 - pass #10 fillup run (280 inserts) ok 11 - pass #11 fillup run (280 inserts) ok 12 - pass #12 fillup run (280 inserts) ok 13 - pass #13 fillup run (280 inserts) ok 14 - pass #14 fillup run (280 inserts) ok 15 - pass #15 fillup run (280 inserts) ok 16 - pass #16 fillup run (280 inserts) ok 17 - pass #17 fillup run (280 inserts) ok 18 - pass #18 fillup run (108 inserts) not ok 19 - pass #19 fillup run (0 inserts) not ok 20 - pass #20 fillup run (0 inserts)
The test is run 4 times with different approaches that we thought could have had some impact. These approaches combine opening the DB per insert or per single fillup, and reopening the DB just for the DB drop. In this case these options have no effect and the results are the same, but originally a unit test that this originates from had some differences so I kept them in.
Note that I also tried calling mdb_drop with "1" as the last parameter, but it had no effect on the result.
I am linking against the latest LMDB from sources.
I also tried adding a constant number of pages to the reserve before the insert function fails. This only changes the amount of inserted items but doesn't change the functionality.
If you compile the program you can supply it with an argument to change the default size of the item value size. Lower values break the DB sooner (20kB requires only 12 repeated fillups and drops).
If anyone has any idea what might be the issue, please let me know. I know deleting the DB from filesystem and creating a new one would work, but that's a hack, not a fix.
Thanks.
Dominik
Dominik Taborsky wrote:
Hello,
I've tried stress-testing LMDB in our use case and I've discovered something I consider a bug.
In our use case we need to keep inserting to the DB until it's full, flush it and repeat. We've designed the insert function to fail if there are not enough headroom pages for safe delete / DB drop. During the tests the DB works correctly for some amount of repeats, but then it suddenly refuses to insert anything. It seems LMDB doesn't clear or reuse all of its pages. I would like to get anyone's opinion on this.
I have attached the source file containing the test. The DB size is set to 20MB, the inserted values have by default 25kB. In the first run the DB accepts 365 inserts, which decreases over time and stabilizes at 280. After 18 repeats the DB does not accept any inserts at all: ok 1 - pass #1 fillup run (365 inserts) ok 2 - pass #2 fillup run (300 inserts) ok 3 - pass #3 fillup run (286 inserts) ok 4 - pass #4 fillup run (283 inserts) ok 5 - pass #5 fillup run (281 inserts) ok 6 - pass #6 fillup run (280 inserts) ok 7 - pass #7 fillup run (280 inserts) ok 8 - pass #8 fillup run (280 inserts) ok 9 - pass #9 fillup run (280 inserts) ok 10 - pass #10 fillup run (280 inserts) ok 11 - pass #11 fillup run (280 inserts) ok 12 - pass #12 fillup run (280 inserts) ok 13 - pass #13 fillup run (280 inserts) ok 14 - pass #14 fillup run (280 inserts) ok 15 - pass #15 fillup run (280 inserts) ok 16 - pass #16 fillup run (280 inserts) ok 17 - pass #17 fillup run (280 inserts) ok 18 - pass #18 fillup run (108 inserts) not ok 19 - pass #19 fillup run (0 inserts) not ok 20 - pass #20 fillup run (0 inserts)
The test is run 4 times with different approaches that we thought could have had some impact. These approaches combine opening the DB per insert or per single fillup, and reopening the DB just for the DB drop. In this case these options have no effect and the results are the same, but originally a unit test that this originates from had some differences so I kept them in.
Note that I also tried calling mdb_drop with "1" as the last parameter, but it had no effect on the result.
I am linking against the latest LMDB from sources.
I also tried adding a constant number of pages to the reserve before the insert function fails. This only changes the amount of inserted items but doesn't change the functionality.
If you compile the program you can supply it with an argument to change the default size of the item value size. Lower values break the DB sooner (20kB requires only 12 repeated fillups and drops).
If anyone has any idea what might be the issue, please let me know. I know deleting the DB from filesystem and creating a new one would work, but that's a hack, not a fix.
Your calculation of max_pages vs used_pages is totally bogus. There's nothing wrong with LMDB.
On 11/25/2015 02:27 PM, Howard Chu wrote:
Dominik Taborsky wrote:
Hello,
I've tried stress-testing LMDB in our use case and I've discovered something I consider a bug.
In our use case we need to keep inserting to the DB until it's full, flush it and repeat. We've designed the insert function to fail if there are not enough headroom pages for safe delete / DB drop. During the tests the DB works correctly for some amount of repeats, but then it suddenly refuses to insert anything. It seems LMDB doesn't clear or reuse all of its pages. I would like to get anyone's opinion on this.
I have attached the source file containing the test. The DB size is set to 20MB, the inserted values have by default 25kB. In the first run the DB accepts 365 inserts, which decreases over time and stabilizes at 280. After 18 repeats the DB does not accept any inserts at all: ok 1 - pass #1 fillup run (365 inserts) ok 2 - pass #2 fillup run (300 inserts) ok 3 - pass #3 fillup run (286 inserts) ok 4 - pass #4 fillup run (283 inserts) ok 5 - pass #5 fillup run (281 inserts) ok 6 - pass #6 fillup run (280 inserts) ok 7 - pass #7 fillup run (280 inserts) ok 8 - pass #8 fillup run (280 inserts) ok 9 - pass #9 fillup run (280 inserts) ok 10 - pass #10 fillup run (280 inserts) ok 11 - pass #11 fillup run (280 inserts) ok 12 - pass #12 fillup run (280 inserts) ok 13 - pass #13 fillup run (280 inserts) ok 14 - pass #14 fillup run (280 inserts) ok 15 - pass #15 fillup run (280 inserts) ok 16 - pass #16 fillup run (280 inserts) ok 17 - pass #17 fillup run (280 inserts) ok 18 - pass #18 fillup run (108 inserts) not ok 19 - pass #19 fillup run (0 inserts) not ok 20 - pass #20 fillup run (0 inserts)
The test is run 4 times with different approaches that we thought could have had some impact. These approaches combine opening the DB per insert or per single fillup, and reopening the DB just for the DB drop. In this case these options have no effect and the results are the same, but originally a unit test that this originates from had some differences so I kept them in.
Note that I also tried calling mdb_drop with "1" as the last parameter, but it had no effect on the result.
I am linking against the latest LMDB from sources.
I also tried adding a constant number of pages to the reserve before the insert function fails. This only changes the amount of inserted items but doesn't change the functionality.
If you compile the program you can supply it with an argument to change the default size of the item value size. Lower values break the DB sooner (20kB requires only 12 repeated fillups and drops).
If anyone has any idea what might be the issue, please let me know. I know deleting the DB from filesystem and creating a new one would work, but that's a hack, not a fix.
Your calculation of max_pages vs used_pages is totally bogus. There's nothing wrong with LMDB.
Thanks for checking it out, Howard, but can you please advise on how to compute a safe buffer then?
Dominik Taborsky wrote:
On 11/25/2015 02:27 PM, Howard Chu wrote:
Dominik Taborsky wrote:
If anyone has any idea what might be the issue, please let me know. I know deleting the DB from filesystem and creating a new one would work, but that's a hack, not a fix.
Your calculation of max_pages vs used_pages is totally bogus. There's nothing wrong with LMDB.
Thanks for checking it out, Howard, but can you please advise on how to compute a safe buffer then?
Read mdb_stat.c, follow how it computes the number of free pages.
Howard Chu wrote:
Dominik Taborsky wrote:
If anyone has any idea what might be the issue, please let me know. I know deleting the DB from filesystem and creating a new one would work, but that's a hack, not a fix.
Your calculation of max_pages vs used_pages is totally bogus. There's nothing wrong with LMDB.
Also, you never reset the "k" counter, which is why you eventually get to zero inserts per attempt, regardless of the page usage computations.
openldap-technical@openldap.org
-
Dominik Taborsky -
Howard Chu