https://bugs.openldap.org/show_bug.cgi?id=10406
Issue ID: 10406 Summary: Write regression since 0.9.19 Product: LMDB Version: 0.9.33 Hardware: All OS: All Status: UNCONFIRMED Keywords: needs_review Severity: normal Priority: --- Component: liblmdb Assignee: bugs@openldap.org Reporter: ben.alex@acegi.com.au Target Milestone: ---
I maintain LmdbJava, a Java wrapper for LMDB. We have an extensive benchmark suite covering read, cursor, and write pathways, with isolated testing of LMDB versions from 0.9.17 through 0.9.33.
Results: https://lmdb-benchmark.lmdbjava.org/
============================================== SUMMARY ==============================================
Read performance: 0.9.33 consistently outperforms 0.9.17 across all read benchmarks.
Write performance: Consistent regression since 0.9.19:
---------------------------------- Tag ms/op vs 0.9.17 ---------------------------------- LMDB_0.9.17 75.790 baseline LMDB_0.9.18 74.909 -1.2% LMDB_0.9.19 81.404 +7.4% LMDB_0.9.20 83.459 +10.1% LMDB_0.9.21 83.237 +9.8% LMDB_0.9.22 83.027 +9.5% LMDB_0.9.23 82.870 +9.3% LMDB_0.9.24 82.952 +9.4% LMDB_0.9.27 82.989 +9.5% LMDB_0.9.28 83.025 +9.5% LMDB_0.9.29 83.063 +9.6% LMDB_0.9.30 83.237 +9.8% LMDB_0.9.31 88.419 +16.7% LMDB_0.9.33 83.304 +9.9%
Excluding 0.9.31 as an outlier, write performance degraded by approximately 9% from 0.9.20 and has remained at that level through 0.9.33.
============================================== BENCHMARK DETAILS ==============================================
The write benchmark sequentially inserts 1,000,000 entries (100-byte values, 4-byte integer keys) into a database opened with MDB_INTEGERKEY, using a single cursor with MDB_APPEND flag. The benchmark includes 2 warmup iterations and 3 measurement iterations of 120 seconds each.
============================================== ANALYSIS ==============================================
Reviewing the commit history, the regressions appear linked to correctness fixes: - 0.9.19: ITS#8406 (xcursor fixup after cursor_del) - 0.9.20: ITS#8557 (cursor_last and C_EOF handling)
Both issues added cursor state management overhead in write-heavy code paths.
============================================== REQUEST ==============================================
Could you please reproduce this regression using a native C benchmark to confirm it's not JNI-specific? If confirmed, this may be a necessary performance/correctness tradeoff given that most LMDB workloads are read-heavy.
https://bugs.openldap.org/show_bug.cgi?id=10406
Quanah Gibson-Mount quanah@openldap.org changed:
What |Removed |Added ---------------------------------------------------------------------------- Keywords|needs_review | Assignee|bugs@openldap.org |hyc@openldap.org
https://bugs.openldap.org/show_bug.cgi?id=10406
--- Comment #1 from Howard Chu hyc@openldap.org --- Created attachment 1091 --> https://bugs.openldap.org/attachment.cgi?id=1091&action=edit Benchmark output
I'm not really seeing any performance regression here.
https://bugs.openldap.org/show_bug.cgi?id=10406
--- Comment #2 from Howard Chu hyc@openldap.org --- Created attachment 1092 --> https://bugs.openldap.org/attachment.cgi?id=1092&action=edit Benchmark code
Here's the C source I used. I don't know how comparable it is to your test.
https://bugs.openldap.org/show_bug.cgi?id=10406
--- Comment #3 from Ben Alex ben.alex@acegi.com.au --- Thanks Howard. I'll give it a run with your code and report back.
https://bugs.openldap.org/show_bug.cgi?id=10406
--- Comment #4 from Ben Alex ben.alex@acegi.com.au --- Thanks for your benchmark code and giving me a few days to try it.
Your original code (using mdb_put) showed no regression, which initially puzzled me. However, I noticed our Java benchmark uses cursor operations, whereas your code uses mdb_put directly. When I modified your benchmark to use mdb_cursor_put instead (the only change), the regression appeared:
Benchmark Results (ms per operation):
Version C (ms) Java (ms) C vs 0.9.17 Java vs 0.9.17 =============================================================================== 0.9.17 56 75.79 baseline baseline 0.9.18 57 74.91 +0.7% -1.2% 0.9.19 58 81.40 +3.6% +7.4% 0.9.20 66 83.46 +16.7% +10.1% 0.9.21 63 83.24 +11.3% +9.8% 0.9.22 66 83.03 +17.8% +9.5% 0.9.23 63 82.87 +12.3% +9.3% 0.9.24 64 82.95 +13.7% +9.5% 0.9.27 64 82.99 +14.2% +9.5% 0.9.28 64 83.02 +12.7% +9.5% 0.9.29 62 83.06 +10.6% +9.6% 0.9.30 66 83.24 +16.9% +9.8% 0.9.31 64 88.42 +13.4% +16.7% 0.9.33 66 83.30 +17.5% +9.9%
The regression patterns match closely between C and Java when cursor operations are used.
Modified benchmark code is attached. The only changes from your original are: - Added MDB_cursor *cursor = NULL; - Replaced mdb_put(txn, dbi, &key, &data, MDB_APPEND) with cursor equivalent - Open/close cursor around the batch (maintaining your 1,000 entry transaction pattern)
Thank you for your help in tracking this down.
https://bugs.openldap.org/show_bug.cgi?id=10406
--- Comment #5 from Ben Alex ben.alex@acegi.com.au --- Created attachment 1095 --> https://bugs.openldap.org/attachment.cgi?id=1095&action=edit Benchmark with mdb_cursor_put
https://bugs.openldap.org/show_bug.cgi?id=10406
--- Comment #6 from Howard Chu hyc@openldap.org --- Running valgrind/callgrind on your code, I see that extra time is spent in mdb_cursor_last(). Because of ITS#8557 we no longer trust that the cursor is actually on the last page of the DB and so we explicitly move it there. This was changed in c97f4ed1aef858ba352a7bfcc67532c774fd434f.
You're welcome to suggest an alternate fix.
https://bugs.openldap.org/show_bug.cgi?id=10406
--- Comment #7 from Ben Alex ben.alex@acegi.com.au --- Thanks Howard for looking into this further.
It appears we've isolated it to a specific write path (mdb_cursor_put) and there is no regression on the alternate write path (mdb_put). As such I am happy if you wish to close this ticket. I'll just adjust our benchmark to use mdb_put.
https://bugs.openldap.org/show_bug.cgi?id=10406
Howard Chu hyc@openldap.org changed:
What |Removed |Added ---------------------------------------------------------------------------- Resolution|--- |INVALID Status|UNCONFIRMED |RESOLVED
https://bugs.openldap.org/show_bug.cgi?id=10406
Quanah Gibson-Mount quanah@openldap.org changed:
What |Removed |Added ---------------------------------------------------------------------------- Status|RESOLVED |VERIFIED
-
openldap-its＠openldap.org