openldap-bugs July 2024

openldap-bugs@openldap.org

1 participants
3 discussions

[Issue 10236] New: fragmentation makes mdb_page_alloc slow
by openldap-its＠openldap.org 02 Jul '24

02 Jul '24

https://bugs.openldap.org/show_bug.cgi?id=10236 Issue ID: 10236 Summary: fragmentation makes mdb_page_alloc slow Product: LMDB Version: 0.9.31 Hardware: All OS: All Status: UNCONFIRMED Keywords: needs_review Severity: normal Priority: --- Component: liblmdb Assignee: bugs(a)openldap.org Reporter: aalekseyev(a)janestreet.com Target Milestone: --- Created attachment 1022 --> https://bugs.openldap.org/attachment.cgi?id=1022&action=edit patch is relative to LMDB_0.9.31 It's a known problem that mdb_page_alloc can be slow when the free list is large and fragmented. [1] [2] [3] I'm not sure it's known *how* slow it can be. In our workload we saw a fragmented freelist leading to a pathological O(n^2) behavior. To handle a multi-page allocation we iterate loading chunks of the free list one by one, and at every iteration we do O(n) work to check if the allocation can succeed. Even small-ish allocations (tens of pages) are repeatedly hitting this edge case, with free list growing to ~1000000, and the outer loop taking ~2000 iterations (10^9 worth of work in total, just to allocate a few pages). Even though I'm sure there are ways to avoid hitting this pathological scenario so much (avoid values larger than 4k, or fix whatever causes fragmentation), it seems unacceptable to have a performance cliff this bad. I made a patch to make the allocation take ~O(n*log(n)), by loading and merging multiple chunks at once instead of doing it one-by-one. I'd appreciate it if someone could review the patch (attached), improve it, and/or come up with an alternative fix. The code in `midl.c` is kinda meme-y, including a contribution from GPT-4o, but it performs well enough to speed up our pathological workload by ~20x (which is still ~3x away from the non-fragmented case). Anyway, the main thing that warrants scrutiny is the change in `mdb.c`: I understand very little about lmdb internals and I worry that loading multiple pages at once instead of one-by-one might break something. [1] issue #8664 [2] https://lists.openldap.org/hyperkitty/list/openldap-bugs@openldap.org/threa… [3] https://lists.openldap.org/hyperkitty/list/openldap-technical@openldap.org/… -- You are receiving this mail because: You are on the CC list for the issue.

1 0

[Issue 10233] New: wrong idl intersection
by openldap-its＠openldap.org 02 Jul '24

02 Jul '24

https://bugs.openldap.org/show_bug.cgi?id=10233 Issue ID: 10233 Summary: wrong idl intersection Product: OpenLDAP Version: 2.6.8 Hardware: All OS: All Status: UNCONFIRMED Keywords: needs_review Severity: normal Priority: --- Component: backends Assignee: bugs(a)openldap.org Reporter: hamano(a)osstech.co.jp Target Milestone: --- The `mdb_idl_intersection()` and `wt_idl_intersection()` functions derived from back-bdb return wrong results. expect: [1, 3] ∩ [2] = [] actual: [1, 3] ∩ [2] = [2] -- You are receiving this mail because: You are on the CC list for the issue.

1 1

[Issue 10234] New: syncrepl does not reset the retrynum
by openldap-its＠openldap.org 02 Jul '24

02 Jul '24

https://bugs.openldap.org/show_bug.cgi?id=10234 Issue ID: 10234 Summary: syncrepl does not reset the retrynum Product: OpenLDAP Version: 2.6.8 Hardware: All OS: All Status: UNCONFIRMED Keywords: needs_review Severity: normal Priority: --- Component: slapd Assignee: bugs(a)openldap.org Reporter: hamano(a)osstech.co.jp Target Milestone: --- ``` syncrepl retry="5 10 30 +" ``` When replication fails with the above settings, syncrepl retries "10 times at 5 second intervals". Then, the retry count should be reset on the next replication failure. In actual, it does not reset. The behavior is as follows: ``` (first time replication failure) do_syncrepl: rid=001 rc -1 retrying (9 retries left) do_syncrepl: rid=001 rc -1 retrying (8 retries left) (resume replication) (second time replication failure) do_syncrepl: rid=001 rc -1 retrying (7 retries left) do_syncrepl: rid=001 rc -1 retrying (6 retries left) ``` -- You are receiving this mail because: You are on the CC list for the issue.

1 1

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs July 2024