https://bugs.openldap.org/show_bug.cgi?id=10465
Issue ID: 10465 Summary: accesslog_purge mdb scheduling is unfair Product: OpenLDAP Version: unspecified Hardware: All OS: All Status: UNCONFIRMED Keywords: needs_review Severity: normal Priority: --- Component: overlays Assignee: bugs@openldap.org Reporter: ondra@mistotebe.net Target Milestone: ---
On most systems, a running accesslog purge task on a back-mdb data will starve other modifications for oxygen:
It issues deletes to the DB with a very narrow gap where it might be usefully preempted, on a multi-CPU systems it usually manages to reacquire the TXN mutex before another waiting thread can even wake up. Meanwhile a live modification has to issue a modification to the original DB, then get to the point where it waits to acquire a write TXN on the accesslog DB, all the while holding li_rmutex preventing any other modification from getting under way. So even one wins the mutex, another one is unlikely to do so straight away.
This means that write latencies can and usually do shoot through the roof until the purge task finishes. If 1% of time is spent on write ops on average and an accesslog purge is scheduled to run every 30 mins, that is ~18 seconds of work for the purge task to remove those entries. On a busy system, write latencies of above 1 second (which this can easily spiral to) might be unacceptable, especially when ppolicy/lastbind etc. are configured.
https://bugs.openldap.org/show_bug.cgi?id=10465
--- Comment #1 from Ondřej Kuzník ondra@mistotebe.net --- Considering our options, let's keep to Linux because in some ways that's where this is most pronounced and start with the least intrusive upwards: 1. changing nice() levels increases the chances of being rescheduled but in practice we just woke up after being suspended for I/O (SYNC for lmdb TXN commit), so we don't get rescheduled or the stuck thread is not likely be parked on our CPU? At least that's my assessment from the tests I've ran.
2. lowering thread priority is not a thing on Linux, SCHED_OTHER is the default and lowest apart from SCHED_IDLE which makes no difference in practice
3. We could bump the priority of all other threads by default, switching the purge task to SCHED_OTHER and back on completion
4. We let the purge task sleep for a little every so often, sacrificing throughput for (guaranteed) responsiveness.
5. We could make changes to back-mdb to detect contention of the write txn mutex and enforce fairness on that level. This might be invasive and costly.
1, 2 and 3 have a massive drawback, changing the scheduler policy (SCHED_RR/FIFO or even SCHED_IDLE) seems to be a privileged operation so is bumping nice() levels back to original.
None of them are very appealing. Of them just pausing for a little might just be the least bad? Take this scheme for an idea:
If every N deletes we measure how long it took to process them and wait (sleep) for 1/N of that afterwards, we lose a fraction (1/(N+1)) of purge throughput while bounding the latency impact to a factor of N at most.
https://bugs.openldap.org/show_bug.cgi?id=10465
Quanah Gibson-Mount quanah@openldap.org changed:
What |Removed |Added ---------------------------------------------------------------------------- Assignee|bugs@openldap.org |ondra@mistotebe.net Target Milestone|--- |2.7.0 Keywords|needs_review |
https://bugs.openldap.org/show_bug.cgi?id=10465
--- Comment #2 from Ondřej Kuzník ondra@mistotebe.net --- https://git.openldap.org/openldap/openldap/-/merge_requests/844
https://bugs.openldap.org/show_bug.cgi?id=10465
Quanah Gibson-Mount quanah@openldap.org changed:
What |Removed |Added ---------------------------------------------------------------------------- Status|UNCONFIRMED |RESOLVED Resolution|--- |TEST
--- Comment #3 from Quanah Gibson-Mount quanah@openldap.org --- • a6be90c6 by Ondřej Kuzník at 2026-03-23T19:34:31+00:00 ITS#10465 Suspend purge task sometimes to improve fairness
https://bugs.openldap.org/show_bug.cgi?id=10465
Quanah Gibson-Mount quanah@openldap.org changed:
What |Removed |Added ---------------------------------------------------------------------------- Resolution|TEST |FIXED Status|RESOLVED |VERIFIED
-
openldap-its＠openldap.org