Hallvard B Furuseth wrote:
Howard Chu wrote:
Hallvard B Furuseth wrote:
Then I'll at least reduce pool_wrapper a bit.
Done. Noticed a few other things underway:
Seems to have helped with 16 threads on the T5120, back-null peak went from 17,900 auths/sec at 32 connections to 18,500 auths/sec at 96 connections.
Not much improvement with 24 threads, from a peak of 17,500 at 32 connections to a peak of 17,000 at 60 connections. So the overall peak is a little slower, but it can handle a heavier load before maxing out.
I haven't thought about any of the following suggestions yet.
slapd does not set ltp_max_pending, we could drop it or add some slapd.conf option to set it. Or make it ltp_max_tasks instead, then the test in _submit() can be moved inside the branch which mallocs a new task. (The freelist size will enforce the limit.)
It's slightly less work to maintain number of pending + active tasks than number of pending tasks, or for that matter a count which reduces the "paused or create new thread?" test to a single compare, but if _submit() grows smarter it may need to undo that. (That's the change I thought was best to delay for now.)
Unless pool_purgekey() gets called from the main thread and expects to update other threads (I have no idea), we can kill thread_keys[] now and support multiple pools. Is that useful?
Replace thread_keys[] with: a circular list with prev/next pointers in the thread contexts, a pool->ltp_ctx_list which points to one of them, and a pool_context() call in pool_purgekey() to get at the list.
(...) a single shared resource like this [ltp_mutex] is just a really bad idea.
True enough. Still, slapd has a lot of mutexes. Should perhaps check if this one stands out before rearraning scheduling around it.
Using back-null makes this fairly clear. There are no globally shared mutexes in the connection manager at all, so this is the only remaining culprit.
After grepping around a bit I'm not sure what you mean. There are several other singled shared mutexes which seem to be frequently used. I'm far from sure of all of these, but here are some in slapd:
attr.c: attr_mutex, for attr_alloc() and attrs_alloc().
Doesn't affect back-null.
daemon.c: slap_daemon.sd_mutex via slapd_<set/clr>_<read/write>() and in slapd_daemon_task().
True.
daemon.c: slapd_rq.rq_mutex?
Doesn't affect my current test configuration.
operation.c: slap_op_mutex, for slap_op_time().
True.
gmtime_mutex if !HAVE_GMTIME_R, per connection in back-monitor at least. slap_get_csn() does not #ifndef HAVE_GMTIME_R, should it?
No, read the comment there.