Aaron Richton wrote:
On Tue, 11 May 2010, ST Wong (ITSC) wrote:
I currently set thread in slapd.conf to 32 (default x 2). While I think shouldn't modify SLAP_MAX_WORKER_THREADS, I tried to increase no. of threads, at the cost of read performance. Besides, after increasing thread to over 32, say 100, I got warning of "threads larger than twice the default".
Would like to know the recommended way to tune the system in our case. Would anyone pls help?
"Recommended way to tune the system" is quite the slippery slope, but here goes nothing...
Note that the "too many executing" message refers to the number of operations executing on a single connection. The first thing to do is find out why this single connection is so busy. Well behaved clients would not be spitting out hundreds of operations over a single connection all at once.
It sounds like you're focusing primarily on parallelism, which certainly has its merits. But I might sooner look at latency per operation. For example, with loglevel stats, you should have visiblity of client operations that are taking too long (i.e. delay until RESULT lines). Perhaps an application started searching a field that isn't properly indexed, for example.
Also, you don't mention results from looking for obvious bottlenecks (spindles, bus, cpu, etc.). For example, if you're waiting on hard drives, adding more threads is unlikely to help and in many cases would hurt. The likely solution there, short of a hardware upgrade, is adding more replicas (which also has benefits for redundancy/HA concerns, in addition to alleviating hardware limitations).
Finally, you mention that you're using BerkeleyDB. You should post the size of your working set and your DB_CONFIG parameters. A 4-core system sounds fairly modern, it generally shouldn't be going to disk...if DB_CONFIG is not tuned accordingly, that would drastically affect performance. You can get some ideas for DB_CONFIG tuning in the FAQ-O-Matic and list archives.