I have a patch that can speed up parallel ldap add/delete a lot; in my testing about 2.5X.
The idea is described here: the mdb backend does not really support parallelism in rw
transactions, so add/delete are serialized by a mutex inside the mdb backend as small
transactions. In my patch, all add/delete are put in a queue and executed in a dedicated
worker thread. It may seem pointless at first, but by doing them all in the same thread,
now I can merge concurrent ops from different clients into larger transactions, and reduce
the number of expensive txn_commit calls (fsyncs).
my test results are ( your mileage may vary)
10 thread parallel add:
1, unpatched slapd + default config, ~ 2800 op/s (performance stable over time)
2, unpatched slapd + dbnosync ~9000 op/s (lots of fluctuation due to background dirty page
3, unpatched slapd + dpnosync + checkpoint every minute ~8000 op/s, with reduced
4, patched slapd ~7300 op/s (fluctuation nearly gone)
2 is not a recommended config, because crashing or power lost will cause massive data
lost. 3 is better, but still up to 1 minutes of data can be lost at crash. With this
patch, I can achieve >90% of the performance of 3, with data durability as good as 1;
or 2.5X performance of 1 with same data durability guaranty.
The patch still need some tidy up; is this email list the place to send patches?