Re: thread pools, performance

David Boreham wrote: With all due respect David, you've done some nice work on those products but OpenLDAP is still significantly faster than either SunDS or FedoraDS, both in importing data and in the authentication rates. IIRC we had this conversation about parallel import before on this list. I had tried parallel entry importing before and gave up on it because it showed no benefit on my tests. Then based on your hints I tested a variety of other approaches. What I eventually settled on for OL 2.3 - serial entry import with parallel indexing - has been the fastest approach in my tests. One obvious reason is that no entry locking is required because only a single thread ever operates on any given DB. With parallel writes of entries to the same DB, you still need full locking to prevent data corruption. Another reason is that BerkeleyDB's B-trees are optimized for sequential writes; parallelizing the entry writes defeats that optimization because it partially randomizes the I/Os. (I.e., it requires seeks instead of just sequential access.) It's too bad that SunDS still disallows publishing benchmark results. The current license terms just tells the world they have something to hide. -- -- Howard Chu Chief Architect, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/