David Boreham wrote:
Gavin Henry wrote:
> Thanks for all this Howard. It certainly makes it clear where OpenLDAP
> lies in the LDAP world (looking down from the top and around at
> everyone else ;-) ).
Hmm...not so sure about the 'everyone' part because
I added this feature (parallel import with correctly implemented
partent/child entry interlocking) to the product that is now SunDS and
FedoraDS
in 1999.
With all due respect David, you've done some nice work on those products but
OpenLDAP is still significantly faster than either SunDS or FedoraDS, both in
importing data and in the authentication rates.
IIRC we had this conversation about parallel import before on this list. I had
tried parallel entry importing before and gave up on it because it showed no
benefit on my tests. Then based on your hints I tested a variety of other
approaches. What I eventually settled on for OL 2.3 - serial entry import with
parallel indexing - has been the fastest approach in my tests.
One obvious reason is that no entry locking is required because only a single
thread ever operates on any given DB. With parallel writes of entries to the
same DB, you still need full locking to prevent data corruption. Another
reason is that BerkeleyDB's B-trees are optimized for sequential writes;
parallelizing the entry writes defeats that optimization because it partially
randomizes the I/Os. (I.e., it requires seeks instead of just sequential access.)
It's too bad that SunDS still disallows publishing benchmark results. The
current license terms just tells the world they have something to hide.
--
-- Howard Chu
Chief Architect, Symas Corp.
http://www.symas.com
Director, Highland Sun
http://highlandsun.com/hyc/
Chief Architect, OpenLDAP
http://www.openldap.org/project/