Howard Chu wrote:
Ultimately after several more repeated runs, AD peaked at 4800 auths/sec (a bit faster than the 4400 we saw before, apparently it took a while for their caches to get fully primed). We then removed AD and installed ADAM. The import went a bit faster this time, thankfully:
time ldifde.exe -i -h -f examp3.ldif -s localhost -q 8 152.62u 99.32s 27:33.84 15.2%
Using 8 threads, 27-1/2 minutes. The peak authentication rate we got was just under 5500 auths/sec using 52 client threads. At that load a single auth took an average of 9.5 milliseconds, up from 1.5ms at 4 client threads. (slapd's average latency is only 0.5ms at 8 clients, 2.2ms at 52 clients.)
A slapadd -q for back-hdb on 1M entries took only 5 minutes: time ../servers/slapd/slapd -Ta -f slapd.conf.slam -q -l ~hyc/ldif/example.ldif.1mil
real 5m2.669s user 3m59.224s sys 1m55.482s
(Using BDB 4.6.21 and libtcmalloc)
An ldapadd (with back-bdb) of those 1M entries took 19 minutes: time ../clients/tools/ldapmodify -a -x -D dc=example,dc=com -w secret -f ~hyc/ldif/example.ldif.1mil
real 19m3.044s user 1m12.435s sys 0m38.401s
So, even restricted to a single thread, we still import entries faster than AD or ADAM with 8 threads. Online or offline, they don't even come close. (Which should be no surprise...)
It'll be even more fun when we come back and run the tests on a 5 million entry DB. It's pretty clear from looking at their memory usage that they won't be able to cache all of that in the 16GB of RAM on this box, while OpenLDAP will. As such, their throughput rate will just reflect the speed of our disks, while slapd will still be running at full RAM speed.
The time to slapadd 5 million entries for back-hdb was 29 minutes: time ../servers/slapd/slapd -Ta -f slapd.conf.5M -l ~hyc/5Mssha.ldif -q
real 29m15.828s user 20m22.110s sys 6m56.433s
For ADAM it was about 3 hours:
time ldifde.exe -i -h -f 5M.ldif -s localhost -q 8 743.57u 475.56s 2:59:44.78 11.3%
The authrate for back-hdb was again 25,700/sec with SSHA passwords, pretty much the same as for 1 million entries. The authrate for ADAM was 2359/sec.
It's also interesting to note the database and process sizes. The 5M ldif was about 2.9GB; the hdb database was about 6.4GB. The ADAM DB was about 22GB.
The slapd process size was about 8GB. The ADAM process size was about 14GB. We could easily handle a 10 million entry database on this machine and still be running at full cache speed; ADAM is already disk limited here.
This weekend we'll try loading 5M into AD. I figure we'll start up the import at the end of the day today and come back to look at it again in a couple of days...