Howard Chu wrote:
Howard Chu wrote:
I've tested again with libhoard 3.5.1, and it's actually superior to libumem for both speed and fragmentation. Here are some results against glibc 2.3.3, libhoard, and libumem:
I've retested with current HEAD, adding Google's tcmalloc to the mix.
The initial size is the size of the slapd process right after startup, with the process totally idle. The id2entry DB is 1.3GB with about 360,000 entries, BDB cache at 512M, entry cache at 70,000 entries, cachefree 7000. The subsequent statistics are the size of the slapd process after running a single search filtering on an unindexed attribute, basically spanning the entire DB. The entries range in size from a few K to a couple megabytes.
To be exact, there are 380836 entries in the DB, varying in size from 1K to 10MB.
The other thing to note is that for this run with libhoard, I doubled its SUPERBLOCK_SIZE from 64K to 128K. That's probably why it's less space-efficient here than umem, while it was more efficient in the August results. I guess I'll have to recompile that with the original size to see what difference that makes. For now I'd say my preference would be tcmalloc...
I reran the tests one more time with hoard in its default configuration (64K SUPERBLOCK_SIZE), in single-user mode, with no network connected. The results are here http://highlandsun.com/hyc/#Malloc
Kinda interesting - with hoard this shows us processing 23000 entries per second in the single-threaded case, vs only 3521 per second with glibc malloc. In the multi-threaded case we get peaks up to 51136 entries per second vs 19645/second with glibc. It's understandable that the multi-threaded case may get greater than 2x boost over the single-threaded numbers on this dual-core box, since some portion of entries will be in-cache and won't take any malloc hit at all. But still the difference against glibc is staggering. All of the other throughput benchmarks we've published so far have just used the default libc malloc and we're already faster than all other LDAP products. Re-running those with hoard or umem will be a real eye-opener.