Howard Chu wrote:
Howard Chu wrote:
> It seems to me this can only be a viable mode of operation if you're always
> going to run asynch and don't care much about transaction durability or DB
> recoverability. Running in this mode offers absolutely zero crash resistance;
> the entire DB will almost always be irreparably damaged after a system crash.
>
> Would you run like that, if it offered you the potential of maybe 10x faster
> write performance? (It could be useful for slapadd -q, certainly.)
OK, 10x speedup was far too optimistic. Quickly cobbling the changes together,
it looks more like about a 70% speedup.
slapadd -q with 5 million entries took 24m16s as originally reported at
LDAPCon last year.
With current mdb.master it takes 22m24s:
real 22m23.984s
user 26m1.658s
sys 8m17.415s
With the writable mmap and no msyncs it took 13m17s.
real 13m17.225s
user 22m15.511s
sys 1m12.533s
This code is currently available on the map2 branch of my git repo on
ada.openldap.org. I'll clean it up a bit further then push it to mdb.master
after some more testing.
The speedup seems to be proportional to the number of indices that are defined
on the database. With Quanah's torture-test LDIF (~6 million entries, 4.9GB),
there's only a small difference between the two when no indices are defined:
Original mdb.master:
real 11m27.385s
user 15m5.489s
sys 6m47.825s
Writemap:
real 10m27.447s
user 14m35.663s
sys 6m10.767s
But with 31 indices defined...
Original:
real 94m35.862s
user 93m31.755s
sys 20m39.693s
Writemap:
real 42m53.499s
user 54m35.509s
sys 7m23.364s
Over a 2:1 speedup.
--
-- Howard Chu
CTO, Symas Corp.
http://www.symas.com
Director, Highland Sun
http://highlandsun.com/hyc/
Chief Architect, OpenLDAP
http://www.openldap.org/project/