Volker Lendecke wrote:
On Sun, May 18, 2014 at 12:15:45PM -0700, Howard Chu wrote:
But even tdb only allows one write transaction at a time. I looked into writing a back-tdb for OpenLDAP back in 2009, before I started writing LMDB. I know pretty well how tdb works...
Okay, transactioned, safe writes are slow. True. But the non-transactioned ones have significantly improved in the very recent past. We get a lot by mutexes (we had to find out how badly the linux fcntl locks really suck ...), and also by spreading the load from the freelist to the dead records in neighboring hash chains. I don't have any microbenchmarks, but larger-scale benchmark benefit a lot from those two.
(And of course, fcntl only works for inter-process locking. We needed thread support, which also required mutexes.)
I would like to give lmdb a try in Samba, really. I see that for 32-bit systems we will probably still need tdb for the future (pread/pwrite in lmdb anyone in the meantime? :-)).
That will require app-level buffer cache mgmt and lots of memcpys. Kinda defeats the design of LMDB.
The other blocker when I last took a serious look is that crashed processes can have harmful effects. Has this changed in the meantime with automatic cleanup and/or robust mutexes? I know those might be a bit slower, but I would love to offer our users the choice at least.
Hallvard has a test branch with robust mutex support, we need to look into merging it...