LMDB proposed changes
by Howard Chu
Summarizing some discussions from IRC...
The hardcoded limit on the size of the dirty page list in a transaction is a
problem, there should not be limits on the effective size of a transaction.
The plan is to change LMDB's disk page format to include the txnID in the page
header. This way, when the dirty page list gets full we can flush it to disk
without losing track of which pages were dirtied. Then if a subsequent access
in the same txn revisits one of these pages, when we read it back from the DB
we'll know that it came from the current txn and doesn't need to be copied
again before making further modifications.
The P_DIRTY bit in the page header will no longer be needed - if the txnID
matches, the page can be used directly. If not, the page is clean and a new
page must be allocated before writing.
For WRITEMAP mode the dirty page list can be completely eliminated, the only
reason we keep it now is to know which pages' P_DIRTY bit we need to clear at
commit time.
Increasing the size of the page header by 8 bytes is a bit annoying, this will
require a full slapcat/slapadd reload of existing back-mdb databases. It would
be nice if we can avoid this but I don't see how.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
10 years, 4 months
- 1
- 1
Current master broken with back-mdb
by Quanah Gibson-Mount
Just FYI, current master is utterly broken with back-mdb. Most
sub-databases are missing.
mdb database with 2.4.35:
zimbra@zre-ldap002:~/data/ldap/mdb/db$ mdb_stat -eaf . | grep "Status of"
Status of Main DB
Status of DKIMIdentity
Status of DKIMSelector
Status of ad2i
Status of cn
Status of displayName
Status of dn2i
Status of entryCSN
Status of entryUUID
Status of givenName
Status of id2e
Status of mail
Status of objectClass
Status of sn
Status of uid
Status of zimbraACE
Status of zimbraAuthKerberos5Realm
Status of zimbraCalResBuilding
Status of zimbraCalResCapacity
Status of zimbraCalResFloor
Status of zimbraCalResRoom
Status of zimbraCalResSite
Status of zimbraDomainAliasTargetId
Status of zimbraDomainName
Status of zimbraForeignPrincipal
Status of zimbraId
Status of zimbraMailAlias
Status of zimbraMailCatchAllAddress
Status of zimbraMailDeliveryAddress
Status of zimbraMailForwardingAddress
Status of zimbraMailHost
Status of zimbraMailTransport
Status of zimbraMemberOf
Status of zimbraShareInfo
Status of zimbraSharedItem
Status of zimbraUCServiceId
Status of zimbraVirtualHostname
Status of zimbraVirtualIPAddress
Status of zimbraYahooId
mdb database with master:
zimbra@zre-ldap001:~/data/ldap/mdb/db$ mdb_stat -eaf . | grep "Status of"
Status of Main DB
Status of DKIMIdentity
Status of DKIMSelector
Status of ad2i
Status of cn
--Quanah
--
Quanah Gibson-Mount
Sr. Member of Technical Staff
Zimbra, Inc
A Division of VMware, Inc.
--------------------
Zimbra :: the leader in open source messaging and collaboration
10 years, 8 months
- 2
- 3
- ← Newer
- 1
- Older →