LMDB proposed changes
by Howard Chu
Summarizing some discussions from IRC...
The hardcoded limit on the size of the dirty page list in a transaction is a
problem, there should not be limits on the effective size of a transaction.
The plan is to change LMDB's disk page format to include the txnID in the page
header. This way, when the dirty page list gets full we can flush it to disk
without losing track of which pages were dirtied. Then if a subsequent access
in the same txn revisits one of these pages, when we read it back from the DB
we'll know that it came from the current txn and doesn't need to be copied
again before making further modifications.
The P_DIRTY bit in the page header will no longer be needed - if the txnID
matches, the page can be used directly. If not, the page is clean and a new
page must be allocated before writing.
For WRITEMAP mode the dirty page list can be completely eliminated, the only
reason we keep it now is to know which pages' P_DIRTY bit we need to clear at
commit time.
Increasing the size of the page header by 8 bytes is a bit annoying, this will
require a full slapcat/slapadd reload of existing back-mdb databases. It would
be nice if we can avoid this but I don't see how.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
10 years, 2 months
lmdb.h defining mode_t
by Hallvard Breien Furuseth
openldap-commit2devel(a)OpenLDAP.org writes:
> commit 0c568117cea6764c3f12a0d69ddf54ee467d2dec
> Author: Howard Chu <hyc(a)symas.com>
> Date: Sun Mar 31 08:10:43 2013 -0800
>
> MSVC doesn't define mode_t
Nor should ldmb.h. That can break if it meets a header from another
package doing almost the same thing. E.g. #define mode_t unsigned int.
Try:
#ifndef _MSC_VER
typedef mode_t mdb_mode_t;
#else
typedef int mdb_mode_t;
#endif
...
int mdb_env_open(... mdb_mode_t mode);
--
Hallvard
10 years, 6 months
RE24 Testing call #1 (OpenLDAP 2.4.35)
by Quanah Gibson-Mount
Current RE24 is ready for testing for the 2.4.35 release.
Thanks!
--Quanah
--
Quanah Gibson-Mount
Sr. Member of Technical Staff
Zimbra, Inc
A Division of VMware, Inc.
--------------------
Zimbra :: the leader in open source messaging and collaboration
10 years, 6 months
Re: Notes
by Howard Chu
Brett Maxfield wrote:
> On 12/03/2013, at 8:04 AM, Howard Chu <hyc(a)symas.com> wrote:
>
>> We're talking about deprecating the BDB-based backends Real Soon Now. I
suppose this should be done in OpenLDAP 2.5, but we've pulled a lot into 2.4
that we had originally slated for 2.5, which keeps pushing any actual 2.5 work
further into the future as we keep extending 2.4.
>
> I imagine dropping bdb completely will leave a multitude of systems which
apart from suffering from distribution lag, will now suffer additionally from
end of line itus, as they will be unable to patch to the newest minor version
if bdb is dropped entirely in some future 2.4.x version..
>
> I'd say deprecate it in 2.4.x great, but dont drop it completely until
> 2.5.
If you are in a rush, drop bdb backend from the default configure options to
get the point across, or print a warning or some such notice in configure, so
package maintainers get the hint before 2.6 is released..
>
> Given users have the option to drop bdb today, why force the issue?
>
> Most people don't expect huge changes in behavior, for only minor version
releases.
>
> If 2.4.x is stable and feature complete / no new features, then at some
point there will be a natural finish, when there will be few changes for 2.4.x
right? Bug fix only tree for 2.4 would leave resources to work on 2.5 release ?
I only said "deprecating", I didn't say anything about dropping.
back-bdb has already been turned off by default in HEAD. Looks like it's still
on by default in RE24, that seems to be an oversight since we'd been
recommending back-hdb for a long time already.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
10 years, 6 months
Notes
by Howard Chu
We're talking about deprecating the BDB-based backends Real Soon Now. I
suppose this should be done in OpenLDAP 2.5, but we've pulled a lot into 2.4
that we had originally slated for 2.5, which keeps pushing any actual 2.5 work
further into the future as we keep extending 2.4.
At any rate, LMDB is settling down pretty well now. The only reason left to
use BDB today is if you're working with a >2GB DB on a 32-bit machine, and
frankly if you're doing that, instead of just upgrading to a 64-bit machine,
you've got other problems.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
10 years, 6 months
