We should also walk thru the Software Enhancement requests and decide which to accept and which to reject. Currently there are 37 outstanding.
Here's a few things we talked about on IRC:
* Re-design and re-implemenation of the C LDAP (and LBER) API.
per http://scratchpad.wikia.com/wiki/LDAP_C_API
** No global state; everything in app or connection handles. No exceptions, no mercy! ** Use function pointers to allow override of *** Memory allocation *** Non-reentrant functions *** Have sane internal defaults as well as defaults for nspr, apr, glib, etc ** Better defaults! (v3 etc) ** Simple function alternatives for simple apps ** Use structures instead of many arguments.
* Overhaul Debug():
** Clean up multi-arg issues ** Revise (create) log formatting guidelines ** Better macros to normalize function trace enter/leave ** Support adding an optional high-res timestamp
* Revised memory allocation:
** Common memory allocator interface with explicit scoping/extent tags ** Consider built-in substitutes for malloc with better performance ** Possible interactions/conflicts/advantages of back-mdb integration
* Back-MDB concerns:
** Schema info in non-mmap ram pointed to from an mmap()d database? (no, and you mentioned this, but we'll need to re-arrange things, as discussed on IRC) Per DB schemas?
* Ensure fork() can based safely in more places - Essential for a fully operational slapo-(shell, perl, lua, python, etc)
* With API cleanups here and there, start supporting more languages like those.
* liblforth for dynamic syntax extensions and other features. (back-forth just sounds like an apr1 proposal...)
* Entry cache overhaul
** The right fix for this is MDB instead, but... ** Would be very nice to specify limits in terms of memory size rather than object count. Yes, this is hard. - Allocation of entry blocks into usage pools with generational mark/sweep collections?
- If support for slapd.conf is completely dropped would it also
be possible to implement a more client-friendly back-config schema which does not have to be backward compatible to slapd.conf?
Describe what would be more client-friendly.
* Make database suffixes be the RDN for database configuration - Use unique RDNs with inherently ordered characteristics where-ever possible Obviously not for ACLs, but many other things are fixable
* Similarly, Drop the concept of backend ordering entirely and move slapo-glue into more generic VFS-like support of the DIT
* Databases should only store one suffix. No, I'm not sure how that should integrate with people putting "" in a database yet. Need to figure that out.
* slapmodify - too useful to not implement now that we have a configdb. Bonus points if it (and slapadd) can be made safe to use with a running directory in all cases. (Perhaps a cache- invalidation-hint unix domain socket?)
* Full two-phase commit support at all layers, including read-transaction support. Back-MDB should make this feasible, and it offers lots of nice data reliability guaranties.
** Use multiple dbroots in MDB to track read/write transactions as they develop.
** Commited data in MDB is effectively read-only. Read transactions just hold their relevant generational root open.
** Write transactions hold open a parent-generation and maintain a log of updates as well as a local difference-tree holding just modified branch/leaf nodes. On prepare, this is reconciled with the new master-commited root and entered into the mmap zone. (or rejected) On commit, that new root is promoted to master-commited root. (Copy-on-write)
** Roots disappear when no one is watching them anymore. If all roots and temp-roots are known, we can easily remove internal nodes and leaves at the same time, so this is much simpler than full garbage collection.
** Root cleanup can be done asynchronously with respect to the triggering reads or writes, potentially in another thread.
** This method is equivalent to, and replaces a journal file. Commited MDB tree pages could be mprotect()ed if desirable.
Matthew Backes Symas Corporation mbackes@symas.com