On Sun, Mar 09, 2008 at 01:21:22PM -0700, Howard Chu wrote:

Howard Chu wrote:

...

I recall we had a few ideas last year, but they came up a bit late. Anyone still have some wish list items for this year?

I think it would be feasible for a student to develop a back-tdb backend. The basic backend functionality could be copied from back-hdb and the caching layers could be omitted, which would leave the code pretty straightforward. Since back-tdb is primarily an in-memory database it wouldn't be much of a performance issue, and it would get us something easy to configure for the very common, less demanding deployments.

Another possibility which I started discussing with Tridge is to modify the tdb code to always mmap to a fixed address space. In that case we could store Entry objects directly, with all pointers intact, which would eliminate the need for a deserialization step on Reads. Then the need for entry caching would be completely eliminated as well.

One thing that could also be kept in mind is a potential ctdb backend to go clustered. ctdb right now lacks transactions the way you would probably need them, but this is something that will be added.

Volker

Howard Chu wrote:

Howard Chu wrote:

...

I recall we had a few ideas last year, but they came up a bit late. Anyone still have some wish list items for this year?

The application deadline is two days from now, so let's try to summarize and elaborate where needed within the next 24 hours.

I think it would be feasible for a student to develop a back-tdb backend. The basic backend functionality could be copied from back-hdb and the caching layers could be omitted, which would leave the code pretty straightforward. Since back-tdb is primarily an in-memory database it wouldn't be much of a performance issue, and it would get us something easy to configure for the very common, less demanding deployments.

I think the above would be a pretty useful project as-is.

Another possibility which I started discussing with Tridge is to modify the tdb code to always mmap to a fixed address space. In that case we could store Entry objects directly, with all pointers intact, which would eliminate the need for a deserialization step on Reads. Then the need for entry caching would be completely eliminated as well.

This is probably a refinement that can wait.

There are also occasional requests for additional matching rules support, that may be something to add.

Without specifics, I don't see what we can do with this idea. What matching rules?

The Forth based engine also sounds promising.

At the moment, I'm guessing this is bigger than a summer student project.

At least with LDIF, the format is already well-defined. (Though I agree that it's a pretty miserable format...) I'm not sure what a useful LDIF parsing library should look like. Perhaps one that parsed it into a chain of LDAPMessages.

(Perhaps that should be an SoC project too.)

This kind of ties in to my point that the current LDAP C API sucks. It would be mildly useful to have a library that parses LDIF input into LDAP Request structures. To make it completely useful, the LDAP library itself needs to be built to use LDAP Request structures instead of its current "one function with lots of parameters-and oops we forgot one-design."

fork/exec in a threaded program is still a dicy proposition, which is one of the reasons we haven't gone there yet. It's also a good reason to phase out back-shell, and promote back-sock instead.

It would probably be a small enough job to put an overlay wrapper around back-sock (just like the chain overlay is a wrapper around back-ldap) with some filtering/selection keywords.

I think this is a pretty straightforward project. One just needs to define the desired configuration language for specifying how to select when to trigger an external call. I think it may be as simple as specifying a list of LDAP URLs - any operation whose target entry matches the base/scope/filter of the URL is forwarded to the back-sock listener.

...

...

You mean like the USN code currently in head? ;) IIRC, it still needs a little bit of work to polish it off.

/repo/OpenLDAP/pkg/ldap/contrib/slapd-modules/usn

Actually, looking at it again, I'd do it a bit differently now. It should probably be a global overlay instead of a per-database overlay, so that it can publish a highestCommittedUSN in the rootDSE. It also should not maintain its own counters, it should just map CSNs to USNs (which are 64 bit unsigned integers).

By the way, yet another bug in ActiveDirectory, both the Interval syntax and the LargeInteger syntax have the same OID, even though they're distinct syntaxes. Idiots... http://msdn2.microsoft.com/en-us/library/ms684426(VS.85).aspx http://msdn2.microsoft.com/en-us/library/ms684427(VS.85).aspx

This should be pretty straightforward too. Hardly a summer's worth of effort though.

That looks like all the ideas so far. For each idea, we will also have to have mentors to work with the students. So post here if you're interested in that as well. I'll set up a FAQ-o-Matic page for whatever we decide on.

...

<quote who="Howard Chu"> > I recall we had a few ideas last year, but they came up a bit late. > Anyone still have some wish list items for this year? What about the todo list? Anything good in there?

There are also occasional requests for additional matching rules support, that may be something to add. The Forth based engine also sounds promising.

Maybe an overlay to implement USN, like the uSNChanged attribute in Active Directory http://support.microsoft.com/kb/891995 I'd imagine USN support is required for Samba4 to use OpenLDAP as a backend, and it would just be handy since some tools/scripts assume AD-isms.

Just a thought.

Quanah Gibson-Mount wrote:

--On Monday, March 10, 2008 8:50 AM -0400 Adam Tauno Williams adamtaunowilliams@gmail.com wrote:

...

...

...

<quote who="Howard Chu"> > I recall we had a few ideas last year, but they came up a bit late. > Anyone still have some wish list items for this year? What about the todo list? Anything good in there?

There are also occasional requests for additional matching rules support, that may be something to add. The Forth based engine also sounds promising.

Maybe an overlay to implement USN, like the uSNChanged attribute in Active Directoryhttp://support.microsoft.com/kb/891995 I'd imagine USN support is required for Samba4 to use OpenLDAP as a backend, and it would just be handy since some tools/scripts assume AD-isms.

Just a thought.

You mean like the USN code currently in head? ;) IIRC, it still needs a little bit of work to polish it off.

/repo/OpenLDAP/pkg/ldap/contrib/slapd-modules/usn

Actually what it needs is for someone to define an actual use case for it. I wrote that overlay up when Andrew Bartlett said "it would be nice if we had USN support" but beyond that, no one has stated what "support" is needed.

Quanah Gibson-Mount wrote:

--On Monday, March 10, 2008 8:50 AM -0400 Adam Tauno Williams adamtaunowilliams@gmail.com wrote:

...

...

...

<quote who="Howard Chu"> > I recall we had a few ideas last year, but they came up a bit late. > Anyone still have some wish list items for this year? What about the todo list? Anything good in there?

There are also occasional requests for additional matching rules support, that may be something to add. The Forth based engine also sounds promising.

Maybe an overlay to implement USN, like the uSNChanged attribute in Active Directoryhttp://support.microsoft.com/kb/891995 I'd imagine USN support is required for Samba4 to use OpenLDAP as a backend, and it would just be handy since some tools/scripts assume AD-isms.

Just a thought.

You mean like the USN code currently in head? ;) IIRC, it still needs a little bit of work to polish it off.

/repo/OpenLDAP/pkg/ldap/contrib/slapd-modules/usn

Actually, looking at it again, I'd do it a bit differently now. It should probably be a global overlay instead of a per-database overlay, so that it can publish a highestCommittedUSN in the rootDSE. It also should not maintain its own counters, it should just map CSNs to USNs (which are 64 bit unsigned integers).

By the way, yet another bug in ActiveDirectory, both the Interval syntax and the LargeInteger syntax have the same OID, even though they're distinct syntaxes. Idiots... http://msdn2.microsoft.com/en-us/library/ms684426(VS.85).aspx http://msdn2.microsoft.com/en-us/library/ms684427(VS.85).aspx

Emmanuel Dreyfus wrote:

Howard Chuhyc@symas.com wrote:

...

I recall we had a few ideas last year, but they came up a bit late. Anyone still have some wish list items for this year?

External command execution overlay: it would enable calling fork()/execve() for a set of (operation, baseDN, pré-condition, post-condition). execve argument would allow substitution for various input: %{dn}, %{uid}, and so on.

For now this can be acheived by setting up an accesslog overlay on a shell backend and having a program filtering the produced LDIF, but it is rather suboptimal, and everyone has to reinvent the wheel for the LDIF filtering.

Have you looked at using back-sock instead of back-shell?

re: reinventing the wheel - this is true regardless. In order to provide a rich enough interface to be useful, you need to be able to pass a lot of parameters to the script/process on the other side, and that is going to require a significant chunk of parsing code. At least with LDIF, the format is already well-defined. (Though I agree that it's a pretty miserable format...) I'm not sure what a useful LDIF parsing library should look like. Perhaps one that parsed it into a chain of LDAPMessages.

(Perhaps that should be an SoC project too.)

I use it to allocate UID and GID automatically when a user is created, and to setup homes and quotas on machines that need it.

fork/exec in a threaded program is still a dicy proposition, which is one of the reasons we haven't gone there yet. It's also a good reason to phase out back-shell, and promote back-sock instead.

It would probably be a small enough job to put an overlay wrapper around back-sock (just like the chain overlay is a wrapper around back-ldap) with some filtering/selection keywords.

--On Monday, March 10, 2008 5:18 AM +0100 Emmanuel Dreyfus manu@netbsd.org wrote:

For now this can be achieved by setting up an accesslog overlay on a shell backend and having a program filtering the produced LDIF, but it is rather suboptimal, and everyone has to reinvent the wheel for the LDIF filtering.

You can also use Net::LDAPapi as a delta-syncrepl "listener" that acts on changes, whether it is new account creation, or otherwise. And then have perl fire off whatever it is you want done for any variety of situations.

--Quanah

--

Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

Buchan Milne bgmilne@staff.telkomsa.net wrote:

While I doubt the utility for creating home directories (pam_mkhomedir is sufficient for all but the NFS-homes case IMHO),

I use it to address the NFS-home case.