openldap-devel March 2010

openldap-devel@openldap.org

9 participants
9 discussions

OpenLDAP git repo?
by Volker Lendecke 29 Mar '11

29 Mar '11

Hi! Out of several rumors I've heard that OpenLDAP converted it source repo to git. Is that true? www.openldap.org still speaks about cvs. If you changed to git, is the repo publically available? Thanks, Volker

5 12

OpenLDAP 2.5
by Howard Chu 28 Apr '10

28 Apr '10

With 2.4.21 out, and hopefully stable enough to promote to the next Stable release, it's time to feature-freeze 2.4 and prepare for the 2.5 branch. As I already announced to the OpenLDAP-Committers, we're also planning to switch from CVS to GIT in mid-January. Commits for 2.5 will begin after we've settled into GIT. We haven't really laid out a formal roadmap for 2.5 yet, but I think most of it has been discussed here or in Development ITSs already. I would like to be able to resolve all outstanding Development ITSs - we will either implement them or reject/close them. There are 42 outstanding at the moment. Likewise for all outstanding ITSs in Software Bugs - many of them have been deferred because a proper fix would require invasive changes to large parts of the code base. There are 26 outstanding. With 2.5 beginning we are free to make these large scale changes. We should also walk thru the Software Enhancement requests and decide which to accept and which to reject. Currently there are 37 outstanding. I also have a number of specific areas I want to see worked on; some of these are included in the above ITSs but I'll outline them here: syncrepl config - this is pretty unwieldy already; syncrepl needs to be moved outside of the slapd core and into an overlay. That will allow us a lot more flexibility in configuring while also eliminating a lot of redundant parsing code. suffixmassage - we at the very least need to be able to point a consumer at some non-homogeneous suffix in the provider. We may go for complete librewrite support as well, although at this point I don't see as strong a need. config TLS certs and keys should be stored as LDAP attributes, not pointers to filesystem locations. This is a prereq to using some of the dynamic cert generation features of the CA overlay. (This can be troublesome as there may not be clean APIs for reading certs from memory in all of the TLS APIs we support.) Disabling individual config attribute values and entries. At the moment I'm thinking of adding an ";x-disabled" tag to those values. back-mdb Using a single-level store for Entries will impact all of the schema engine as well. I think the simplest solution here is going to be using an mmap'd file for all of the schema elements. The actual design of back-mdb still needs to be defined in several areas. The single-level store approach exposes us to some new failure modes that the current multi-level backends don't have. (E.g., corruptions due to bad RAM / wild pointer writes are very likely to get persisted on disk, implicitly.) The solution I'm considering is based on a mirroring strategy. Every database will be stored twice on disk: once in the file that is actively mmap'd into the process, and once in a write-only file. On every intentional update of a memory page, we will also store a checksum of the page, and manually write the page to the mirror. If we detect a checksum failure on any in-memory page we can still retrieve a valid copy from the mirror file. This of course doubles our potential I/O load, but I don't believe it's any worse than the load from performing write-ahead logging on a traditional database. (And yes, mirroring will take the place of writing transaction log files.) Some of these same considerations apply to the schema storage, but not entirely. At runtime, the schema is effectively read-only. When we do dynamic schema changes thru cn=config, all other threads are suspended. For the mmap purposes, we can mark all of the schema pages as read-only during runtime, and only make them read-write when cn=config is actually trying to perform an update. As such, the only sticky issue is dealing with changes made to the back-config internal files by plain text editors and such. These are the things I'm interested in. But as always, this Project is driven forward by the particular interests of each individual contributor. If you have other ideas you want to pursue, speak up. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

9 13

User-level network stack
by Howard Chu 30 Mar '10

30 Mar '10

Quite a while back I talked about optimizing select/epoll/whatever using mmap'd memory shared between user space and the kernel for the event table. http://www.openldap.org/lists/openldap-devel/200411/msg00088.html These guys have gone even further, and use mapped memory to bring the entire network stack up to user level: http://www.openonload.org/openonload-google-talk.pdf Their paper talks about the work they did with two particular 10Gbit NICs. I find this interesting because in these benchmarks I ran http://connexitor.com/blog/pivot/entry.php?id=191 the NIC interrupt overhead was 100% of a CPU core, and that was only on 1Gbit ethernet. Clearly for heavy-load deployments a better network solution is needed. It sounds like maybe the OpenOnload guys have a better approach. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

logging "reqEntryUUID" in accesslog?
by Michael Ströder 29 Mar '10

29 Mar '10

HI! In web2ldap when displaying a single entry I have a button which directly forms a filter for searching the audit log (accesslog DB) for the entry based on the attribute 'reqDN'. This is handy but does not work anymore when entries were renamed. So how about logging the value of entryUUID of an entry affected by a write operation in a separate attribute 'reqEntryUUID'? Ciao, Michael.

2 2

pcache logging
by masarati＠aero.polimi.it 26 Mar '10

26 Mar '10

I'd like to enhance pcache logging by adding the conn= op= prefix. Objections? p.

2 1

liblutil/ultils.c: lutil_localtime() <g-differential> timestamp format bug?
by Daniel 20 Mar '10

20 Mar '10

Hi, there seems to be a small problem in "lutil_localtime()" regarding the creation of <g-differential> timestamp formated bervals: The returned length "ret" (of type: size_t) of a timestamp value in <g-differential> format contains the terminating "\0" which leads to base64 encoded presentation... For demonstration I've attached a minimal (and probably very dirty) patch which fixed this issue for me, but I'm quite unsure whether: - it's really a bug or I'm doing something completely wrong - my patch represents a clean solution. Therefore I've not opened an ITS. First of all I would appreciate your valuable feedback, whether lutil_localtime() creates <g-differential> timestamps like intended? Thanks a lot! Best regards Daniel Index: libraries/liblutil/utils.c =================================================================== RCS file: /repo/OpenLDAP/pkg/ldap/libraries/liblutil/utils.c,v retrieving revision 1.70 diff -u -r1.70 utils.c --- libraries/liblutil/utils.c 20 Nov 2009 19:43:50 -0000 1.70 +++ libraries/liblutil/utils.c 20 Mar 2010 17:15:12 -0000 @@ -147,7 +147,7 @@ snprintf( p, smax - 15, "%02ld%02ld", delta / 3600, ( delta % 3600 ) / 60 ); - return ret + 5; + return ret + 4; } int lutil_tm2time( struct lutil_tm *tm, struct lutil_timet *tt )

1 0

Altering the behavior of pcache to improve cache hit/miss ratio
by Ryan Steele 10 Mar '10

10 Mar '10

Hey folks, I've been playing with pcache for about a week (in combination with back-ldap), testing various things and trying to create a configuration that helps preserve system stability if access to the LDAP server disappears. Many things seem to work well, but I noticed that quite a good number of Unix commands, like "id", request a user's full entry (i.e., no filter string). In its current implementation, this will always fail to hit the cache, because (as the man page states) you cannot specify an empty list of attributes. This means that even if you have all the information required to return what was requested in the local cache, you will never be able to leverage that information. Given that, I wanted to ask if it could be considered to change the design of the overlay a bit, such that if an empty attribute list is specified, pcache will return every attribute *in the cache* for that entry, instead of blindly assuming it doesn't have the information to accomplish the task. I understand the shortcomings this could introduce, most notably if one did not want to cache every attribute for a particular entry and still wanted to be able to omit the filter string and get the entry's full attribute list, but I think the benefits would make it worthwhile. I say that for a few reasons: if you're using pcache and are interested in truly optimizing performance or keeping a system that is LDAP-dependent from having major issues if the network disappears, it would make sense to cache every attribute for entries used in common system tasks (e.g., using utilities like "id"). I believe this could also significantly improve the hit/miss ratios, especially if you take a look at how many of these common system utilities make queries without specifying any kind of filter. I would think that this could be accomplished by adding a case for the empty filter string, such that it would take a unison of each set of attributes corresponding to the cached entry for that particular LDAP filter, and returning the result. If the same attribute is present in more than one cache entry matching the specified LDAP filter, the most recent would be favored and returned, thereby preventing pcache from having choose between several query-string-dependent entries for that entry, if more than one are present. I'm interested to hear what you guys think of this. Even if my implementation ideas aren't 100% spot-on, I would think that the wiser minds could adapt the general idea in to something more artful, provided they agree with the idea in principle. Thanks! -Ryan

2 2

Re: commit: openldap-guide/admin backends.sdf overlays.sdf
by Gavin Henry 06 Mar '10

06 Mar '10

----- hyc(a)OpenLDAP.org wrote: > Update of /repo/OpenLDAP/pkg/openldap-guide/admin > > Modified Files: > backends.sdf 1.17 -> 1.18 > overlays.sdf 1.47 -> 1.48 > > Log Message: > ITS#6486 - avoid explicit moduleload in examples > > > CVS Web URLs: > > http://www.openldap.org/devel/cvsweb.cgi/admin/?cvsroot=OpenLDAP-guide > > http://www.openldap.org/devel/cvsweb.cgi/admin/backends.sdf?cvsroot=OpenLDA… > > http://www.openldap.org/devel/cvsweb.cgi/admin/overlays.sdf?cvsroot=OpenLDA… > Thanks! -- Kind Regards, Gavin Henry. OpenLDAP Engineering Team. E ghenry(a)OpenLDAP.org Community developed LDAP software. http://www.openldap.org/project/

1 0

back-config delete support (syncprov overlay)
by Ralf Haferkamp 02 Mar '10

02 Mar '10

Hi, while taking up some loose ends on my work on delete support for back- config (enabled with -DSLAP_CONFIG_DELETE) I wondered how we should deal with the deletion of the syncprov overlay when there are active refreshAndPersist sessions. What errorcode should we sent when closing such a connection? To me LDAP_UNAVAILABLE sound like the best choice. LDAP_UNWILLING_TO_PERFORM might also be ok. -- Ralf

4 9

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-devel March 2010