openldap-devel

openldap-devel@openldap.org

1416 discussions

New plugin: slapd-kinit
by Ralf Haferkamp 14 Oct '10

14 Oct '10

Hi, I am currently coding a small slapd plugin named kinit. All it does is requesting a Kerberos TGT for a configurable principal (default: ldap/<hostname>@REALM) and renews/reinits it when needed using slapd's runqueue. I'd like to submit it to HEAD soon. Where should it go? contrib/slapd- modules/kinit ? -- regards, Ralf

2 1

(Round 3) Proposed patch to support: draft-zeilenga-ldap-c-api-concurrency
by Doug Leavitt 11 Oct '10

11 Oct '10

Since my last patch submission (Round2) I spent a considerable amount of time performing performance analysis of my proposed both client side and server side. The client side testing primarily consisted of running parallel copies of slapd-mtread and monitoring in real time with dtrace to monitor the behavior of all the concurrent threads in clients looking for signs of thread starvation. I was not able to find any. On the server side I performed similar testing while using modified versions of the test036-meta-concurrency and test020-proxycache test cases. Again looking for thread starvation. Additionally, on the server side I setup single server and proxy cache server environments, in combination with slamd to perform head to head comparisons of HEAD (most recently as of 10/7/10) vs. HEAD (most recently as of 10/7/10) and the round 3 MT patches to see if there was any performance degradations or other performance issues, especially related to situations where slapd was using libldap_r as part of various backends. The results of the head to head comparisons, especially in the proxycache situation, show a 3% performance improvement with the MT patches over the HEAD without the patches. The reason according to dtrace outputs is that there are now cases, especially in ldap_result where my proposed patch has up to 50% less calls to thread_lock/unlock than are performed in the current HEAD. All of the head to head dslam/dtrace results are included along with the other patch materials. The proposed patch details are located here: http://cr.opensolaris.org/~djl/openldap-codereview-r3/ Details about the contents of http://cr.opensolaris.org/~djl/openldap-codereview-r3/dtrace is located here: http://cr.opensolaris.org/~djl/openldap-codereview-r3/README.txt Doug.

3 4

Re: Proposed patch to support: draft-zeilenga-ldap-c-api-concurrency
by masarati＠aero.polimi.it 28 Sep '10

28 Sep '10

> Doug, > > a list of relatively quick comments, as I didn't go into too much detail > in the analysis of each bit of your work. > > I think your patch contains at least three levels of modifications; they > all make sense separately and incrementally, so I think we should revise > them separately. > > 1) "cosmetic" cleanup (like wrapping locks in specific macros); they do > not strictly relate to the support of > draft-zeilenga-ldap-c-api-concurrency (but they make sense, and I like > them); they should be considered first, regardless of the rest (I note > that according to the way you define LDAP_NEXT_MSGID when #ifdef > LDAP_R_COMPILE, you don't need to #ifdef its definition :) > > 2) a set of issues that fix the concurrent behavior of libldap relatively > irrespective of the support of draft-zeilenga-ldap-c-api-concurrency (see > for example the private discussion we had on fixing concurrent access to > lconn_refcnt). > > 3) specific support to draft-zeilenga-ldap-c-api-concurrency, including > but not limited to the addition of ldap_dup(), ldap_destroy() and related > stuff. > > As you may have understood, I'm especially interested in (2), because it > fixes a number of potential (or actual) issues in OpenLDAP code that uses > libldap concurrently, like the proxy backends. So far, that code used > libldap taking into account known weaknesses from a concurrency > standpoint. For example, proxy backends protect handlers until a bind > succeeds, to prevent the undesired simultaneous creation of default > connections. However, some issues may slip in (as I believe is the case > of ITS#6574). > > One thing I'd like to ask is: you introduce a few additional mutexes: > > ldapoptions -> ldo_mutex > > ldapcommon -> ldc_msgid_mutex, ldc_abandon_mutex > > in addition of the already existing > > ldap -> ld_conn_mutex, ld_req_mutex, ld_res_mutex > > that move to ldap_common. > > My concern is: can you guarantee that the occurrences of locking/unlocking > those additional mutexes, combined to the existing ones, do not result in > deadlocks? I mean: did you explicitly check all possible logical paths or > so, or take measures to avoid this possibility? Another mostly "style" comment: I think you shouldn't be polluting libldap's space with thread implementation specific details, like init.c: #if defined(HAVE_THR) #elif defined(HAVE_PTHREADS) PTHREAD_MUTEX_INITIALIZER #if !(defined(HAVE_THR) || defined(HAVE_PTHREADS)) and so. I understand PTHREAD_MUTEX_INITIALIZER is handy, but the whole thing should remain confined in libldap_r specific files. Also, but since it's a test tool it is not a priority, slapd-mtread.c should use libldap_r and hide thread implementation details under OpenLDAP's portable interface. p.

2 5

Question on chasing referrals
by Lucas C. Villa Real 22 Sep '10

22 Sep '10

Hello folks, We have been running openldap-2.3.32 on SLES64 for a while with no problem. However, recently a couple of packages were upgraded (not including openldap), and we noticed that some applications started to consume 100% of CPU in a loop that never ended. After doing some investigation we found that it was due to a bug in a list traversal of OpenLDAP that had been fixed in mainline already. This is the patch in question, which was introduced in the commit referenced at the end of the email [1]: --- openldap-2.3.32/libraries/libldap/request.c.orig 2010-05-27 02:22:38.000000000 +0200 +++ openldap-2.3.32/libraries/libldap/request.c 2010-05-27 02:22:50.000000000 +0200 @@ -916,7 +916,7 @@ ldap_chase_v3referrals( LDAP *ld, LDAPRe if ( lp == origreq ) { lp = lp->lr_child; } else { - lp = lr->lr_refnext; + lp = lp->lr_refnext; } } if ( looped ) { So, these apps probably started to interact differently with OpenLDAP. The thing is that now these applications are taking much more time to complete execution than before. As a simple test, I tried to replace the above fix with a call to "break", which would mimic the old behavior back when applications didn't execute that code path: } else { - lp = lr->lr_refnext; + break; } To my surprise, the execution time of those apps went back to normal. Since I don't have experience with OpenLDAP nor I have a good understanding on many of its concepts, I'd like to ask you what's the impact on not chasing all referrals from the linked list? I read in the comments of try_read1msg() that if chasing fails then the referral is passed back to the application. Can I assume that try_read1msg() is basically caching that information just in case the caller needs it? What is that chasing used for? Thank you very much! Lucas [1] http://www.openldap.org/devel/cvsweb.cgi/libraries/libldap/request.c.diff?r…

1 0

Re: commit: ldap/servers/slapd/overlays sssvlv.c
by Howard Chu 19 Sep '10

19 Sep '10

ando(a)OpenLDAP.org wrote: > Update of /repo/OpenLDAP/pkg/ldap/servers/slapd/overlays > > Modified Files: > sssvlv.c 1.18 -> 1.19 > > Log Message: > make sure so is correctly initialized (spotted by valgrind, possibly related to ITS#6649) This patch ought to have no visible effect, since all relevant sort_op fields are initialized as needed in lines 815-834. Where does valgrind say any misuse occurs? -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

2 1

sssvlv + ordering
by Dieter Kluenter 19 Sep '10

19 Sep '10

Hi, i just updated cvs HEAD in order to test sssvlv. I realised 2 issues. 1. It a search is limited to a set of attributes, the first result chunk only presents this defined attributes, but while walking through the tree by just pressing the return tab, all consecutive chunks present all user attributes, 2. ordering rules are not followed anymore (it has been at some time), something like 0/3/19/90:=>K , or 0/1/10/90:=<H, the result allways presents all DN from a-Z -Dieter -- Dieter Klünter | Systemberatung sip: 7770535(a)sipgate.de http://www.dpunkt.de/buecher/2104.html GPG Key ID:8EF7B6C6

1 0

sssvlv & pr
by masarati＠aero.polimi.it 18 Sep '10

18 Sep '10

While tracking ITS#6649 I noted that sssvlv contains some code to interact with paged results; I haven't looked too deep into it, but it makes sense that results are first sorted and then returned paged accordingly. However, if this was the intention, it looks broken. The first page is returned; the second page bails out with sizelimit exceeded without returning any entry, although no limit was set. I'd open an ITS, if this behavior is confirmed and can be considered erroneous. p.

1 0

common pessimization: ber_mem*_x(...,NULL)
by Hallvard B Furuseth 10 Sep '10

10 Sep '10

The OpenLDAP code is a bit too clever sometimes. If someone has too much time on their hands, it could be useful to look for places where the code tries and fails to be more clever than the compiler, and simplify that. E.g. compilers are commonly very good at optimizing array operations - i.e. when a function takes a pointer which it *does not modify* and walks an index up and down the pointed-to array. However once the function starts doing pointer games, a number of compilers seem to be scared away from array optimizations and instead just do plain pointer operations. Another example: OpenLDAP has a fair number of calls like ber_memfree_x(ptr, NULL) which I guess is intended as an optimization - it avoids the ber_memfree(ptr) wrapper call. Actually that's a pessimization if liblber/memory.c is optimized to inline ber_memfree_x() into ber_memfree(): ber_memfree_x() cannot be optimized on ctx==NULL. For an optimization, assuming it makes any difference, it would make more sense remove the _x from the call and inline by hand some of the ber_*_x() functions in memory.c. gcc even has some builtins that can be use to turn ber_foo_x(bar,NULL) calls into ber_foo(bar). Someething like this: #ifdef __GNUC__ # define ber_memfree_x(ptr, ctx) \ ((__builtin_constant_p(ctx) && ((ctx) == NULL)) \ ? ber_memfree(ptr) : (ber_memfree_x)(ptr, ctx)) #endif That'll also help code like this: void foo(void *bar) { foo_x(bar, NULL); } void foo_x(void *bar, void *ctx) { ber_memfree_x(bar, ctx); } If foo_x() is into foo(), that will turn ber_memfree_x into ber_memfree. -- Hallvard

1 0

Proposed patch to support: draft-zeilenga-ldap-c-api-concurrency
by Doug Leavitt 23 Aug '10

23 Aug '10

I would like to propose a patch to the OpenLDAP C APIs to support the operation thread safe features described in the draft-zeilenga-ldap-c-api-concurrency drafts. The enhancements add upwardly compatible interfaces and should have no behavioral regressions w.r.t. any existing libldap interfaces. The new concurrency APIs include: ldap_dup and ldap_destroy They are described in more detail in the concurrency draft and in the applicable man paged. The related feature options are also provided. Additionally I would like to propose 1 new public error API ldap_get_lderror which returns all three ldap_get_option results: LDAP_OPT_MATCHED_DN LDAP_OPT_DIAGNOSTIC_MESSAGE LDAP_OPT_RESULT_CODE in a single API, and 1 new depreciated public API: ldap_get_lderrno which performs almost the same function as ldap_get_error but does so in a mt-unsafe manner, but that is compatible with Mozilla's libldap API of the same name. The point of the two new error APIs is to provide a transition path from Mozilla libldap users that still use ldap_get_lderrno to either a similar API that obeys OpenLDAP like practices (ldap_get_error) or as a transition until any existing code can be converted away from ldap_get_lderrno to just using ldap_get_options as needed. The patch includes a new test case to test the new functionality and man pages. The new test case needs additional work for some of the threading models. Any input here is requested, since I do not have testing resources for all the threading combination and do not have the ability to test the Windows model. I am looking forward to any and all comments. I plan to submit a formal ITS with a finalized version of changes once any and all issues are thrashed out and resolved to a level that they are acceptable for integration. Just for the record: This patch file is derived from OpenLDAP Software. All of the modifications to OpenLDAP Software represented in the following patch(es) were developed by Oracle Corporation.Oracle Corporation has not assigned rights and/or interest in this work to any party. I, Douglas Leavitt am authorized by Oracle Corporation, my employer, to release this work under the following terms. The initial version of my proposed patch can be found here: http://cr.opensolaris.org/~djl/openldap-codereview This includes a diff -U of the full patch (/diffU) a webrev diff (/webrev) for on-line review and a tar ball image of the OpenLDAP HEAD as of 8/13/10 with the current patch applied. Thank you for you consideration, and I look forward to any and all comments. Doug Leavitt

4 8

Re: commit: ldap/servers/slapd bconfig.c daemon.c proto-slap.h
by Howard Chu 16 Aug '10

16 Aug '10

hyc(a)OpenLDAP.org wrote: > Update of /repo/OpenLDAP/pkg/ldap/servers/slapd > > Modified Files: > bconfig.c 1.413 -> 1.414 > daemon.c 1.444 -> 1.445 > proto-slap.h 1.802 -> 1.803 > > Log Message: > Add support for multiple listener threads. Lightly tested on Linux, > Winsock needs a couple more tweaks Well, it doesn't look like this patch caused any harm for the default case. I'm only seeing about a 10% gain in throughput using two listener threads on a 16 core machine. Not earth-shattering, not bad. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

2 8

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-devel