openldap-devel December 2007

openldap-devel@openldap.org

16 participants
26 discussions

dITStructureRules/nameForms in subschema subentry for informational purpose
by Michael Ströder 21 Mar '09

21 Mar '09

HI! Discussed this very briefly with Howard at LDAPcon 2007 based on an idea of Steve: Support for dITStructureRules and nameForms is still in OpenLDAP's TODO. In the meanwhile slapd could accept definitions for both in slapd.conf and simply pass them on to a schema-aware LDAP client for informational purpose without enforcing them. Same function like rootDSE <file> in slapd.conf. Opinions? Ciao, Michael. -- Michael Ströder E-Mail: michael(a)stroeder.com http://www.stroeder.com

2 4

Re: thread pools, performance
by Howard Chu 14 Feb '08

14 Feb '08

Rick Jones wrote: > There are definitely interrupt coalescing settings available with tg3-driven > cards, as well as bnx2 driven ones: > > ftp://ftp.cup.hp.com/dist/networking/briefs/nic_latency_vs_tput.txt Yep, that helped. Raising rx-usecs from default 20 to 1000, and rx-frames from default 5 to 100, I'm getting 43k auths/sec with back-null (in 4 separate thread pools) and the core fielding the interrupts is only about 80% busy now instead of 100%. I'm afraid my load generators may be maxed out now, because I can't seem to drive up the load on the server any higher even though there's more idle CPU. The current code in HEAD (with only 1 thread pool) is reaching 36k auths/sec with back-null, so it's actually not far off from my experimental peak rate. Considering that HEAD was at 25k/sec last week (and now in 2.4.6) that's pretty decent. With back-bdb and 1 million users I'm getting 26.1k/sec with plaintext passwords (up from 19.3k/sec last week). With {SSHA} passwords that drops to 25.7k/sec (~1.5% difference). I have to put this tinkering on hold for a bit, to run some authrate tests against ActiveDirectory on this machine (using W2K3sp2 X64). Later on we'll do a W2K3 OpenLDAP build for comparison as well. Should be entertaining... -- -- Howard Chu Chief Architect, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

7 22

CPU scaling
by Howard Chu 05 Jan '08

05 Jan '08

Has anyone got a dual or quad socket Intel Xeon based server for testing? I've been testing on two AMD systems, one quad socket dual core and one dual socket quad core. There are a lot of different ways to tune these systems... slapd currently uses a single listener thread and a pool of some number of worker threads. I've found that performance improves significantly when the listener thread is pinned to a single core, and no other threads are allowed to run there. I've also found that performance improves somewhat when all worker threads are pinned to specific cores, instead of being free to run on any of the remaining cores. This has made testing a bit more complicated than I expected. I originally was just pinning the entire process to a set number of cores (first 1, then 2, incrementing up to 8) to see how performance changed with additional cores. But due to the motherboard layout and the fact that the I/O bridges are directly attached to particular sockets, it makes a big difference exactly which cores you use. Another item I noticed is that while we scale perfectly linearly from 1 core to 2 cores in a socket (with a dual-core processor), as we start spreading across multiple sockets the scaling tapers off drastically. That makes sense given the constraints of the Hypertransport connections between the sockets. On the quad-core system we scale pretty linearly from 1 to 4 cores (in one socket) but again the improvement tapers off drastically when the 2nd socket is added in. I don't have any Xeon systems to test on at the moment, but I'm curious to see how they do given that all CPUs should have equal access to the northbridge. (Of course, given that both memory and I/O traffic go over the bus, I'm not expecting any miracles...) The quad-core system I'm using is a Supermicro AS-2021M-UR+B; it's based on an Nvidia MCP55 chipset. The gigabit ethernet is integrated in this chipset. Using back-null we can drive this machine to over 54,000 authentications/second, at which point 100% of a core is consumed by interrupt processing in the ethernet driver. The driver doesn't support interrupt coalescing, unfortunately. (By the way, that represents somewhere between 324,000pps and 432,000pps. While there's only 5 LDAP packets per transaction, some of the client machines choose to send separate TCP ACKs, while others don't, which makes the packet count somewhere between 5-8 packets per transaction. I hadn't taken those ACKs into account when I discussed these figures before. At these packet sizes (80-140 bytes), I think the network would be 100% saturated at around 900,000pps.) Interestingly, while 2 cores can get over 13,000 auths/second, and 4 cores can get around 25,000 auths/second (using back-hdb), with all 8 cores it's only peaking at 29,000 auths/second. This tells me it's better to run two separate slapds in a mirrormode configuration on this box (4 cores per process) than to run a single process across all of the cores. Then I'd expect to hit 50,000 auths/second total, pretty close to the limits of the ethernet device/driver. -- -- Howard Chu Chief Architect, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

2 3

OpenLDAP memberof plugin and Samba4
by Andrew Bartlett 04 Jan '08

04 Jan '08

I've been working with current CVS OpenLDAP and the memberof plugin, for Samba4 integration. Following your suggestion, I'm trying to load multiple memberof instances, but the syntax doesn't seem to work for me. Attached is how I'm currently configuring the overlay. It causes this when loading: overlay_config(): overlay "memberof" already in list overlay_config(): overlay "memberof" already in list ... It also only appears to work for the first entry (happily that is member/memberof, and this seems to have worked). Is the syntax I'm using correct, or does the module need to be reworked for this operation? Finally, I'm wondering if the error returns can be adjusted: When I add invalid member to a group, OpenLDAP returns LDAP_CONSTRAINT_VIOLATION <adding non-existing object as group member>, but AD returns error 32, LDAP_NO_SUCH_OBJECT for this situation. Would it be reasonable to change this, or could it be made configurable. Having the LDAP server give me the error the client expects would avoid the need for a translation layer. (it might be nobody ever looks at this, but I don't like to make that assumption). Thanks, Andrew Bartlett -- Andrew Bartlett http://samba.org/~abartlet/ Authentication Developer, Samba Team http://samba.org Samba Developer, Red Hat Inc. http://redhat.com

4 12

HEADS-UP: pvt API change in ldap_pvt_thread_pool_setkey()?
by Pierangelo Masarati 29 Dec '07

29 Dec '07

Occasionally, I need to modify an already set value, which needs de-allocation. This right now seems to require a call to ldap_pvt_thread_pool_getkey() to get the old value first, followed by a call to ldap_pvt_thread_pool_setkey(). This requires running twice through the list of keys (not much a hassle, but not a clean interface either). I see two options: 1) let ldap_pvt_thread_pool_setkey() call the free handler, if defined, passing the old value if not null 2) change the API of ldap_pvt_thread_pool_setkey() so that it returns the old value, if passed a non-null pointer to hold it. Option 2 is more intrusive (requires multiple changes to existing code) but possibly more versatile. Comments? p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it --------------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Email: pierangelo.masarati(a)sys-net.it ---------------------------------------

1 1

liblber, sockbuf drivers
by Howard Chu 23 Dec '07

23 Dec '07

One of the things that often concerned me is that we have socket calls and references to errno in liblber, which is built as non-threaded code, but "errno" changes on many platforms in a threaded process. I was thinking that we should move the actual network I/O drivers out of liblber and into libldap, so that they'll be compiled correctly (with libldap_r) for their actual runtime environment. Not that I can point to any specific platform where this has been a problem. Just thinking out loud. -- -- Howard Chu Chief Architect, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

ldap_ntlm_bind patch from Evolution
by Russ Allbery 21 Dec '07

21 Dec '07

The Debian packaging of OpenLDAP has been carrying around the attached patch for many years now, and I'd really like to make it go away. Adding new functions to the exposed ABI of a library is really not kosher. However, it looks like Evolution really does use this patch for its Exchange addressbook feature, it falls back on doing simple binds with passwords without it, and it is apparently still a recommended patch. I'm attaching the patch as shipped with the evolution-exchange package, although the code that actually uses it is now in evolution-data-server. What should we do with this? Is what Evolution wants to do just broken? Obsolete in some way? Is there some other API that they could now use? I'm happy to file bugs against the corresponding Debian packages so that we can get rid of this patch, but I don't know what to tell them and don't even entirely understand what they're trying to accomplish. === begin quote of patch from evolution-exchange === (Note that this patch is not useful on its own... it just adds some hooks to work with the LDAP authentication process at a lower level than the API otherwise allows. The code that calls these hooks and actually drives the NTLM authentication process is in lib/e2k-global-catalog.c, and the code that actually implements the NTLM algorithms is in xntlm/.) This is a patch against OpenLDAP 2.2.6. Apply with -p0 --- include/ldap.h.orig 2004-01-01 13:16:28.000000000 -0500 +++ include/ldap.h 2004-07-14 11:58:49.000000000 -0400 @@ -1753,5 +1753,26 @@ LDAPControl **cctrls )); +/* + * hacks for NTLM + */ +#define LDAP_AUTH_NTLM_REQUEST ((ber_tag_t) 0x8aU) +#define LDAP_AUTH_NTLM_RESPONSE ((ber_tag_t) 0x8bU) +LDAP_F( int ) +ldap_ntlm_bind LDAP_P(( + LDAP *ld, + LDAP_CONST char *dn, + ber_tag_t tag, + struct berval *cred, + LDAPControl **sctrls, + LDAPControl **cctrls, + int *msgidp )); +LDAP_F( int ) +ldap_parse_ntlm_bind_result LDAP_P(( + LDAP *ld, + LDAPMessage *res, + struct berval *challenge)); + + LDAP_END_DECL #endif /* _LDAP_H */ --- libraries/libldap/Makefile.in.orig 2004-01-01 13:16:29.000000000 -0500 +++ libraries/libldap/Makefile.in 2004-07-14 13:37:23.000000000 -0400 @@ -20,7 +20,7 @@ SRCS = bind.c open.c result.c error.c compare.c search.c \ controls.c messages.c references.c extended.c cyrus.c \ modify.c add.c modrdn.c delete.c abandon.c \ - sasl.c sbind.c kbind.c unbind.c cancel.c \ + sasl.c ntlm.c sbind.c kbind.c unbind.c cancel.c \ filter.c free.c sort.c passwd.c whoami.c \ getdn.c getentry.c getattr.c getvalues.c addentry.c \ request.c os-ip.c url.c sortctrl.c vlvctrl.c \ @@ -29,7 +29,7 @@ OBJS = bind.lo open.lo result.lo error.lo compare.lo search.lo \ controls.lo messages.lo references.lo extended.lo cyrus.lo \ modify.lo add.lo modrdn.lo delete.lo abandon.lo \ - sasl.lo sbind.lo kbind.lo unbind.lo cancel.lo \ + sasl.lo ntlm.lo sbind.lo kbind.lo unbind.lo cancel.lo \ filter.lo free.lo sort.lo passwd.lo whoami.lo \ getdn.lo getentry.lo getattr.lo getvalues.lo addentry.lo \ request.lo os-ip.lo url.lo sortctrl.lo vlvctrl.lo \ --- /dev/null 2004-06-30 15:04:37.000000000 -0400 +++ libraries/libldap/ntlm.c 2004-07-14 13:44:18.000000000 -0400 @@ -0,0 +1,137 @@ +/* $OpenLDAP: pkg/ldap/libraries/libldap/ntlm.c,v 1.1.4.10 2002/01/04 20:38:21 kurt Exp $ */ +/* + * Copyright 1998-2002 The OpenLDAP Foundation, All Rights Reserved. + * COPYING RESTRICTIONS APPLY, see COPYRIGHT file + */ + +/* Mostly copied from sasl.c */ + +#include "portable.h" + +#include <stdlib.h> +#include <stdio.h> + +#include <ac/socket.h> +#include <ac/string.h> +#include <ac/time.h> +#include <ac/errno.h> + +#include "ldap-int.h" + +int +ldap_ntlm_bind( + LDAP *ld, + LDAP_CONST char *dn, + ber_tag_t tag, + struct berval *cred, + LDAPControl **sctrls, + LDAPControl **cctrls, + int *msgidp ) +{ + BerElement *ber; + int rc; + ber_int_t id; + + Debug( LDAP_DEBUG_TRACE, "ldap_ntlm_bind\n", 0, 0, 0 ); + + assert( ld != NULL ); + assert( LDAP_VALID( ld ) ); + assert( msgidp != NULL ); + + if( msgidp == NULL ) { + ld->ld_errno = LDAP_PARAM_ERROR; + return ld->ld_errno; + } + + /* create a message to send */ + if ( (ber = ldap_alloc_ber_with_options( ld )) == NULL ) { + ld->ld_errno = LDAP_NO_MEMORY; + return ld->ld_errno; + } + + assert( LBER_VALID( ber ) ); + + LDAP_NEXT_MSGID( ld, id ); + rc = ber_printf( ber, "{it{istON}" /*}*/, + id, LDAP_REQ_BIND, + ld->ld_version, dn, tag, + cred ); + + /* Put Server Controls */ + if( ldap_int_put_controls( ld, sctrls, ber ) != LDAP_SUCCESS ) { + ber_free( ber, 1 ); + return ld->ld_errno; + } + + if ( ber_printf( ber, /*{*/ "N}" ) == -1 ) { + ld->ld_errno = LDAP_ENCODING_ERROR; + ber_free( ber, 1 ); + return ld->ld_errno; + } + + /* send the message */ + *msgidp = ldap_send_initial_request( ld, LDAP_REQ_BIND, dn, ber, id ); + + if(*msgidp < 0) + return ld->ld_errno; + + return LDAP_SUCCESS; +} + +int +ldap_parse_ntlm_bind_result( + LDAP *ld, + LDAPMessage *res, + struct berval *challenge) +{ + ber_int_t errcode; + ber_tag_t tag; + BerElement *ber; + ber_len_t len; + + Debug( LDAP_DEBUG_TRACE, "ldap_parse_ntlm_bind_result\n", 0, 0, 0 ); + + assert( ld != NULL ); + assert( LDAP_VALID( ld ) ); + assert( res != NULL ); + + if ( ld == NULL || res == NULL ) { + return LDAP_PARAM_ERROR; + } + + if( res->lm_msgtype != LDAP_RES_BIND ) { + ld->ld_errno = LDAP_PARAM_ERROR; + return ld->ld_errno; + } + + if ( ld->ld_error ) { + LDAP_FREE( ld->ld_error ); + ld->ld_error = NULL; + } + if ( ld->ld_matched ) { + LDAP_FREE( ld->ld_matched ); + ld->ld_matched = NULL; + } + + /* parse results */ + + ber = ber_dup( res->lm_ber ); + + if( ber == NULL ) { + ld->ld_errno = LDAP_NO_MEMORY; + return ld->ld_errno; + } + + tag = ber_scanf( ber, "{ioa" /*}*/, + &errcode, challenge, &ld->ld_error ); + ber_free( ber, 0 ); + + if( tag == LBER_ERROR ) { + ld->ld_errno = LDAP_DECODING_ERROR; + return ld->ld_errno; + } + + ld->ld_errno = errcode; + + return( ld->ld_errno ); +} -- Russ Allbery (rra(a)stanford.edu) <http://www.eyrie.org/~eagle/>

3 4

Re: commit: ldap/doc/man/man5 slapd-bdb.5
by Howard Chu 17 Dec '07

17 Dec '07

hyc(a)OpenLDAP.org wrote: > Update of /repo/OpenLDAP/pkg/ldap/doc/man/man5 > > Modified Files: > slapd-bdb.5 1.38 -> 1.39 > > Log Message: > Support DB encryption When this topic was first raised, I thought it was pretty useless: http://www.openldap.org/lists/openldap-software/200202/msg00232.html And in general, it's not even a necessary feature: http://www.openldap.org/lists/openldap-devel/200211/msg00045.html But it seems to be a checklist feature these days. It may actually provide some value to sites that do regular backups of their raw DB files. It may actually be useful in some cases where you provide an encryption key on separate removable media (e.g. a USB flash drive). It might actually prevent a news article down the road on how some organization lost their 5 million record customer database and now all that unprotected data is now being exploited by criminals. I doubt it, of course. It exacts a performance penalty on every DB operation, so I don't think anyone will be able to use this long-term. For the off-site backup scenario, it makes more sense to just encrypt the backup images (tar format or whatever backup utility is used). That way you only spend cycles on encryption once, at backup time. Any site that's savvy enough to do automated backups can certainly figure out how to protect those backups with encryption. But the question comes up from time to time, why we don't offer this feature in the DB itself, and sometimes it's easier to just say "ok" than try to educate people. (In fact we did a custom build of OpenLDAP for a bank a few years ago, that requested this feature from us. They didn't even care about the key management, the key was just a 96 character string hardcoded into the back-bdb patch. The current patch in CVS is obviously a little better than that.) So anyway, if you're wondering, no, I still think it's a dumb solution. It's here as a marketing gimmick, for feature list checkboxes, not for any technical merit. -- -- Howard Chu Chief Architect, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

3 4

Re: commit: ldap/libraries/libldap url.c
by Hallvard B Furuseth 17 Dec '07

17 Dec '07

hyc(a)OpenLDAP.org writes: > url.c 1.104 -> 1.105 > Better fix to prev commit Actually the code was OK with not calling hex_escape_len() when there is a port number; it's only ldapi URLs which are escaped in desc2str(). Good call with IPv6 "[]" though. I guess it's time to walk through desc2str_len() and check that it matches desc2str(). -- Hallvard

1 0

Re: commit: ldap/libraries/libldap url.c
by Howard Chu 16 Dec '07

16 Dec '07

hallvard(a)OpenLDAP.org wrote: > Update of /repo/OpenLDAP/pkg/ldap/libraries/libldap > > Modified Files: > url.c 1.103 -> 1.104 > > Log Message: > Declare enough buffer space for out-of-range URL port numbers It would have been better simply to never accept out-of-range port numbers. lud_port should have been an unsigned short instead of int. Or just test for the correct range on assignment and return an error as necessary. -- -- Howard Chu Chief Architect, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

2 2

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-devel December 2007