dITStructureRules/nameForms in subschema subentry for informational purpose
by Michael Ströder
HI!
Discussed this very briefly with Howard at LDAPcon 2007 based on an idea
of Steve:
Support for dITStructureRules and nameForms is still in OpenLDAP's TODO.
In the meanwhile slapd could accept definitions for both in slapd.conf
and simply pass them on to a schema-aware LDAP client for informational
purpose without enforcing them. Same function like rootDSE <file> in
slapd.conf.
Opinions?
Ciao, Michael.
--
Michael Ströder
E-Mail: michael(a)stroeder.com
http://www.stroeder.com
14 years, 2 months
Re: thread pools, performance
by Howard Chu
Rick Jones wrote:
> There are definitely interrupt coalescing settings available with tg3-driven
> cards, as well as bnx2 driven ones:
>
> ftp://ftp.cup.hp.com/dist/networking/briefs/nic_latency_vs_tput.txt
Yep, that helped. Raising rx-usecs from default 20 to 1000, and rx-frames from
default 5 to 100, I'm getting 43k auths/sec with back-null (in 4 separate
thread pools) and the core fielding the interrupts is only about 80% busy now
instead of 100%. I'm afraid my load generators may be maxed out now, because I
can't seem to drive up the load on the server any higher even though there's
more idle CPU.
The current code in HEAD (with only 1 thread pool) is reaching 36k auths/sec
with back-null, so it's actually not far off from my experimental peak rate.
Considering that HEAD was at 25k/sec last week (and now in 2.4.6) that's
pretty decent.
With back-bdb and 1 million users I'm getting 26.1k/sec with plaintext
passwords (up from 19.3k/sec last week). With {SSHA} passwords that drops to
25.7k/sec (~1.5% difference).
I have to put this tinkering on hold for a bit, to run some authrate tests
against ActiveDirectory on this machine (using W2K3sp2 X64). Later on we'll do
a W2K3 OpenLDAP build for comparison as well. Should be entertaining...
--
-- Howard Chu
Chief Architect, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
15 years, 3 months
CPU scaling
by Howard Chu
Has anyone got a dual or quad socket Intel Xeon based server for testing? I've
been testing on two AMD systems, one quad socket dual core and one dual socket
quad core. There are a lot of different ways to tune these systems...
slapd currently uses a single listener thread and a pool of some number of
worker threads. I've found that performance improves significantly when the
listener thread is pinned to a single core, and no other threads are allowed
to run there. I've also found that performance improves somewhat when all
worker threads are pinned to specific cores, instead of being free to run on
any of the remaining cores. This has made testing a bit more complicated than
I expected.
I originally was just pinning the entire process to a set number of cores
(first 1, then 2, incrementing up to 8) to see how performance changed with
additional cores. But due to the motherboard layout and the fact that the I/O
bridges are directly attached to particular sockets, it makes a big difference
exactly which cores you use.
Another item I noticed is that while we scale perfectly linearly from 1 core
to 2 cores in a socket (with a dual-core processor), as we start spreading
across multiple sockets the scaling tapers off drastically. That makes sense
given the constraints of the Hypertransport connections between the sockets.
On the quad-core system we scale pretty linearly from 1 to 4 cores (in one
socket) but again the improvement tapers off drastically when the 2nd socket
is added in.
I don't have any Xeon systems to test on at the moment, but I'm curious to see
how they do given that all CPUs should have equal access to the northbridge.
(Of course, given that both memory and I/O traffic go over the bus, I'm not
expecting any miracles...)
The quad-core system I'm using is a Supermicro AS-2021M-UR+B; it's based on an
Nvidia MCP55 chipset. The gigabit ethernet is integrated in this chipset.
Using back-null we can drive this machine to over 54,000
authentications/second, at which point 100% of a core is consumed by interrupt
processing in the ethernet driver. The driver doesn't support interrupt
coalescing, unfortunately. (By the way, that represents somewhere between
324,000pps and 432,000pps. While there's only 5 LDAP packets per transaction,
some of the client machines choose to send separate TCP ACKs, while others
don't, which makes the packet count somewhere between 5-8 packets per
transaction. I hadn't taken those ACKs into account when I discussed these
figures before. At these packet sizes (80-140 bytes), I think the network
would be 100% saturated at around 900,000pps.)
Interestingly, while 2 cores can get over 13,000 auths/second, and 4 cores can
get around 25,000 auths/second (using back-hdb), with all 8 cores it's only
peaking at 29,000 auths/second. This tells me it's better to run two separate
slapds in a mirrormode configuration on this box (4 cores per process) than to
run a single process across all of the cores. Then I'd expect to hit 50,000
auths/second total, pretty close to the limits of the ethernet device/driver.
--
-- Howard Chu
Chief Architect, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
15 years, 4 months
OpenLDAP memberof plugin and Samba4
by Andrew Bartlett
I've been working with current CVS OpenLDAP and the memberof plugin, for
Samba4 integration.
Following your suggestion, I'm trying to load multiple memberof
instances, but the syntax doesn't seem to work for me. Attached is how
I'm currently configuring the overlay. It causes this when loading:
overlay_config(): overlay "memberof" already in list
overlay_config(): overlay "memberof" already in list
...
It also only appears to work for the first entry (happily that is
member/memberof, and this seems to have worked).
Is the syntax I'm using correct, or does the module need to be reworked
for this operation?
Finally, I'm wondering if the error returns can be adjusted:
When I add invalid member to a group, OpenLDAP returns
LDAP_CONSTRAINT_VIOLATION <adding non-existing object as group member>,
but AD returns error 32, LDAP_NO_SUCH_OBJECT for this situation. Would
it be reasonable to change this, or could it be made configurable.
Having the LDAP server give me the error the client expects would avoid
the need for a translation layer. (it might be nobody ever looks at
this, but I don't like to make that assumption).
Thanks,
Andrew Bartlett
--
Andrew Bartlett http://samba.org/~abartlet/
Authentication Developer, Samba Team http://samba.org
Samba Developer, Red Hat Inc. http://redhat.com
15 years, 4 months
HEADS-UP: pvt API change in ldap_pvt_thread_pool_setkey()?
by Pierangelo Masarati
Occasionally, I need to modify an already set value, which needs
de-allocation. This right now seems to require a call to
ldap_pvt_thread_pool_getkey() to get the old value first, followed by a
call to ldap_pvt_thread_pool_setkey(). This requires running twice
through the list of keys (not much a hassle, but not a clean interface
either). I see two options:
1) let ldap_pvt_thread_pool_setkey() call the free handler, if defined,
passing the old value if not null
2) change the API of ldap_pvt_thread_pool_setkey() so that it returns
the old value, if passed a non-null pointer to hold it.
Option 2 is more intrusive (requires multiple changes to existing code)
but possibly more versatile.
Comments?
p.
Ing. Pierangelo Masarati
OpenLDAP Core Team
SysNet s.r.l.
via Dossi, 8 - 27100 Pavia - ITALIA
http://www.sys-net.it
---------------------------------------
Office: +39 02 23998309
Mobile: +39 333 4963172
Email: pierangelo.masarati(a)sys-net.it
---------------------------------------
15 years, 4 months
liblber, sockbuf drivers
by Howard Chu
One of the things that often concerned me is that we have socket calls and
references to errno in liblber, which is built as non-threaded code, but
"errno" changes on many platforms in a threaded process. I was thinking that
we should move the actual network I/O drivers out of liblber and into libldap,
so that they'll be compiled correctly (with libldap_r) for their actual
runtime environment.
Not that I can point to any specific platform where this has been a problem.
Just thinking out loud.
--
-- Howard Chu
Chief Architect, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
15 years, 5 months
ldap_ntlm_bind patch from Evolution
by Russ Allbery
The Debian packaging of OpenLDAP has been carrying around the attached
patch for many years now, and I'd really like to make it go away. Adding
new functions to the exposed ABI of a library is really not kosher.
However, it looks like Evolution really does use this patch for its
Exchange addressbook feature, it falls back on doing simple binds with
passwords without it, and it is apparently still a recommended patch. I'm
attaching the patch as shipped with the evolution-exchange package,
although the code that actually uses it is now in evolution-data-server.
What should we do with this? Is what Evolution wants to do just broken?
Obsolete in some way? Is there some other API that they could now use?
I'm happy to file bugs against the corresponding Debian packages so that
we can get rid of this patch, but I don't know what to tell them and don't
even entirely understand what they're trying to accomplish.
=== begin quote of patch from evolution-exchange ===
(Note that this patch is not useful on its own... it just adds some
hooks to work with the LDAP authentication process at a lower level
than the API otherwise allows. The code that calls these hooks and
actually drives the NTLM authentication process is in
lib/e2k-global-catalog.c, and the code that actually implements the
NTLM algorithms is in xntlm/.)
This is a patch against OpenLDAP 2.2.6. Apply with -p0
--- include/ldap.h.orig 2004-01-01 13:16:28.000000000 -0500
+++ include/ldap.h 2004-07-14 11:58:49.000000000 -0400
@@ -1753,5 +1753,26 @@
LDAPControl **cctrls ));
+/*
+ * hacks for NTLM
+ */
+#define LDAP_AUTH_NTLM_REQUEST ((ber_tag_t) 0x8aU)
+#define LDAP_AUTH_NTLM_RESPONSE ((ber_tag_t) 0x8bU)
+LDAP_F( int )
+ldap_ntlm_bind LDAP_P((
+ LDAP *ld,
+ LDAP_CONST char *dn,
+ ber_tag_t tag,
+ struct berval *cred,
+ LDAPControl **sctrls,
+ LDAPControl **cctrls,
+ int *msgidp ));
+LDAP_F( int )
+ldap_parse_ntlm_bind_result LDAP_P((
+ LDAP *ld,
+ LDAPMessage *res,
+ struct berval *challenge));
+
+
LDAP_END_DECL
#endif /* _LDAP_H */
--- libraries/libldap/Makefile.in.orig 2004-01-01 13:16:29.000000000 -0500
+++ libraries/libldap/Makefile.in 2004-07-14 13:37:23.000000000 -0400
@@ -20,7 +20,7 @@
SRCS = bind.c open.c result.c error.c compare.c search.c \
controls.c messages.c references.c extended.c cyrus.c \
modify.c add.c modrdn.c delete.c abandon.c \
- sasl.c sbind.c kbind.c unbind.c cancel.c \
+ sasl.c ntlm.c sbind.c kbind.c unbind.c cancel.c \
filter.c free.c sort.c passwd.c whoami.c \
getdn.c getentry.c getattr.c getvalues.c addentry.c \
request.c os-ip.c url.c sortctrl.c vlvctrl.c \
@@ -29,7 +29,7 @@
OBJS = bind.lo open.lo result.lo error.lo compare.lo search.lo \
controls.lo messages.lo references.lo extended.lo cyrus.lo \
modify.lo add.lo modrdn.lo delete.lo abandon.lo \
- sasl.lo sbind.lo kbind.lo unbind.lo cancel.lo \
+ sasl.lo ntlm.lo sbind.lo kbind.lo unbind.lo cancel.lo \
filter.lo free.lo sort.lo passwd.lo whoami.lo \
getdn.lo getentry.lo getattr.lo getvalues.lo addentry.lo \
request.lo os-ip.lo url.lo sortctrl.lo vlvctrl.lo \
--- /dev/null 2004-06-30 15:04:37.000000000 -0400
+++ libraries/libldap/ntlm.c 2004-07-14 13:44:18.000000000 -0400
@@ -0,0 +1,137 @@
+/* $OpenLDAP: pkg/ldap/libraries/libldap/ntlm.c,v 1.1.4.10 2002/01/04 20:38:21 kurt Exp $ */
+/*
+ * Copyright 1998-2002 The OpenLDAP Foundation, All Rights Reserved.
+ * COPYING RESTRICTIONS APPLY, see COPYRIGHT file
+ */
+
+/* Mostly copied from sasl.c */
+
+#include "portable.h"
+
+#include <stdlib.h>
+#include <stdio.h>
+
+#include <ac/socket.h>
+#include <ac/string.h>
+#include <ac/time.h>
+#include <ac/errno.h>
+
+#include "ldap-int.h"
+
+int
+ldap_ntlm_bind(
+ LDAP *ld,
+ LDAP_CONST char *dn,
+ ber_tag_t tag,
+ struct berval *cred,
+ LDAPControl **sctrls,
+ LDAPControl **cctrls,
+ int *msgidp )
+{
+ BerElement *ber;
+ int rc;
+ ber_int_t id;
+
+ Debug( LDAP_DEBUG_TRACE, "ldap_ntlm_bind\n", 0, 0, 0 );
+
+ assert( ld != NULL );
+ assert( LDAP_VALID( ld ) );
+ assert( msgidp != NULL );
+
+ if( msgidp == NULL ) {
+ ld->ld_errno = LDAP_PARAM_ERROR;
+ return ld->ld_errno;
+ }
+
+ /* create a message to send */
+ if ( (ber = ldap_alloc_ber_with_options( ld )) == NULL ) {
+ ld->ld_errno = LDAP_NO_MEMORY;
+ return ld->ld_errno;
+ }
+
+ assert( LBER_VALID( ber ) );
+
+ LDAP_NEXT_MSGID( ld, id );
+ rc = ber_printf( ber, "{it{istON}" /*}*/,
+ id, LDAP_REQ_BIND,
+ ld->ld_version, dn, tag,
+ cred );
+
+ /* Put Server Controls */
+ if( ldap_int_put_controls( ld, sctrls, ber ) != LDAP_SUCCESS ) {
+ ber_free( ber, 1 );
+ return ld->ld_errno;
+ }
+
+ if ( ber_printf( ber, /*{*/ "N}" ) == -1 ) {
+ ld->ld_errno = LDAP_ENCODING_ERROR;
+ ber_free( ber, 1 );
+ return ld->ld_errno;
+ }
+
+ /* send the message */
+ *msgidp = ldap_send_initial_request( ld, LDAP_REQ_BIND, dn, ber, id );
+
+ if(*msgidp < 0)
+ return ld->ld_errno;
+
+ return LDAP_SUCCESS;
+}
+
+int
+ldap_parse_ntlm_bind_result(
+ LDAP *ld,
+ LDAPMessage *res,
+ struct berval *challenge)
+{
+ ber_int_t errcode;
+ ber_tag_t tag;
+ BerElement *ber;
+ ber_len_t len;
+
+ Debug( LDAP_DEBUG_TRACE, "ldap_parse_ntlm_bind_result\n", 0, 0, 0 );
+
+ assert( ld != NULL );
+ assert( LDAP_VALID( ld ) );
+ assert( res != NULL );
+
+ if ( ld == NULL || res == NULL ) {
+ return LDAP_PARAM_ERROR;
+ }
+
+ if( res->lm_msgtype != LDAP_RES_BIND ) {
+ ld->ld_errno = LDAP_PARAM_ERROR;
+ return ld->ld_errno;
+ }
+
+ if ( ld->ld_error ) {
+ LDAP_FREE( ld->ld_error );
+ ld->ld_error = NULL;
+ }
+ if ( ld->ld_matched ) {
+ LDAP_FREE( ld->ld_matched );
+ ld->ld_matched = NULL;
+ }
+
+ /* parse results */
+
+ ber = ber_dup( res->lm_ber );
+
+ if( ber == NULL ) {
+ ld->ld_errno = LDAP_NO_MEMORY;
+ return ld->ld_errno;
+ }
+
+ tag = ber_scanf( ber, "{ioa" /*}*/,
+ &errcode, challenge, &ld->ld_error );
+ ber_free( ber, 0 );
+
+ if( tag == LBER_ERROR ) {
+ ld->ld_errno = LDAP_DECODING_ERROR;
+ return ld->ld_errno;
+ }
+
+ ld->ld_errno = errcode;
+
+ return( ld->ld_errno );
+}
--
Russ Allbery (rra(a)stanford.edu) <http://www.eyrie.org/~eagle/>
15 years, 5 months
Re: commit: ldap/doc/man/man5 slapd-bdb.5
by Howard Chu
hyc(a)OpenLDAP.org wrote:
> Update of /repo/OpenLDAP/pkg/ldap/doc/man/man5
>
> Modified Files:
> slapd-bdb.5 1.38 -> 1.39
>
> Log Message:
> Support DB encryption
When this topic was first raised, I thought it was pretty useless:
http://www.openldap.org/lists/openldap-software/200202/msg00232.html
And in general, it's not even a necessary feature:
http://www.openldap.org/lists/openldap-devel/200211/msg00045.html
But it seems to be a checklist feature these days.
It may actually provide some value to sites that do regular backups of their
raw DB files. It may actually be useful in some cases where you provide an
encryption key on separate removable media (e.g. a USB flash drive). It might
actually prevent a news article down the road on how some organization lost
their 5 million record customer database and now all that unprotected data is
now being exploited by criminals.
I doubt it, of course. It exacts a performance penalty on every DB operation,
so I don't think anyone will be able to use this long-term. For the off-site
backup scenario, it makes more sense to just encrypt the backup images (tar
format or whatever backup utility is used). That way you only spend cycles on
encryption once, at backup time. Any site that's savvy enough to do automated
backups can certainly figure out how to protect those backups with encryption.
But the question comes up from time to time, why we don't offer this feature
in the DB itself, and sometimes it's easier to just say "ok" than try to
educate people. (In fact we did a custom build of OpenLDAP for a bank a few
years ago, that requested this feature from us. They didn't even care about
the key management, the key was just a 96 character string hardcoded into the
back-bdb patch. The current patch in CVS is obviously a little better than that.)
So anyway, if you're wondering, no, I still think it's a dumb solution. It's
here as a marketing gimmick, for feature list checkboxes, not for any
technical merit.
--
-- Howard Chu
Chief Architect, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
15 years, 5 months
Re: commit: ldap/libraries/libldap url.c
by Hallvard B Furuseth
hyc(a)OpenLDAP.org writes:
> url.c 1.104 -> 1.105
> Better fix to prev commit
Actually the code was OK with not calling hex_escape_len() when there is
a port number; it's only ldapi URLs which are escaped in desc2str().
Good call with IPv6 "[]" though. I guess it's time to walk through
desc2str_len() and check that it matches desc2str().
--
Hallvard
15 years, 5 months
Re: commit: ldap/libraries/libldap url.c
by Howard Chu
hallvard(a)OpenLDAP.org wrote:
> Update of /repo/OpenLDAP/pkg/ldap/libraries/libldap
>
> Modified Files:
> url.c 1.103 -> 1.104
>
> Log Message:
> Declare enough buffer space for out-of-range URL port numbers
It would have been better simply to never accept out-of-range port numbers.
lud_port should have been an unsigned short instead of int. Or just test for
the correct range on assignment and return an error as necessary.
--
-- Howard Chu
Chief Architect, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
15 years, 5 months
