openldap-devel

openldap-devel@openldap.org

2 participants
1422 discussions

slapd size and file descriptors
by Quanah Gibson-Mount 28 Feb '08

28 Feb '08

After some users reported that slapd had changed size enormously between two zimbra releases, I spent some time tracking down why. Apparently, at least on linux, the startup size of slapd is directly related to the number of file descriptors it can access. For example, with 1024 file descriptors, slapd was 18MB resident and around 100MB virtual in size. With some 550,000 file descriptors, slapd was 500MB+ virtual and 327MB resident. This seems a bit odd. Is this simply a "feature" of epoll() (I'm on a Linux 2.6 kernel)? --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

4 7

glue ehancement?
by Howard Chu 28 Feb '08

28 Feb '08

Seems like it would be a good idea to define a new option "glue-peer" or somesuch that allows multiple peer-level DBs to be glued together. http://www.openldap.org/lists/openldap-software/200802/msg00128.html It wouldn't really be much work in backglue.c if the set of entries is disjoint. In case of duplicate entries, we'd have to track them and either drop some of them or merge them. -- -- Howard Chu Chief Architect, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

4 6

Re: GnuTLS considered harmful
by Howard Chu 27 Feb '08

27 Feb '08

Simon Josefsson wrote: > Howard Chu<hyc(a)symas.com> writes: > >> The recent trouble in ITS#5361 prompted me to look into the GnuTLS >> code a little deeper. It turns out that their corresponding >> set_subject_alt_name() API only takes a char * pointer as input, >> without a corresponding length. As such, this API will only work for >> string-form alternative names, and will typically break with IP >> addresses and other alternatives. > > I've assigned this a ticket: > > http://trac.gnutls.org/cgi-bin/trac.cgi/ticket/18 > > It seems most SAN's in wide use are string-like, so nobody had > discovered this before. It seems clear gnutls should have a new API to > allow setting arbitrary SAN values. Not really trying to be antagonistic here, but this kind of proves my point about inexperience - SANs are pretty well documented in X.509v3. (Heck, if there's anything you can count on in ITU specs, it's that there are mountains of them.) Anyone who has spent any time with ASN.1 and X.500 knows that your safest, most uniform coding approach is to use explicit (length,data) tuples everywhere. Uniformity in the API design prevents this sort of problem ever occurring in the first place. Your bug report text is interesting because it's only concerned with truncated binary values. There's a flip side to this problem that you didn't mention, which is the potential to SEGV from accessing unmapped memory due to use of a non-terminated string. And again, an experienced C programmer would understand this issue without needing to be bludgeoned over the head with it (as I am certainly beating you up about it now). There's another issue here, due to usability. Anyone who has spent time with X.509 certificates has run into the problem of needing to specify multiple SANs to get their certificates working. Your current API only allows a single SAN value to be stored. Again, the API appears to have been designed by somebody who has never used these concepts in the real world, and doesn't understand how these items actually need to work. That's hardly reassuring in the general case, doubly worrying for a piece of security software. >> Looking across more of their APIs, I see that the code makes liberal >> use of strlen and strcat, when it needs to be using counted-length >> data blobs everywhere. In short, the code is fundamentally broken; >> most of its external and internal APIs are incapable of passing binary >> data without mangling it. The code is completely unsafe for handling >> binary data, and yet the nature of TLS processing is almost entirely >> dependent on secure handling of binary data. >> >> I strongly recommend that GnuTLS not be used. All of its APIs would >> need to be overhauled to correct its flaws and it's clear that the >> developers there are too naive and inexperienced to even understand >> that it's broken. > > I looked at the X.509 API's (x509.h) and I couldn't find any other that > didn't take buffer length arguments. I didn't look carefully though. > > There is 1 (one) use of 'strcat' in the X.509 code, and it looks correct > to me. There was 20 uses of 'strlen' in the X.509 code, and I went over > the first matches but when they looked correct I didn't look further. > (For reference, the X.509 code size is around 21000 lines of code.) > If you can give more details, that would be appreciated. Aside from the inherently unsafe nature of using strlen() on strings of unknown origin, the overall code quality is extremely poor. It looks like it was written by a Pascal programmer, someone who's accustomed to strings with embedded lengths and for which strlen() is essentially free. I hesitate to mention this aspect of it since you'll probably consider it a premature optimization. For example, minitasn1/coding.c: asn1_retCode _asn1_time_der (unsigned char *str, unsigned char *der, int *der_len) { int len_len; int max_len; max_len = *der_len; asn1_length_der (strlen (str), (max_len > 0) ? der : NULL, &len_len); if ((len_len + (int) strlen (str)) <= max_len) memcpy (der + len_len, str, strlen (str)); *der_len = len_len + strlen (str); if ((*der_len) > max_len) return ASN1_MEM_ERROR; return ASN1_SUCCESS; } There are 4 calls to strlen(str) here, which ... I can't even find the words to express how gross this is. You note that there's really a small number of instances of strcat() in the code. That's true, but that's because you've provided your own _gnutls_str_cat() function instead, which is also heavily used. Assuming that strlen() isn't going to SEGV on you (which depends on dumb luck) this becomes just a question of efficiency. E.g. in x509/dn.c you have in str_escape() str_length = MIN (strlen (str), buffer_size - 1); which is already bad because the MIN macro may evaluate that strlen() call twice. This function is called from _gnutls_x509_parse_dn() which makes heavy use of _gnutls_string_append_str() which guess what, calls strlen() on its arguments again. All of your string processing code has O(n^2) efficiency. It's incredibly sloppy and there's no justifiable reason for it. In x509/output.c your gnutls_x509_crt_print() function constructs a gnutls_string structure with a series of inefficient calls and then, the final capper: _gnutls_string_append_data (&str, "\0", 1); out->data = str.data; out->size = strlen (str.data); It walks all across the length of the string to append a single terminating NUL byte, and then it calls strlen() on the result *again*, even though str.length is already calculated for you in _gnutls_string_append_data(). If you actually had used gnutls_string structures everywhere, you *could* do all string construction operations in O(n) time (n ~ number of characters being added) by simply memcpy'ing to str.data+str.length. As it is, anything based on strcat is inherently O(n^2) and you do so many repeated traversals of the same data that I suspect the actual compute cost is even higher than n^2. -- -- Howard Chu Chief Architect, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

3 4

Re: GnuTLS considered harmful
by Howard Chu 27 Feb '08

27 Feb '08

Simon Josefsson wrote: > Howard Chu<hyc(a)symas.com> writes: >> I strongly recommend that GnuTLS not be used. All of its APIs would >> need to be overhauled to correct its flaws and it's clear that the >> developers there are too naive and inexperienced to even understand >> that it's broken. > > I looked at the X.509 API's (x509.h) and I couldn't find any other that > didn't take buffer length arguments. I didn't look carefully though. > > There is 1 (one) use of 'strcat' in the X.509 code, and it looks correct > to me. There was 20 uses of 'strlen' in the X.509 code, and I went over > the first matches but when they looked correct I didn't look further. > (For reference, the X.509 code size is around 21000 lines of code.) > > If you can give more details, that would be appreciated. And while we're at it, x509/x509.c uses memmem() which on my system says: man memmem: CONFORMING TO This function is a GNU extension. BUGS This function was broken in Linux libraries up to and including libc 5.0.9; there the `needle' and `haystack' arguments were interchanged, and a pointer to the end of the first occurrence of needle was returned. Since libc 5.0.9 is still widely used, this is a dan‐ gerous function to use. Both old and new libc's have the bug that if needle is empty haystack-1 (instead of haystack) is returned. And glibc 2.0 makes it worse, and returns a pointer to the last byte of `haystack'. This is fixed in glibc 2.1. While I would expect that most Linux sites are running something newer than glibc 2.1 these days, it's still a poor choice to use a GNU-specific library function in portable code. Not all the world runs Linux. I've been involved with the FSF since 1988 and I still have to accept the fact that the GNU way isn't the only way. -- -- Howard Chu Chief Architect, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

2 1

test050 failures
by Luca Scamoni 25 Feb '08

25 Feb '08

I'm seeing random failures with test050 and current HEAD. this time I saved the testrun dirs. There seem to be two different kind of failures: - one kind can simply be related to uncomplete synch between the different instances of slapd (database differs); - the other one worries me more: [luca@luca-nb tests]$ ./run -b hdb test050 Cleaning up test run directory leftover from previous run. Running ./scripts/test050-syncrepl-multimaster... running defines.sh Initializing server configurations... Starting producer slapd on TCP/IP port 9011... Using ldapsearch to check that producer slapd is running... Inserting syncprov overlay on producer... Starting consumer slapd on TCP/IP port 9012... Using ldapsearch to check that consumer slapd is running... Configuring syncrepl on consumer... Starting consumer2 slapd on TCP/IP port 9013... Using ldapsearch to check that consumer2 slapd is running... Configuring syncrepl on consumer2... Adding schema and databases on producer... Using ldapadd to populate producer... Waiting 20 seconds for syncrepl to receive changes... Using ldapsearch to check that syncrepl received database changes... Using ldapsearch to check that syncrepl received database changes on consumer2... Waiting 5 seconds for syncrepl to receive changes... Waiting 5 seconds for syncrepl to receive changes... Waiting 5 seconds for syncrepl to receive changes... Waiting 5 seconds for syncrepl to receive changes... Waiting 5 seconds for syncrepl to receive changes... Waiting 5 seconds for syncrepl to receive changes... ldapsearch failed (32)! see attachment Ing. Luca Scamoni Responsabile Ricerca e Sviluppo SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it ----------------------------------- Office: +39 0382 573859 (137) Mobile: +39 347 1014425 Email: luca.scamoni(a)sys-net.it -----------------------------------

3 3

GnuTLS considered harmful
by Howard Chu 22 Feb '08

22 Feb '08

The recent trouble in ITS#5361 prompted me to look into the GnuTLS code a little deeper. It turns out that their corresponding set_subject_alt_name() API only takes a char * pointer as input, without a corresponding length. As such, this API will only work for string-form alternative names, and will typically break with IP addresses and other alternatives. Looking across more of their APIs, I see that the code makes liberal use of strlen and strcat, when it needs to be using counted-length data blobs everywhere. In short, the code is fundamentally broken; most of its external and internal APIs are incapable of passing binary data without mangling it. The code is completely unsafe for handling binary data, and yet the nature of TLS processing is almost entirely dependent on secure handling of binary data. I strongly recommend that GnuTLS not be used. All of its APIs would need to be overhauled to correct its flaws and it's clear that the developers there are too naive and inexperienced to even understand that it's broken. -- -- Howard Chu Chief Architect, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

7 13

Re: commit: openldap-guide/admin overlays.sdf
by Quanah Gibson-Mount 20 Feb '08

20 Feb '08

I think it's wrong. "each overlay's function" seems correct to me. The function of a given overlay, so possessive is correct. The function belongs to the particular overlay in question. "each overlays function" seems quite incorrect to me, as now you've made overlays plural and lost the ownership of function. "Each dog's fur" for example. Not "Each dogs fur". Try googling that and look at the suggestion. ;) --Quanah --On February 16, 2008 4:55:57 AM -0800 ghenry(a)OpenLDAP.org wrote: > Update of /repo/OpenLDAP/pkg/openldap-guide/admin > > Modified Files: > overlays.sdf 1.26 -> 1.27 > > Log Message: > One change, picky ;-) > > CVS Web URLs: > http://www.openldap.org/devel/cvsweb.cgi/admin/?cvsroot=OpenLDAP-guide > > http://www.openldap.org/devel/cvsweb.cgi/admin/overlays.sdf?cvsroot=OpenL > DAP-guide > > Changes are generally available on cvs.openldap.org (and CVSweb) > within 30 minutes of being committed. -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

2 1

appendix-changes.sdf
by Gavin Henry 20 Feb '08

20 Feb '08

Hi All, Can I remove the monitor section in this now? Also, is it worth noting in that doc that you need to wipe indexes due in integer changes etc. from 2.4.6 to 2.4 current? Thanks, Gavin. -- Kind Regards, Gavin Henry. Managing Director. T +44 (0) 1224 279484 M +44 (0) 7930 323266 F +44 (0) 1224 824887 E ghenry(a)suretecsystems.com Open Source. Open Solutions(tm). http://www.suretecsystems.com/

1 0

icu -- is it the "recommended" OpenLDAP unicode implementation?
by Xin LI 18 Feb '08

18 Feb '08

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, I'd like to get an inquiry about the state of OpenLDAP's unicode implementation. Recently there is some reports that the FreeBSD port of OpenLDAP 2.4.x will pick up icu automatically if it is installed, according to the configure script, it seems that there is no way to disable this behavior (i.e. by --disable-icu or something similar), so it seems that a patch will be necessary if the user does not want to use icu. So my question is that, is it recommended to use icu? I am thinking about making icu an mandatory dependency of the port, but I guess it would be good to ask the developers before making such changes :) Cheers, - -- Xin LI <delphij(a)delphij.net> http://www.delphij.net/ FreeBSD - The Power to Serve! -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFHuP63i+vbBBjt66ARAiqcAKCuxW9xJW3qVataZpBeKVjBhbXHcQCffYy/ JhU4ZOghLdrGzYHTzQt7Ey8= =xTGY -----END PGP SIGNATURE-----

2 2

Anyone use ccache?
by Gavin Henry 18 Feb '08

18 Feb '08

Just curious, does anyone use ccache when compile/developing etc.? -- Kind Regards, Gavin Henry. Managing Director. T +44 (0) 1224 279484 M +44 (0) 7930 323266 F +44 (0) 1224 824887 E ghenry(a)suretecsystems.com Open Source. Open Solutions(tm). http://www.suretecsystems.com/

2 1

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-devel