openldap-bugs July 2019

openldap-bugs@openldap.org

18 participants
59 discussions

Re: (ITS#9055) contrib/slapd-modules/passwd/totp improvements
by quanah＠symas.com 18 Jul '19

18 Jul '19

--On Thursday, July 18, 2019 7:37 PM +0000 gv(a)members.scinet.supercomputing.org wrote: > - Allow the OTP from the previous time window to be accepted, provided > there has been no successful bind in or after that time window. This > avoids false authentication failures if for example the time window rolls > over as the OTP is being entered or transmitted. This should be a configuration item that is an integer value of the number of seconds to allow outside of the timeslice, with 0 meaning only the default time slice is allowed. Allowing people to authenticate outside of the time slice is of course a security issue and should not be allowed by default (So the default value of the parameter should be 0). Regards, Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

Re: (ITS#9055) contrib/slapd-modules/passwd/totp improvements
by hyc＠symas.com 18 Jul '19

18 Jul '19

gv(a)members.scinet.supercomputing.org wrote: > Full_Name: Greg Veldman > Version: HEAD > OS: CentOS 7 > URL: https://scinet.supercomputing.org/~gv/slapd-totp.txt > Submission from: (NULL) (128.210.189.74) > > > Improve the contrib/slapd-modules/passwd/totp module in the following ways: > > - Add support for two-factor (password+OTP) authentication, where password can > be defined via any currently-supported scheme. Your implementation of this feature is problematic, as it doesn't support setting the password using the PasswordModify exop. That seems to imply that users are required to generate their passwords using some other tool, and set them using a normal Modify op, but doing so is deprecated. Password changes should only be done using the PasswordModify exop. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#9055) contrib/slapd-modules/passwd/totp improvements
by gv＠members.scinet.supercomputing.org 18 Jul '19

18 Jul '19

Full_Name: Greg Veldman Version: HEAD OS: CentOS 7 URL: https://scinet.supercomputing.org/~gv/slapd-totp.txt Submission from: (NULL) (128.210.189.74) Improve the contrib/slapd-modules/passwd/totp module in the following ways: - Add support for two-factor (password+OTP) authentication, where password can be defined via any currently-supported scheme. - Allow the OTP from the previous time window to be accepted, provided there has been no successful bind in or after that time window. This avoids false authentication failures if for example the time window rolls over as the OTP is being entered or transmitted. The attached patch file is derived from OpenLDAP Software. All of the modifications to OpenLDAP Software represented in the following patch(es) were developed by Greg Veldman <gv(a)members.scinet.supercomputing.org>. I have not assigned rights and/or interest in this work to any party. I, Greg Veldman, hereby place the following modifications to OpenLDAP Software (and only these modifications) into the public domain. Hence, these modifications may be freely used and/or redistributed for any purpose with or without attribution and/or other notice.

1 0

Re: (ITS#8875) [Patch] Performance problems in back-mdb with large DITs and many aliases
by henrik.bohnenkamp＠ionos.com 18 Jul '19

18 Jul '19

On Mon, Jul 15, 2019 at 07:44:05PM +0100, Howard Chu wrote: > Howard Chu wrote: > > Henrik Bohnenkamp wrote: > >> On Mon, Jul 15, 2019 at 02:26:59PM +0100, Howard Chu wrote: > >>> > >>> Fyi, on our problematic test database with 11M entries and 3.7M aliases, a search with -a always , starting from the > >>> DB suffix, took 4 minutes without this patch, and 1235 minutes with this patch. > >>> > >>> Needless to say, that's not looking good. Still checking other test cases. I start to understand why this is the case. I believe that our two approaches to solve the problem each have their Nemesis... I have constructed a test case where the patched slapd needs 2 minutes (1 minute on a repeat) for a search for people objects, but the master branch has yet to finish after 30 minutes (and might not finish today). The test case is modeled similar to what is done in my company: group objects have not only member entries for people, but each member has an alias to the corresponding person object, these aliases being children to the respective group object. Furthermore, subgroups/group inclusion is modeled by adding aliases (also as children) to the group object, pointing to the group objects representing the included groups. So there is some recursive descent to get all person objects of all subgroups. In total I have 5 mio entries, 3 mio of which are aliases. There are no cycles, but in this test case the depth of the recursion is absolutely unrealistic. I believe, in the master-slapd, each step down in the recursion needs a call to mdb_idscope, which in turn goes through all 3 mio aliases to compute the intersection with the scope of the group object (that's what I remember, but I have not looked at the algorithm for a year now). The patch traverses in this case just very small sub-trees (although many of those) to collect the relevant aliases and continues from there. That seems to be much more efficient here. If I understand your description above correctly, my patch sucks with your test case because it does a full traversal of the LDAP tree to collect all aliases ... with 11 mio entries that takes a while. The master branch, however, just computes intersection with the scope once and continues from there, which must be much faster. I actually do not know what to conclude from all this. I wonder however, if a combination of both approaches could be beneficial. With the patch the diskNode of an entry contains not only the number of children in the scope, but also the number of aliases. If the total number of aliases in the database is large, but the search scope is small, the tree traversal might be faster. If the number of entries in the scope is big, and there are aliases in the search scope, the intersection in mdb_idscope might be more efficient. If there are no aliases in the search scope, nothing has to be done at all.

1 0

Re: (ITS#9054) Add support for multiple EECDH curves
by quanah＠symas.com 16 Jul '19

16 Jul '19

--On Tuesday, July 16, 2019 9:45 PM +0000 quanah(a)openldap.org wrote: > Full_Name: Quanah Gibson-Mount > Version: 2.4.47 > OS: N/A > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (47.208.128.44) > > > Currently OpenLDAP only allows for a single EECDH curve to be configured. > However, OpenSSL 1.0.2 released in January 2015 was the first release to > implement negotiation of supported curves in TLS servers. OpenLDAP needs > updating to support this functionality. tls_dh.c in postfix/src/tls_dh.c gives some insight into how to correctly do this with OpenSSL, in the tls_auto_eecdh_curves fucntion. --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

(ITS#9054) Add support for multiple EECDH curves
by quanah＠openldap.org 16 Jul '19

16 Jul '19

Full_Name: Quanah Gibson-Mount Version: 2.4.47 OS: N/A URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (47.208.128.44) Currently OpenLDAP only allows for a single EECDH curve to be configured. However, OpenSSL 1.0.2 released in January 2015 was the first release to implement negotiation of supported curves in TLS servers. OpenLDAP needs updating to support this functionality.

1 0

Re: (ITS#7657) Alias dereferencing with MDB slow compared with BDB
by Mark.Cairney＠ed.ac.uk 16 Jul '19

16 Jul '19

Hi Howard, Brilliant- I notice that there's a 2.4.48 RC due out shortly so Ill roll that out on a Dev box and see how it performs. Kind regards, Mark On 15/07/2019 17:07, Howard Chu wrote: > Fyi, we recently revisited this issue and are seeing improved results with > commit e90e8c7d3c12d897bb0584ba04dc519d4f23acf9 in master. > -- /**************************** Mark Cairney ITI Enterprise Services Information Services University of Edinburgh Tel: 0131 650 6565 Email: Mark.Cairney(a)ed.ac.uk PGP: 0x435A9621 *******************************/ The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.

1 0

Re: (ITS#8875) [Patch] Performance problems in back-mdb with large DITs and many aliases
by hyc＠symas.com 15 Jul '19

15 Jul '19

This is a multi-part message in MIME format. --------------AED2FFEE73F73B1613190B64 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit hyc(a)symas.com wrote: > The tool to generate the test LDIF is attached. It reads an LDIF containing 500,000 users on stdin, and outputs the same LDIF, > with aliases interspersed, on stdout. > Slightly tweaked, creates the alias after the target entry. In case the server does referential integrity on loading. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/ --------------AED2FFEE73F73B1613190B64 Content-Type: text/x-csrc; name="mkalias.c" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="mkalias.c" #include <stdio.h> #include <stdlib.h> #include <string.h> #include <time.h> #define NUMUSERS 500000 #define NUMALIASES 30000 #define NUMSUBALIASES 435 #define SUFFIX "dc=example,dc=com" static const char ou1[] = "\ dn: ou=alias1," SUFFIX "\n\ objectclass: top\n\ objectclass: organizationalUnit\n\ ou: alias1\n"; static const char ou2[] = "\ dn: ou=alias2," SUFFIX "\n\ objectclass: top\n\ objectclass: organizationalUnit\n\ ou: alias2\n"; int qcmp(const void *a, const void *b) { const int *i = a, *j = b; return *i - *j; } int main() { char line[1024], dn[1024], *ou; int aliases[NUMALIASES]; int subaliases[NUMSUBALIASES]; int i; int nusers=0, naliases=0, nsubaliases=0, showalias=0; /* select a random subset of users to generate aliases */ srand(time(0L)); for (i=0; i<NUMALIASES; i++) { aliases[i] = rand() % NUMUSERS; } uniq1: qsort(aliases, NUMALIASES, sizeof(int), qcmp); /* make sure they're unique */ for (i=1; i<NUMALIASES; i++) { if (aliases[i-1] == aliases[i]) { aliases[i] = rand() % NUMUSERS; goto uniq1; } } /* select a random susbset of aliases for the target subtree */ for (i=0; i<NUMSUBALIASES; i++) { subaliases[i] = rand() % NUMALIASES; } uniq2: qsort(subaliases, NUMSUBALIASES, sizeof(int), qcmp); /* make sure they're unique */ for (i=1; i<NUMSUBALIASES; i++) { if (subaliases[i-1] == subaliases[i]) { subaliases[i] = rand() % NUMALIASES; goto uniq2; } } for (i=0; i<NUMSUBALIASES; i++) { subaliases[i] = aliases[subaliases[i]]; } /* read LDIF */ while (fgets(line, sizeof(line), stdin) != NULL) { if (line[0] == '#') continue; if (!strncmp(line, "dn: ", 4)) { /* we assume all users' DNs use uid for RDN */ if (!strncmp(line+4, "uid=", 4)) { if (!nusers) { puts(ou1); puts(ou2); } if (showalias) { printf("dn: uid=x.%d,ou=%s," SUFFIX "\n", nusers, ou); puts("objectclass: alias"); puts("objectclass: extensibleObject"); printf("aliasedObjectName: %s\n", dn); showalias = 0; } if (nusers == aliases[naliases] || nusers == subaliases[nsubaliases]) { if (nusers == subaliases[nsubaliases]) { ou = "alias2"; nsubaliases++; } else { ou = "alias1"; } naliases++; strcpy(dn, line+4); showalias = 1; } nusers++; } } fputs(line, stdout); } return 0; } --------------AED2FFEE73F73B1613190B64--

1 0

Re: (ITS#8875) [Patch] Performance problems in back-mdb with large DITs and many aliases
by hyc＠symas.com 15 Jul '19

15 Jul '19

This is a multi-part message in MIME format. --------------0F1DE22CD9273A65E1AE5118 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Howard Chu wrote: > Henrik Bohnenkamp wrote: >> On Mon, Jul 15, 2019 at 02:26:59PM +0100, Howard Chu wrote: >>> >>> Fyi, on our problematic test database with 11M entries and 3.7M aliases, a search with -a always , starting from the >>> DB suffix, took 4 minutes without this patch, and 1235 minutes with this patch. >>> >>> Needless to say, that's not looking good. Still checking other test cases. >> >> Interesting, so the behavior is reversed now :-). I assume you have >> found an alternative approach to solve the problem. That's fine with >> me, I want the problem solved, not my patch integrated. I'm of course >> interested in how you do it. Surely you did not get the 4 minutes with >> a stock 2.4.48 slapd? > > For this size of DB we needed the ITS#8977 patches to accommodate larger IDLs. > (I used 24 bits for IDLs, 16.7M slots) > Also at this size, the IDL processing itself is the main bottleneck now. We would > need to switch to bitmaps or trees to avoid this bottleneck, but that's also a > much larger change than we can consider for this release. > I've set up a more modest test database along the lines of ITS#7657. It has 500,000 users, 30,000 aliases total, and 435 in ou=alias2 (all the rest under ou=alias1). For unpatched back-mdb: time ../clients/tools/ldapsearch -x -H ldap://:9012 -D cn=manager,dc=example,dc=com -w secret -b ou=alias1,dc=example,dc=com -a always # search result search: 2 result: 0 Success # numResponses: 29567 # numEntries: 29566 real 0m42.504s user 0m1.344s sys 0m2.996s time ../clients/tools/ldapsearch -x -H ldap://:9012 -D cn=manager,dc=example,dc=com -w secret -b ou=alias2,dc=example,dc=com -a always # search result search: 2 result: 0 Success # numResponses: 437 # numEntries: 436 real 0m48.406s user 0m0.040s sys 0m0.076s For back-mdb with e90e8c7d3c12d897bb0584ba04dc519d4f23acf9 time ../clients/tools/ldapsearch -x -H ldap://:9012 -D cn=manager,dc=example,dc=com -w secret -b ou=alias1,dc=example,dc=com -a always # search result search: 2 result: 0 Success # numResponses: 29567 # numEntries: 29566 real 0m5.500s user 0m1.516s sys 0m2.944s time ../clients/tools/ldapsearch -x -H ldap://:9012 -D cn=manager,dc=example,dc=com -w secret -b ou=alias2,dc=example,dc=com -a always # search result search: 2 result: 0 Success # numResponses: 437 # numEntries: 436 real 0m0.399s user 0m0.048s sys 0m0.060s For back-mdb with this ITS#8875 patch time ../clients/tools/ldapsearch -x -H ldap://:9012 -D cn=manager,dc=example,dc=com -w secret -b ou=alias1,dc=example,dc=com -a always # search result search: 2 result: 0 Success # numResponses: 29567 # numEntries: 29566 real 0m6.020s user 0m1.640s sys 0m3.372s time ../clients/tools/ldapsearch -x -H ldap://:9012 -D cn=manager,dc=example,dc=com -w secret -b ou=alias2,dc=example,dc=com -a always # search result search: 2 result: 0 Success # numResponses: 437 # numEntries: 436 real 0m0.203s user 0m0.052s sys 0m0.048s It seems close enough in this case (I didn't do enough repeated runs to average out any measurement error) while the committed patch performs better on the really ugly test case. The tool to generate the test LDIF is attached. It reads an LDIF containing 500,000 users on stdin, and outputs the same LDIF, with aliases interspersed, on stdout. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/ --------------0F1DE22CD9273A65E1AE5118 Content-Type: text/x-csrc; name="mkalias.c" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="mkalias.c" #include <stdio.h> #include <stdlib.h> #include <string.h> #include <time.h> #define NUMUSERS 500000 #define NUMALIASES 30000 #define NUMSUBALIASES 435 static const char ou1[] = "\ dn: ou=alias1,dc=example,dc=com\n\ objectclass: top\n\ objectclass: organizationalUnit\n\ ou: alias1\n"; static const char ou2[] = "\ dn: ou=alias2,dc=example,dc=com\n\ objectclass: top\n\ objectclass: organizationalUnit\n\ ou: alias2\n"; int qcmp(const void *a, const void *b) { const int *i = a, *j = b; return *i - *j; } int main() { char line[1024]; int aliases[NUMALIASES]; int subaliases[NUMSUBALIASES]; int i; int gotuser = 0, nusers=0, naliases=0, nsubaliases=0; /* select a random subset of users to generate aliases */ srand(time(0L)); for (i=0; i<NUMALIASES; i++) { aliases[i] = rand() % NUMUSERS; } uniq1: qsort(aliases, NUMALIASES, sizeof(int), qcmp); /* make sure they're unique */ for (i=1; i<NUMALIASES; i++) { if (aliases[i-1] == aliases[i]) { aliases[i] = rand() % NUMUSERS; goto uniq1; } } /* select a radnom susbset of aliases for the target subtree */ for (i=0; i<NUMSUBALIASES; i++) { subaliases[i] = rand() % NUMALIASES; } uniq2: qsort(subaliases, NUMSUBALIASES, sizeof(int), qcmp); /* make sure they're unique */ for (i=1; i<NUMSUBALIASES; i++) { if (subaliases[i-1] == subaliases[i]) { subaliases[i] = rand() % NUMALIASES; goto uniq2; } } for (i=0; i<NUMSUBALIASES; i++) { subaliases[i] = aliases[subaliases[i]]; } while (fgets(line, sizeof(line), stdin) != NULL) { if (line[0] == '#') continue; if (!strncmp(line, "dn: ", 4)) { if (!gotuser) { if (!strncmp(line+4, "uid=", 4)) { gotuser = 1; puts(ou1); puts(ou2); } } else { if (nusers == aliases[naliases] || nusers == subaliases[nsubaliases]) { char *ou; int id; sscanf(line, "dn: uid=user.%d,", &id); if (nusers == subaliases[nsubaliases]) { ou = "alias2"; nsubaliases++; } else { ou = "alias1"; } naliases++; printf("dn: uid=x.%d,ou=%s,dc=example,dc=com\n", id, ou); puts("objectclass: alias"); puts("objectclass: extensibleObject"); printf("aliasedObjectName: %s\n", line+4); } nusers++; } } fputs(line, stdout); } return 0; } --------------0F1DE22CD9273A65E1AE5118--

1 0

Re: (ITS#7657) Alias dereferencing with MDB slow compared with BDB
by Howard Chu 15 Jul '19

15 Jul '19

Fyi, we recently revisited this issue and are seeing improved results with commit e90e8c7d3c12d897bb0584ba04dc519d4f23acf9 in master.

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs July 2019