Re: (ITS#9055) contrib/slapd-modules/passwd/totp improvements
by quanah＠symas.com
--On Thursday, July 18, 2019 7:37 PM +0000
gv(a)members.scinet.supercomputing.org wrote:
> - Allow the OTP from the previous time window to be accepted, provided
> there has been no successful bind in or after that time window. This
> avoids false authentication failures if for example the time window rolls
> over as the OTP is being entered or transmitted.
This should be a configuration item that is an integer value of the number
of seconds to allow outside of the timeslice, with 0 meaning only the
default time slice is allowed. Allowing people to authenticate outside of
the time slice is of course a security issue and should not be allowed by
default (So the default value of the parameter should be 0).
Regards,
Quanah
--
Quanah Gibson-Mount
Product Architect
Symas Corporation
Packaged, certified, and supported LDAP solutions powered by OpenLDAP:
<http://www.symas.com>
4 years, 2 months
Re: (ITS#9055) contrib/slapd-modules/passwd/totp improvements
by hyc＠symas.com
gv(a)members.scinet.supercomputing.org wrote:
> Full_Name: Greg Veldman
> Version: HEAD
> OS: CentOS 7
> URL: https://scinet.supercomputing.org/~gv/slapd-totp.txt
> Submission from: (NULL) (128.210.189.74)
>
>
> Improve the contrib/slapd-modules/passwd/totp module in the following ways:
>
> - Add support for two-factor (password+OTP) authentication, where password can
> be defined via any currently-supported scheme.
Your implementation of this feature is problematic, as it doesn't support setting
the password using the PasswordModify exop. That seems to imply that users are
required to generate their passwords using some other tool, and set them using a
normal Modify op, but doing so is deprecated. Password changes should only be done
using the PasswordModify exop.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
4 years, 2 months
(ITS#9055) contrib/slapd-modules/passwd/totp improvements
by gv＠members.scinet.supercomputing.org
Full_Name: Greg Veldman
Version: HEAD
OS: CentOS 7
URL: https://scinet.supercomputing.org/~gv/slapd-totp.txt
Submission from: (NULL) (128.210.189.74)
Improve the contrib/slapd-modules/passwd/totp module in the following ways:
- Add support for two-factor (password+OTP) authentication, where password can
be defined via any currently-supported scheme.
- Allow the OTP from the previous time window to be accepted, provided there has
been no successful bind in or after that time window. This avoids false
authentication failures if for example the time window rolls over as the OTP is
being entered or transmitted.
The attached patch file is derived from OpenLDAP Software. All of the
modifications to OpenLDAP Software represented in the following patch(es) were
developed by Greg Veldman <gv(a)members.scinet.supercomputing.org>. I have not
assigned rights and/or interest in this work to any party.
I, Greg Veldman, hereby place the following modifications to OpenLDAP Software
(and only these modifications) into the public domain. Hence, these
modifications may be freely used and/or redistributed for any purpose with or
without attribution and/or other notice.
4 years, 2 months
Re: (ITS#8875) [Patch] Performance problems in back-mdb with large DITs and many aliases
by henrik.bohnenkamp＠ionos.com
On Mon, Jul 15, 2019 at 07:44:05PM +0100, Howard Chu wrote:
> Howard Chu wrote:
> > Henrik Bohnenkamp wrote:
> >> On Mon, Jul 15, 2019 at 02:26:59PM +0100, Howard Chu wrote:
> >>>
> >>> Fyi, on our problematic test database with 11M entries and 3.7M aliases, a search with -a always , starting from the
> >>> DB suffix, took 4 minutes without this patch, and 1235 minutes with this patch.
> >>>
> >>> Needless to say, that's not looking good. Still checking other test cases.
I start to understand why this is the case. I believe that our two
approaches to solve the problem each have their Nemesis... I have
constructed a test case where the patched slapd needs 2 minutes (1
minute on a repeat) for a search for people objects, but the master
branch has yet to finish after 30 minutes (and might not finish today).
The test case is modeled similar to what is done in my company: group
objects have not only member entries for people, but each member has
an alias to the corresponding person object, these aliases being
children to the respective group object. Furthermore, subgroups/group
inclusion is modeled by adding aliases (also as children) to the group
object, pointing to the group objects representing the included
groups. So there is some recursive descent to get all person objects
of all subgroups. In total I have 5 mio entries, 3 mio of which are
aliases. There are no cycles, but in this test case the depth of the
recursion is absolutely unrealistic.
I believe, in the master-slapd, each step down in the recursion needs a call to
mdb_idscope, which in turn goes through all 3 mio aliases to compute
the intersection with the scope of the group object (that's what I
remember, but I have not looked at the algorithm for a year now).
The patch traverses in this case just very small sub-trees (although many of those)
to collect the relevant aliases and continues from there.
That seems to be much more efficient here.
If I understand your description above correctly, my patch sucks with
your test case because it does a full traversal of the LDAP tree to
collect all aliases ... with 11 mio entries that takes a while. The
master branch, however, just computes intersection with the scope once
and continues from there, which must be much faster.
I actually do not know what to conclude from all this. I wonder
however, if a combination of both approaches could be beneficial. With
the patch the diskNode of an entry contains not only the number of
children in the scope, but also the number of aliases. If the
total number of aliases in the database is large, but the search scope
is small, the tree traversal might be faster. If the number of entries in
the scope is big, and there are aliases in the search scope, the
intersection in mdb_idscope might be more efficient. If there are no
aliases in the search scope, nothing has to be done at all.
4 years, 2 months
Re: (ITS#9054) Add support for multiple EECDH curves
by quanah＠symas.com
--On Tuesday, July 16, 2019 9:45 PM +0000 quanah(a)openldap.org wrote:
> Full_Name: Quanah Gibson-Mount
> Version: 2.4.47
> OS: N/A
> URL: ftp://ftp.openldap.org/incoming/
> Submission from: (NULL) (47.208.128.44)
>
>
> Currently OpenLDAP only allows for a single EECDH curve to be configured.
> However, OpenSSL 1.0.2 released in January 2015 was the first release to
> implement negotiation of supported curves in TLS servers. OpenLDAP needs
> updating to support this functionality.
tls_dh.c in postfix/src/tls_dh.c gives some insight into how to correctly
do this with OpenSSL, in the tls_auto_eecdh_curves fucntion.
--Quanah
--
Quanah Gibson-Mount
Product Architect
Symas Corporation
Packaged, certified, and supported LDAP solutions powered by OpenLDAP:
<http://www.symas.com>
4 years, 2 months
(ITS#9054) Add support for multiple EECDH curves
by quanah＠openldap.org
Full_Name: Quanah Gibson-Mount
Version: 2.4.47
OS: N/A
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (47.208.128.44)
Currently OpenLDAP only allows for a single EECDH curve to be configured.
However, OpenSSL 1.0.2 released in January 2015 was the first release to
implement negotiation of supported curves in TLS servers. OpenLDAP needs
updating to support this functionality.
4 years, 2 months
Re: (ITS#7657) Alias dereferencing with MDB slow compared with BDB
by Mark.Cairney＠ed.ac.uk
Hi Howard,
Brilliant- I notice that there's a 2.4.48 RC due out shortly so Ill roll
that out on a Dev box and see how it performs.
Kind regards,
Mark
On 15/07/2019 17:07, Howard Chu wrote:
> Fyi, we recently revisited this issue and are seeing improved results with
> commit e90e8c7d3c12d897bb0584ba04dc519d4f23acf9 in master.
>
--
/****************************
Mark Cairney
ITI Enterprise Services
Information Services
University of Edinburgh
Tel: 0131 650 6565
Email: Mark.Cairney(a)ed.ac.uk
PGP: 0x435A9621
*******************************/
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
4 years, 2 months
Re: (ITS#8875) [Patch] Performance problems in back-mdb with large DITs and many aliases
by hyc＠symas.com
This is a multi-part message in MIME format.
--------------AED2FFEE73F73B1613190B64
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
hyc(a)symas.com wrote:
> The tool to generate the test LDIF is attached. It reads an LDIF containing 500,000 users on stdin, and outputs the same LDIF,
> with aliases interspersed, on stdout.
>
Slightly tweaked, creates the alias after the target entry. In case the server does referential integrity on loading.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
--------------AED2FFEE73F73B1613190B64
Content-Type: text/x-csrc;
name="mkalias.c"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename="mkalias.c"
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>
#define NUMUSERS 500000
#define NUMALIASES 30000
#define NUMSUBALIASES 435
#define SUFFIX "dc=example,dc=com"
static const char ou1[] = "\
dn: ou=alias1," SUFFIX "\n\
objectclass: top\n\
objectclass: organizationalUnit\n\
ou: alias1\n";
static const char ou2[] = "\
dn: ou=alias2," SUFFIX "\n\
objectclass: top\n\
objectclass: organizationalUnit\n\
ou: alias2\n";
int qcmp(const void *a, const void *b)
{
const int *i = a, *j = b;
return *i - *j;
}
int main() {
char line[1024], dn[1024], *ou;
int aliases[NUMALIASES];
int subaliases[NUMSUBALIASES];
int i;
int nusers=0, naliases=0, nsubaliases=0, showalias=0;
/* select a random subset of users to generate aliases */
srand(time(0L));
for (i=0; i<NUMALIASES; i++) {
aliases[i] = rand() % NUMUSERS;
}
uniq1:
qsort(aliases, NUMALIASES, sizeof(int), qcmp);
/* make sure they're unique */
for (i=1; i<NUMALIASES; i++) {
if (aliases[i-1] == aliases[i]) {
aliases[i] = rand() % NUMUSERS;
goto uniq1;
}
}
/* select a random susbset of aliases for the target subtree */
for (i=0; i<NUMSUBALIASES; i++) {
subaliases[i] = rand() % NUMALIASES;
}
uniq2:
qsort(subaliases, NUMSUBALIASES, sizeof(int), qcmp);
/* make sure they're unique */
for (i=1; i<NUMSUBALIASES; i++) {
if (subaliases[i-1] == subaliases[i]) {
subaliases[i] = rand() % NUMALIASES;
goto uniq2;
}
}
for (i=0; i<NUMSUBALIASES; i++) {
subaliases[i] = aliases[subaliases[i]];
}
/* read LDIF */
while (fgets(line, sizeof(line), stdin) != NULL) {
if (line[0] == '#')
continue;
if (!strncmp(line, "dn: ", 4)) {
/* we assume all users' DNs use uid for RDN */
if (!strncmp(line+4, "uid=", 4)) {
if (!nusers) {
puts(ou1);
puts(ou2);
}
if (showalias) {
printf("dn: uid=x.%d,ou=%s," SUFFIX "\n", nusers, ou);
puts("objectclass: alias");
puts("objectclass: extensibleObject");
printf("aliasedObjectName: %s\n", dn);
showalias = 0;
}
if (nusers == aliases[naliases] ||
nusers == subaliases[nsubaliases]) {
if (nusers == subaliases[nsubaliases]) {
ou = "alias2";
nsubaliases++;
} else {
ou = "alias1";
}
naliases++;
strcpy(dn, line+4);
showalias = 1;
}
nusers++;
}
}
fputs(line, stdout);
}
return 0;
}
--------------AED2FFEE73F73B1613190B64--
4 years, 2 months
Re: (ITS#8875) [Patch] Performance problems in back-mdb with large DITs and many aliases
by hyc＠symas.com
This is a multi-part message in MIME format.
--------------0F1DE22CD9273A65E1AE5118
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Howard Chu wrote:
> Henrik Bohnenkamp wrote:
>> On Mon, Jul 15, 2019 at 02:26:59PM +0100, Howard Chu wrote:
>>>
>>> Fyi, on our problematic test database with 11M entries and 3.7M aliases, a search with -a always , starting from the
>>> DB suffix, took 4 minutes without this patch, and 1235 minutes with this patch.
>>>
>>> Needless to say, that's not looking good. Still checking other test cases.
>>
>> Interesting, so the behavior is reversed now :-). I assume you have
>> found an alternative approach to solve the problem. That's fine with
>> me, I want the problem solved, not my patch integrated. I'm of course
>> interested in how you do it. Surely you did not get the 4 minutes with
>> a stock 2.4.48 slapd?
>
> For this size of DB we needed the ITS#8977 patches to accommodate larger IDLs.
> (I used 24 bits for IDLs, 16.7M slots)
> Also at this size, the IDL processing itself is the main bottleneck now. We would
> need to switch to bitmaps or trees to avoid this bottleneck, but that's also a
> much larger change than we can consider for this release.
>
I've set up a more modest test database along the lines of ITS#7657. It has 500,000 users,
30,000 aliases total, and 435 in ou=alias2 (all the rest under ou=alias1).
For unpatched back-mdb:
time ../clients/tools/ldapsearch -x -H ldap://:9012 -D cn=manager,dc=example,dc=com -w secret -b ou=alias1,dc=example,dc=com -a always
# search result
search: 2
result: 0 Success
# numResponses: 29567
# numEntries: 29566
real 0m42.504s
user 0m1.344s
sys 0m2.996s
time ../clients/tools/ldapsearch -x -H ldap://:9012 -D cn=manager,dc=example,dc=com -w secret -b ou=alias2,dc=example,dc=com -a always
# search result
search: 2
result: 0 Success
# numResponses: 437
# numEntries: 436
real 0m48.406s
user 0m0.040s
sys 0m0.076s
For back-mdb with e90e8c7d3c12d897bb0584ba04dc519d4f23acf9
time ../clients/tools/ldapsearch -x -H ldap://:9012 -D cn=manager,dc=example,dc=com -w secret -b ou=alias1,dc=example,dc=com -a always
# search result
search: 2
result: 0 Success
# numResponses: 29567
# numEntries: 29566
real 0m5.500s
user 0m1.516s
sys 0m2.944s
time ../clients/tools/ldapsearch -x -H ldap://:9012 -D cn=manager,dc=example,dc=com -w secret -b ou=alias2,dc=example,dc=com -a always
# search result
search: 2
result: 0 Success
# numResponses: 437
# numEntries: 436
real 0m0.399s
user 0m0.048s
sys 0m0.060s
For back-mdb with this ITS#8875 patch
time ../clients/tools/ldapsearch -x -H ldap://:9012 -D cn=manager,dc=example,dc=com -w secret -b ou=alias1,dc=example,dc=com -a always
# search result
search: 2
result: 0 Success
# numResponses: 29567
# numEntries: 29566
real 0m6.020s
user 0m1.640s
sys 0m3.372s
time ../clients/tools/ldapsearch -x -H ldap://:9012 -D cn=manager,dc=example,dc=com -w secret -b ou=alias2,dc=example,dc=com -a always
# search result
search: 2
result: 0 Success
# numResponses: 437
# numEntries: 436
real 0m0.203s
user 0m0.052s
sys 0m0.048s
It seems close enough in this case (I didn't do enough repeated runs to average out any measurement error) while
the committed patch performs better on the really ugly test case.
The tool to generate the test LDIF is attached. It reads an LDIF containing 500,000 users on stdin, and outputs the same LDIF,
with aliases interspersed, on stdout.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
--------------0F1DE22CD9273A65E1AE5118
Content-Type: text/x-csrc;
name="mkalias.c"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename="mkalias.c"
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>
#define NUMUSERS 500000
#define NUMALIASES 30000
#define NUMSUBALIASES 435
static const char ou1[] = "\
dn: ou=alias1,dc=example,dc=com\n\
objectclass: top\n\
objectclass: organizationalUnit\n\
ou: alias1\n";
static const char ou2[] = "\
dn: ou=alias2,dc=example,dc=com\n\
objectclass: top\n\
objectclass: organizationalUnit\n\
ou: alias2\n";
int qcmp(const void *a, const void *b)
{
const int *i = a, *j = b;
return *i - *j;
}
int main() {
char line[1024];
int aliases[NUMALIASES];
int subaliases[NUMSUBALIASES];
int i;
int gotuser = 0, nusers=0, naliases=0, nsubaliases=0;
/* select a random subset of users to generate aliases */
srand(time(0L));
for (i=0; i<NUMALIASES; i++) {
aliases[i] = rand() % NUMUSERS;
}
uniq1:
qsort(aliases, NUMALIASES, sizeof(int), qcmp);
/* make sure they're unique */
for (i=1; i<NUMALIASES; i++) {
if (aliases[i-1] == aliases[i]) {
aliases[i] = rand() % NUMUSERS;
goto uniq1;
}
}
/* select a radnom susbset of aliases for the target subtree */
for (i=0; i<NUMSUBALIASES; i++) {
subaliases[i] = rand() % NUMALIASES;
}
uniq2:
qsort(subaliases, NUMSUBALIASES, sizeof(int), qcmp);
/* make sure they're unique */
for (i=1; i<NUMSUBALIASES; i++) {
if (subaliases[i-1] == subaliases[i]) {
subaliases[i] = rand() % NUMALIASES;
goto uniq2;
}
}
for (i=0; i<NUMSUBALIASES; i++) {
subaliases[i] = aliases[subaliases[i]];
}
while (fgets(line, sizeof(line), stdin) != NULL) {
if (line[0] == '#')
continue;
if (!strncmp(line, "dn: ", 4)) {
if (!gotuser) {
if (!strncmp(line+4, "uid=", 4)) {
gotuser = 1;
puts(ou1);
puts(ou2);
}
} else {
if (nusers == aliases[naliases] ||
nusers == subaliases[nsubaliases]) {
char *ou;
int id;
sscanf(line, "dn: uid=user.%d,", &id);
if (nusers == subaliases[nsubaliases]) {
ou = "alias2";
nsubaliases++;
} else {
ou = "alias1";
}
naliases++;
printf("dn: uid=x.%d,ou=%s,dc=example,dc=com\n", id, ou);
puts("objectclass: alias");
puts("objectclass: extensibleObject");
printf("aliasedObjectName: %s\n", line+4);
}
nusers++;
}
}
fputs(line, stdout);
}
return 0;
}
--------------0F1DE22CD9273A65E1AE5118--
4 years, 2 months
