dynlist vs memberof performance issues
by Mark Cairney
Hi,
I've been out the LDAP loop for a bit but the recent discussion of the
memberof overlay on 2.5 piqued my curiosity. Having upgraded a Dev box,
removed the memberof elements from the database and replaced the
memberof overlay with dynlist the queries appear to work as expected but
are both a) slow and b) heavily CPU-intensive on the LDAP server.
2021-09-01T12:47:17.603513+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 fd=12 ACCEPT from IP=192.168.152.33:58738
(IP=129.215.17.9:636)
2021-09-01T12:47:17.687488+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 fd=12 TLS established tls_ssf=256 ssf=256
tls_proto=TLSv1.3 tls_cipher=TLS_AES_256_GCM_SHA384
2021-09-01T12:47:17.688032+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=0 SRCH base="" scope=0 deref=0
filter="(objectClass=*)"
2021-09-01T12:47:17.688470+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=0 SRCH attr=supportedSASLMechanisms
2021-09-01T12:47:17.688878+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=0 SEARCH RESULT tag=101 err=0 qtime=0.000014
etime=0.000214 nentries=1 text=
2021-09-01T12:47:17.811279+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=1 BIND dn="" method=163
2021-09-01T12:47:17.819249+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=1 RESULT tag=97 err=14 qtime=0.000030
etime=0.009084 text=SASL(0): successful result:
2021-09-01T12:47:17.908889+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=2 BIND dn="" method=163
2021-09-01T12:47:17.909836+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=2 RESULT tag=97 err=14 qtime=0.000031
etime=0.000181 text=SASL(0): successful result:
2021-09-01T12:47:17.938839+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=3 BIND dn="" method=163
2021-09-01T12:47:17.939621+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=3 BIND authcid="mcairney(a)EASE.ED.AC.UK"
authzid="mcairney(a)EASE.ED.AC.UK"
2021-09-01T12:47:17.940213+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=3 BIND
dn="uid=mcairney,ou=people,ou=central,dc=authorise-dev,dc=ed,dc=ac,dc=uk"
mech=GSSAPI bind_ssf=256 ssf=256
2021-09-01T12:47:17.940616+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=3 RESULT tag=97 err=0 qtime=0.000024
etime=0.000409 text=
2021-09-01T12:47:18.227342+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=4 SRCH
base="dc=authorise-dev,dc=ed,dc=ac,dc=uk" scope=2 deref=0
filter="(uid=mcairney)"
2021-09-01T12:47:18.227703+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=4 SRCH attr=* +
2021-09-01T12:47:31.392255+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=5 UNBIND
2021-09-01T12:47:31.460705+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=4 SEARCH RESULT tag=101 err=0 qtime=0.000031
etime=13.233679 nentries=1 text=
2021-09-01T12:47:31.461098+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 fd=12 closed
I'm guessing that as the values are computed that this will be heavier
on the CPU but it seems a bit excessive? Has anyone else noticed any
similar performance issues?
This is a relatively low-specced DEV server (2 vCPUs, 4GB RAM) so I
guess this could be a factor but there's no io waiting on the server and
no swapping?
The database is on a par in size with our Production service ( about
400K user objects with 1 group object per user and then about 80K actual
groups of users)
The config for the primary DB (ACLs and rootPW redacted) is:
dn: olcDatabase={1}mdb,cn=config
objectClass: olcDatabaseConfig
objectClass: olcMdbConfig
olcDatabase: {1}mdb
olcDbDirectory: /opt/openldap/var/openldap-data/authorise
olcSuffix: dc=authorise-dev,dc=ed,dc=ac,dc=uk
olcAddContentAcl: FALSE
olcLastMod: TRUE
olcMaxDerefDepth: 2
olcReadOnly: FALSE
olcSecurity: ssf=1
olcSecurity: update_ssf=112
olcSecurity: simple_bind=64
olcSizeLimit: unlimited
olcSyncUseSubentry: FALSE
olcTimeLimit: unlimited
olcMonitoring: TRUE
olcDbEnvFlags: writemap
olcDbEnvFlags: nometasync
olcDbNoSync: FALSE
olcDbIndex: objectClass eq
olcDbIndex: entryUUID eq
olcDbIndex: entryCSN eq
olcDbIndex: cn pres,eq,sub
olcDbIndex: uid pres,eq,sub
olcDbIndex: uidNumber pres,eq
olcDbIndex: gidNumber pres,eq
olcDbIndex: eduniType eq
olcDbIndex: gecos pres,eq,sub
olcDbIndex: eduniCategory eq
olcDbIndex: mail pres,eq,sub
olcDbIndex: eduniSchoolCode eq
olcDbIndex: eduniIDStatus eq
olcDbIndex: eduniCollegeCode eq
olcDbIndex: eduniOrgCode eq
olcDbIndex: memberOf pres,eq
olcDbIndex: eduniLibraryBarcode pres,eq
olcDbIndex: eduniOrganisation pres,eq,sub
olcDbIndex: eduniServiceCode pres,eq
olcDbIndex: krbName pres,eq
olcDbIndex: eduPersonAffiliation pres,eq
olcDbIndex: eduPersonEntitlement pres,eq
olcDbIndex: sn pres,eq,sub
olcDbIndex: eduniIdmsId pres,eq
olcDbIndex: member pres,eq
olcDbIndex: memberUid pres,eq
olcDbIndex: eduniRefNo pres,eq
olcDbIndex: eduniTitle pres,eq
olcDbIndex: title pres,eq,sub
olcDbIndex: eduniCardNumber pres,eq
olcDbIndex: eduniYearOfStudy eq
olcDbIndex: description pres,eq,sub
olcDbIndex: givenName pres,eq,sub
olcDbIndex: aliasedObjectName eq
olcDbIndex: yubiKeyId pres,eq
olcDbIndex: isMemberOf pres,eq
olcDbIndex: hasMember pres,eq
olcDbIndex: proxyAddresses pres,eq,sub
olcDbMaxReaders: 96
olcDbMaxSize: 32212254720
olcDbMode: 0600
olcDbSearchStack: 16
structuralObjectClass: olcMdbConfig
dn: olcOverlay={0}syncprov,olcDatabase={1}mdb,cn=config
objectClass: olcOverlayConfig
objectClass: olcConfig
objectClass: top
objectClass: olcSyncProvConfig
olcOverlay: {0}syncprov
structuralObjectClass: olcSyncProvConfig
dn: olcOverlay={1}accesslog,olcDatabase={1}mdb,cn=config
objectClass: olcOverlayConfig
objectClass: olcAccessLogConfig
olcOverlay: {1}accesslog
olcAccessLogDB: cn=accesslog
olcAccessLogOps: writes
olcAccessLogPurge: 02+00:00 00+04:00
olcAccessLogSuccess: TRUE
structuralObjectClass: olcAccessLogConfig
dn: olcOverlay={2}dynlist,olcDatabase={1}mdb,cn=config
objectClass: olcOverlayConfig
objectClass: olcDynListConfig
olcOverlay: {2}dynlist
olcDynListAttrSet: {0}groupOfURLs memberURL member+memberOf@groupOfNames
structuralObjectClass: olcDynListConfig
--
/****************************
Mark Cairney
ITI Enterprise Services
Information Services
University of Edinburgh
Tel: 0131 650 6565
Email: Mark.Cairney(a)ed.ac.uk
*******************************/
The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. Is e buidheann carthannais a th’ ann an Oilthigh Dhùn Èideann, clàraichte an Alba, àireamh clàraidh SC005336.
4 months, 3 weeks
Delta-sync replication: is it possible to force resync delta?
by skeletor
Hi.
I use delta-sync replication on version 2.4. Sometimes, some records
don't send to slave. Insofar as this is delta-sync after a new update
slave receive only last update. Therefore slave and master are not
consistent. Is it possible run force resync from accesslog to consistent
check if all records are present on slave?
May be this is a bug on version 2.4 and it already has been fixed in
newer version?
master 2.4.57
slave 2.4.55
1 year, 3 months
2.6 MMR 4-way with one master different flavor of linux
by Dave Macias
Hello,
Hope everyone is doing well.
Our current environment:
3-way MMR (syncrepl) with centos 7 using v2.6
We want to add a 4th master but with Rocky Linux 8.
Why rocky? because eventually we will migrate the 3 masters to rocky linux too. But that will take some time…
Any concerns with mixing linux flavor but using same openldap version?
Dont do it?
Any input is appreciated.
Thank you,
Dave
1 year, 3 months
[LMDB] mdb_env_set_mapsize and read transactions
by Ken Wenzel
Hello,
I like to implement an autogrow functionality for LMDB.
The documentation for mdb_env_set_mapsize says that no transactions should
be active when using this function.
When looking at the code I can see that the function only checks if there is
an active WRITE transaction and in this case it returns an error.
Is it possible to reuse existing READ transactions or even associated
cursors after mdb_env_set_mapsize has been called?
Thank you and best regards,
Ken
1 year, 4 months
Minor typo in slapo-ppolicy
by Ulrich Windl
Hi!
I just discovered a minor typo in my version of the slapo-ppolicy manual page (possibly it's fixed alrerady):
The manual page lists "pwdGraceAuthnLimit", but the attribute returned by slapcat is "pwdGraceAuthNLimit" (different case for 'N')
The name from the schema also is "pwdGraceAuthNLimit".
Regards,
Ulrich
1 year, 4 months
Question regarding password dictionary rule in OpenLDAP
by Alan Andrea
I have a question regarding password rules that are enforced when a user changes their password in OpenLDAP. We have a need to implement a dictionary rule whereby words and phrases in a dictionary are not allowed in a users password. I am not able to see currently where such functionality exists in OpenLDAP and am wondering if there are any extensions to OPenLDAP that were developed to support this or if it would be required to write code to support this feature?
Thanks,Alan
1 year, 4 months
Need to define behaviour when storing pwdChangedTime
by David Coutadeur
Hi,
When doing a backup / restore on my OpenLDAP 2.5.9 instance, I faced a
behaviour that I think must be defined explicitely, in
draft-behera-ldap-password-policy, or at least in OpenLDAP documentation.
My backup contains an entry like this:
dn: uid=test,ou=people,ou=branch,dc=example,dc=com
cn: test
sn: test
givenName: test
uid: test
userPassword: secret
pwdChangedTime: 20220110153431Z
mail: test(a)domain.com
objectClass: inetOrgPerson
objectClass: organizationalPerson
objectClass: person
There is also a valid default password policy: (which must be defined
before the users in the backup file)
dn: cn=default,ou=ppolicies,dc=example,dc=com
objectClass: pwdPolicy
objectClass: pwdPolicyChecker
objectClass: organizationalRole
cn: default
pwdMaxAge: 7776000
pwdAttribute: userPassword
pwdCheckQuality: 2
pwdLockout: TRUE
pwdMaxFailure: 5
pwdMinLength: 6
pwdMustChange: TRUE
pwdCheckModule: /usr/local/openldap/lib64/ppm.so
When restoring the backup with this command:
ldapadd -x -h '127.0.0.1:389' -D 'cn=Manager,dc=example,dc=com' -w
'secret' -f backup.ldif -e relax
I have an error showing that the attribute pwdChangedTime is duplicated
and must not be defined twice.
I assume that the password policy does not replace my pwdChangedTime
value with the current date, but duplicates the attribute.
Could you define this behaviour somewhere?
1/ Is it possible to update the pwdChangedTime attribute along with the
userPassword ?
2/ If so, what value should be stored? (the given value or the current
date?)
3/ Optionally, update OpenLDAP code according to the defined behaviour
Thanks in advance for your answer.
Regards,
David
1 year, 4 months
How to properly test whether an asynchronous ldap_connect() has fully connected?
by nickpelling＠nickpelling.com
I'm trying to run OpenLDAP on a Cortex-M7 processor, but to be a "good citizen" my LDAP client code needs to connect asynchronously. (The code connects synchronously fine.)
Stripped down to the bare minimum (i.e. no error handling) for clarity here (the actual code has tons of error handling), my asynchronous connection code looks like this:
ldap_set_option(pstContext->ld, LDAP_OPT_CONNECT_ASYNC, LDAP_OPT_ON); // Enable async connection
ldap_connect(ld); // Do the LDAP connection
As you'd expect, the next step is to test (asynchronously) whether the connection has completed, but it can't see any OpenLDAP functionality to do this directly.
So here's what my test currently looks like:
/**
* Determine whether a connection has yet connected
*
* @param[in] LDAP connection instance to test
*
* @retval 0 if connection has connected
* @retval -2 if connection is still waiting
* @return ...otherwise an unexpected problem was encountered
*/
int ldap_connection_connected(LDAP * ld)
{
ber_socket_t sd;
int iRetVal;
iRetVal = -99; // Default to "an unexpected problem was encountered"
if (ld->ld_defconn != NULL)
{
if (ld->ld_defconn->lconn_status == LDAP_CONNST_CONNECTED)
{
iRetVal = 0; // i.e. the connection has already connected, so exit early!
}
else
{
// Get the connection's sockbuffer
ber_sockbuf_ctrl( ld->ld_defconn->lconn_sb, LBER_SB_OPT_GET_FD, &sd );
// Check to see if a result has yet been received via this sockbuffer
iRetVal = ldap_int_check_async_open( ld, sd );
}
}
return iRetVal;
}
However, my bind code (i.e. that executes immediately after this) fails to connect to the server after an asynchronous ldap_connect(), even though it works fine after a synchronous ldap_connect(). Moreover, it looks to me (from the timings) as though my test for asynchronous connection is signalling completion earlier than I'm expecting, so I'm wondering if this is not waiting properly, e.g. if it's returning after the very first message response when it should be waiting for a whole series of message responses to arrive.
Am I missing some clever OpenLDAP trick for correctly detecting when asynchronous ldap_connect() calls have fully connected? Thanks!
1 year, 4 months
mdb_txn_begin() before mdb_dbi_open()
by Sam Dave
Hello,
1. I create a read-only transaction on an environment with mdb_txn_begin().
2.Then I get a database handle on the same environment (using mdb_dbi_open() within a separate transaction).
3. Finally I try to create and use a cursor using the database in 2. and transaction in 1. but I get various runtime errors ("Segmentation fault" or "double free or corruption").
However, if I switch 1. and 2., everything works.
Must a database really be created before a transaction? If so, curious to learn if this is mentioned anywhere in the docs (I didn't see it)
Regards,
Sam
1 year, 4 months
mdb_cursor_close and mdb_txn_close/abort for read-only transactions
by Sam Dave
Hello,
The mdb_cursor_close() documentation says: "Its transaction must still be live if it is a write-transaction."
This leads me to my question: For *read-only* transactions, is it allowed to run mdb_cursor_close() *after* mdb_txn_close() or mdb_txn_abort()?
Regards,
Sam
1 year, 4 months
