dynlist vs memberof performance issues
by Mark Cairney
Hi,
I've been out the LDAP loop for a bit but the recent discussion of the
memberof overlay on 2.5 piqued my curiosity. Having upgraded a Dev box,
removed the memberof elements from the database and replaced the
memberof overlay with dynlist the queries appear to work as expected but
are both a) slow and b) heavily CPU-intensive on the LDAP server.
2021-09-01T12:47:17.603513+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 fd=12 ACCEPT from IP=192.168.152.33:58738
(IP=129.215.17.9:636)
2021-09-01T12:47:17.687488+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 fd=12 TLS established tls_ssf=256 ssf=256
tls_proto=TLSv1.3 tls_cipher=TLS_AES_256_GCM_SHA384
2021-09-01T12:47:17.688032+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=0 SRCH base="" scope=0 deref=0
filter="(objectClass=*)"
2021-09-01T12:47:17.688470+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=0 SRCH attr=supportedSASLMechanisms
2021-09-01T12:47:17.688878+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=0 SEARCH RESULT tag=101 err=0 qtime=0.000014
etime=0.000214 nentries=1 text=
2021-09-01T12:47:17.811279+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=1 BIND dn="" method=163
2021-09-01T12:47:17.819249+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=1 RESULT tag=97 err=14 qtime=0.000030
etime=0.009084 text=SASL(0): successful result:
2021-09-01T12:47:17.908889+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=2 BIND dn="" method=163
2021-09-01T12:47:17.909836+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=2 RESULT tag=97 err=14 qtime=0.000031
etime=0.000181 text=SASL(0): successful result:
2021-09-01T12:47:17.938839+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=3 BIND dn="" method=163
2021-09-01T12:47:17.939621+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=3 BIND authcid="mcairney(a)EASE.ED.AC.UK"
authzid="mcairney(a)EASE.ED.AC.UK"
2021-09-01T12:47:17.940213+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=3 BIND
dn="uid=mcairney,ou=people,ou=central,dc=authorise-dev,dc=ed,dc=ac,dc=uk"
mech=GSSAPI bind_ssf=256 ssf=256
2021-09-01T12:47:17.940616+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=3 RESULT tag=97 err=0 qtime=0.000024
etime=0.000409 text=
2021-09-01T12:47:18.227342+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=4 SRCH
base="dc=authorise-dev,dc=ed,dc=ac,dc=uk" scope=2 deref=0
filter="(uid=mcairney)"
2021-09-01T12:47:18.227703+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=4 SRCH attr=* +
2021-09-01T12:47:31.392255+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=5 UNBIND
2021-09-01T12:47:31.460705+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 op=4 SEARCH RESULT tag=101 err=0 qtime=0.000031
etime=13.233679 nentries=1 text=
2021-09-01T12:47:31.461098+01:00 bonsai.authorise-dev.is.ed.ac.uk
slapd[30075]: conn=1019 fd=12 closed
I'm guessing that as the values are computed that this will be heavier
on the CPU but it seems a bit excessive? Has anyone else noticed any
similar performance issues?
This is a relatively low-specced DEV server (2 vCPUs, 4GB RAM) so I
guess this could be a factor but there's no io waiting on the server and
no swapping?
The database is on a par in size with our Production service ( about
400K user objects with 1 group object per user and then about 80K actual
groups of users)
The config for the primary DB (ACLs and rootPW redacted) is:
dn: olcDatabase={1}mdb,cn=config
objectClass: olcDatabaseConfig
objectClass: olcMdbConfig
olcDatabase: {1}mdb
olcDbDirectory: /opt/openldap/var/openldap-data/authorise
olcSuffix: dc=authorise-dev,dc=ed,dc=ac,dc=uk
olcAddContentAcl: FALSE
olcLastMod: TRUE
olcMaxDerefDepth: 2
olcReadOnly: FALSE
olcSecurity: ssf=1
olcSecurity: update_ssf=112
olcSecurity: simple_bind=64
olcSizeLimit: unlimited
olcSyncUseSubentry: FALSE
olcTimeLimit: unlimited
olcMonitoring: TRUE
olcDbEnvFlags: writemap
olcDbEnvFlags: nometasync
olcDbNoSync: FALSE
olcDbIndex: objectClass eq
olcDbIndex: entryUUID eq
olcDbIndex: entryCSN eq
olcDbIndex: cn pres,eq,sub
olcDbIndex: uid pres,eq,sub
olcDbIndex: uidNumber pres,eq
olcDbIndex: gidNumber pres,eq
olcDbIndex: eduniType eq
olcDbIndex: gecos pres,eq,sub
olcDbIndex: eduniCategory eq
olcDbIndex: mail pres,eq,sub
olcDbIndex: eduniSchoolCode eq
olcDbIndex: eduniIDStatus eq
olcDbIndex: eduniCollegeCode eq
olcDbIndex: eduniOrgCode eq
olcDbIndex: memberOf pres,eq
olcDbIndex: eduniLibraryBarcode pres,eq
olcDbIndex: eduniOrganisation pres,eq,sub
olcDbIndex: eduniServiceCode pres,eq
olcDbIndex: krbName pres,eq
olcDbIndex: eduPersonAffiliation pres,eq
olcDbIndex: eduPersonEntitlement pres,eq
olcDbIndex: sn pres,eq,sub
olcDbIndex: eduniIdmsId pres,eq
olcDbIndex: member pres,eq
olcDbIndex: memberUid pres,eq
olcDbIndex: eduniRefNo pres,eq
olcDbIndex: eduniTitle pres,eq
olcDbIndex: title pres,eq,sub
olcDbIndex: eduniCardNumber pres,eq
olcDbIndex: eduniYearOfStudy eq
olcDbIndex: description pres,eq,sub
olcDbIndex: givenName pres,eq,sub
olcDbIndex: aliasedObjectName eq
olcDbIndex: yubiKeyId pres,eq
olcDbIndex: isMemberOf pres,eq
olcDbIndex: hasMember pres,eq
olcDbIndex: proxyAddresses pres,eq,sub
olcDbMaxReaders: 96
olcDbMaxSize: 32212254720
olcDbMode: 0600
olcDbSearchStack: 16
structuralObjectClass: olcMdbConfig
dn: olcOverlay={0}syncprov,olcDatabase={1}mdb,cn=config
objectClass: olcOverlayConfig
objectClass: olcConfig
objectClass: top
objectClass: olcSyncProvConfig
olcOverlay: {0}syncprov
structuralObjectClass: olcSyncProvConfig
dn: olcOverlay={1}accesslog,olcDatabase={1}mdb,cn=config
objectClass: olcOverlayConfig
objectClass: olcAccessLogConfig
olcOverlay: {1}accesslog
olcAccessLogDB: cn=accesslog
olcAccessLogOps: writes
olcAccessLogPurge: 02+00:00 00+04:00
olcAccessLogSuccess: TRUE
structuralObjectClass: olcAccessLogConfig
dn: olcOverlay={2}dynlist,olcDatabase={1}mdb,cn=config
objectClass: olcOverlayConfig
objectClass: olcDynListConfig
olcOverlay: {2}dynlist
olcDynListAttrSet: {0}groupOfURLs memberURL member+memberOf@groupOfNames
structuralObjectClass: olcDynListConfig
--
/****************************
Mark Cairney
ITI Enterprise Services
Information Services
University of Edinburgh
Tel: 0131 650 6565
Email: Mark.Cairney(a)ed.ac.uk
*******************************/
The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. Is e buidheann carthannais a th’ ann an Oilthigh Dhùn Èideann, clàraichte an Alba, àireamh clàraidh SC005336.
2 months, 2 weeks
Empty error when configuring OpenLDAP
by Cezary Drożak
Hello,
I am trying to set up OpenLDAP on Arch Linux on my server, following
instruction on Arch Wiki[1]. I prepared the config.ldif file, replacing
every $BASEDN and $PASSWD in the example configuration:
# The root config entry
dn: cn=config
objectClass: olcGlobal
cn: config
olcArgsFile: /run/openldap/slapd.args
olcPidFile: /run/openldap/slapd.pid
# Schemas
dn: cn=schema,cn=config
objectClass: olcSchemaConfig
cn: schema
# TODO: Include further schemas as necessary
include: file:///etc/openldap/schema/core.ldif
# The config database
dn: olcDatabase=config,cn=config
objectClass: olcDatabaseConfig
olcDatabase: config
olcRootDN: cn=Manager,dc=example,dc=com
# The database for our entries
dn: olcDatabase=mdb,cn=config
objectClass: olcDatabaseConfig
objectClass: olcMdbConfig
olcDatabase: mdb
olcSuffix: dc=example,dc=com
olcRootDN: cn=Manager,dc=example,dc=com
olcRootPW: {SSHA}xZqSQN4wG4+C5I57dB/Qm02vJ+kQcwd7
olcDbDirectory: /var/lib/openldap/openldap-data
# TODO: Create further indexes
olcDbIndex: objectClass eq
Then I executed the following command:
sudo -u ldap slapadd -n 0 -F /etc/openldap/slapd.d/ -l ./config.ldif
This gave me the following error:
invalid config directory /etc/openldap/slapd.d/, error 2
slapadd: bad configuration directory!
I checked that the directory did not exist, so I created it and changed
owner to `ldap`. The wiki page did not mention that the directory should
be created earlier, so maybe it should have been created by a post
installation script. If that's the case, I will report it to package
maintainers.
After I created the directory, I ran the command again, this time having
a different error message:
slapadd: could not add entry dn="cn=config" (line=1):
Closing DB...
I have no idea what is wrong now and I cannot find anything useful on
the internet. Does anyone have an idea what I may be doing wrong here?
[1]: https://wiki.archlinux.org/title/OpenLDAP
7 months, 1 week
How to relay read and write requests to different ldap servers
by nagamani.chinnapaiyan＠viasat.com
Hi,
I am new to ldap. We have 4 ldap servers, 2 of them are in mirror-mode providers, 2 of them are just consumers/replicas.
I am working on loadbalancer for these 4 ldap servers using ldap/meta backend.
I want to the ldap proxy/loadbalancer to,
redirect write requests to one of the 2 mirror-mode providers.
redirect read requests to any of the 2 replicas/consumers.
I know ldap backend has uri list which can be used to redirect to mirror-mode providers. But I want to redirect only the write requests.
Regards,
Nagamani Chinnapaiyan
8 months, 2 weeks
Re: dynlist overlay, memberURL not working on spcified attribiutes
by Bog Dan
Thanks for your reply, I try to explain.
Populating memberOf attribiute to users entry working well using this:
*olcDynListAttrSet: {1}inetOrgPerson labeledURI memberOf*
User entry look as follow:
*dn: uid=test1,ou=people,dc=test,dc=comsambaAcctFlags: [U
]sambaPwdLastSet: 9999999999sambaNTPassword: passo: testsambaSID:
S-1-5-21-3945181060-1826002392-430723570pwdPolicySubentry:
cn=noexpire,ou=ppolicy,dc=test,dc=comcreateTimestamp:
20220529070624.324Zdescription: User accountuserPassword::
passsambaPwdCanChange: 1528009736sambaPwdMustChange: 0mail: test1(a)test.com
<test1(a)test.com>loginShell: /bin/bashgivenName: Test1sambaLogonTime: 0sn:
Testcn: Test1 TestobjectClass: posixAccountobjectClass: topobjectClass:
inetOrgPersonobjectClass: personobjectClass:
organizationalPersonobjectClass: sambaSamAccountobjectClass:
shadowAccounthomeDirectory: /home/test1pwdChangedTime:
20220529070856.504ZgidNumber: 1002uidNumber: 1002uid:
test1structuralObjectClass: inetOrgPersonentryUUID:
348cd83e-7c6a-103c-8612-1918ce7a0bc4creatorsName:
cn=admin,dc=test,dc=comlabeledURI:
ldap:///ou=groups,dc=test,dc=com??sub?(|(&(objectclass=groupOfUniqueNames)(uniqueMember=uid=test1,ou=people,dc=test,dc=com))(&(objectClass=posixGroup)(memberUid=test1)))entryCSN:
20220609180738.487916Z#000000#001#000000modifiersName:
cn=admin,dc=test,dc=commodifyTimestamp: 20220609180738ZmemberOf:
cn=devops,ou=groups,dc=test,dc=comentryDN:
uid=test1,ou=people,dc=test,dc=comsubschemaSubentry:
cn=SubschemahasSubordinates: FALSE*
Static group entry:
*cn=devops,ou=groups,dc=test,dc=comcn: devopsobjectClass:
groupOfUniqueNamesobjectClass: topdescription: devops groupuniqueMember:
uid=test1,ou=people,dc=test,dc=comuniqueMember:
uid=test2,ou=people,dc=test,dc=com*
Next what I want to do is agregate multiple groups to one virtual using
this:
*olcDynListAttrSet: {0}groupOfURLs memberURL member*
Then I create appropiate group:
*cn=testluri,ou=groups,dc=test,dc=comcn: testluriobjectClass:
topobjectClass: groupOfURLsdescription: test groupmemberURL:
ldap:///ou=people,dc=test,dc=com??sub?(memberOf=cn=devops,ou=groups,dc=test,dc=com)*
but this don't add memeber entry to this group. When I changing memberURL
as follow:
*memberURL: ldap:///ou=people,dc=test,dc=com??sub?(|(uid=test1)(uid=test2))*
member attribute was added to testluri group:
*cn=testluri,ou=groups,dc=test,dc=comcn: testluriobjectClass:
topobjectClass: groupOfURLsdescription: test groupmemberURL:
ldap:///ou=people,dc=test,dc=com??sub?(memberOf=cn=devops,ou=groups,dc=test,dc=com)*
*member: uid=test1,ou=people,dc=test,dc=com*
*member: uid=test2,ou=people,dc=test,dc=com*
but this is not the goal. As I mentioned I want to agregate multiple group
to one using memberOf attribute in memberURL:
*memberURL:
ldap:///ou=people,dc=test,dc=com??sub?(memberOf=cn=devops,ou=groups,dc=test,dc=com)*
but this not working. What I'm doing wrong?
Reagrds
BS
śr., 29 cze 2022 o 19:17 Quanah Gibson-Mount <quanah(a)fast-mail.org>
napisał(a):
>
>
> --On Tuesday, June 28, 2022 12:18 PM +0200 Bog Dan <bsiara.cgi(a)gmail.com>
> wrote:
>
> >
> > Hi All,
> > I have problem with dynlist overlay, this is my configuration:
> >
> >
> >
> > olcOverlay={1}dynlist,olcDatabase={1}mdb,cn=config
> > objectClass: olcOverlayConfig
> > objectClass: olcDynListConfig
> > olcOverlay: {1}dynlist
> > olcDynListAttrSet: {0}groupOfURLs memberURL member
> > olcDynListAttrSet: {1}inetOrgPerson labeledURI memberOf
> >
> >
> >
> > First I create static group:
> >
> >
> > cn=devops,ou=groups,dc=test,dc=com
> > cn: devops
> > objectClass: groupOfUniqueNames
> > objectClass: top
> > description: devops group
> > uniqueMember: uid=test1,ou=people,dc=test,dc=com
> > uniqueMember: uid=test2,ou=people,dc=test,dc=com
> >
> >
> >
> > When I create new dynamic group:
> >
> > cn=testluri,ou=groups,dc=test,dc=com
> > cn: testluri
> > objectClass: top
> > objectClass: groupOfURLs
> > description: test group
> > memberURL:
> > ldap:///ou=people,dc=test,dc=com??sub?(memberOf=cn=devops,ou=groups,dc=te
> > st,dc=com)
> >
> >
> >
> > I don't get any member of group. Users test1 and test2 already have
> > memberOf attribute:
> >
> > memberOf: cn=devops,ou=groups,dc=test,dc=com
> >
> >
> >
> > When I change memberURL to use not dynamic attributes (memberOf):
> >
> >
> > memberURL:
> ldap:///ou=people,dc=test,dc=com??sub?(|(uid=test1)(uid=test2))
> >
> >
> >
> > users added to testluri group and dynlist works well.
> > What I should do to configure dynlist with memberOf?
>
> I've read your email multiple times, and quite frankly I don't understand
> what your end goal is.
>
> If your end goal is to have static groups, where memberOf is dynamically
> populated on the user entries (which is the usual use case for replacing
> the 2.4 memberOf), then your configs are clearly incorrect.
>
> Can you better explain what your end goal is?
>
> Thanks,
> Quanah
>
>
>
>
8 months, 3 weeks
dynlist overlay, memberURL not working on spcified attribiutes
by Bog Dan
Hi All,
I have problem with dynlist overlay, this is my configuration:
olcOverlay={1}dynlist,olcDatabase={1}mdb,cn=config
objectClass: olcOverlayConfig
objectClass: olcDynListConfig
olcOverlay: {1}dynlist
olcDynListAttrSet: {0}groupOfURLs memberURL member
olcDynListAttrSet: {1}inetOrgPerson labeledURI memberOf
First I create static group:
cn=devops,ou=groups,dc=test,dc=com
cn: devops
objectClass: groupOfUniqueNames
objectClass: top
description: devops group
uniqueMember: uid=test1,ou=people,dc=test,dc=com
uniqueMember: uid=test2,ou=people,dc=test,dc=com
When I create new dynamic group:
cn=testluri,ou=groups,dc=test,dc=com
cn: testluri
objectClass: top
objectClass: groupOfURLs
description: test group
memberURL:
ldap:///ou=people,dc=test,dc=com??sub?(memberOf=cn=devops,ou=groups,dc=test,dc=com)
I don't get any member of group. Users test1 and test2 already have
memberOf attribute:
memberOf: cn=devops,ou=groups,dc=test,dc=com
When I change memberURL to use not dynamic attributes (memberOf):
memberURL: ldap:///ou=people,dc=test,dc=com??sub?(|(uid=test1)(uid=test2))
users added to testluri group and dynlist works well.
What I should do to configure dynlist with memberOf?
Openldap 2.5.12
Regards
BS
9 months
role manage can bypass pwdCheckQuality with MOD but not with ADD op
by tempo＠net-c.com
Hi,
I'm doing some testing on userPassword management actually with openldap 2.5.9
I noticed that I could MOD a userPassword without checking quality if my admin role was "manage"
However, if I try to ADD a user with its attribute userPassword set, then quality is checked although the role "manage"
ppolicy in both cases are the default one (policy subentry not set)
Is it normal behavior ?
Regards,
9 months
setting slapd logging to STDOUT only
by Alceu Rodrigues de Freitas Junior
Greetings,
My name is Alceu and I'm new to this list.
I'm experimenting with OpenLDAP at a Google Cloud, running it as a Pod
inside a GKE cluster.
I've being struggling to setup slapd to send it's logs to STDOUT only,
but it seems in most cases the logs are sent to STDERR. The problem is,
for GCP Log Explorer, everything that is sent to STDERR is considered an
error, even when it's not. :-)
Here is a reference:
https://stackoverflow.com/questions/71158475/gcp-log-explorer-shows-wrong....
I'm using this open source project
<https://github.com/osixia/docker-openldap> to deploy OpenLDAP as a
Docker container, but there is no configuration option for that. I would
like to make it clear that this project is not using the latest OpenLDAP
stable version.
AFAIK, slapd will use syslog facility for logging and will write to the
terminal if the "-d" option is in use, which in fact it is:
$ docker exec -ti openldap /bin/bash
root@56529dea5931:/# ps aux
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 0.7 0.2 76528 47232 ? Ss 20:54 0:00
/usr/bin/python -u /container/tool/run --loglevel debug
openldap 21 0.1 0.0 294932 14844 ? Sl 20:54 0:00
/usr/sbin/slapd -h ldap://56529dea5931 ldaps://56529dea5931 ldapi:/// -u
openldap -g openldap -d 256
root 25 1.0 0.0 19868 3600 pts/0 Ss 20:54 0:00 /bin/bash
root 32 0.0 0.0 38312 3364 pts/0 R+ 20:54 0:00 ps aux
Is there any way to configure slapd to use STDOUT for everything,
besides using shell redirection (2>&1)?
Thanks in advance,
Alceu
9 months
Setting to suppress "connection_read(XXX): no connection!"
by Uwe Sauter
Dear list,
is there a way to configure slapd to not emit messages regarding "connection_read(XXX): no connection!"?
Currently the configuration contails "loglevel none" but these messages are sent to syslog
local4.debug regardless.
In case the version is relevant, this is 2.4.59 on RHEL 8.4. slapd is started with URL list
"ldap:/// ldaps:/// ldapi:///"
Thanks,
Uwe
9 months, 1 week
rwm bindDN context and ppolicy issues
by Kartik Subbarao
I'm able to specify rwm bindDN rules without password-policy enabled
just fine, like this one:
rwm-rewriteContext bindDN
rwm-rewriteRule
"^([^=]+)=([^@]+)@olddomain.com(.+),dc=olddomain,dc=com$"
"$1=$2(a)newdomain.com$3,dc=newdomain,dc=com" ":@"
However, when I enable password policy (which also works fine on its
own), slapd segfaults. From doing a backtrace and stepping through the
code, it looks like the crux of the issue is that the mdb_info struct
ends up with garbage data:
struct mdb_info *mdb = (struct mdb_info *) op->o_bd->be_private;
mi_dbenv_home and mi_monitor have random stuff in them.
I'm mulling over how much additional time to spend on this. rwm is a
very elegant solution to a current issue that could save me a bunch of
time to set up additional LDAP servers with the renamed data. If this is
an isolated bug for which a quick fix might be possible, I might
investigate further.
But if it's a thorny issue or just the tip of the iceberg of things
where rwm might break unexpectedly, then it may be better for me to
consider other options. OpenLDAP developers, what do your instincts say
on this?
Regards,
-Kartik
9 months, 1 week
