(ITS#4805) memory leak in syncrepl code
by ando@sys-net.it
Full_Name: Pierangelo Masarati
Version: HEAD/re23
OS: irrelevant
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (131.175.154.35)
Submitted by: ando
There seems to be few memory leaks in syncrepl code (detected by valgrind when
running test048); one is when parsing an entry and sync "present" or "done".
It's now fixed in HEAD/re23. Others are being investigated
p.
15 years, 5 months
Re: (ITS#4591) back-meta does not propagate response controls
by ando@sys-net.it
ando(a)sys-net.it wrote:
> In most cases, all response payload (matched, referrals) is ignored when calling
> ldap_parse_result(); only in searches matched and referrals are handled.
For most operations this should now be fixed, since response passes thru
meta_back_op_result(), which takes care of controls. However, in
case of searches, handling this issue may be problematic, as different
targets may return different, possibly overlapping or incompatible
controls in response, so ignoring them might be safe. A minimal,
forgiving approach could be to record response controls, and, in case
only one target returned any, use them; otherwise, either ignore them or
complain somehow.
p.
15 years, 5 months
Re: Antwort: Re: (ITS#4758) valsort + dynlist can cause 100% cpu utilization causing slapd to become unresponsive
by ando@sys-net.it
Michael.Heep(a)o2.com wrote:
> Today I deployed a patched 2.3.32 version on one of our replicas and
> changed the config to include the aforementioned dynlist/valsort
> construct.
> It's been running nicely for a few hours now. Seems like the patch fixed
> this issue. Thank you very much!
Thanks for the feedback. The fix will appear in 2.3.33.
p.
Ing. Pierangelo Masarati
OpenLDAP Core Team
SysNet s.n.c.
Via Dossi, 8 - 27100 Pavia - ITALIA
http://www.sys-net.it
------------------------------------------
Office: +39.02.23998309
Mobile: +39.333.4963172
Email: pierangelo.masarati(a)sys-net.it
------------------------------------------
15 years, 5 months
Antwort: Re: (ITS#4758) valsort + dynlist can cause 100% cpu utilization causing slapd to become unresponsive
by Michael.Heep@o2.com
Dies ist eine mehrteilige Nachricht im MIME-Format.
--=_alternative 005BCC39C1257261_=
Content-Type: text/plain; charset="US-ASCII"
Today I deployed a patched 2.3.32 version on one of our replicas and
changed the config to include the aforementioned dynlist/valsort
construct.
It's been running nicely for a few hours now. Seems like the patch fixed
this issue. Thank you very much!
With kind regards
Michael Heep
Pierangelo Masarati <ando(a)sys-net.it>
11.01.2007 23:37
An
michael.heep(a)o2.com
Kopie
openldap-its(a)openldap.org
Thema
Re: (ITS#4758) valsort + dynlist can cause 100% cpu utilization causing
slapd to become unresponsive
michael.heep(a)o2.com wrote:
> the following valsort + dynlist combination causes slapd to utilize 100%
cpu
> time when issueing searches against parts of the DIT containing
attribute-value
> pairs "created" by dynlist:
>
> overlay dynlist
> dynlist-attrset extensibleObject memberURL uniqueMember
>
> overlay valsort
> valsort-attr uniqueMember dc=o2online,dc=de alpha-ascend
>
> When run independent both overlays work flawlessly.
>
A bug (ITS#4801) was recently fixed in slapo-dynlist(5) which appeared
in conjunction with slapo-valsort(5). The fix is available both in HEAD
and in re23 CVS (tag OPENLDAP_REL_ENG_2_3). The relevant change is in
servers/slapd/overlays/dynlist.c 1.28 -> 1.29 (HEAD) or 1.5.2.9 ->
1.5.2.10 (re23). Could you please check if it does any good to the
issue you reported?
Thanks, p.
Ing. Pierangelo Masarati
OpenLDAP Core Team
SysNet s.n.c.
Via Dossi, 8 - 27100 Pavia - ITALIA
http://www.sys-net.it
------------------------------------------
Office: +39.02.23998309
Mobile: +39.333.4963172
Email: pierangelo.masarati(a)sys-net.it
------------------------------------------
--=_alternative 005BCC39C1257261_=
Content-Type: text/html; charset="US-ASCII"
<br><font size=2 face="sans-serif">Today I deployed a patched 2.3.32 version
on one of our replicas and changed the config to include the aforementioned
dynlist/valsort construct.</font>
<br><font size=2 face="sans-serif">It's been running nicely for a few hours
now. Seems like the patch fixed this issue. Thank you very much!</font>
<br>
<br><font size=2 face="sans-serif">With kind regards</font>
<br><font size=2 face="sans-serif">Michael Heep<br>
</font>
<br>
<br>
<br>
<table width=100%>
<tr valign=top>
<td width=40%><font size=1 face="sans-serif"><b>Pierangelo Masarati <ando(a)sys-net.it></b>
</font>
<p><font size=1 face="sans-serif">11.01.2007 23:37</font>
<td width=59%>
<table width=100%>
<tr valign=top>
<td>
<div align=right><font size=1 face="sans-serif">An</font></div>
<td><font size=1 face="sans-serif">michael.heep(a)o2.com</font>
<tr valign=top>
<td>
<div align=right><font size=1 face="sans-serif">Kopie</font></div>
<td><font size=1 face="sans-serif">openldap-its(a)openldap.org</font>
<tr valign=top>
<td>
<div align=right><font size=1 face="sans-serif">Thema</font></div>
<td><font size=1 face="sans-serif">Re: (ITS#4758) valsort + dynlist can
cause 100% cpu utilization causing slapd to become unresponsive</font><font size=2 face="sans-serif">
</font></table>
<br>
<table>
<tr valign=top>
<td>
<td></table>
<br></table>
<br>
<br>
<br><font size=2><tt>michael.heep(a)o2.com wrote:<br>
> the following valsort + dynlist combination causes slapd to utilize
100% cpu<br>
> time when issueing searches against parts of the DIT containing attribute-value<br>
> pairs "created" by dynlist:<br>
><br>
> overlay dynlist <br>
> dynlist-attrset extensibleObject
memberURL uniqueMember <br>
><br>
> overlay valsort <br>
> valsort-attr uniqueMember dc=o2online,dc=de alpha-ascend<br>
><br>
> When run independent both overlays work flawlessly.<br>
> <br>
A bug (ITS#4801) was recently fixed in slapo-dynlist(5) which appeared
<br>
in conjunction with slapo-valsort(5). The fix is available both in
HEAD <br>
and in re23 CVS (tag OPENLDAP_REL_ENG_2_3). The relevant change is
in <br>
servers/slapd/overlays/dynlist.c 1.28 -> 1.29 (HEAD) or 1.5.2.9 ->
<br>
1.5.2.10 (re23). Could you please check if it does any good to the
<br>
issue you reported?<br>
<br>
Thanks, p.<br>
<br>
<br>
<br>
Ing. Pierangelo Masarati<br>
OpenLDAP Core Team<br>
<br>
SysNet s.n.c.<br>
Via Dossi, 8 - 27100 Pavia - ITALIA<br>
http://www.sys-net.it<br>
------------------------------------------<br>
Office: +39.02.23998309<br>
Mobile: +39.333.4963172<br>
Email: pierangelo.masarati(a)sys-net.it<br>
------------------------------------------<br>
<br>
</tt></font>
<br>
--=_alternative 005BCC39C1257261_=--
15 years, 5 months
Deny bind for subtree not working?
by Daniel Hasler
Hi
I try to deni BIND for all entries in a subtree. I compiled openldap with the LDAP backend, because this is only a proxy that forwards request to another directory.
Following is my configuration:
>include /local/home/hasleda4/openldap/etc/openldap/schema/core.schema
>include /local/home/hasleda4/openldap/etc/openldap/schema/cosine.schema
>include /local/home/hasleda4/openldap/etc/openldap/schema/inetorgperson.schema
>
>pidfile /local/home/hasleda4/openldap/var/run/gaad-slapd.pid
>argsfile /local/home/hasleda4/openldap/var/run/gaad-slapd.args
>
>database ldap
>suffix "dc=company,dc=com"
>uri "ldaps://other-dir.net:26930"
>
>access to dn.subtree="ou=people,ou=intranet,dc=company,dc=com" by dn.subtree="ou=applications,ou=intranet,dc=company,dc=com" read
> by * none
>access to dn.subtree="ou=applications,ou=intranet,dc=company,dc=com" by users read
> by anonymous auth
> by * none
>access to * by * read
As by the first ACL, anonymous users are not allowed to bind against "ou=people,ou=intranet,dc=novartis,dc=com".
If I now try to bind, the ACL seems not to be evaluated (I run slapd with -d 128 to see ACL processing, and there is no output during the BIND) and the BIND operation succeeds if I give the correct password.
Is this a bug? Or just how openldap behaves for bind operations?
Is there another way to deny bind operations for a subtree?
Thanks for any response.
Cheers
Dani
15 years, 5 months
Re: (ITS#4758) valsort + dynlist can cause 100% cpu utilization causing slapd to become unresponsive
by ando@sys-net.it
michael.heep(a)o2.com wrote:
> the following valsort + dynlist combination causes slapd to utilize 100% cpu
> time when issueing searches against parts of the DIT containing attribute-value
> pairs "created" by dynlist:
>
> overlay dynlist
> dynlist-attrset extensibleObject memberURL uniqueMember
>
> overlay valsort
> valsort-attr uniqueMember dc=o2online,dc=de alpha-ascend
>
> When run independent both overlays work flawlessly.
>
A bug (ITS#4801) was recently fixed in slapo-dynlist(5) which appeared
in conjunction with slapo-valsort(5). The fix is available both in HEAD
and in re23 CVS (tag OPENLDAP_REL_ENG_2_3). The relevant change is in
servers/slapd/overlays/dynlist.c 1.28 -> 1.29 (HEAD) or 1.5.2.9 ->
1.5.2.10 (re23). Could you please check if it does any good to the
issue you reported?
Thanks, p.
Ing. Pierangelo Masarati
OpenLDAP Core Team
SysNet s.n.c.
Via Dossi, 8 - 27100 Pavia - ITALIA
http://www.sys-net.it
------------------------------------------
Office: +39.02.23998309
Mobile: +39.333.4963172
Email: pierangelo.masarati(a)sys-net.it
------------------------------------------
15 years, 5 months
Re: (ITS#4801) segfault in dynlist
by quanah@stanford.edu
--On Thursday, January 11, 2007 9:15 PM +0000 ando(a)sys-net.it wrote:
> quanah(a)stanford.edu wrote:
>> Okay, I can confirm it works as expected once valsort is no longer
>> used....
>>
> Please try servers/slapd/overlays/dynlist.c 1.28 -> 1.29
Now it works with valsort also loaded. :)
--Quanah
--
Quanah Gibson-Mount
Principal Software Developer
ITS/Shared Application Services
Stanford University
GnuPG Public Key: http://www.stanford.edu/~quanah/pgp.html
15 years, 5 months
Re: (ITS#4801) segfault in dynlist
by ando@sys-net.it
quanah(a)stanford.edu wrote:
> Okay, I can confirm it works as expected once valsort is no longer used....
>
Please try servers/slapd/overlays/dynlist.c 1.28 -> 1.29
p.
Ing. Pierangelo Masarati
OpenLDAP Core Team
SysNet s.n.c.
Via Dossi, 8 - 27100 Pavia - ITALIA
http://www.sys-net.it
------------------------------------------
Office: +39.02.23998309
Mobile: +39.333.4963172
Email: pierangelo.masarati(a)sys-net.it
------------------------------------------
15 years, 5 months
Re: (ITS#4801) segfault in dynlist
by quanah@stanford.edu
--On Thursday, January 11, 2007 8:57 PM +0000 quanah(a)stanford.edu wrote:
>
>
> --On Thursday, January 11, 2007 9:53 PM +0100 Pierangelo Masarati
> <ando(a)sys-net.it> wrote:
>
>> quanah(a)stanford.edu wrote:
>>> Full_Name: Quanah Gibson-Mount
>>> Version: 2.3.32
>>> OS: Linux 2.6 (64-bit)
>>> URL: ftp://ftp.openldap.org/incoming/
>>> Submission from: (NULL) (171.64.19.81)
>>>
>> I'm not seeing anything like that, neither with HEAD nor with re23. It
>> might be worth having a bit more details on your configuration, since
>> that resulting from test044 (which basically complies with the info you
>> provided) doesn't even hit that line of code (because rs->sr_flags == 0).
>> So there must be something in between that causes the entry to be freed.
>> That could be slapo-dynlist in some cases, but also slapo-translucent,
>> slapo-collect, slapo-rwm or slapo-valsort. I guess at this point you
>> should share the conf, the data and the op that's causing trouble.
>
> The search I'm performing is:
Okay, I can confirm it works as expected once valsort is no longer used....
--Quanah
--
Quanah Gibson-Mount
Principal Software Developer
ITS/Shared Application Services
Stanford University
GnuPG Public Key: http://www.stanford.edu/~quanah/pgp.html
15 years, 5 months
Re: (ITS#4801) segfault in dynlist
by quanah@stanford.edu
--On Thursday, January 11, 2007 9:53 PM +0100 Pierangelo Masarati
<ando(a)sys-net.it> wrote:
> quanah(a)stanford.edu wrote:
>> Full_Name: Quanah Gibson-Mount
>> Version: 2.3.32
>> OS: Linux 2.6 (64-bit)
>> URL: ftp://ftp.openldap.org/incoming/
>> Submission from: (NULL) (171.64.19.81)
>>
> I'm not seeing anything like that, neither with HEAD nor with re23. It
> might be worth having a bit more details on your configuration, since
> that resulting from test044 (which basically complies with the info you
> provided) doesn't even hit that line of code (because rs->sr_flags == 0).
> So there must be something in between that causes the entry to be freed.
> That could be slapo-dynlist in some cases, but also slapo-translucent,
> slapo-collect, slapo-rwm or slapo-valsort. I guess at this point you
> should share the conf, the data and the op that's causing trouble.
The search I'm performing is:
ldapsearch -LLL -Q -h ldap-dev1.stanford.edu -b
"cn=groups,cn=applications,dc=stanford,dc=edu"
the slapd.conf is as follows, and the group configurations are as
previously noted.
My principal has full read into the LDAP database, so the only ACL parsed
is access to * for it. I am using valsort, so perhaps it is an interaction
between those two?
# $Id: slapd.conf.dev,v 1.7 2007/01/11 00:05:44 quanah Exp $
#
# See slapd.conf(5) for details on configuration options.
# This file should NOT be world readable.
#
include /usr/local/etc/openldap/schema/core.schema
include /usr/local/etc/openldap/schema/cosine.schema
include /usr/local/etc/openldap/schema/dyngroup.schema
include /usr/local/etc/openldap/schema/krb5-kdc.schema
include /usr/local/etc/openldap/schema/inetorgperson.schema
include /usr/local/etc/openldap/schema/misc.schema
include /usr/local/etc/openldap/schema/nis.schema
include /usr/local/etc/openldap/schema/eduperson.schema
include /usr/local/etc/openldap/schema/suacct.schema
include /usr/local/etc/openldap/schema/superson.schema
include /usr/local/etc/openldap/schema/suapplication.schema
# Allow V2 binds
allow bind_v2
# Use star cert
TLSCertificateFile /usr/local/etc/openldap/stardomain.crt
TLSCertificateKeyFile /usr/local/etc/openldap/stardomain.key
TLSCACertificateFile /usr/local/etc/openldap/comodo.pem
# Define global ACLs
include /usr/local/etc/openldap/slapd.acl
#
pidfile /var/run/slapd.pid
argsfile /var/run/slapd.args
# Set the default search base for clients that don't specify a base.
defaultsearchbase "dc=stanford,dc=edu"
# Turn gentlehup off, it takes too long.
gentlehup off
# Read slapd.conf(5) for possible values
loglevel 256
# Set the number of threads (8 seems to work best)
threads 8
# Set the number of threads to use in tool mode
tool-threads 2
# Set the timeout for idle connections
#idletimeout 30
# SASL conf
sasl-realm stanford.edu
sasl-authz-policy both
sasl-regexp uid=(.*)/cgi,cn=stanford.edu,cn=gssapi,cn=auth
ldap:///cn=cgi,cn=applications,dc=stanford,dc=edu??sub?krb5PrincipalName=$1/cgi@stanford.edu
sasl-regexp uid=service/(.*),cn=stanford.edu,cn=gssapi,cn=auth
ldap:///cn=Service,cn=Applications,dc=stanford,dc=edu??sub?krb5PrincipalName=service/$1@stanford.edu
sasl-regexp uid=webauth/(.*),cn=stanford.edu,cn=gssapi,cn=auth
ldap:///cn=Webauth,cn=Applications,dc=stanford,dc=edu??sub?krb5PrincipalName=webauth/$1@stanford.edu
sasl-regexp uid=(.*),cn=stanford.edu,cn=gssapi,cn=auth
ldap:///uid=$1,cn=Accounts,dc=stanford,dc=edu??sub?suSeasStatus=active
# Load dynamic backend modules:
modulepath /usr/local/lib/openldap
moduleload back_hdb.la
moduleload back_monitor.la
moduleload valsort.la
moduleload dynlist.la
#######################################################################
# stanford.edu database definitions
#######################################################################
database hdb
suffix "dc=stanford,dc=edu"
rootdn "cn=manager,dc=stanford,dc=edu"
# Valsort Overlay
overlay valsort
valsort-attr ou cn=people,dc=stanford,dc=edu weighted
valsort-attr suAffiliation cn=people,dc=stanford,dc=edu weighted
valsort-attr suDisplayAffiliation cn=people,dc=stanford,dc=edu weighted
# Dynlist Overlay
overlay dynlist
dynlist-attrset groupOfURLS memberURL member
# Let ldapadmin have limitless searches
limits group="cn=ldapadmin,cn=applications,dc=stanford,dc=edu"
time.soft=unlimited time.hard=unlimited size.soft=unlimited
size.hard=unlimited
# Let the Athletics principal have limitless searches
limits
dn.exact="cn=athletics,cn=service,cn=applications,dc=stanford,dc=edu"
time.soft=unlimited time.hard=unlimited size.soft=unlimited
size.hard=unlimited
# Let the Authority audit principal have limitless searches
limits
dn.exact="cn=workgroup-audit,cn=service,cn=applications,dc=stanford,dc=edu"
time.soft=unlimited time.hard=unlimited size.soft=unlimited
size.hard=unlimited
# Let the Registry Data Auditor principal have limitless searches
limits
dn.exact="cn=RegistryDataAuditor,cn=Service,cn=Applications,dc=stanford,dc=edu"
time.soft=unlimited time.hard=unlimited size.soft=unlimited
size.hard=unlimited
# Let the ispace prinicpal have a search of 5000 entries
limits dn.exact="cn=ispace,cn=Service,cn=Applications,dc=stanford,dc=edu"
time.soft=unlimited time.hard=unlimited size.soft=5000 size.hard=5000
# Let the GSB person principal have unlimited searches
limits
dn.exact="cn=gsb-person,cn=service,cn=applications,dc=stanford,dc=edu"
time.soft=unlimited time.hard=unlimited size.soft=unlimited
size.hard=unlimited
# Save the time that the entry gets modified
lastmod on
include /usr/local/etc/openldap/syncrepl.conf
# Set the location of where the database storage files go.
directory /var/lib/ldap
dbconfig set_cachesize 3 536870912 1
dbconfig set_lg_regionmax 262144
dbconfig set_lg_bsize 2097152
dbconfig set_lg_dir /var/log/bdb
dbconfig set_lk_max_locks 3000
dbconfig set_lk_max_objects 1500
dbconfig set_lk_max_lockers 1500
#
# Automatically remove log files that are no longer needed.
dbconfig set_flags DB_LOG_AUTOREMOVE
#
# Setting set_tas_spins reduces resource contention from multiple clients
on systems with multiple CPU's.
dbconfig set_tas_spins 1
# Checkpoint the database to prevent transaction loss in unclean shutdowns,
and speed up slapd shutdowns.
checkpoint 1024 5
# Entries to cache in memory
cachesize 50000
# IDL Entries to cache in memory
idlcachesize 50000
# Entries to free up when cache gets full
cachefree 1000
# Change the sub_any index length from 4 to 3 so that searches like *lee*
work.
index_substr_any_len 3
# Indices to maintain
index default eq
index cn eq,sub
index dc
index displayName
index entryUUID
index givenName eq,sub
index homePhone eq,sub
index krb5PrincipalName
index mail eq,sub
index mobile eq,sub
index modifyTimestamp
index o
index objectClass
index pager eq,sub
index sn eq,sub,approx
index suAffiliation
index suCalendarStatus
index suCardNumber pres,eq
index suCN eq,sub
index suDialinStatus
index suDisplayAffiliation
index suEmailPager eq,sub
index suGeneralID eq,sub
index suGivenName eq,sub
index suGwAffilFax1 eq,sub
index suGwAffilFax2 eq,sub
index suGwAffilFax3 eq,sub
index suGwAffilFax4 eq,sub
index suGwAffilFax5 eq,sub
index suGwAffilPhone1 eq,sub
index suGwAffilPhone2 eq,sub
index suGwAffilPhone3 eq,sub
index suGwAffilPhone4 eq,sub
index suGwAffilPhone5 eq,sub
index suKerberosStatus
index suLelandStatus
index suLocalPhone eq,sub
index suMaildrop
index suOtherName
index suPermanentPhone eq,sub
index suPrimaryOrganizationID
index suPrivilegeGroup eq,sub
index suProxyCardNumber pres,eq
index suRegID
index suRegisteredName eq,sub
index suResidencePhone eq,sub
index suSearchID
index suSeasStatus
index suSeasSunetID
index suSN eq,sub,approx
index suSunetID
index suUniqueIdentifier
index suUnivID
index suVisibAffilAddress1
index suVisibAffilAddress2
index suVisibAffilAddress3
index suVisibAffilAddress4
index suVisibAffilAddress5
index suVisibAffilFax1
index suVisibAffilFax2
index suVisibAffilFax3
index suVisibAffilFax4
index suVisibAffilFax5
index suVisibAffiliation1
index suVisibAffiliation2
index suVisibAffiliation3
index suVisibAffiliation4
index suVisibAffiliation5
index suVisibAffilPhone1
index suVisibAffilPhone2
index suVisibAffilPhone3
index suVisibAffilPhone4
index suVisibAffilPhone5
index suVisibEmail
index suVisibFacsimileTelephoneNumber
index suVisibHomeAddress
index suVisibHomePage
index suVisibHomePhone
index suVisibIdentity
index suVisibLocalAddress
index suVisibMailAddress
index suVisibMailCode
index suVisibMobilePhone
index suVisibPagerEmail
index suVisibPagerPhone
index suVisibPermanentAddress
index suVisibProfile
index suVisibStreet
index suVisibSunetID
index suVisibTelephoneNumber
index telephoneNumber eq,sub
index uid pres,eq
index uidNumber
#######################################################################
# back-monitor database definitions
#######################################################################
database monitor
reverse-lookup on
--
Quanah Gibson-Mount
Principal Software Developer
ITS/Shared Application Services
Stanford University
GnuPG Public Key: http://www.stanford.edu/~quanah/pgp.html
15 years, 5 months