openldap-bugs May 2015

openldap-bugs@openldap.org

26 participants
101 discussions

Re: (ITS#8147) syncrepl fails when encountering bad data, master leaks memory
by hyc＠symas.com 19 May '15

19 May '15

mark.bannister(a)morganstanley.com wrote: > Full_Name: Mark Bannister > Version: 2.4.30 > OS: Oracle Solaris 11.2 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (205.228.82.171) > > > I have a master server and a replica configured with syncrepl in > refreshAndPersist mode. Im using bdb (not mdb), simple auth (not SASL), > standard syncrepl (not delta-syncrepl). > > I noticed that the directory has about 5 erroneous entries in 300,000 where > there is a multi-valued attribute containing two identical values. These > entries were added by slapadd -q. Here is an example: > > dn: cn=test,ou=rpc,dc=mycompany,dc=com > objectClass: oncRpc > cn: test > cn: test > oncRpcNumber: 12345678 The docs already tell you to only use slapadd with known-good data. > I did see that a number of memory leaks have been fixed since 2.4.30, but I > didn't see anything that looked like this profile. Sorry I don't have a newer > version of OpenLDAP to hand to re-test. Closing this ITS. You can followup again if you see the same problem in the current 2.4.41 release candidate. 2.4.30 is over 3 years old, no one is going to investigate this. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#8147) syncrepl fails when encountering bad data, master leaks memory
by mark.bannister＠morganstanley.com 19 May '15

19 May '15

Full_Name: Mark Bannister Version: 2.4.30 OS: Oracle Solaris 11.2 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (205.228.82.171) I have a master server and a replica configured with syncrepl in refreshAndPersist mode. Im using bdb (not mdb), simple auth (not SASL), standard syncrepl (not delta-syncrepl). I noticed that the directory has about 5 erroneous entries in 300,000 where there is a multi-valued attribute containing two identical values. These entries were added by slapadd -q. Here is an example: dn: cn=test,ou=rpc,dc=mycompany,dc=com objectClass: oncRpc cn: test cn: test oncRpcNumber: 12345678 When the replica attempts to copy this data using syncrepl from the master server, it fails. All entries up to that point are synchronised fine, but any entries from that point onwards are missing. I didn't see any log entries telling me about this failure, although I admit I didn't look very hard or tweak the log levels. This then causes a memory leak in the master server: $ while :; do ps -p 9025 -o pid,ppid,pmem,rss,vsz,args | tail +2; sleep 60; done 9025 1 5.2 434204 463892 /usr/lib/slapd -f /etc/openldap/slapd.conf -u openldap -g openldap -h ldap:/// 9025 1 5.3 443216 472900 /usr/lib/slapd -f /etc/openldap/slapd.conf -u openldap -g openldap -h ldap:/// 9025 1 5.4 444384 474076 /usr/lib/sla - -f /etc/openldap/slapd.conf -u openldap -g openldap -h ldap:/// 9025 1 5.5 454680 484364 /usr/lib/slapd -f /etc/openldap/slapd.conf -u openldap -g openldap -h ldap:/// 9025 1 5.5 458288 487972 /usr/lib/slapd -f /etc/openldap/slapd.conf -u openldap -g openldap -h ldap:/// 9025 1 5.6 464996 494684 /usr/lib/slapd -f /etc/openldap/slapd.conf -u openldap -g openldap -h ldap:/// 9025 1 5.7 472952 502636 /usr/lib/slapd -f /etc/openldap/slapd.conf -u openldap -g openldap -h ldap:/// ... until memory is exhausted, and I get: ch_malloc of 606572 bytes failed ... followed by a 3GB core dump file. When the server restarts, the cycle starts over again, ad infinitum until the filesystem is full of core dump files. When I remove the 5 duplicate attribute values and restart the master and replica servers, the entire directory is then successfully replicated and the memory footprint of the slapd process remains stable. I am therefore assuming that the duplicate attribute values were causing syncrepl not to complete and a memory leak in the master server. I did see that a number of memory leaks have been fixed since 2.4.30, but I didn't see anything that looked like this profile. Sorry I don't have a newer version of OpenLDAP to hand to re-test.

1 0

Re: (ITS#8117) Bugs related to key-size in lmdb and backend
by h.b.furuseth＠usit.uio.no 18 May '15

18 May '15

On 04/05/15 02:13, h.b.furuseth(a)usit.uio.no wrote: > (...) If I understand correctly this time, the doc should > simply say that the two types size_t and unsigned int are supported. > The "typically" is plain misleading. > > That is, if mdb_node_search() shows the intent when it picks either > int-sized or size_t-sized for IS_BRANCH. OTOH, another bug: > > default_cmp pick cmp_int for INTEGERDUP|DUPFIXED and some other > replaces md_dcmp = cmp_int with cmp_long, which makes no sense unless > cmp_int is just a placeholder. Yet at least mdb_dcmp() calls that > placeholder directly without checking the data size. If that's the intent, branch "mdb/keysize" @ my UiO repo fixes it. Should it do the following too? > And if it's > just a placeholder, how about instead setting dcmp to something which > just abort()s, to make sure the bug of not replacing it is caught? Hallvard

1 0

Re: (ITS#8146) Slapd with mdb consumes 100% CPU if deref aliases is requested
by andrew.findlay＠skills-1st.co.uk 18 May '15

18 May '15

On Mon, May 18, 2015 at 04:58:29PM +0000, Howard Chu wrote: > > (On my laptop, "all possible" is 2^64 - 1 so yes, this will spin for .. a while.) 584942 years at 1M tests/sec - I am not waiting around for the timeout :-) (or the electricity bill: over £10M at todays prices) > Fixed now in git master. Thanks Andrew -- ----------------------------------------------------------------------- | From Andrew Findlay, Skills 1st Ltd | | Consultant in large-scale systems, networks, and directory services | | http://www.skills-1st.co.uk/ +44 1628 782565 | -----------------------------------------------------------------------

1 0

Re: (ITS#8146) Slapd with mdb consumes 100% CPU if deref aliases is requested
by hyc＠symas.com 18 May '15

18 May '15

hyc(a)symas.com wrote: > Andrew Findlay wrote: >> On Mon, May 18, 2015 at 04:48:13PM +0100, Howard Chu wrote: >> >>> ITS#8103 fix crash with more than 65535 aliases in a scope >> >> Ah - my test case does not have any aliases at all so it may be an >> unrelated issue. > > I see, your issue is caused by the fact you haven't indexed the objectClass > attribute. When back-mdb does a search to find all the alias objects in the > DB, instead of returning zero, it returns unknown which causes it to loop > through all possible entryIDs looking for alias entries. > > (On my laptop, "all possible" is 2^64 - 1 so yes, this will spin for .. a while.) > Fixed now in git master. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8146) Slapd with mdb consumes 100% CPU if deref aliases is requested
by hyc＠symas.com 18 May '15

18 May '15

Andrew Findlay wrote: > On Mon, May 18, 2015 at 04:48:13PM +0100, Howard Chu wrote: > >> ITS#8103 fix crash with more than 65535 aliases in a scope > > Ah - my test case does not have any aliases at all so it may be an > unrelated issue. I see, your issue is caused by the fact you haven't indexed the objectClass attribute. When back-mdb does a search to find all the alias objects in the DB, instead of returning zero, it returns unknown which causes it to loop through all possible entryIDs looking for alias entries. (On my laptop, "all possible" is 2^64 - 1 so yes, this will spin for .. a while.) -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8146) Slapd with mdb consumes 100% CPU if deref aliases is requested
by andrew.findlay＠skills-1st.co.uk 18 May '15

18 May '15

On Mon, May 18, 2015 at 04:48:13PM +0100, Howard Chu wrote: > ITS#8103 fix crash with more than 65535 aliases in a scope Ah - my test case does not have any aliases at all so it may be an unrelated issue. Andrew -- ----------------------------------------------------------------------- | From Andrew Findlay, Skills 1st Ltd | | Consultant in large-scale systems, networks, and directory services | | http://www.skills-1st.co.uk/ +44 1628 782565 | -----------------------------------------------------------------------

1 0

Re: (ITS#8146) Slapd with mdb consumes 100% CPU if deref aliases is requested
by hyc＠symas.com 18 May '15

18 May '15

Andrew Findlay wrote: > On Mon, May 18, 2015 at 12:39:17PM +0100, Howard Chu wrote: > >> Already fixed in RE24. > > Not in the version that I pulled 10 minutes ago... > > openldap-OPENLDAP_REL_ENG_2_4-e445af3 Hmm. violino:~/OD/gr24/servers/slapd> git log -1 back-mdb commit e1fdac9db59e4677b9a2403a072558bd89e44321 Author: Howard Chu <hyc(a)openldap.org> Date: Mon Apr 13 13:11:29 2015 -0700 ITS#8103 fix crash with more than 65535 aliases in a scope I'll look at your test case shortly, thanks. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8146) Slapd with mdb consumes 100% CPU if deref aliases is requested
by andrew.findlay＠skills-1st.co.uk 18 May '15

18 May '15

On Mon, May 18, 2015 at 12:39:17PM +0100, Howard Chu wrote: > Already fixed in RE24. Not in the version that I pulled 10 minutes ago... openldap-OPENLDAP_REL_ENG_2_4-e445af3 Andrew -- ----------------------------------------------------------------------- | From Andrew Findlay, Skills 1st Ltd | | Consultant in large-scale systems, networks, and directory services | | http://www.skills-1st.co.uk/ +44 1628 782565 | -----------------------------------------------------------------------

1 0

Re: (ITS#8146) Slapd with mdb consumes 100% CPU if deref aliases is requested
by hyc＠symas.com 18 May '15

18 May '15

andrew.findlay(a)skills-1st.co.uk wrote: > Full_Name: Andrew Findlay > Version: 2.4.40 > OS: Linux: OpenSuSE 13.2 > URL: ftp://ftp.openldap.org/incoming/openldap-mdb-spin-bug.tgz > Submission from: (NULL) (2001:8b0:8d0:f7e1:61a6:169e:b4ba:9954) > > > Some client programs (such as Apache Directory Studio) request alias > dereferencing by default. This puts slapd with mdb into a spin where it consumes > 100% of a CPU. Other threads continue to work and new connections are accepted. > Timeouts do not terminate the spin. Disconnecting the client does not terminate > the spin. Already fixed in RE24. > To reproduce: start slapd and issue a search of the form: > > ldapsearch -a search -x -b dc=example,dc=org objectclass=person > > I have placed a tarball on the FTP server containing: > > slapd.conf > sample data in LDIF > gdb output showing thread trace after breaking in > script of client commands > > The build options were: > > CFLAGS=-g > export CFLAGS > > ./configure --prefix=/meme/andrew/test/openldap-2.4.40 \ > --enable-spasswd \ > --enable-crypt \ > --enable-slapi \ > --enable-overlays \ > --enable-hdb=no \ > --enable-bdb=no \ > --enable-ldap \ > --enable-rewrite \ > --enable-meta \ > --enable-null \ > --enable-monitor \ > --enable-relay \ > --enable-sock \ > --with-cyrus-sasl \ > --with-tls > > > For those that need it, the workaround with Apache Directory Studio is to > disable alias dereferencing in the Browser Options tab for the connection. > > Andrew > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

← Newer
1
2
3
4
5
6
7
...
11
Older →

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs May 2015