openldap-bugs August 2010

openldap-bugs@openldap.org

23 participants
68 discussions

(ITS#6632) Core dump in test030 when using back-ldap
by masarati＠aero.polimi.it 28 Aug '10

28 Aug '10

Full_Name: Pierangelo Masarati Version: HEAD/re24 OS: irrelevant URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (129.72.166.16) Submitted by: ando When compiled with -DSLAP_NO_SL_MALLOC=1 test030 cores when performing passwd extop with back-ldap; a gdb stack backtrace is provided below. (gdb) bt #0 0x00115402 in __kernel_vsyscall () #1 0x001c4d20 in raise () from /lib/libc.so.6 #2 0x001c6631 in abort () from /lib/libc.so.6 #3 0x001fce6b in __libc_message () from /lib/libc.so.6 #4 0x00204b16 in _int_free () from /lib/libc.so.6 #5 0x00208070 in free () from /lib/libc.so.6 #6 0x0826ae7a in ber_memfree_x (p=0x8b84fe0, ctx=0x0) at ../../../ldap-src/libraries/liblber/memory.c:152 #7 0x080f537c in slap_sl_free (ptr=0x8b84fe0, ctx=0x8b15138) at ../../../ldap-src/servers/slapd/sl_malloc.c:481 #8 0x080c8c58 in do_extended (op=0x8b2cde0, rs=0xb79820e4) at ../../../ldap-src/servers/slapd/extended.c:184 #9 0x0808acba in connection_operation (ctx=0xb79821d0, arg_v=0x8b2cde0) at ../../../ldap-src/servers/slapd/connection.c:1109 #10 0x0808b1fa in connection_read_thread (ctx=0xb79821d0, argv=0xd) at ../../../ldap-src/servers/slapd/connection.c:1245 #11 0x08234cc9 in ldap_int_thread_pool_wrapper (xpool=0x8a53b28) at ../../../ldap-src/libraries/libldap_r/tpool.c:685 #12 0x0011b46b in start_thread () from /lib/libpthread.so.0 #13 0x0026cdbe in clone () from /lib/libc.so.6 The issue, according to valgrind, consists in a double-free, which does not surface when dealt with by the slab. conn=1021 fd=17 ACCEPT from IP=127.0.0.1:45744 (IP=127.0.0.1:9011) conn=1021 op=0 BIND dn="cn=Manager,o=Example,c=US" method=128 conn=1022 fd=22 ACCEPT from IP=127.0.0.1:45745 (IP=127.0.0.1:9011) conn=1022 op=0 BIND dn="cn=Manager,dc=example,dc=com" method=128 conn=1022 op=0 BIND dn="cn=Manager,dc=example,dc=com" mech=SIMPLE ssf=0 conn=1022 op=0 RESULT tag=97 err=0 text= conn=1021 op=0 BIND dn="cn=Manager,dc=example,dc=com" mech=SIMPLE ssf=0 conn=1021 op=0 RESULT tag=97 err=0 text= conn=1021 op=1 EXT oid=1.3.6.1.4.1.4203.1.11.1 conn=1021 op=1 PASSMOD id="cn=Added User,ou=Alumni Association,ou=People,o=Example,c=US" new conn=1022 op=1 EXT oid=1.3.6.1.4.1.4203.1.11.1 conn=1022 op=1 PASSMOD id="cn=Added User,ou=Alumni Association,ou=People,dc=example,dc=com" new conn=1021 op=1 RESULT oid= err=0 text= ==4946== ==4946== Thread 7: ==4946== Invalid free() / delete / delete[] ==4946== at 0x4004FDA: free (vg_replace_malloc.c:233) ==4946== by 0x826AE79: ber_memfree_x (memory.c:152) ==4946== by 0x80F537B: slap_sl_free (sl_malloc.c:481) ==4946== by 0x80C8C57: do_extended (extended.c:184) ==4946== by 0x808ACB9: connection_operation (connection.c:1109) ==4946== by 0x808B1F9: connection_read_thread (connection.c:1245) ==4946== by 0x8234CC8: ldap_int_thread_pool_wrapper (tpool.c:685) ==4946== by 0xDEB46A: start_thread (in /lib/libpthread-2.5.so) ==4946== by 0xD42DBD: clone (in /lib/libc-2.5.so) ==4946== Address 0x45DD018 is 0 bytes inside a block of size 62 free'd ==4946== at 0x4004FDA: free (vg_replace_malloc.c:233) ==4946== by 0x826AE79: ber_memfree_x (memory.c:152) ==4946== by 0x80F537B: slap_sl_free (sl_malloc.c:481) ==4946== by 0x80CA323: passwd_extop (passwd.c:321) ==4946== by 0x80C8E6E: fe_extended (extended.c:222) ==4946== by 0x80C8C05: do_extended (extended.c:177) ==4946== by 0x808ACB9: connection_operation (connection.c:1109) ==4946== by 0x808B1F9: connection_read_thread (connection.c:1245) ==4946== by 0x8234CC8: ldap_int_thread_pool_wrapper (tpool.c:685) ==4946== by 0xDEB46A: start_thread (in /lib/libpthread-2.5.so) ==4946== by 0xD42DBD: clone (in /lib/libc-2.5.so) The issue *might* be related to slapo-rwm playing with the rewriting of the request dn/ndn, but it does not surface when the same test is performed using back-relay, which also uses slapo-rwm. p.

1 0

Re: (ITS#6612) Back SQL / Mysql / InnoDB
by masarati＠aero.polimi.it 28 Aug '10

28 Aug '10

The requested feature has been added to HEAD code; please test and report. Thanks, p.

1 0

Re: (ITS#6631) back-sql attribute order incorrect
by masarati＠aero.polimi.it 28 Aug '10

28 Aug '10

> Also, that portion of the code has been recently modified (to allow 64-bit > integer keys); could you test with back-sql code from HEAD? It should > build and work straight with 2.3.43 code. I need to partly amend myself: that improvement requires long long parsing capabilities that were added to liblutil, so you'll probably be better off pulling the entire HEAD from the CVS to test it. p.

1 0

ITS#6630 dupent
by masarati＠aero.polimi.it 28 Aug '10

28 Aug '10

I've recently committed an overlay that implements the (expired) <draft-ietf-ldapext-ldapv3-dupent-08.txt> (ITS#6630); it's in contrib/slapd-modules/dupent/; configuration is pretty straightforward: load the module, and add overlay dupent in the database you want dupent to be enabled. I've only tested it as database specific, although it should also work as a global overlay. Could anybody with access to a dupent-capable client please test it and report (through the ITS, in case)? I'm not 100% sure about the request/response encoding/decoding. Thanks, p.

1 0

Re: (ITS#6631) back-sql attribute order incorrect
by masarati＠aero.polimi.it 27 Aug '10

27 Aug '10

> slapd log: > backsql_add_attr("radiuslogin=xyzzy2,ou=Dialup,ou=External,o=Integra"): > executing "UPDATE radiusaccount SET radiuslogin=? WHERE id=?" val[0], id=1411 > > MySQL log: > 173 Query UPDATE radiusaccount SET radiuslogin=1411 WHERE id='xyzzy2' > Note that this appears to be a logging issue rather than a functional one. > = > The parameter order changes according to the param_order field, but the > sl= > apd log always indicates the order as if param_order=3D0 I'm not sure I understand this report. Can you please make the example more esplicative? Also, please note that "id" in the update procedure is the name of a specific field in the RDBMS, while "id" in slapd's log is a generic name for the key that is being used. The name coincidence is purely incidental. Also, that portion of the code has been recently modified (to allow 64-bit integer keys); could you test with back-sql code from HEAD? It should build and work straight with 2.3.43 code. Thanks, p.

1 0

Re: (ITS#6631) back-sql attribute order incorrect
by david.stults＠integratelecom.com 26 Aug '10

26 Aug '10

Note that this appears to be a logging issue rather than a functional one. = The parameter order changes according to the param_order field, but the sl= apd log always indicates the order as if param_order=3D0 On Aug 26, 2010, at 11:38 AM, <openldap-its(a)OpenLDAP.org> <openldap-its@Ope= nLDAP.org> wrote: >=20 > *** THIS IS AN AUTOMATICALLY GENERATED REPLY *** >=20 > Thanks for your report to the OpenLDAP Issue Tracking System. Your > report has been assigned the tracking number ITS#6631. >=20 > One of our support engineers will look at your report in due course. > Note that this may take some time because our support engineers > are volunteers. They only work on OpenLDAP when they have spare > time. >=20 > If you need to provide additional information in regards to your > issue report, you may do so by replying to this message. Note that > any mail sent to openldap-its(a)openldap.org with (ITS#6631) > in the subject will automatically be attached to the issue report. >=20 > mailto:openldap-its@openldap.org?subject=3D(ITS#6631) >=20 > You may follow the progress of this report by loading the following > URL in a web browser: > http://www.OpenLDAP.org/its/index.cgi?findid=3D6631 >=20 > Please remember to retain your issue tracking number (ITS#6631) > on any further messages you send to us regarding this report. If > you don't then you'll just waste our time and yours because we > won't be able to properly track the report. >=20 > Please note that the Issue Tracking System is not intended to > be used to seek help in the proper use of OpenLDAP Software. > Such requests will be closed. >=20 > OpenLDAP Software is user supported. > http://www.OpenLDAP.org/support/ >=20 > -------------- > Copyright 1998-2007 The OpenLDAP Foundation, All Rights Reserved. >=20

1 0

(ITS#6631) back-sql attribute order incorrect
by david.stults＠integratelecom.com 26 Aug '10

26 Aug '10

Full_Name: David Stults Version: 2.4.23 OS: CentOS 5.5 URL: Submission from: (NULL) (64.122.164.12) When executing the add_proc SQL query, back-sql claims to provide the attributes in a particular order, but reality is reversed: slapd log: backsql_add_attr("radiuslogin=xyzzy2,ou=Dialup,ou=External,o=Integra"): executing "UPDATE radiusaccount SET radiuslogin=? WHERE id=?" val[0], id=1411 MySQL log: 173 Query UPDATE radiusaccount SET radiuslogin=1411 WHERE id='xyzzy2'

1 0

Re: (ITS#6629) send_search_entry() does not clear controls
by h.b.furuseth＠usit.uio.no 26 Aug '10

26 Aug '10

hyc(a)symas.com writes: > If these two behaviors are related, that says to me that we should > just go the rest of the way with the slab allocator and allocate > additional slabs when the current one is exhausted. By itself that solves one problem, but introduces another. The slab allocator does almost no garbage collection until the memory context is reset. So a task could use up an arbitrary number of slabs, slap_sl_free()ing almost all memory in each, before it ends and its slabs are all reset. The fallback to plain malloc() avoids this. A fix would be for extra slabs to manage their memory with a proper malloc algorithm. The first slab can use the current fast algorithm. Or, would it get too fragile to require slapd to track whether a memory block comes from plain malloc() or one of the OpenLDAP malloc functions? That would allow each slapd memory block to identify the slab it came from, if any. A two-byte header would allow 65535 slabs, if 8 bytes for a ctx pointer per sl_malloced block is too much. > The purpose of the slab allocator was two-fold - to provide > thread-local memory that needs no locking, and also to provide > local memory that needs no housekeeping/explicit frees. And #3: to avoid memory fragmentation from plain malloc. -- Hallvard

1 0

Re: (ITS#6629) send_search_entry() does not clear controls
by hyc＠symas.com 25 Aug '10

25 Aug '10

quanah(a)zimbra.com wrote: > --On August 26, 2010 3:44:02 AM +0000 masarati(a)aero.polimi.it wrote: > >> Full_Name: Pierangelo Masarati >> Version: HEAD/re24 >> OS: irrelevant >> URL: ftp://ftp.openldap.org/incoming/ >> Submission from: (NULL) (129.72.175.5) >> Submitted by: ando >> >> >> REP_CTRLS_MUSTBEFREED is only honored by send_ldap_response(); when >> response controls are attached to slap_send_search_entry() it is not >> honored. This does not surface because controls must be malloc'ed on the >> slab; however, it clearly appears when built with SLAP_NO_SL_MALLOC. > > Interesting -- I'm seeing an issue where I can get slapd to consume all > available memory on the system by using paged results. It takes a while to > do so (depending on how much memory is available), but it's 100% > repeatable. The more mixes of paged and non paged results there are with > large result sets, the more quickly it happens. However, I do not build > with SLAP_NO_SL_MALLOC (or at least, not intentionally ;) ). If these two behaviors are related, that says to me that we should just go the rest of the way with the slab allocator and allocate additional slabs when the current one is exhausted. The purpose of the slab allocator was two-fold - to provide thread-local memory that needs no locking, and also to provide local memory that needs no housekeeping/explicit frees. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#6629) send_search_entry() does not clear controls
by masarati＠aero.polimi.it 25 Aug '10

25 Aug '10

> > > --On August 26, 2010 3:44:02 AM +0000 masarati(a)aero.polimi.it wrote: > >> Full_Name: Pierangelo Masarati >> Version: HEAD/re24 >> OS: irrelevant >> URL: ftp://ftp.openldap.org/incoming/ >> Submission from: (NULL) (129.72.175.5) >> Submitted by: ando >> >> >> REP_CTRLS_MUSTBEFREED is only honored by send_ldap_response(); when >> response controls are attached to slap_send_search_entry() it is not >> honored. This does not surface because controls must be malloc'ed on >> the >> slab; however, it clearly appears when built with SLAP_NO_SL_MALLOC. > > Interesting -- I'm seeing an issue where I can get slapd to consume all > available memory on the system by using paged results. It takes a while > to > do so (depending on how much memory is available), but it's 100% > repeatable. The more mixes of paged and non paged results there are with > large result sets, the more quickly it happens. pagedResults only sends a control response with the searchResultDone response, which uses send_ldap_response() and thus clears the controls; moreover, we are talking about few bytes, so I don't think this is the issue. There might be something else related to slab allocation that runs out of slab memory and starts using the heap without freeing it. The present issue might affect syncrepl, since in that case a control is attached to each entry/intermediate response (afaik). It might even be intentional; in fact, often when a control is attached to an entry in a searchResultEntry the same control needs to be applied to many entries returned by the same operation. However, in this case we should probably define a specific API for slap_add_ctrls() that does not set the REP_CTRLS_MUSTBEFREED, or something like that. > However, I do not build > with SLAP_NO_SL_MALLOC (or at least, not intentionally ;) ). I occasionally do (you only need to recompile slapd/sl_malloc.c) to trace possible leaks when developing something. Since we make an intensive use of slab mallocs, usual minimal tests with valgrind do not show leaks otherwise. p.

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs August 2010