- openldap-bugs - openldap.org

(ITS#6203) Problem & solution for back-ndb issue connecting to Cluster data nodes
by andrew.morgan＠sun.com 09 Jul '09

09 Jul '09

Full_Name: Andrew Morgan Version: 2.4.16 OS: Ubuntu URL: Submission from: (NULL) (90.193.196.76) I am running MySQL Cluster with data nodes all running on the same host. I configured, built and installed OpenLDAP with back-ndb but when I run "slapd -d 5", it failed with "ndb_db_open: ni_cluster[0]->wait failed (1)". Looking at the code in "back-ndb/init.cpp" this implies that it was able to connect to the Cluster management node but only the first of the data nodes. The cause of this is line 133: "ni->ni_cluster[i]->wait_until_ready( 30, 0 );" - the 30,0 means wait up to 30 seconds for first data node but then no time at all for the others; the fact that the method returned 1 means that the first node was found to be live. The second parameter should be set to a non-zero value to give the other data nodes chance to show as being live. I changed the 0 to 30, recompiled/installed and slapd came up correctly.

1 0

Re: (ITS#6139) slapd: password-hash rejects multiple values
by h.b.furuseth＠usit.uio.no 09 Jul '09

09 Jul '09

It has been fixed in CVS (and in next OpenLDAP release): http://www.openldap.org/devel/cvsweb.cgi/servers/slapd/bconfig.c.diff?r1=1.… -- Hallvard

1 0

(ITS#6202) adding command line option to disable line breaking in ldif output
by openldap.org＠bloms.de 09 Jul '09

09 Jul '09

Full_Name: Dieter Bloms Version: all OS: linux URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (193.27.50.90) I want to do an enhancement request for ldapsearch, because it insert a line break after 78 characters. Many people like me use the ldapsearch command in shell script and have to workaround this issue. Maybe it make sense to add a command line option to deactivate this line breaks.

1 0

RE: (ITS#6201) Double free in syncprov
by Alister.Winfield＠sns.bskyb.com 09 Jul '09

09 Jul '09

I think I have it (bad configuration)... I have the 'accesslogs' at every level which makes sense because I'm replic= ating individual branches separately. I forgot to remove the "overlay glue" line from the configurati= on (previously this configuration had the whole database replicated with just one provider). Now that puts th= e same accesslog entry into two db's which might be where the source of the double free begins. NB: Testing this setup without that extraneous line of config has, so far, = not crashed. After adding it again it crashed almost instantly after I started removing = entries from LDAP. Okay so it might be a configuration 'error' but I guess its worth trying to= avoid the crash. The last crash looks like. syncprov_search_response: cookie=3Drid=3D099,csn=3D20090709104334.923157Z#0= 00000#000#000000 *** glibc detected *** /data/ldap-re24/libexec/slapd: double free or corrup= tion (out): 0x0000000041c406f0 *** =3D=3D=3D=3D=3D=3D=3D Backtrace: =3D=3D=3D=3D=3D=3D=3D=3D=3D /lib/libc.so.6[0x7f5256652948] /lib/libc.so.6(cfree+0x76)[0x7f5256654a56] /data/ldap-re24/libexec/ldap/syncprov-2.4-releng.so.2[0x7f5254260821] /data/ldap-re24/libexec/slapd[0x441c50] /data/ldap-re24/libexec/slapd[0x44478b] /data/ldap-re24/libexec/slapd(slap_send_ldap_result+0x185)[0x4453f5] /data/ldap-re24/libexec/ldap/syncprov-2.4-releng.so.2[0x7f5254263733] /data/ldap-re24/libexec/slapd(overlay_op_walk+0x4a)[0x496bda] /data/ldap-re24/libexec/slapd[0x4976d3] /data/ldap-re24/libexec/slapd[0x49555b] /data/ldap-re24/libexec/slapd(overlay_op_walk+0x4a)[0x496bda] /data/ldap-re24/libexec/slapd[0x4976d3] /data/ldap-re24/libexec/slapd(fe_op_search+0x2c1)[0x434bb1] /data/ldap-re24/libexec/slapd(do_search+0x6fc)[0x4353ac] /data/ldap-re24/libexec/slapd[0x43292f] /data/ldap-re24/lib/libldap_r-2.4-releng.so.2[0x7f5258d0f7e0] /lib/libpthread.so.0[0x7f525831cfc7] /lib/libc.so.6(clone+0x6d)[0x7f52566ae5ad] =3D=3D=3D=3D=3D=3D=3D Memory map: =3D=3D=3D=3D=3D=3D=3D=3D 00400000-00527000 r-xp 00000000 08:06 4841518 /d= ata/ldap-re24/libexec/slapd 00726000-0072e000 rw-p 00126000 08:06 4841518 /d= ata/ldap-re24/libexec/slapd 0072e000-00772000 rw-p 0072e000 00:00 0 01511000-09566000 rw-p 01511000 00:00 0 [h= eap] 4032a000-4032b000 ---p 4032a000 00:00 0 4032b000-40b2b000 rw-p 4032b000 00:00 0 40c41000-40c42000 ---p 40c41000 00:00 0 40c42000-41442000 rw-p 40c42000 00:00 0 41442000-41443000 ---p 41442000 00:00 0 41443000-41c43000 rw-p 41443000 00:00 0 41c43000-41c44000 ---p 41c43000 00:00 0 41c44000-42444000 rw-p 41c44000 00:00 0 42444000-42445000 ---p 42444000 00:00 0 42445000-42c45000 rw-p 42445000 00:00 0 42c45000-42c46000 ---p 42c45000 00:00 0 42c46000-43446000 rw-p 42c46000 00:00 0 43446000-43447000 ---p 43446000 00:00 0 43447000-43c47000 rw-p 43447000 00:00 0 43c47000-43c48000 ---p 43c47000 00:00 0 43c48000-44448000 rw-p 43c48000 00:00 0 44448000-44449000 ---p 44448000 00:00 0 44449000-44c49000 rw-p 44449000 00:00 0 44c49000-44c4a000 ---p 44c49000 00:00 0 44c4a000-4544a000 rw-p 44c4a000 00:00 0 4544a000-4544b000 ---p 4544a000 00:00 0 4544b000-45c4b000 rw-p 4544b000 00:00 0 45c4b000-45c4c000 ---p 45c4b000 00:00 0 45c4c000-4644c000 rw-p 45c4c000 00:00 0 4644c000-4644d000 ---p 4644c000 00:00 0 4644d000-46c4d000 rw-p 4644d000 00:00 0 46c4d000-46c4e000 ---p 46c4d000 00:00 0 46c4e000-4744e00Aborted x4440:~/junk# Information in this email including any attachments may be privileged, conf= idential and is intended exclusively for the addressee. The views expressed= may not be official policy, but the personal views of the originator. If y= ou have received it in error, please notify the sender by return e-mail and= delete it from your system. You should not reproduce, distribute, store, r= etransmit, use or disclose its contents to anyone. Please note we reserve t= he right to monitor all e-mail communication through our internal and exter= nal networks. SKY and the SKY marks are trade marks of British Sky Broadcas= ting Group plc and are used under licence. British Sky Broadcasting Limited= (Registration No. 2906991), Sky Interactive Limited (Registration No. 3554= 332), Sky-In-Home Service Limited (Registration No. 2067075) and Sky Subscr= ibers Services Limited (Registration No. 2340150) are direct or indirect su= bsidiaries of British Sky Broadcasting Group plc (Registration No. 2247735)= . All of the companies mentioned in this paragraph are incorporated in Engl= and and Wales and share the same registered office at Grant Way, Isleworth,= Middlesex TW7 5QD.

1 0

Re: (ITS#5836) hung writers
by jorge.perez＠adaptia.net 09 Jul '09

09 Jul '09

Hi, On Tue, Jul 7, 2009 at 6:40 PM, Quanah Gibson-Mount<quanah(a)zimbra.com> wrote: > > Hi Jorge, > > We're going to need more information from you than you are providing. Please > make sure you: > > (a) build openldap without optimization (-O0) > (b) are gdbing the process, not a core file > (c) Send a full backtrace of everything: thr apply all bt full > > Also, please confirm what BDB version you are using, including if it has all > patches applied and compile options for BDB, and which backend you are > using. The last testing was wrong due to an old slapd.conf without writetimeout keyword, my apologies. We have been testing for 2 days to reproduce this problem and we have not seen the problem again, even though with debug symbols the assert ""assert( SLAP_SOCK_IS_ACTIVE( s ))"" still fails. Reading symbols from /usr/lib64/sasl2/libanonymous.so...done. Loaded symbols for /usr/lib64/sasl2/libanonymous.so Core was generated by `/local/slapd_L5B8_17-7 -f /home/cudb/dataAccess/ldapAccess/ldapFe/config/slapd.'. Program terminated with signal 6, Aborted. #0 0x00002b18e9dd5d45 in raise () from /lib64/libc.so.6 (gdb) bt #0 0x00002b18e9dd5d45 in raise () from /lib64/libc.so.6 #1 0x00002b18e9dd7140 in abort () from /lib64/libc.so.6 #2 0x00002b18e9dcf286 in __assert_fail () from /lib64/libc.so.6 #3 0x00000000004641b0 in slapd_clr_write (s=410, wake=0) at daemon.c:939 #4 0x000000000046db7d in connection_write (s=410) at connection.c:1855 #5 0x00000000004688e0 in slapd_daemon_task (ptr=0x0) at daemon.c:2567 #6 0x00002b18e9264143 in start_thread () from /lib64/libpthread.so.0 #7 0x00002b18e9e66b8d in clone () from /lib64/libc.so.6 #8 0x0000000000000000 in ?? () Best regards.

1 0

Re: (ITS#6139) slapd: password-hash rejects multiple values
by fumiyas＠osstech.jp 08 Jul '09

08 Jul '09

Hi, At Mon, 25 May 2009 13:22:47 GMT, michael(a)stroeder.com wrote: > fumiyas(a)osstech.co.jp wrote: > > My /etc/openldap/slapd.conf has the following line: > > password-hash {CRYPT} {SSHA} > > What exactly do you want to achieve by this? I use 'password-hash {CLEARTEXT} {CRYPT}' in my slapd.conf on OpenLDAP 2.3.43 to maintain the clear text password and the crypt(3)-ed password. The {CLEARTEXT} password can be used for: 1. Generating MD5 has for Digest/CRAM-MD5 authentication. 2. Notifying a user of the user's current password. (This is a rare case, I think.) 3. Migrating users and their passwords to another system in the future. 4. ... and so on. The {CRYPT} password can be used for old NIS clients. -- -- Name: SATOH Fumiyasu (fumiyas @ osstech co jp) -- Business Home: http://www.OSSTech.co.jp/ -- Personal Home: http://www.SFO.jp/blog/

1 0

Re: (ITS#6201) Double free in syncprov
by hyc＠symas.com 08 Jul '09

08 Jul '09

hyc(a)symas.com wrote: > Alister.Winfield(a)sns.bskyb.com wrote: >> Failed in the same way.... > > Can you please provide the full text of the error message that glibc printed? Also, can you reproduce this with a slapd compiled without optimization? The stack trace is missing some information here. >> >> >> (gdb) bt >> #0 0x00007f2b031b1ed5 in *__GI_raise (sig=3D<value optimized out>) at ../n= >> ptl/sysdeps/unix/sysv/linux/raise.c:64 >> #1 0x00007f2b031b33f3 in *__GI_abort () at abort.c:88 >> #2 0x00007f2b031ee3a8 in __libc_message (do_abort=3D2, fmt=3D0x7f2b0329d5f= >> 0 "*** glibc detected *** %s: %s: 0x%s ***\n") at ../sysdeps/unix/sysv/linu= >> x/libc_fatal.c:170 >> #3 0x00007f2b031f3948 in malloc_printerr (action=3D2, str=3D0x7f2b0329d6d8= >> "double free or corruption (out)", ptr=3D<value optimized out>) at malloc.= >> c:5891 >> #4 0x00007f2b031f5a56 in *__GI___libc_free (mem=3D0x7f2b03293fa0) at mallo= >> c.c:3626 >> #5 0x00007f2b00e01821 in syncprov_search_cleanup (op=3D0x7f279a205010, rs= >> =3D0x40efbd30) at ../../../../openldap-re24/servers/slapd/overlays/syncprov= >> .c:2024 >> #6 0x0000000000441c50 in ?? () >> #7 0x000000000044478b in ?? () >> #8 0x00000000004453f5 in slap_send_ldap_result () >> #9 0x00007f2b00e04733 in syncprov_op_search (op=3D0x7f279a205010, rs=3D0x4= >> 0efbd30) at ../../../../openldap-re24/servers/slapd/overlays/syncprov.c:243= >> 7 >> #10 0x0000000000496bda in overlay_op_walk () >> #11 0x00000000004976d3 in ?? () >> #12 0x000000000049555b in ?? () >> #13 0x0000000000496bda in overlay_op_walk () >> #14 0x00000000004976d3 in ?? () >> #15 0x0000000000434bb1 in fe_op_search () >> #16 0x00000000004353ac in do_search () >> #17 0x000000000043292f in ?? () >> #18 0x00007f2b058b07e0 in ldap_int_thread_pool_wrapper (xpool=3D<value opti= >> mized out>) at ../../../openldap-re24/libraries/libldap_r/tpool.c:685 >> #19 0x00007f2b04ebdfc7 in start_thread (arg=3D<value optimized out>) at pth= >> read_create.c:297 >> #20 0x00007f2b0324f5ad in clone () from /usr/lib/debug/libc.so.6 >> #21 0x0000000000000000 in ?? () >> (gdb) -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#6201) Double free in syncprov
by hyc＠symas.com 08 Jul '09

08 Jul '09

Alister.Winfield(a)sns.bskyb.com wrote: > Failed in the same way.... Can you please provide the full text of the error message that glibc printed? > > > (gdb) bt > #0 0x00007f2b031b1ed5 in *__GI_raise (sig=3D<value optimized out>) at ../n= > ptl/sysdeps/unix/sysv/linux/raise.c:64 > #1 0x00007f2b031b33f3 in *__GI_abort () at abort.c:88 > #2 0x00007f2b031ee3a8 in __libc_message (do_abort=3D2, fmt=3D0x7f2b0329d5f= > 0 "*** glibc detected *** %s: %s: 0x%s ***\n") at ../sysdeps/unix/sysv/linu= > x/libc_fatal.c:170 > #3 0x00007f2b031f3948 in malloc_printerr (action=3D2, str=3D0x7f2b0329d6d8= > "double free or corruption (out)", ptr=3D<value optimized out>) at malloc.= > c:5891 > #4 0x00007f2b031f5a56 in *__GI___libc_free (mem=3D0x7f2b03293fa0) at mallo= > c.c:3626 > #5 0x00007f2b00e01821 in syncprov_search_cleanup (op=3D0x7f279a205010, rs= > =3D0x40efbd30) at ../../../../openldap-re24/servers/slapd/overlays/syncprov= > .c:2024 > #6 0x0000000000441c50 in ?? () > #7 0x000000000044478b in ?? () > #8 0x00000000004453f5 in slap_send_ldap_result () > #9 0x00007f2b00e04733 in syncprov_op_search (op=3D0x7f279a205010, rs=3D0x4= > 0efbd30) at ../../../../openldap-re24/servers/slapd/overlays/syncprov.c:243= > 7 > #10 0x0000000000496bda in overlay_op_walk () > #11 0x00000000004976d3 in ?? () > #12 0x000000000049555b in ?? () > #13 0x0000000000496bda in overlay_op_walk () > #14 0x00000000004976d3 in ?? () > #15 0x0000000000434bb1 in fe_op_search () > #16 0x00000000004353ac in do_search () > #17 0x000000000043292f in ?? () > #18 0x00007f2b058b07e0 in ldap_int_thread_pool_wrapper (xpool=3D<value opti= > mized out>) at ../../../openldap-re24/libraries/libldap_r/tpool.c:685 > #19 0x00007f2b04ebdfc7 in start_thread (arg=3D<value optimized out>) at pth= > read_create.c:297 > #20 0x00007f2b0324f5ad in clone () from /usr/lib/debug/libc.so.6 > #21 0x0000000000000000 in ?? () > (gdb) > > > -----Original Message----- > From: Quanah Gibson-Mount [mailto:quanah@zimbra.com] > Sent: 08 July 2009 19:15 > To: Alister Winfield; openldap-its(a)openldap.org > Subject: Re: (ITS#6201) Double free in syncprov > > --On Wednesday, July 08, 2009 4:59 PM +0000 alister.winfield(a)sns.bskyb.com > wrote: > >> Full_Name: Alister Winfield >> Version: 2.4.16 >> OS: Debian Lenny >> URL: >> Submission from: (NULL) (90.207.245.9) >> >> >> I'm testing 2.4.16 and have stumbled across a problem. > > Please re-test and follow up with current RE24 CVS, as numerous fixes have = > been made since the 2.4.16 release. Thanks! > > --Quanah -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

RE: (ITS#6201) Double free in syncprov
by Alister.Winfield＠sns.bskyb.com 08 Jul '09

08 Jul '09

Failed in the same way.... (gdb) bt #0 0x00007f2b031b1ed5 in *__GI_raise (sig=3D<value optimized out>) at ../n= ptl/sysdeps/unix/sysv/linux/raise.c:64 #1 0x00007f2b031b33f3 in *__GI_abort () at abort.c:88 #2 0x00007f2b031ee3a8 in __libc_message (do_abort=3D2, fmt=3D0x7f2b0329d5f= 0 "*** glibc detected *** %s: %s: 0x%s ***\n") at ../sysdeps/unix/sysv/linu= x/libc_fatal.c:170 #3 0x00007f2b031f3948 in malloc_printerr (action=3D2, str=3D0x7f2b0329d6d8= "double free or corruption (out)", ptr=3D<value optimized out>) at malloc.= c:5891 #4 0x00007f2b031f5a56 in *__GI___libc_free (mem=3D0x7f2b03293fa0) at mallo= c.c:3626 #5 0x00007f2b00e01821 in syncprov_search_cleanup (op=3D0x7f279a205010, rs= =3D0x40efbd30) at ../../../../openldap-re24/servers/slapd/overlays/syncprov= .c:2024 #6 0x0000000000441c50 in ?? () #7 0x000000000044478b in ?? () #8 0x00000000004453f5 in slap_send_ldap_result () #9 0x00007f2b00e04733 in syncprov_op_search (op=3D0x7f279a205010, rs=3D0x4= 0efbd30) at ../../../../openldap-re24/servers/slapd/overlays/syncprov.c:243= 7 #10 0x0000000000496bda in overlay_op_walk () #11 0x00000000004976d3 in ?? () #12 0x000000000049555b in ?? () #13 0x0000000000496bda in overlay_op_walk () #14 0x00000000004976d3 in ?? () #15 0x0000000000434bb1 in fe_op_search () #16 0x00000000004353ac in do_search () #17 0x000000000043292f in ?? () #18 0x00007f2b058b07e0 in ldap_int_thread_pool_wrapper (xpool=3D<value opti= mized out>) at ../../../openldap-re24/libraries/libldap_r/tpool.c:685 #19 0x00007f2b04ebdfc7 in start_thread (arg=3D<value optimized out>) at pth= read_create.c:297 #20 0x00007f2b0324f5ad in clone () from /usr/lib/debug/libc.so.6 #21 0x0000000000000000 in ?? () (gdb) -----Original Message----- From: Quanah Gibson-Mount [mailto:quanah@zimbra.com] Sent: 08 July 2009 19:15 To: Alister Winfield; openldap-its(a)openldap.org Subject: Re: (ITS#6201) Double free in syncprov --On Wednesday, July 08, 2009 4:59 PM +0000 alister.winfield(a)sns.bskyb.com wrote: > Full_Name: Alister Winfield > Version: 2.4.16 > OS: Debian Lenny > URL: > Submission from: (NULL) (90.207.245.9) > > > I'm testing 2.4.16 and have stumbled across a problem. Please re-test and follow up with current RE24 CVS, as numerous fixes have = been made since the 2.4.16 release. Thanks! --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration Information in this email including any attachments may be privileged, conf= idential and is intended exclusively for the addressee. The views expressed= may not be official policy, but the personal views of the originator. If y= ou have received it in error, please notify the sender by return e-mail and= delete it from your system. You should not reproduce, distribute, store, r= etransmit, use or disclose its contents to anyone. Please note we reserve t= he right to monitor all e-mail communication through our internal and exter= nal networks. SKY and the SKY marks are trade marks of British Sky Broadcas= ting Group plc and are used under licence. British Sky Broadcasting Limited= (Registration No. 2906991), Sky Interactive Limited (Registration No. 3554= 332), Sky-In-Home Service Limited (Registration No. 2067075) and Sky Subscr= ibers Services Limited (Registration No. 2340150) are direct or indirect su= bsidiaries of British Sky Broadcasting Group plc (Registration No. 2247735)= . All of the companies mentioned in this paragraph are incorporated in Engl= and and Wales and share the same registered office at Grant Way, Isleworth,= Middlesex TW7 5QD.

1 0

Re: (ITS#6201) Double free in syncprov
by quanah＠zimbra.com 08 Jul '09

08 Jul '09

--On Wednesday, July 08, 2009 4:59 PM +0000 alister.winfield(a)sns.bskyb.com wrote: > Full_Name: Alister Winfield > Version: 2.4.16 > OS: Debian Lenny > URL: > Submission from: (NULL) (90.207.245.9) > > > I'm testing 2.4.16 and have stumbled across a problem. Please re-test and follow up with current RE24 CVS, as numerous fixes have been made since the 2.4.16 release. Thanks! --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

Re: (ITS#6200) slapd crashes under load w/ syncrepl
by quanah＠zimbra.com 08 Jul '09

08 Jul '09

--On Wednesday, July 08, 2009 4:44 PM +0000 imf(a)u.washington.edu wrote: > Full_Name: Tracy Stenvik > Version: 2.4.16 (+db-4.7.25pl4) > OS: Red Hat Enterprise Linux Server release 5.3 (Tikanga) > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (140.142.111.41) > > > In evaluating 2.4.16 (to upgrade our 2.3.39-based directory service) we > have encountered some serious issues. A similated production-level load > that 2.3.39 (with slurpd running) handles easily causes 2.4.16 (and > 2.4.15) to crash within 5 minutes to 24 hours. Please re-test with current RE24 from CVS, as numerous fixes have been made and 2.4.17 is imminent. Thanks! --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

(ITS#6201) Double free in syncprov
by alister.winfield＠sns.bskyb.com 08 Jul '09

08 Jul '09

Full_Name: Alister Winfield Version: 2.4.16 OS: Debian Lenny URL: Submission from: (NULL) (90.207.245.9) I'm testing 2.4.16 and have stumbled across a problem. compiled: openldap2.4.16 db4.7.25 Imported ~2,500,000 entries then selected 80000 random entries from the ou=<n>,ou=radiusUsers,dc=sky,dc=com trees and started making lots of deletes. WIthout a slave active I haven't 'yet' seen any crashes. With an active syncrepl slave, however, I get the following crash at random points (unfortunately it can take many 1000's of updates to trigger this which takes a while. (gdb) c Continuing. [New Thread 0x471f8950 (LWP 20035)] [New Thread 0x479f9950 (LWP 20036)] Program received signal SIGABRT, Aborted. [Switching to Thread 0x439f1950 (LWP 19973)] 0x00007fe0426a6ed5 in *__GI_raise (sig=<value optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64 64 ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory. in ../nptl/sysdeps/unix/sysv/linux/raise.c (gdb) bt #0 0x00007fe0426a6ed5 in *__GI_raise (sig=<value optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64 #1 0x00007fe0426a83f3 in *__GI_abort () at abort.c:88 #2 0x00007fe0426e33a8 in __libc_message (do_abort=2, fmt=0x7fe0427925f0 "*** glibc detected *** %s: %s: 0x%s ***\n") at ../sysdeps/unix/sysv/linux/libc_fatal.c:170 #3 0x00007fe0426e8948 in malloc_printerr (action=2, str=0x7fe0427926d8 "double free or corruption (out)", ptr=<value optimized out>) at malloc.c:5891 #4 0x00007fe0426eaa56 in *__GI___libc_free (mem=0x7fe042788fa0) at malloc.c:3626 #5 0x00007fe040750801 in syncprov_search_cleanup (op=0x7fdce01eb4a0, rs=0x439f0d30) at ../../../../openldap-2.4.16/servers/slapd/overlays/syncprov.c:2024 #6 0x00000000004415c0 in ?? () #7 0x00000000004440a9 in ?? () #8 0x0000000000444cf5 in slap_send_ldap_result () #9 0x00007fe040753713 in syncprov_op_search (op=0x7fdce01eb4a0, rs=0x439f0d30) at ../../../../openldap-2.4.16/servers/slapd/overlays/syncprov.c:2437 #10 0x000000000049629a in overlay_op_walk () #11 0x0000000000496d93 in ?? () #12 0x0000000000494c10 in ?? () #13 0x000000000049629a in overlay_op_walk () #14 0x0000000000496d93 in ?? () #15 0x0000000000434581 in fe_op_search () #16 0x0000000000434d7c in do_search () #17 0x00000000004323d6 in ?? () #18 0x00007fe044da566a in ldap_int_thread_pool_wrapper (xpool=<value optimized out>) at ../../../openldap-2.4.16/libraries/libldap_r/tpool.c:663 #19 0x00007fe0443b2fc7 in start_thread (arg=<value optimized out>) at pthread_create.c:297 #20 0x00007fe0427445ad in clone () from /usr/lib/debug/libc.so.6 #21 0x0000000000000000 in ?? () (gdb) slapd.conf # # See slapd.conf(5) for details on configuration options. # This file should NOT be world readable. # include /data/ldap-2.4.16/etc/ldap/schema/core.schema include /data/ldap-2.4.16/etc/ldap/schema/cosine.schema include /data/ldap-2.4.16/etc/ldap/schema/nis.schema include /data/ldap-2.4.16/etc/ldap/schema/inetorgperson.schema include /data/ldap-2.4.16/etc/ldap/schema/sky.schema # Define global ACLs to disable default read access. # Do not enable referrals until AFTER you have a working directory # service AND an understanding of referrals. #referral ldap://root.openldap.org loglevel 0 concurrency 512 threads 1024 tool-threads 128 idletimeout 5 pidfile /data/ldap-2.4.16/var/run/slapd.pid argsfile /data/ldap-2.4.16/var/run/slapd.args # Load dynamic backend modules: modulepath /data/ldap-2.4.16/libexec/ldap #moduleload back_bdb.la moduleload back_hdb.la moduleload syncprov.la moduleload accesslog.la ####################################################################### # BDB database definitions ####################################################################### access to * by dn.base="cn=admin,dc=test" read by * break database hdb suffix cn=accesslog directory /bigdata/ldap-2.4.16-splitdb/accesslog rootdn cn=accesslog index default eq index entryCSN,objectClass,reqEnd,reqResult,reqStart overlay syncprov syncprov-nopresent TRUE syncprov-reloadhint TRUE limits dn.exact="cn=admin,dc=test" time.soft=unlimited time.hard=unlimited size.soft=unlimited size.hard=unlimited # RADIUS DB-0 ACCESSLOG DB database hdb suffix cn=accesslog0 directory /bigdata/ldap-2.4.16-splitdb/accesslog0 rootdn cn=accesslog0 index default eq index entryCSN,objectClass,reqEnd,reqResult,reqStart overlay syncprov syncprov-nopresent TRUE syncprov-reloadhint TRUE cachesize 100000 dncachesize 100000 idlcachesize 500000 limits dn.exact="cn=admin,dc=test" time.soft=unlimited time.hard=unlimited size.soft=unlimited size.hard=unlimited # RADIUS DB-0 database hdb subordinate suffix "ou=0,ou=radiusUsers,dc=test" directory /bigdata/ldap-2.4.16-splitdb/radiusUsers/0 rootdn "cn=admin,dc=test" cachesize 400000 dncachesize 400000 idlcachesize 800000 # Indices to maintain index objectClass eq index entryUUID eq index entryCSN,contextCSN eq index cn eq,sub,pres overlay syncprov syncprov-checkpoint 100 1 syncprov-sessionlog 40000 overlay accesslog logdb cn=accesslog0 logops writes logsuccess TRUE logpurge 00+20:00 17+12:00 limits dn.exact="cn=admin,dc=test" time.soft=unlimited time.hard=unlimited size.soft=unlimited size.hard=unlimited # ------------------------------------------------------- # RADIUS DB-1 ACCESSLOG DB # ------------------------------------------------------- database hdb suffix cn=accesslog1 directory /bigdata/ldap-2.4.16-splitdb/accesslog1 rootdn cn=accesslog1 index default eq index entryCSN,objectClass,reqEnd,reqResult,reqStart cachesize 100000 dncachesize 100000 idlcachesize 500000 overlay syncprov syncprov-nopresent TRUE syncprov-reloadhint TRUE limits dn.exact="cn=admin,dc=test" time.soft=unlimited time.hard=unlimited size.soft=unlimited size.hard=unlimited # RADIUS DB-1 database hdb subordinate suffix "ou=1,ou=radiusUsers,dc=test" rootdn "cn=admin,dc=test" directory /bigdata/ldap-2.4.16-splitdb/radiusUsers/1 cachesize 400000 dncachesize 400000 idlcachesize 800000 # Indices to maintain index objectClass eq index entryUUID eq index entryCSN,contextCSN eq index cn eq,sub,pres overlay syncprov syncprov-checkpoint 100 1 syncprov-sessionlog 40000 overlay accesslog logdb cn=accesslog1 logops writes logsuccess TRUE logpurge 00+20:00 17+12:00 limits dn.exact="cn=admin,dc=test" time.soft=unlimited time.hard=unlimited size.soft=unlimited size.hard=unlimited # ------------------------------------------------------- # RADIUS DB-2 ACCESSLOG DB # ------------------------------------------------------- database hdb suffix cn=accesslog2 directory /bigdata/ldap-2.4.16-splitdb/accesslog2 rootdn cn=accesslog2 index default eq index entryCSN,objectClass,reqEnd,reqResult,reqStart cachesize 100000 dncachesize 100000 idlcachesize 500000 overlay syncprov syncprov-nopresent TRUE syncprov-reloadhint TRUE limits dn.exact="cn=admin,dc=test" time.soft=unlimited time.hard=unlimited size.soft=unlimited size.hard=unlimited # RADIUS DB-2 database hdb subordinate suffix "ou=2,ou=radiusUsers,dc=test" rootdn "cn=admin,dc=test" directory /bigdata/ldap-2.4.16-splitdb/radiusUsers/2 cachesize 400000 dncachesize 400000 idlcachesize 800000 # Indices to maintain index objectClass eq index entryUUID eq index entryCSN,contextCSN eq index cn eq,sub,pres overlay syncprov syncprov-checkpoint 100 1 syncprov-sessionlog 40000 overlay accesslog logdb cn=accesslog2 logops writes logsuccess TRUE logpurge 00+20:00 17+12:00 limits dn.exact="cn=admin,dc=test" time.soft=unlimited time.hard=unlimited size.soft=unlimited size.hard=unlimited ... ... Repeated up until ou=7,ou=radiusUsers,dc=test. # ---------- # ROOT DB # ---------- database hdb suffix "dc=test" rootdn "cn=admin,dc=test" # Cleartext passwords, especially for the rootdn, should # be avoid. See slappasswd(8) and slapd.conf(5) for details. # Use of strong authentication encouraged. rootpw password # The database directory MUST exist prior to running slapd AND # should only be accessible by the slapd and slap tools. # Mode 700 recommended. directory /bigdata/ldap-2.4.16-splitdb/root cachesize 1000000 dncachesize 5000000 idlcachesize 8000000 # Indices to maintain index objectClass eq index entryUUID eq index entryCSN,contextCSN eq index cn eq,sub,pres overlay glue overlay syncprov syncprov-checkpoint 100 1 syncprov-sessionlog 40000 overlay accesslog logdb cn=accesslog logops writes logsuccess TRUE logpurge 00+20:00 17+12:00 limits dn.exact="cn=admin,dc=test" time.soft=unlimited time.hard=unlimited size.soft=unlimited size.hard=unlimited

1 0

(ITS#6200) slapd crashes under load w/ syncrepl
by imf＠u.washington.edu 08 Jul '09

08 Jul '09

Full_Name: Tracy Stenvik Version: 2.4.16 (+db-4.7.25pl4) OS: Red Hat Enterprise Linux Server release 5.3 (Tikanga) URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (140.142.111.41) In evaluating 2.4.16 (to upgrade our 2.3.39-based directory service) we have encountered some serious issues. A similated production-level load that 2.3.39 (with slurpd running) handles easily causes 2.4.16 (and 2.4.15) to crash within 5 minutes to 24 hours. Without replication active, 2.4.16 survived 24 hours under load. As soon as replication was activated however, the server crashed within 5 minutes. We've tried several syncrepl variants (as described below) but none appear stable. In analyzing the various crash dumps, most of the trouble appears to lie in the caching code. It has a nasty habit of returing nulls when nulls are not expected. Our simulated load consists of an arbitrary mix of adds/modifies/searches to various objects in the directory. This load is defined as "N" instances of the "Stressor" test, as discussed on the "GDS Stress Test Findings" page at, http://wiki.cac.washington.edu/display/infra/GDS+Stress+Test+Findings Following, you will find a series of crash dumps, along with a few comments. Also included are sections the various slapd.confs we were testing with. Please contact me if you require additional information. delta syncrepl crash -------------------- (gdb) where #0 0xb7b7abab in strncmp () from /lib/libc.so.6 #1 0x08143ea4 in bdb_rdn_cmp (v_e1=0xf069be8, v_e2=0xf73cb48) at cache.c:301 #2 0x081d00ee in avl_insert (root=0x8622088, data=0xf069be8, fcmp=0x8143e55 <bdb_rdn_cmp>, fdup=0x81d10a0 <avl_dup_error>) at avl.c:95 #3 0x08144692 in hdb_cache_find_parent (op=0xf95ea78, txn=0xe7def98, id=343637, res=0x14b3ade0) at cache.c:593 #4 0x08144dc9 in hdb_cache_find_id (op=0xf95ea78, tid=0xe7def98, id=343637, eip=0x14b3ade0, flag=0, lock=0x14b3adc4) at cache.c:905 #5 0x0810b6a0 in hdb_search (op=0xf95ea78, rs=0x14bfc0e8) at search.c:705 #6 0x080e9b02 in overlay_op_walk (op=0xf95ea78, rs=0x14bfc0e8, which=op_search, oi=0x85aeed8, on=0x0) at backover.c:669 #7 0x080e9cb7 in over_op_func (op=0xf95ea78, rs=0x14bfc0e8, which=op_search) at backover.c:721 #8 0x080e9d3b in over_op_search (op=0xf95ea78, rs=0x14bfc0e8) at backover.c:743 #9 0x0806f0b3 in fe_op_search (op=0xf95ea78, rs=0x14bfc0e8) at search.c:366 #10 0x0806ea53 in do_search (op=0xf95ea78, rs=0x14bfc0e8) at search.c:217 #11 0x0806b959 in connection_operation (ctx=0x14bfc1d0, arg_v=0xf95ea78) at connection.c:1097 #12 0x0806be33 in connection_read_thread (ctx=0x14bfc1d0, argv=0x23) at connection.c:1223 #13 0x081d74a1 in ldap_int_thread_pool_wrapper (xpool=0x8561178) at tpool.c:663 #14 0xb7e4d49b in start_thread () from /lib/libpthread.so.0 #15 0xb7bdc42e in clone () from /lib/libc.so.6 (gdb) p *(EntryInfo *)v_e1 $2 = {bei_parent = 0x8622058, bei_id = 343637, bei_lockpad = 0, bei_state = 132, bei_finders = 0, bei_nrdn = {bv_len = 45, bv_val = 0xf7af1f8 "serialNumber=40442595174b5ebf53167252b5cbcc34"}, bei_rdn = {bv_len = 45, bv_val = 0xf36d700 "serialNumber=40442595174B5EBF53167252B5CBCC34"}, bei_modrdns = 0, bei_ckids = 0, bei_dkids = 0, bei_e = 0x0, bei_kids = 0x0, bei_kids_mutex = {__data = {__lock = 1, __count = 0, __owner = 32550, __kind = 0, __nusers = 1, {__spins = 0, __list = {__next = 0x0}}}, __size = "\001\000\000\000\000\000\000\000&\177\000\000\000\000\000\000\001\000\000\000\000\000\000", __align = 1}, bei_lrunext = 0x8978ad0, bei_lruprev = 0x0} (gdb) p *(EntryInfo *)v_e2 $3 = {bei_parent = 0x0, bei_id = 410257, bei_lockpad = 0, bei_state = 0, bei_finders = 0, bei_nrdn = {bv_len = 45, bv_val = 0x0}, bei_rdn = { bv_len = 45, bv_val = 0x0}, bei_modrdns = 0, bei_ckids = 0, bei_dkids = 0, bei_e = 0x0, bei_kids = 0x0, bei_kids_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __kind = 0, __nusers = 0, {__spins = 0, __list = {__next = 0x0}}}, __size = '\0' <repeats 23 times>, __align = 0}, bei_lrunext = 0x8978ad0, bei_lruprev = 0x0} In bdb_rdn_cmp() you're comparing e1->bei_nrdn.bv_val to e2->bei_nrdn.bv_val but the latter is null even though e2->bei_nrdn.bv_len is not zero. standard syncrepl crash ----------------------- (gdb) where #0 0xb7d9f300 in pthread_mutex_lock () from /lib/libpthread.so.0 #1 0x081d8645 in ldap_pvt_thread_mutex_lock (mutex=0x34) at thr_posix.c:296 #2 0x08145bc0 in bdb_cache_delete_internal (cache=0xa25cc04, e=0x147e6d38, decr=0) at cache.c:1372 #3 0x08144b08 in hdb_cache_lru_purge (bdb=0xa25cbc8) at cache.c:776 #4 0x081452d1 in hdb_cache_find_id (op=0x11923ca8, tid=0x1316aa48, id=280869, eip=0x2bffcbfc, flag=0, lock=0x2bffcc54) at cache.c:1053 #5 0x0814b2f7 in hdb_dn2entry (op=0x11923ca8, tid=0x1316aa48, dn=0xa25e3bc, e=0x2bffcc64, matched=0, lock=0x2bffcc54) at dn2entry.c:67 #6 0x08151845 in hdb_entry_get (op=0x11923ca8, ndn=0xa25e3bc, oc=0xa2549f0, at=0xa23ebb0, rw=0, ent=0x2bffceac) at id2entry.c:354 #7 0x080e936d in overlay_entry_get_ov (op=0x11923ca8, dn=0xa25e3bc, oc=0xa2549f0, ad=0xa23ebb0, rw=0, e=0x2bffceac, on=0x0) at backover.c:364 #8 0x080e941f in over_entry_get_rw (op=0x11923ca8, dn=0xa25e3bc, oc=0xa2549f0, ad=0xa23ebb0, rw=0, e=0x2bffceac) at backover.c:396 #9 0x0807cfc3 in be_entry_get_rw (op=0x11923ca8, ndn=0xa25e3bc, oc=0xa2549f0, at=0xa23ebb0, rw=0, e=0x2bffceac) at backend.c:1346 #10 0x0807d1a2 in fe_acl_group (op=0x11923ca8, target=0x0, gr_ndn=0xa25e3bc, op_ndn=0x11923d44, group_oc=0xa2549f0, group_at=0xa23ebb0) at backend.c:1402 #11 0x0807d8c3 in backend_group (op=0x11923ca8, target=0x0, gr_ndn=0xa25e3bc, op_ndn=0x11923d44, group_oc=0xa2549f0, group_at=0xa23ebb0) at backend.c:1597 #12 0x080d32e2 in limits_get (op=0x11923ca8, limit=0x11923cdc) at limits.c:117 #13 0x080d588d in limits_check (op=0x11923ca8, rs=0x2bffe0e8) at limits.c:1106 #14 0x0806f095 in fe_op_search (op=0x11923ca8, rs=0x2bffe0e8) at search.c:364 #15 0x0806ea53 in do_search (op=0x11923ca8, rs=0x2bffe0e8) at search.c:217 #16 0x0806b959 in connection_operation (ctx=0x2bffe1d0, arg_v=0x11923ca8) at connection.c:1097 #17 0x0806be33 in connection_read_thread (ctx=0x2bffe1d0, argv=0x1d) at connection.c:1223 #18 0x081d74a1 in ldap_int_thread_pool_wrapper (xpool=0xa211178) at tpool.c:663 #19 0xb7d9d49b in start_thread () from /lib/libpthread.so.0 #20 0xb7b2c42e in clone () from /lib/libc.so.6 (gdb) p *e->bei_parent Cannot access memory at address 0x0 In bdb_cache_delete_internal(), e->bei_parent is null. 2-way multimaster syncrepl crash -------------------------------- (gdb) where #0 0x08149c1a in hdb_delete (op=0xc01caa8, rs=0x2ecfc0e8) at delete.c:217 #1 0x080e9b02 in overlay_op_walk (op=0xc01caa8, rs=0x2ecfc0e8, which=op_delete, oi=0x88b4550, on=0x0) at backover.c:669 #2 0x080e9cb7 in over_op_func (op=0xc01caa8, rs=0x2ecfc0e8, which=op_delete) at backover.c:721 #3 0x080e9de5 in over_op_delete (op=0xc01caa8, rs=0x2ecfc0e8) at backover.c:773 #4 0x0808ae88 in fe_op_delete (op=0xc01caa8, rs=0x2ecfc0e8) at delete.c:174 #5 0x0808abdb in do_delete (op=0xc01caa8, rs=0x2ecfc0e8) at delete.c:95 #6 0x0806b959 in connection_operation (ctx=0x2ecfc1d0, arg_v=0xc01caa8) at connection.c:1097 #7 0x0806be33 in connection_read_thread (ctx=0x2ecfc1d0, argv=0x22) at connection.c:1223 #8 0x081d74a1 in ldap_int_thread_pool_wrapper (xpool=0x8867178) at tpool.c:663 #9 0xb7e4749b in start_thread () from /lib/libpthread.so.0 #10 0xb7bd642e in clone () from /lib/libc.so.6 (gdb) p eip $1 = (EntryInfo *) 0x0 In hdb_delete(), eip is null. 2-way multimaster syncrepl crash -------------------------------- (gdb) where #0 0x080956db in slap_acl_get (a=0x8b9ba80, count=0x37f5d824, op=0x8c5d948, e=0x37f5dad0, desc=0x8b9b9f0, val=0x0, matches=0x37f5d1c4, state=0x37f5d97c) at acl.c:599 #1 0x080949b9 in slap_access_allowed (op=0x8c5d948, e=0x37f5dad0, desc=0x8b9b9f0, val=0x0, access=ACL_WDEL, state=0x37f5d97c, maskp=0x37f5d8e8) at acl.c:236 #2 0x08094cd5 in fe_access_allowed (op=0x8c5d948, e=0x37f5dad0, desc=0x8b9b9f0, val=0x0, access=ACL_WDEL, state=0x37f5d97c, maskp=0x37f5d8e8) at acl.c:360 #3 0x0809503c in access_allowed_mask (op=0x8c5d948, e=0x37f5dad0, desc=0x8b9b9f0, val=0x0, access=ACL_WDEL, state=0x37f5d97c, maskp=0x0) at acl.c:462 #4 0x08098b8f in acl_check_modlist (op=0x8c5d948, e=0x37f5dad0, mlist=0x34afa7c8) at acl.c:1986 #5 0x0810373a in hdb_modify_internal (op=0x8c5d948, tid=0x961aa10, modlist=0x34afa7c8, e=0x37f5dad0, text=0x37f5f0fc, textbuf=0x37f5db18 "", textlen=256) at modify.c:88 #6 0x08105292 in hdb_modify (op=0x8c5d948, rs=0x37f5f0e8) at modify.c:590 #7 0x080e9b02 in overlay_op_walk (op=0x8c5d948, rs=0x37f5f0e8, which=op_modify, oi=0x8bbb550, on=0x0) at backover.c:669 #8 0x080e9cb7 in over_op_func (op=0x8c5d948, rs=0x37f5f0e8, which=op_modify) at backover.c:721 #9 0x080e9d7f in over_op_modify (op=0x8c5d948, rs=0x37f5f0e8) at backover.c:755 #10 0x08088a89 in fe_op_modify (op=0x8c5d948, rs=0x37f5f0e8) at modify.c:301 #11 0x080884c8 in do_modify (op=0x8c5d948, rs=0x37f5f0e8) at modify.c:175 #12 0x0806b959 in connection_operation (ctx=0x37f5f1d0, arg_v=0x8c5d948) at connection.c:1097 #13 0x0806be33 in connection_read_thread (ctx=0x37f5f1d0, argv=0x22) at connection.c:1223 #14 0x081d74a1 in ldap_int_thread_pool_wrapper (xpool=0x8b6e178) at tpool.c:663 #15 0xb7e6749b in start_thread () from /lib/libpthread.so.0 #16 0xb7bf642e in clone () from /lib/libc.so.6 (gdb) p e->e_nname $3 = {bv_len = 77, bv_val = 0x0} In slap_acl_get(), e->e_nname.bv_val (e->e_ndn) is null. delta syncrepl slapd.conf ------------------------- serverID 1 ldap://rufus01.u.washington.edu:290 serverID 2 ldap://rufus04.u.washington.edu:290 gentlehup on idletimeout 600 sizelimit 20000 sortvals member # ===== Replication db ==== database hdb suffix "cn=log" directory /data/au20aa_log rootdn "cn=Manager,dc=washington,dc=edu" #index objectClass eq #index entryUUID,entryCSN eq index default eq index entryCSN,objectClass,reqEnd,reqResult,reqStart overlay syncprov syncprov-reloadhint true syncprov-nopresent true # ===== Primary db === database hdb suffix "dc=washington,dc=edu" rootdn "cn=Manager,dc=washington,dc=edu" include /usr/local/openldap-2.4.16/etc/openldap/private/slapd-pw.conf directory /data/au20aa index entryCSN eq index entryUUID eq checkpoint 10000 15 cachesize 90000 idlcachesize 180000 overlay syncprov #syncprov-sessionlog 100 syncprov-checkpoint 1000 60 # accesslog overlay definitions for primary database overlay accesslog logdb cn=log logops writes logsuccess true # scan the accesslog DB every day, and purge entries older than 7 days logpurge 07+00:00 01+00:00 standard syncrepl slapd.conf ---------------------------- serverID 1 ldap://rufus01.u.washington.edu:290 serverID 2 ldap://rufus04.u.washington.edu:290 gentlehup on idletimeout 600 sizelimit 20000 sortvals member # ===== Primary db ===== database hdb suffix "dc=washington,dc=edu" rootdn "cn=Manager,dc=washington,dc=edu" include /usr/local/openldap-2.4.16/etc/openldap/private/slapd-pw.conf directory /data/au20aa index entryCSN eq index entryUUID eq checkpoint 10000 15 cachesize 90000 idlcachesize 180000 overlay syncprov #syncprov-sessionlog 100 syncprov-checkpoint 1000 60 2-way multimaster syncrepl slapd.conf ------------------------------------- serverID 1 ldap://rufus01.u.washington.edu:290 serverID 2 ldap://rufus04.u.washington.edu:290 gentlehup on idletimeout 600 sizelimit 20000 sortvals member # ===== Primary db ===== database hdb suffix "dc=washington,dc=edu" rootdn "cn=Manager,dc=washington,dc=edu" include /usr/local/openldap-2.4.16/etc/openldap/private/slapd-pw.conf directory /data/au20aa index entryCSN eq index entryUUID eq checkpoint 10000 15 cachesize 90000 idlcachesize 180000 overlay syncprov #syncprov-sessionlog 100 syncprov-checkpoint 1000 60 ### ### ----------- Replication ### syncrepl rid=101 provider=ldap://rufus01.u.washington.edu:290 searchbase="dc=washington,dc=edu" bindmethod=sasl saslmech=external starttls=yes tls_cert=/usr/local/openldap-2.4.16/etc/openldap/rufus04.crt tls_key=/usr/local/openldap-2.4.16/etc/openldap/private/rufus04.key filter="(objectClass=*)" attrs="*,+" schemachecking=off scope=sub type=refreshAndPersist retry="60 +" syncrepl rid=102 provider=ldap://rufus04.u.washington.edu:290 searchbase="dc=washington,dc=edu" bindmethod=sasl saslmech=external starttls=yes tls_cert=/usr/local/openldap-2.4.16/etc/openldap/rufus04.crt tls_key=/usr/local/openldap-2.4.16/etc/openldap/private/rufus04.key filter="(objectClass=*)" attrs="*,+" schemachecking=off scope=sub type=refreshAndPersist retry="60 +" mirrormode yes

1 0

(ITS#6199) [JLDAP] SASL integrity & confidentiality
by ioplex＠gmail.com 07 Jul '09

07 Jul '09

Full_Name: Michael B Allen Version: JLDAP OS: Linux URL: http://www.ioplex.com/~miallen/openldap-jldap-saslwrap.diff Submission from: (NULL) (71.187.189.103) It would be fabulous if SASL integrity and confidentiality worked in JLDAP. I just tried and failed to add support for it. I could be just misguided but it seems that because the ASN.1 routines encoded and decoded BER directly to and from the socket element by element, there is a fairly low-level issue with wrapping and unwrapping entire message buffers. But I was unable to pinpoint the precise cause. See the URL for my crude patch to test SASL wrap / unwrap. Of course you'll need a SASL client and mechanism that supports at least integrity. Since com.novell.ldap.Connection.writeMessage encodes the entire message before writing it to the socket, the SASL wrap works. However, reading data from the input stream has a strange off-by-one problem. The first byte is missing. But since the first 4 bytes are the big endian SASL buffer size, I can hack to code to skip it and it is unwrapped properly (although it looks like a referral so I think perhaps my search() test program isn't quite right either). It then hangs in LDAPSearchResults.hasMore: at java.lang.Object.wait(Object.java:474) at com.novell.ldap.MessageAgent.getLDAPMessage(MessageAgent.java:399) - locked <0x89152240> (a com.novell.ldap.MessageVector) at com.novell.ldap.LDAPMessageQueue.getResponse(LDAPMessageQueue.java:175) at com.novell.ldap.LDAPMessageQueue.getResponse(LDAPMessageQueue.java:126) at com.novell.ldap.LDAPSearchResults.getBatchOfResults(LDAPSearchResults.java:338) at com.novell.ldap.LDAPSearchResults.resetVectors(LDAPSearchResults.java:213) at com.novell.ldap.LDAPSearchResults.hasMore(LDAPSearchResults.java:175) at org.ietf.ldap.LDAPSearchResults.hasMore(LDAPSearchResults.java:89) Unfortunately this looks like a fairly low-level architectual issue so I'll have to fall-back to JNDI. But hopefully this will help someone who wants to add this critical functionality in the future. Using the builtin integrity and confidentiality offered by mechanisms like GSSAPI is pretty important IMO. At least it's quite a bit more graceful than using TLS.

1 0

Re: (ITS#6195) Replications with 2.3.43 master to 2.4.16 slave is failing
by quanah＠zimbra.com 07 Jul '09

07 Jul '09

--On Thursday, July 02, 2009 10:49 PM +0000 quanah(a)zimbra.com wrote: > --On Thursday, July 02, 2009 3:40 PM -0700 Quanah Gibson-Mount > <quanah(a)zimbra.com> wrote: Fixed with the CSN normalization commit. --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

Re: (ITS#6194) Patch - Enhancement - provide LDIF support as libldif
by rmeggins＠redhat.com 07 Jul '09

07 Jul '09

Howard Chu wrote: > rmeggins(a)redhat.com wrote: >> Full_Name: Rich Megginson >> Version: CVS HEAD (2.4.16+) >> OS: Fedora >> URL: >> ftp://ftp.openldap.org/incoming/openldap-2.4.16-libldif-20090702.patch >> Submission from: (NULL) (76.113.59.19) >> >> >> This patch allows liblutil to provide the LDIF reading and writing >> functions as >> public APIs in a libldif, and exposes ldif.h to the public API. > > See also ITS#4033; ldif.h only parses LDIF into chunks/records, it's > still up to the caller to turn these records into something > meaningful. In fact libldif used to be a library of its own, but in > RE22 we made the decision to merge it into liblutil because its > functionality is so limited. I don't expect we'll reverse this > decision unless/until it gets enough parsing added to it to make it > more generally useful. I think it's very useful as it is - the mozldap libldif (which is almost identical in functionality to my proposed openldap libldif) is used in several different applications in the 389 directory server suite. What more does it need? Some sort of higher level "entry" and "attribute" abstraction? > >> This patch file is derived from OpenLDAP Software. All of the >> modifications to OpenLDAP Software represented in the following >> patch(es) were developed by Red Hat, Inc.. Red Hat, Inc. has not >> assigned rights and/or interest in this work to any party. I, Richard >> Megginson am authorized by Red Hat, Inc., my employer, to release this >> work under the following terms. >> >> Red Hat, Inc. hereby place the following modifications to OpenLDAP >> Software (and only these modifications) into the public domain. Hence, >> these modifications may be freely used and/or redistributed for any >> purpose with or without attribution and/or other notice. >

1 0

Re: (ITS#6194) Patch - Enhancement - provide LDIF support as libldif
by hyc＠symas.com 07 Jul '09

07 Jul '09

rmeggins(a)redhat.com wrote: > Full_Name: Rich Megginson > Version: CVS HEAD (2.4.16+) > OS: Fedora > URL: ftp://ftp.openldap.org/incoming/openldap-2.4.16-libldif-20090702.patch > Submission from: (NULL) (76.113.59.19) > > > This patch allows liblutil to provide the LDIF reading and writing functions as > public APIs in a libldif, and exposes ldif.h to the public API. See also ITS#4033; ldif.h only parses LDIF into chunks/records, it's still up to the caller to turn these records into something meaningful. In fact libldif used to be a library of its own, but in RE22 we made the decision to merge it into liblutil because its functionality is so limited. I don't expect we'll reverse this decision unless/until it gets enough parsing added to it to make it more generally useful. > This patch file is derived from OpenLDAP Software. All of the > modifications to OpenLDAP Software represented in the following > patch(es) were developed by Red Hat, Inc.. Red Hat, Inc. has not > assigned rights and/or interest in this work to any party. I, Richard > Megginson am authorized by Red Hat, Inc., my employer, to release this > work under the following terms. > > Red Hat, Inc. hereby place the following modifications to OpenLDAP > Software (and only these modifications) into the public domain. Hence, > these modifications may be freely used and/or redistributed for any > purpose with or without attribution and/or other notice. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#5340) REP_ENTRY_MODIFIABLE bug in dynlist
by h.b.furuseth＠usit.uio.no 07 Jul '09

07 Jul '09

I've been trying to factor this MUSTRELEASE/MUSTBEFREED stuff out to some macros. Easier said than done, I ended up with the macro monstrosity below. Do I put that in slap.h? Or a separate slap-response.h file? Generally the idea is that most code which modifies rs->sr_<entry/matched/ref/ctrls> (as opposed to initializing) should use these macros, which obey sr_flag and dispose of the old value if needed. There won't be that many callers actually, mostly overlays. There is overlay code which need to be rearranged a bit or which can't use this, and there is overlay code I don't understand at all and don't dare touch. Oh, and this seems to be wrong for how sr_ctrls are set up, which seems to be allocated differently some places. I'm just including that for reference for now, won't commit it yet in any case. Finally, code which aborts normal processing of some response and instead send an error often needs to be fixed - it often doesn't flush SlapReply data, and the data to be flushed and maybe freeed can be lost because it's overwritten. Generally I think code which modifies sr_type (or in some cases calls a function which will do so) should be responsible for flushing old SlapReply data first. #define REP_ENTRY_MUSTFLUSH (REP_ENTRY_MUSTBEFREED|REP_ENTRY_MUSTRELEASE) #define REP_ENTRY_MASK (REP_ENTRY_MODIFIABLE |REP_ENTRY_MUSTFLUSH) #define REP_MATCHED_MUSTFLUSH REP_MATCHED_MUSTBEFREED #define REP_REF_MUSTFLUSH REP_REF_MUSTBEFREED #define REP_CTRLS_MUSTFLUSH REP_CTRLS_MUSTBEFREED /* Macros updating freeable members of SlapReply, mostly for overlays. */ /* * Ensure rs->sr_entry is modifiable, by duplicating it if necessary. * Obey sr_flags. Set REP_ENTRY_<MODIFIABLE, and MUSTBEFREED if duplicated>. * Do <postmodActions> if the entry was replaced. */ #define RS_MODIFIABLE_ENTRY(op, rs, on, postmodActions) do { \ SlapReply *const rmeRS = (rs); \ if ( rmeRS->sr_flags & REP_ENTRY_MODIFIABLE ) break; \ RS_REPLACE_ENTRY( op, rmeRS, on, entry_dup( rmeRS->sr_entry ), 0 ); \ rmeRS->sr_flags |= REP_ENTRY_MODIFIABLE | REP_ENTRY_MUSTBEFREED; \ postmodActions; \ } while (0) /* Reset rs->sr_<member> if REP_<member>_MUSTBEFREED/MUSTRELEASE says so. */ #define RS_FLUSH_ENTRY(op, rs, on) RS_FLUSH(ENTRY, op, rs, on, 1, {}, {}) #define RS_FLUSH_CTRLS(op, rs) RS_FLUSH(CTRLS, op, rs, -, 1, {}, {}) #define RS_FLUSH_MATCHED( rs) RS_FLUSH(MATCHED, -, rs, -, 1, {}, {}) #define RS_FLUSH_REF( rs) RS_FLUSH(REF, -, rs, -, 1, {}, {}) /* Flush and set rs->sr_<member>. Set REP_<member>_MUSTBEFREED if <isDup>. */ #define RS_REPLACE_ENTRY(op,rs,on,v,isDup) RS_REPLACE(ENTRY,op,rs,on, v,isDup) #define RS_REPLACE_CTRLS(op,rs,v,isDup) RS_REPLACE(CTRLS, op,rs,-, v,isDup) #define RS_REPLACE_MATCHED( rs,v,isDup) RS_REPLACE(MATCHED, -,rs,-, v,isDup) #define RS_REPLACE_REF( rs,v,isDup) RS_REPLACE(REF, -,rs,-, v,isDup) /* Helpers for the above */ #define SR_ENTRY sr_entry #define SR_CTRLS sr_ctrls #define SR_MATCHED sr_matched #define SR_REF sr_ref typedef Entry *SR_ENTRY_t; typedef LDAPControl **SR_CTRLS_t; typedef const char *SR_MATCHED_t; typedef BerVarray SR_REF_t; #define RS_FREE_ENTRY(op, rs, on)\ if ((rs)->sr_flags & REP_ENTRY_MUSTBEFREED) { \ assert (!(rs)->sr_flags & REP_ENTRY_MUSTRELEASE); \ entry_free((rs)->sr_entry); \ } \ else if (on == NULL) be_entry_release_rw(op, (rs)->sr_entry, 0);\ else overlay_entry_release_ov(op, (rs)->sr_entry, 0, on) #define RS_FREE_CTRLS(op, rs, on) slap_free_ctrls( op, (rs)->sr_ctrls ) #define RS_FREE_MATCHED(op, rs, on) ch_free( (char *) (rs)->sr_matched ) #define RS_FREE_REF(op, rs, on) ber_bvarray_free( (rs)->sr_ref ) #define RS_FLUSH(MEMBER, op, rs, on, reset, preActions, postActions) do { \ SlapReply *const rfRS = (rs); \ preActions; \ if ( rfRS->sr_flags & REP_##MEMBER##_MUSTFLUSH ) { \ if ( rfRS->SR_##MEMBER ) { \ RS_FREE_##MEMBER( op, rfRS, on ); \ if ( reset ) rfRS->SR_##MEMBER = NULL; \ } \ } \ rfRS->sr_flags &= ~REP_##MEMBER##_MASK; \ postActions; \ } while (0) #define RS_REPLACE(MEMBER, op, rs, on, val, isDup) \ RS_FLUSH( MEMBER, op, rs, on, 0, \ SR_##MEMBER##_t const rfNewval = (val), \ if ( isDup ) rfRS->sr_flags |= REP_##MEMBER##_MUSTBEFREED; \ rfRS->SR_##MEMBER = rfNewval ) /* Warn if something re#defines the following, which would break RS_REPLACE */ #define ENTRY ENTRY #define CTRLS CTRLS #define MATCHED MATCHED #define REF REF

1 0

Re: (ITS#5892) slapadd doesn't fail on invalid include statements
by hyc＠symas.com 07 Jul '09

07 Jul '09

aredridel(a)nbtsc.org wrote: > Full_Name: Aria Stewart > Version: 2.4.13 > OS: PLD Linux (Th) > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (2001:470:8935:7:21b:fcff:fe77:1757) > > > An ldif file with include statements in the form of: > > include: /filename > > rather than > > include: file:///filename > > silently fail. Slapadd stops at that point, but does not print an error message. Oops, the library has provided an error message for this instance since 2.4.5beta (September 2007) but slapadd wasn't enabling the display of that message by default. This is now fixed in HEAD. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#6101) Uninitialized SlapReply components
by h.b.furuseth＠usit.uio.no 07 Jul '09

07 Jul '09

I wrote: > We need to check > grep -nH -e 'SlapReply[<tab> ]*[a-zA-Z]' *.c */*.c > for other cases. Not for this ITS. Just moved sru_search first in the union. It's not always clear to me which code is responsible for initializing SlapReply anyway. Fixed two other simple cases: - result.c:send_ldap_disconnect() should reset rs->sr_rspdata. - sasl.c: should (re)set rs->sr_sasldata before doing send_ldap_sasl(). -- Hallvard

1 0

Re: (ITS#5836) hung writers
by quanah＠zimbra.com 07 Jul '09

07 Jul '09

--On Tuesday, July 07, 2009 4:41 PM +0000 quanah(a)zimbra.com wrote: > Also, please confirm what BDB version you are using, including if it has > all patches applied and compile options for BDB, and which backend you > are using. Hi Jorge, Please also confirm that you have writetimeout configured in your slapd configuration. --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

Re: (ITS#5836) hung writers
by quanah＠zimbra.com 07 Jul '09

07 Jul '09

--On Tuesday, July 07, 2009 12:51 PM +0000 jorge.perez(a)adaptia.net wrote: Hello, > > The problem still reproduces. Threads hung on the same > ptread_cond_wait.call. This time the assert has not failed. > > Thread 18 (Thread 1390426432 (LWP 17780)): > ---Type <return> to continue, or q <return> to quit--- ># 0=C2=A0 0x00002b5a8515b1c6 in pthread_cond_wait@@GLIBC_2.3.2 () from > /lib64/libpthread.so.0 ># 1=C2=A0 0x00000000004525a4 in send_ldap_ber (op=3D<value optimized out>, > ber=3D0x52e01150) at result.c:156 ># 2=C2=A0 0x000000000045449e in slap_send_search_entry ># (op=3D0x2b5a849138f4, > rs=3D0x52e02da0) at result.c:1259 > Hi Jorge, We're going to need more information from you than you are providing. Please make sure you: (a) build openldap without optimization (-O0) (b) are gdbing the process, not a core file (c) Send a full backtrace of everything: thr apply all bt full Also, please confirm what BDB version you are using, including if it has all patches applied and compile options for BDB, and which backend you are using. Thanks! --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

Re: (ITS#6197) ldap_err2string()/ldap_perror() strangeness
by h.b.furuseth＠usit.uio.no 07 Jul '09

07 Jul '09

Also ldap_err2string(), unlike ldap_perror(), does not pass messages from recognized result codes through _() - i.e. the optional gettext(). I'll fix that too. -- Hallvard

1 0

Re: (ITS#6164) send_ldap_ber() spins holding c_write1_mutex
by h.b.furuseth＠usit.uio.no 07 Jul '09

07 Jul '09

hyc(a)symas.com writes: > This code has changed in HEAD. Closing this ITS. Hm? The unlock-lock is still there. If it still needs to let through another thread to c_write1_mutex I don't see how the connection_valid() test in front of it helps. -- Hallvard

1 0

Re: (ITS#5836) hung writers
by jorge.perez＠adaptia.net 07 Jul '09

07 Jul '09

On Mon, Jul 6, 2009 at 9:32 PM, Quanah Gibson-Mount <quanah(a)zimbra.com> wro= te: > > --On Monday, July 06, 2009 6:07 PM +0000 jorge.perez(a)adaptia.net wrote: > >> >> With the debug symbols there's an assert that fails in slapd_clr_write >> method, ("assert( SLAP_SOCK_IS_ACTIVE( s ))"): > > Thanks. =C2=A0The hang should really be fixed now. :P =C2=A0Please re-syn= c RE24 and test again. =C2=A0Thank you! Hello, The problem still reproduces. Threads hung on the same ptread_cond_wait.call. This time the assert has not failed. Thread 18 (Thread 1390426432 (LWP 17780)): ---Type <return> to continue, or q <return> to quit--- #0=C2=A0 0x00002b5a8515b1c6 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1=C2=A0 0x00000000004525a4 in send_ldap_ber (op=3D<value optimized out>, ber=3D0x52e01150) at result.c:156 #2=C2=A0 0x000000000045449e in slap_send_search_entry (op=3D0x2b5a849138f4, rs=3D0x52e02da0) at result.c:1259 Regards

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs