openldap-bugs

openldap-bugs@openldap.org

1 participants
20967 discussions

(ITS#7717) Sudden memory increase leading to Master LDAP crash
by marcon.bruno＠free.fr 03 Oct '13

03 Oct '13

Full_Name: Marcon Bruno Version: 2.4.36 OS: Ubuntu URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (185.23.92.11) Hi. We have deployed OpenLDAP on our system, using the following architecture : One primary master One failover master (mirror mode) 4 replicas It works very well until the master LDAP receive two many writes /seconds. With 40 writes/seconds, the master LDAP work well about several minutes, then the memory process (slapd) started to increase very quickly, about 20Mo/s ! When all memory and swap is consumed, the OS kill Slapd. Notes : - Even in disabling all overlays except syncprov, the problem persists - If disabling only syncprov, there is no more problem - If all slaves and mirror master are stopped, there is no more problem This problem is very annoying. We are looking to a Microsoft solution, even if we have already deployed our solution but obviously, we would like to keep an OpenLDAP solution. The problem is very easy to replay : one master and one slave are sufficient. Then run 9 windows with a script adding an entry and modifying an entry.

1 0

Re: (ITS#7715) SIGBUS when mdb is configured with writemap
by hyc＠symas.com 03 Oct '13

03 Oct '13

Željko Nejašmić wrote: > On Wed, Oct 2, 2013 at 6:43 PM, <hyc(a)symas.com <mailto:hyc@symas.com>> wrote: > Without any other clues, this feels like ASLR is messing with us but that's > just a wild guess. I can no longer reproduce the SIGBUS in slapd regardless of > compile options, while ldclt itself keeps dying. If you can find some more > reliable way to reproduce the issue that would help. Perhaps using the client > in test060. > > > I ran the whole test suite for mdb, and as far as I can see, every test > returned OK. > > Found a new way to reproduce the SIGBUS using ldapadd on Ubuntu 12.04 firing > to openldap on RH 6.3: > ldapadd -h 172.17.101.150 -p 389 -D "cn=admin,dc=test" -w test -f test.ldif -- > ldif file is the same as the previous ldclt command. Doubt it matters, but the > ldif file is 1M adds. Can you provide your slapd config and this LDIF? > > On the RH box: > - compiled openldap with -g -O0 and previously used flags > gdb `find /root/openldap/ -type d -printf '-d %p '` --args > /opt/openldap/libexec/slapd -h "ldap:/// ldapi:///" -F > /opt/openldap/etc/openldap/slapd.d -g openldap -u openldap -d 0 > > gdb output: > bt -- http://hastebin.com/hefikekaxi.sh > bt 10 full -- http://hastebin.com/vudocosuka.sh Can you also provide the gdb output for print *env > > I am begging to doubt that the problem could be on my end as the bt seems to > point to schema problems (although, I haven't analyzed it in great detail yet). > > > Zeljko -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7715) SIGBUS when mdb is configured with writemap
by nejasmicz＠gmail.com 03 Oct '13

03 Oct '13

--001a11c253c0f33b8604e7d59850 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Wed, Oct 2, 2013 at 6:43 PM, <hyc(a)symas.com> wrote: > hyc(a)symas.com wrote: > > =C5=BDeljko Neja=C5=A1mi=C4=87 wrote: > >> Here you go http://hastebin.com/fukecejuje.tex > > > > Interestingly enough, I got the same result as you on an initial > compile/run > > of slapd. Unfortunately, with optimization, the backtrace wasn't all th= at > > useful. Recompiling back-mdb with just -g, no optimization, gets a > different > > result though - slapd is fine, and ldclt dies with a heap corruption or > > double-free. > > And now I cannot reproduce the original SIGBUS at all. But still getting > various crashes in ldclt. > > ldclt -h localhost -p 9011 -D cn=3Dmanager,dc=3Dexample,dc=3Dcom -w secre= t -b > ou=3Dpeople,dc=3Dexample,dc=3Dcom -e > object=3Dxx.txt,rdn=3D'uid:[A=3DINCRNNOLOOP(200000;999999;6)]' -e > add,commoncounter > -I 68 > ldclt version 4.23 > ldclt[30207]: Starting at Wed Oct 2 09:39:10 2013 > > *** glibc detected *** ldclt: double free or corruption (fasttop): > 0x00007fc2180021e0 *** > =3D=3D=3D=3D=3D=3D=3D Backtrace: =3D=3D=3D=3D=3D=3D=3D=3D=3D > /lib/x86_64-linux-gnu/libc.so.6(+0x7eb96)[0x7fc223ff9b96] > /usr/lib/x86_64-linux-gnu/libnspr4.so(+0x142d1)[0x7fc224f0b2d1] > /usr/lib/x86_64-linux-gnu/libnspr4.so(+0x1ae74)[0x7fc224f11e74] > /usr/lib/x86_64-linux-gnu/libnspr4.so(PR_Malloc+0x49)[0x7fc224f0d0b9] > /usr/lib/x86_64-linux-gnu/libnspr4.so(+0x129f4)[0x7fc224f099f4] > /usr/lib/x86_64-linux-gnu/libnspr4.so(+0x12068)[0x7fc224f09068] > /usr/lib/x86_64-linux-gnu/libnspr4.so(PR_vsmprintf+0x38)[0x7fc224f09b28] > /usr/lib/x86_64-linux-gnu/libnspr4.so(PR_smprintf+0x8c)[0x7fc224f09bec] > ldclt(+0x663a)[0x7fc22556663a] > ldclt(+0x74a4)[0x7fc2255674a4] > ldclt(+0x9d48)[0x7fc225569d48] > ldclt(threadMain+0x329)[0x7fc2255735d9] > /lib/x86_64-linux-gnu/libpthread.so.0(+0x7e9a)[0x7fc224341e9a] > /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7fc22406ecbd] > > Without any other clues, this feels like ASLR is messing with us but that= 's > just a wild guess. I can no longer reproduce the SIGBUS in slapd > regardless of > compile options, while ldclt itself keeps dying. If you can find some mor= e > reliable way to reproduce the issue that would help. Perhaps using the > client > in test060. I ran the whole test suite for mdb, and as far as I can see, every test returned OK. Found a new way to reproduce the SIGBUS using ldapadd on Ubuntu 12.04 firing to openldap on RH 6.3: ldapadd -h 172.17.101.150 -p 389 -D "cn=3Dadmin,dc=3Dtest" -w test -f test.= ldif -- ldif file is the same as the previous ldclt command. Doubt it matters, but the ldif file is 1M adds. On the RH box: - compiled openldap with -g -O0 and previously used flags gdb `find /root/openldap/ -type d -printf '-d %p '` --args /opt/openldap/libexec/slapd -h "ldap:/// ldapi:///" -F /opt/openldap/etc/openldap/slapd.d -g openldap -u openldap -d 0 gdb output: bt -- http://hastebin.com/hefikekaxi.sh bt 10 full -- http://hastebin.com/vudocosuka.sh I am begging to doubt that the problem could be on my end as the bt seems to point to schema problems (although, I haven't analyzed it in great detail yet). Zeljko --001a11c253c0f33b8604e7d59850 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><br><div class=3D"gmail_extra"><br><br><div class=3D"gmail= _quote">On Wed, Oct 2, 2013 at 6:43 PM, <span dir=3D"ltr"><<a href=3D"m= ailto:hyc@symas.com" target=3D"_blank">hyc(a)symas.com</a>></span> wrote:<= br><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;bord= er-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:soli= d;padding-left:1ex"> <div><a href=3D"mailto:hyc@symas.com" target=3D"_blank">hyc(a)symas.com</a> w= rote:<br> > =C5=BDeljko Neja=C5=A1mi=C4=87 wrote:<br> >> Here you go <a href=3D"http://hastebin.com/fukecejuje.tex" target= =3D"_blank">http://hastebin.com/fukecejuje.tex</a><br> ><br> > Interestingly enough, I got the same result as you on an initial compi= le/run<br> > of slapd. Unfortunately, with optimization, the backtrace wasn't a= ll that<br> > useful. Recompiling back-mdb with just -g, no optimization, gets a dif= ferent<br> > result though - slapd is fine, and ldclt dies with a heap corruption o= r<br> > double-free.<br> <br> </div>And now I cannot reproduce the original SIGBUS at all. But still gett= ing<br> various crashes in ldclt.<br> <div><br> ldclt -h localhost -p 9011 -D cn=3Dmanager,dc=3Dexample,dc=3Dcom -w secret = -b<br> ou=3Dpeople,dc=3Dexample,dc=3Dcom -e<br> object=3Dxx.txt,rdn=3D'uid:[A=3DINCRNNOLOOP(200000;999999;6)]' -e a= dd,commoncounter<br> -I 68<br> ldclt version 4.23<br> </div>ldclt[30207]: Starting at Wed Oct =C2=A02 09:39:10 2013<br> <div><br> *** glibc detected *** ldclt: double free or corruption (fasttop):<br> </div>0x00007fc2180021e0 ***<br> =3D=3D=3D=3D=3D=3D=3D Backtrace: =3D=3D=3D=3D=3D=3D=3D=3D=3D<br> /lib/x86_64-linux-gnu/libc.so.6(+0x7eb96)[0x7fc223ff9b96]<br> /usr/lib/x86_64-linux-gnu/libnspr4.so(+0x142d1)[0x7fc224f0b2d1]<br> /usr/lib/x86_64-linux-gnu/libnspr4.so(+0x1ae74)[0x7fc224f11e74]<br> /usr/lib/x86_64-linux-gnu/libnspr4.so(PR_Malloc+0x49)[0x7fc224f0d0b9]<br> /usr/lib/x86_64-linux-gnu/libnspr4.so(+0x129f4)[0x7fc224f099f4]<br> /usr/lib/x86_64-linux-gnu/libnspr4.so(+0x12068)[0x7fc224f09068]<br> /usr/lib/x86_64-linux-gnu/libnspr4.so(PR_vsmprintf+0x38)[0x7fc224f09b28]<br= > /usr/lib/x86_64-linux-gnu/libnspr4.so(PR_smprintf+0x8c)[0x7fc224f09bec]<br> ldclt(+0x663a)[0x7fc22556663a]<br> ldclt(+0x74a4)[0x7fc2255674a4]<br> ldclt(+0x9d48)[0x7fc225569d48]<br> ldclt(threadMain+0x329)[0x7fc2255735d9]<br> /lib/x86_64-linux-gnu/libpthread.so.0(+0x7e9a)[0x7fc224341e9a]<br> /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7fc22406ecbd]<br> <br> Without any other clues, this feels like ASLR is messing with us but that&#= 39;s<br> just a wild guess. I can no longer reproduce the SIGBUS in slapd regardless= of<br> compile options, while ldclt itself keeps dying. If you can find some more<= br> reliable way to reproduce the issue that would help. Perhaps using the clie= nt<br> in test060.</blockquote><div><br></div><div>I ran the whole test suite for = mdb, and as far as I can see, every test returned OK.</div><div><br></div><= div>Found a new way to reproduce the SIGBUS using ldapadd on Ubuntu 12.04 f= iring to openldap on RH 6.3:<br> </div><div>ldapadd -h 172.17.101.150 -p 389 -D "cn=3Dadmin,dc=3Dtest&q= uot; -w test -f test.ldif -- ldif file is the same as the previous ldclt co= mmand. Doubt it matters, but the ldif file is 1M adds.</div></div><br></div= > <div class=3D"gmail_extra">On the RH box:</div><div class=3D"gmail_extra">-= compiled openldap with -g -O0 and previously used flags</div><div class=3D= "gmail_extra"> gdb `find /root/openldap/ -type d -printf '-d %p '` --args /opt/ope= nldap/libexec/slapd -h "ldap:/// ldapi:///" -F /opt/openldap/etc/= openldap/slapd.d -g openldap -u openldap -d 0<br></div><div class=3D"gmail_= extra"> <br></div><div class=3D"gmail_extra">gdb output:</div><div class=3D"gmail_e= xtra">bt --=C2=A0<a href=3D"http://hastebin.com/hefikekaxi.sh" target=3D"_b= lank">http://hastebin.com/hefikekaxi.sh</a></div><div class=3D"gmail_extra"= >bt 10 full --=C2=A0<a href=3D"http://hastebin.com/vudocosuka.sh" target=3D= "_blank">http://hastebin.com/vudocosuka.sh</a></div> <div class=3D"gmail_extra"><br></div><div class=3D"gmail_extra">I am beggin= g to doubt that the problem could be on my end as the bt seems to point to = schema problems (although, I haven't analyzed it in great detail yet).<= /div> <div class=3D"gmail_extra"><br></div><div class=3D"gmail_extra"><br></div><= div class=3D"gmail_extra">Zeljko</div> </div> --001a11c253c0f33b8604e7d59850--

1 0

Re: (ITS#7715) SIGBUS when mdb is configured with writemap
by hyc＠symas.com 02 Oct '13

02 Oct '13

hyc(a)symas.com wrote: > Željko Nejašmić wrote: >> Here you go http://hastebin.com/fukecejuje.tex > > Interestingly enough, I got the same result as you on an initial compile/run > of slapd. Unfortunately, with optimization, the backtrace wasn't all that > useful. Recompiling back-mdb with just -g, no optimization, gets a different > result though - slapd is fine, and ldclt dies with a heap corruption or > double-free. And now I cannot reproduce the original SIGBUS at all. But still getting various crashes in ldclt. ldclt -h localhost -p 9011 -D cn=manager,dc=example,dc=com -w secret -b ou=people,dc=example,dc=com -e object=xx.txt,rdn='uid:[A=INCRNNOLOOP(200000;999999;6)]' -e add,commoncounter -I 68 ldclt version 4.23 ldclt[30207]: Starting at Wed Oct 2 09:39:10 2013 *** glibc detected *** ldclt: double free or corruption (fasttop): 0x00007fc2180021e0 *** ======= Backtrace: ========= /lib/x86_64-linux-gnu/libc.so.6(+0x7eb96)[0x7fc223ff9b96] /usr/lib/x86_64-linux-gnu/libnspr4.so(+0x142d1)[0x7fc224f0b2d1] /usr/lib/x86_64-linux-gnu/libnspr4.so(+0x1ae74)[0x7fc224f11e74] /usr/lib/x86_64-linux-gnu/libnspr4.so(PR_Malloc+0x49)[0x7fc224f0d0b9] /usr/lib/x86_64-linux-gnu/libnspr4.so(+0x129f4)[0x7fc224f099f4] /usr/lib/x86_64-linux-gnu/libnspr4.so(+0x12068)[0x7fc224f09068] /usr/lib/x86_64-linux-gnu/libnspr4.so(PR_vsmprintf+0x38)[0x7fc224f09b28] /usr/lib/x86_64-linux-gnu/libnspr4.so(PR_smprintf+0x8c)[0x7fc224f09bec] ldclt(+0x663a)[0x7fc22556663a] ldclt(+0x74a4)[0x7fc2255674a4] ldclt(+0x9d48)[0x7fc225569d48] ldclt(threadMain+0x329)[0x7fc2255735d9] /lib/x86_64-linux-gnu/libpthread.so.0(+0x7e9a)[0x7fc224341e9a] /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7fc22406ecbd] Without any other clues, this feels like ASLR is messing with us but that's just a wild guess. I can no longer reproduce the SIGBUS in slapd regardless of compile options, while ldclt itself keeps dying. If you can find some more reliable way to reproduce the issue that would help. Perhaps using the client in test060. > All subsequent runs give the same result. Still looking into this; getting > away from the Ubuntu bundled libldap will probably help. >> >> >> Zeljko >> >> >> On Wed, Oct 2, 2013 at 11:34 AM, <hyc(a)symas.com <mailto:hyc@symas.com>> wrote: >> >> nejasmicz(a)gmail.com <mailto:nejasmicz@gmail.com> wrote: >> > Full_Name: Željko Nejašmić >> > Version: latest git pull of OPENLDAP_REL_ENG_2_4 >> > OS: RedHat 6.3 >> > URL: >> > Submission from: (NULL) (213.147.123.33) >> > >> >> > Hardware underneath all of that is: >> > 1) HP ProLiant BL460c Gen8, dual Xeon E5-2658 with attached >> storage blade >> > D2200sb with SSD raid, 256GB RAM -- RedHat 6.3 tests >> > 2) Intel server blade S2400BB, dual Xeon E5-2403, 48GB RAM -- >> Ubuntu 12.04 >> > tests >> > >> > If anything more is required to assist you in troubleshooting, please >> let me >> > know. >> >> Can you also post the gdb output for "bt 5 full" ? >> > >> > >> > Zeljko >> > >> > >> -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7716) slapd crashes after search immediately followed by (abandon+) unbind
by michael＠stroeder.com 02 Oct '13

02 Oct '13

michael.vishchers(a)7p-group.com wrote: > Full_Name: Michael Vishchers > Version: 2.4.23 > OS: Red Hat Enterprise Linux Server release 6.2 Did you check whether this problem still remains with the recent release 2.4.36? (I vaguely remember issues like this being fixed after 2.4.23 but I'm not sure.) Ciao, Michael.

1 0

(ITS#7716) slapd crashes after search immediately followed by (abandon+) unbind
by michael.vishchers＠7p-group.com 02 Oct '13

02 Oct '13

Full_Name: Michael Vishchers Version: 2.4.23 OS: Red Hat Enterprise Linux Server release 6.2 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (178.15.66.50) slapd, running as a proxy to rewrite incoming connections based on user dn for later routing to different back ends, dies sporadically after receiving a (network delayed) search request that is "immediately" followed by an (optional) abandon request and an unbind request. We suspect that the abandon or unbind code tries to clean up data structures that belong to a not yet completely initialized search operation. The problem can unfortunately not easily be reproduced. Last time we had to wait at least two weeks before it appeared. It may be a timing problem between two or more threads. This is the stacktrace, core and other files could be provided if necessary. Program terminated with signal 11, Segmentation fault. #0 0x00007f299c3e71bc in ?? () #0 0x00007f299c3e71bc in ?? () No symbol table info available. #1 0x00007f2999bbd983 in rwm_op_rollback (op=0x7f2984002190, rs=<value optimized out>, ros=0x7f298c003570) at ../../../../servers/slapd/overlays/rwm.c:107 __PRETTY_FUNCTION__ = "rwm_op_rollback" #2 0x00007f2999bbe988 in rwm_op_search (op=0x7f2984002190, rs=0x7f2995bafaa0) at ../../../../servers/slapd/overlays/rwm.c:984 on = 0x7f299fb95210 rwmap = 0x7f299fb94f70 rc = <value optimized out> dc = {rwmap = 0x7f2984002190, conn = 0x7f298c003468, ctx = 0x12 <Address 0x12 out of bounds>, rs = 0x7f299e7963b0} fstr = {bv_len = 0, bv_val = 0x0} f = 0x0 an = 0x0 text = <value optimized out> roc = 0x7f298c003550 #3 0x00007f299e7fe02a in overlay_op_walk (op=0x7f2984002190, rs=0x7f2995bafaa0, which=op_search, oi=0x7f299fb95030, on=0x7f299fb95210) at ../../../servers/slapd/backover.c:659 func = 0x7f299fb95268 rc = 32768 #4 0x00007f299e8d29a1 in slapi_op_func (op=0x7f2984002190, rs=0x7f2995bafaa0) at ../../../../servers/slapd/slapi/slapi_overlay.c:647 pb = 0x7f298c1051b0 which = op_search opinfo = <value optimized out> rc = <value optimized out> oi = <value optimized out> on = <value optimized out> cb = {sc_next = 0x7f2995bae7e0, sc_response = 0x7f299e8d1fc0 <slapi_over_response>, sc_cleanup = 0x7f299e8d1ed0 <slapi_over_cleanup>, sc_private = 0x7f298c1051b0} internal_op = 0 preop_type = <value optimized out> postop_type = 503 be = 0x7f2995bae800 #5 0x00007f299e7fe02a in overlay_op_walk (op=0x7f2984002190, rs=0x7f2995bafaa0, which=op_search, oi=0x7f299fb95030, on=0x7f299fb9e8c0) at ../../../servers/slapd/backover.c:659 func = 0x7f299fb9e918 rc = 32768 #6 0x00007f299e7feb6b in over_op_func (op=0x7f2984002190, rs=<value optimized out>, which=<value optimized out>) at ../../../servers/slapd/backover.c:721 oi = <value optimized out> on = <value optimized out> be = 0x7f299fb940b0 db = {bd_info = 0x7f299fb95210, bd_self = 0x7f299fb940b0, be_ctrls = "\000", '\001' <repeats 17 times>, '\000' <repeats 14 times>, "\001", be_flags = 257, be_restrictops = 0, be_requires = 5, be_ssf_set = {sss_ssf = 0, sss_transport = 0, sss_tls = 0, sss_sasl = 0, sss_update_ssf = 0, sss_update_transport = 0, sss_update_tls = 0, sss_update_sasl = 0, sss_simple_bind = 0}, be_suffix = 0x7f299fb94ed0, be_nsuffix = 0x7f299fb94f00, be_schemadn = {bv_len = 0, bv_val = 0x0}, be_schemandn = {bv_len = 0, bv_val = 0x0}, be_rootdn = {bv_len = 0, bv_val = 0x0}, be_rootndn = {bv_len = 0, bv_val = 0x0}, be_rootpw = {bv_len = 0, bv_val = 0x0}, be_max_deref_depth = 15, be_def_limit = {lms_t_soft = 3600, lms_t_hard = 0, lms_s_soft = 500, lms_s_hard = 0, lms_s_unchecked = -1, lms_s_pr = 0, lms_s_pr_hide = 0, lms_s_pr_total = 0}, be_limits = 0x0, be_acl = 0x0, be_dfltaccess = ACL_READ, be_update_ndn = {bv_len = 0, bv_val = 0x0}, be_update_refs = 0x0, be_pending_csn_list = 0x7f299fc738f0, be_pcl_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, be_syncinfo = 0x0, be_pb = 0x7f299fb9eaa0, be_cf_ocs = 0x7f299eb6da00, be_private = 0x7f299fb94240, be_next = {stqe_next = 0x0}} cb = {sc_next = 0x0, sc_response = 0x7f299e7fdd40 <over_back_response>, sc_cleanup = 0, sc_private = 0x7f299fb95030} sc = <value optimized out> rc = 32768 __PRETTY_FUNCTION__ = "over_op_func" #7 0x00007f299e794999 in fe_op_search (op=0x7f2984002190, rs=0x7f2995bafaa0) at ../../../servers/slapd/search.c:366 bd = 0x7f299eb72760 #8 0x00007f299e795177 in do_search (op=0x7f2984002190, rs=<value optimized out>) at ../../../servers/slapd/search.c:217 base = {bv_len = 55, bv_val = 0x7f298c11fef9 "vfsid=491722472236,ou=subscriber,ou=mmo,c=de,o=vodafone"} siz = 0 off = 0 i = <value optimized out> #9 0x00007f299e7920f9 in connection_operation (ctx=0x7f2995bafb70, arg_v=0x7f2984002190) at ../../../servers/slapd/connection.c:1109 rc = 80 cancel = <value optimized out> op = 0x7f2984002190 rs = {sr_type = REP_RESULT, sr_tag = 0, sr_msgid = 0, sr_err = 80, sr_matched = 0x0, sr_text = 0x7f2999bc4122 "Rewrite error", sr_ref = 0x0, sr_ctrls = 0x0, sr_un = {sru_search = {r_entry = 0x0, r_attr_flags = 0, r_operational_attrs = 0x0, r_attrs = 0x0, r_nentries = 0, r_v2ref = 0x0}, sru_sasl = {r_sasldata = 0x0}, sru_extended = {r_rspoid = 0x0, r_rspdata = 0x0}}, sr_flags = 0} tag = 99 opidx = SLAP_OP_SEARCH conn = 0x7f2996db74d0 memctx = 0x7f298c002820 memctx_null = 0x0 memsiz = 1048576 __PRETTY_FUNCTION__ = "connection_operation" #10 0x00007f299e892678 in ldap_int_thread_pool_wrapper (xpool=0x7f299fae9ae0) at ../../../libraries/libldap_r/tpool.c:685 pool = 0x7f299fae9ae0 task = 0x7f2988000a20 work_list = <value optimized out> ctx = {ltu_id = 139816582448896, ltu_key = {{ltk_key = 0x7f299e790d50, ltk_data = 0x7f298c002d40, ltk_free = 0x7f299e790e30 <conn_counter_destroy>}, {ltk_key = 0x7f299e7eaf70, ltk_data = 0x7f298c002820, ltk_free = 0x7f299e7eae50 <slap_sl_mem_destroy>}, {ltk_key = 0x7f299e7a6b70, ltk_data = 0x0, ltk_free = 0x7f299e7a6940 <slap_op_q_destroy>}, {ltk_key = 0x0, ltk_data = 0x0, ltk_free = 0} <repeats 29 times>}} kctx = <value optimized out> keyslot = <value optimized out> hash = <value optimized out> __PRETTY_FUNCTION__ = "ldap_int_thread_pool_wrapper" #11 0x00007f299c91e7f1 in ?? () No symbol table info available. #12 0x00007f2995bb0700 in ?? () No symbol table info available. #13 0x0000000000000000 in ?? () No symbol table info available.

1 0

Re: (ITS#7715) SIGBUS when mdb is configured with writemap
by hyc＠symas.com 02 Oct '13

02 Oct '13

Željko Nejašmić wrote: > Here you go http://hastebin.com/fukecejuje.tex Interestingly enough, I got the same result as you on an initial compile/run of slapd. Unfortunately, with optimization, the backtrace wasn't all that useful. Recompiling back-mdb with just -g, no optimization, gets a different result though - slapd is fine, and ldclt dies with a heap corruption or double-free. ldclt -h localhost -p 9011 -D cn=manager,dc=example,dc=com -w secret -b ou=people,dc=example,dc=com -e object=xx.txt,rdn='uid:[A=INCRNNOLOOP(200000;999999;6)]' -e add,commoncounter -I 68 ldclt version 4.23 ldclt[10503]: Starting at Wed Oct 2 04:13:03 2013 *** glibc detected *** ldclt: double free or corruption (fasttop): 0x00007fa448003270 *** ======= Backtrace: ========= /lib/x86_64-linux-gnu/libc.so.6(+0x7eb96)[0x7fa463f8cb96] /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2(ldap_pvt_tls_set_option+0x1eb)[0x7fa46472e06b] /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2(ldap_int_tls_config+0x54)[0x7fa46472e234] /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2(+0x2b8b7)[0x7fa4647238b7] /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2(ldap_int_initialize+0x104)[0x7fa464723e84] /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2(ldap_create+0x29)[0x7fa4647074f9] /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2(ldap_initialize+0x2f)[0x7fa464707a7f] ldclt(+0x664a)[0x7fa4656fb64a] ldclt(+0x74a4)[0x7fa4656fc4a4] ldclt(+0x9d48)[0x7fa4656fed48] ldclt(threadMain+0x329)[0x7fa4657085d9] /lib/x86_64-linux-gnu/libpthread.so.0(+0x7e9a)[0x7fa4642d4e9a] /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7fa464001cbd] All subsequent runs give the same result. Still looking into this; getting away from the Ubuntu bundled libldap will probably help. > > > Zeljko > > > On Wed, Oct 2, 2013 at 11:34 AM, <hyc(a)symas.com <mailto:hyc@symas.com>> wrote: > > nejasmicz(a)gmail.com <mailto:nejasmicz@gmail.com> wrote: > > Full_Name: Željko Nejašmić > > Version: latest git pull of OPENLDAP_REL_ENG_2_4 > > OS: RedHat 6.3 > > URL: > > Submission from: (NULL) (213.147.123.33) > > > > > Hardware underneath all of that is: > > 1) HP ProLiant BL460c Gen8, dual Xeon E5-2658 with attached > storage blade > > D2200sb with SSD raid, 256GB RAM -- RedHat 6.3 tests > > 2) Intel server blade S2400BB, dual Xeon E5-2403, 48GB RAM -- > Ubuntu 12.04 > > tests > > > > If anything more is required to assist you in troubleshooting, please > let me > > know. > > Can you also post the gdb output for "bt 5 full" ? > > > > > > Zeljko > > > > > > > -- > -- Howard Chu > CTO, Symas Corp. http://www.symas.com > Director, Highland Sun http://highlandsun.com/hyc/ > Chief Architect, OpenLDAP http://www.openldap.org/project/ > > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7715) SIGBUS when mdb is configured with writemap
by nejasmicz＠gmail.com 02 Oct '13

02 Oct '13

--047d7b67749a03fb9b04e7c010b4 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Here you go http://hastebin.com/fukecejuje.tex Zeljko On Wed, Oct 2, 2013 at 11:34 AM, <hyc(a)symas.com> wrote: > nejasmicz(a)gmail.com wrote: > > Full_Name: =C5=BDeljko Neja=C5=A1mić > > Version: latest git pull of OPENLDAP_REL_ENG_2_4 > > OS: RedHat 6.3 > > URL: > > Submission from: (NULL) (213.147.123.33) > > > > > Hardware underneath all of that is: > > 1) HP ProLiant BL460c Gen8, dual Xeon E5-2658 with attached storag= e > blade > > D2200sb with SSD raid, 256GB RAM -- RedHat 6.3 tests > > 2) Intel server blade S2400BB, dual Xeon E5-2403, 48GB RAM -- > Ubuntu 12.04 > > tests > > > > If anything more is required to assist you in troubleshooting, please > let me > > know. > > Can you also post the gdb output for "bt 5 full" ? > > > > > > Zeljko > > > > > > > -- > -- Howard Chu > CTO, Symas Corp. http://www.symas.com > Director, Highland Sun http://highlandsun.com/hyc/ > Chief Architect, OpenLDAP http://www.openldap.org/project/ > > > --047d7b67749a03fb9b04e7c010b4 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div>Here you go=C2=A0<a href=3D"http://hastebin.com/fukec= ejuje.tex">http://hastebin.com/fukecejuje.tex</a></div><div><br></div><div>= <br></div><div>Zeljko</div></div><div class=3D"gmail_extra"><br><br><div cl= ass=3D"gmail_quote"> On Wed, Oct 2, 2013 at 11:34 AM, <span dir=3D"ltr"><<a href=3D"mailto:h= yc(a)symas.com" target=3D"_blank">hyc(a)symas.com</a>></span> wrote:<br><blo= ckquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #c= cc solid;padding-left:1ex"> <div class=3D"HOEnZb"><div class=3D"h5"><a href=3D"mailto:nejasmicz@gmail.c= om">nejasmicz(a)gmail.com</a> wrote:<br> > Full_Name: =C5=BDeljko Neja=C5=A1mi&#263;<br> > Version: latest git pull of OPENLDAP_REL_ENG_2_4<br> > OS: RedHat 6.3<br> > URL:<br> > Submission from: (NULL) (213.147.123.33)<br> ><br> <br> > Hardware underneath all of that is:<br> > =C2=A0 =C2=A0 =C2=A01) HP ProLiant BL460c Gen8, dual Xeon E5-2658 with= attached storage blade<br> > D2200sb with SSD raid, 256GB RAM -- RedHat 6.3 tests<br> > =C2=A0 =C2=A0 =C2=A02) Intel server blade S2400BB, dual Xeon E5-2403, = 48GB RAM -- Ubuntu 12.04<br> > tests<br> ><br> > If anything more is required to assist you in troubleshooting, please = let me<br> > know.<br> <br> Can you also post the gdb output for "bt 5 full" ?<br> ><br> ><br> > Zeljko<br> ><br> ><br> <br> <br> --<br> =C2=A0 =C2=A0-- Howard Chu<br> =C2=A0 =C2=A0CTO, Symas Corp. =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 <a href=3D= "http://www.symas.com" target=3D"_blank">http://www.symas.com</a><br> =C2=A0 =C2=A0Director, Highland Sun =C2=A0 =C2=A0 <a href=3D"http://highlan= dsun.com/hyc/" target=3D"_blank">http://highlandsun.com/hyc/</a><br> =C2=A0 =C2=A0Chief Architect, OpenLDAP =C2=A0<a href=3D"http://www.openldap= .org/project/" target=3D"_blank">http://www.openldap.org/project/</a><br> <br> <br> </div></div></blockquote></div><br></div> --047d7b67749a03fb9b04e7c010b4--

1 0

Re: (ITS#7715) SIGBUS when mdb is configured with writemap
by hyc＠symas.com 02 Oct '13

02 Oct '13

nejasmicz(a)gmail.com wrote: > Full_Name: eljko Nejamić > Version: latest git pull of OPENLDAP_REL_ENG_2_4 > OS: RedHat 6.3 > URL: > Submission from: (NULL) (213.147.123.33) > > Hardware underneath all of that is: > 1) HP ProLiant BL460c Gen8, dual Xeon E5-2658 with attached storage blade > D2200sb with SSD raid, 256GB RAM -- RedHat 6.3 tests > 2) Intel server blade S2400BB, dual Xeon E5-2403, 48GB RAM -- Ubuntu 12.04 > tests > > If anything more is required to assist you in troubleshooting, please let me > know. Can you also post the gdb output for "bt 5 full" ? > > > Zeljko > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#7715) SIGBUS when mdb is configured with writemap
by nejasmicz＠gmail.com 02 Oct '13

02 Oct '13

Full_Name: eljko Nejamić Version: latest git pull of OPENLDAP_REL_ENG_2_4 OS: RedHat 6.3 URL: Submission from: (NULL) (213.147.123.33) Using ldclt tool to stress test our OpenLDAP mirror sync setup I encountered a SIGBUS. Do note that the same issue occurs on only one node too, without sync. I've tested using the aforementioned tool and the same arguments on both Red Hat 6.3 (2.6.32-279.el6.x86_64) and Ubuntu Server 12.04 (Linux 3.2.0-54-generic x86_64) with the exact outcome. In both cases the OpenLDAP was compiled from sources (origin/OPENLDAP_REL_ENG_2_4), configured with --disable-{hdb,bdb}, --prefix=/opt/openldap, --enable-local=yes and using mdb as a backend, tweaked additionally with: * nometasync * writemap Without the writemap tweak, SIGBUS isn't happening. The command used was: ldclt -h 172.17.101.150 -p 389 -D "cn=xxx,dc=xxx" -w "xxx" -b "ds=USERS,o=STANDARD,dc=xxx" \ -e object=xxx.txt,rdn='uid:[A=INCRNNOLOOP(200000;999999;6)]' -e add,commoncounter -I 68 ...where the xxx.txt has the following content: objectclass: xxxUser The ldclt command uses 10 threads to do the add operation with the incrementing uid parameter on the base dn: ds=USERS,o=STANDARD,dc=xxx. Ulimits are: ulimit -a core file size (blocks, -c) unlimited data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 2066206 max locked memory (kbytes, -l) unlimited max memory size (kbytes, -m) unlimited open files (-n) 4096 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) unlimited real-time priority (-r) 0 stack size (kbytes, -s) unlimited cpu time (seconds, -t) unlimited max user processes (-u) 1024 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited At first sight, gdb seems to point to mdb_page_alloc: Starting program: /opt/openldap/libexec/slapd -h ldap:///\ ldapi:/// -F /opt/openldap/etc/openldap/slapd.d -g openldap -u openldap -d 0 [Thread debugging using libthread_db enabled] [New Thread 0x2aaaac764700 (LWP 20415)] [Thread 0x2aaaac764700 (LWP 20415) exited] [New Thread 0x2aaaac764700 (LWP 20416)] [New Thread 0x2ab3ad168700 (LWP 20417)] [New Thread 0x2ab3ad969700 (LWP 20418)] [New Thread 0x2ab3ae16a700 (LWP 20419)] [New Thread 0x2ab3ae96b700 (LWP 20420)] [New Thread 0x2ab3b8800700 (LWP 20421)] [New Thread 0x2ab3d4800700 (LWP 20422)] Program received signal SIGBUS, Bus error. [Switching to Thread 0x2ab3b8800700 (LWP 20421)] mdb_page_alloc (mc=<value optimized out>, num=1, mp=0x2ab3b87fd8b8) at ./../../../libraries/liblmdb/mdb.c:1759 warning: Source file is more recent than executable. 1759 np->mp_pgno = pgno; And the backtrace is: #0 mdb_page_alloc (mc=<value optimized out>, num=1, mp=0x2ab3b87fd8b8) at ./../../../libraries/liblmdb/mdb.c:1759 #1 0x00000000004afb19 in mdb_page_touch (mc=0x2ab3bc1103f0) at ./../../../libraries/liblmdb/mdb.c:1889 #2 0x00000000004b1c8c in mdb_cursor_touch (mc=0x2ab3bc1103f0) at ./../../../libraries/liblmdb/mdb.c:5597 #3 0x00000000004b3a85 in mdb_cursor_put (mc=0x2ab3bc1103f0, key=0x2ab3b87ff000, data=0x2ab3b87feff0, flags=32) at ./../../../libraries/liblmdb/mdb.c:5727 #4 0x00000000004f8586 in mdb_idl_insert_keys (be=<value optimized out>, cursor=0x2ab3bc1103f0, keys=<value optimized out>, id=13) at idl.c:534 #5 0x00000000004f9116 in indexer (op=0x2ab3bc10dbd0, txn=<value optimized out>, ai=<value optimized out>, ad=0x88e0e0, atname=0x88dfb8, vals=0x2ab3bc110120, id=13, opid=1, mask=4) at index.c:219 #6 0x00000000004f95d1 in index_at_values (op=0x2ab3bc10dbd0, txn=0x2ab3bc10e2f0, ad=<value optimized out>, type=0x88df50, tags=0x88e100, vals=0x2ab3bc110120, id=13, opid=1) at index.c:337 #7 0x00000000004f9627 in mdb_index_values (op=<value optimized out>, txn=<value optimized out>, desc=<value optimized out>, vals=<value optimized out>, id=<value optimized out>, opid=<value optimized out>) at index.c:386 #8 0x00000000004f96f9 in mdb_index_entry (op=0x2ab3bc10dbd0, txn=0x2ab3bc10e2f0, opid=1, e=0x8c18d8) at index.c:558 #9 0x00000000004ed77e in mdb_add (op=0x2ab3bc10dbd0, rs=0x2ab3b87ff950) at add.c:359 #10 0x0000000000487ac7 in overlay_op_walk (op=0x2ab3bc10dbd0, rs=0x2ab3b87ff950, which=op_add, oi=0x932280, on=0x0) at backover.c:671 #11 0x00000000004884a7 in over_op_func (op=0x2ab3bc10dbd0, rs=<value optimized out>, which=<value optimized out>) at backover.c:723 #12 0x00000000004281c0 in fe_op_add (op=0x2ab3bc10dbd0, rs=0x2ab3b87ff950) at add.c:334 #13 0x0000000000428a16 in do_add (op=0x2ab3bc10dbd0, rs=0x2ab3b87ff950) at add.c:194 #14 0x0000000000421259 in connection_operation (ctx=0x2ab3b87ffab0, arg_v=0x2ab3bc10dbd0) at connection.c:1155 #15 0x0000000000421a35 in connection_read_thread (ctx=0x2ab3b87ffab0, argv=<value optimized out>) at connection.c:1291 #16 0x0000000000516380 in ldap_int_thread_pool_wrapper (xpool=0x898160) at tpool.c:688 #17 0x000000384cc07851 in start_thread () from /lib64/libpthread.so.0 #18 0x000000384c8e767d in clone () from /lib64/libc.so.6 For context, the assembly land around the offending pointer dereferencing looks like: 0x4af949 <mdb_page_alloc+665> movslq %r13d,%rax 0x4af94c <mdb_page_alloc+668> lea (%rcx,%rax,1),%rax 0x4af950 <mdb_page_alloc+672> mov %rax,0x10(%r14) 0x4af954 <mdb_page_alloc+676> mov %rcx,0x0(%rbp) Hardware underneath all of that is: 1) HP ProLiant BL460c Gen8, dual Xeon E5-2658 with attached storage blade D2200sb with SSD raid, 256GB RAM -- RedHat 6.3 tests 2) Intel server blade S2400BB, dual Xeon E5-2403, 48GB RAM -- Ubuntu 12.04 tests If anything more is required to assist you in troubleshooting, please let me know. Zeljko

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs