- openldap-bugs - openldap.org

Re: (ITS#9037) observing crash in mdb_cursor_put()
by vporof＠mozilla.com 13 Aug '19

13 Aug '19

--Apple-Mail=_F8BCAB64-A19B-4B00-B41B-03BE06C12A1F Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On 6. Aug 2019, at 13:43, Victor Porof <vporof(a)mozilla.com> wrote: >=20 > Here's everything we know about: = https://crash-stats.mozilla.org/report/index/5d77bd19-41ce-459f-9c1c-7f9fb= 0190324 See the "details" and "telemetry environment" sections for a = breakdown. Since that link is now expired, here=E2=80=99s an up to date place to = see all recent crash reports involving `mdb_cursor_put`: = https://crash-stats.mozilla.org/signature/?signature=3Dmdb_cursor_put#repo= rts = <https://crash-stats.mozilla.org/signature/?signature=3Dmdb_cursor_put#rep= orts>=20 Victor= --Apple-Mail=_F8BCAB64-A19B-4B00-B41B-03BE06C12A1F Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 <html><head><meta http-equiv=3D"Content-Type" content=3D"text/html; = charset=3Dutf-8"></head><body style=3D"word-wrap: break-word; = -webkit-nbsp-mode: space; line-break: after-white-space;" class=3D""><br = class=3D""><div><blockquote type=3D"cite" class=3D""><div class=3D"">On = 6. Aug 2019, at 13:43, Victor Porof <<a = href=3D"mailto:vporof@mozilla.com" class=3D"">vporof(a)mozilla.com</a>> = wrote:</div><div class=3D""><div class=3D""><br class=3D"">Here's = everything we know about: <a = href=3D"https://crash-stats.mozilla.org/report/index/5d77bd19-41ce-459f-9c= 1c-7f9fb0190324" = class=3D"">https://crash-stats.mozilla.org/report/index/5d77bd19-41ce-459f= -9c1c-7f9fb0190324</a> See the "details" and "telemetry environment" = sections for a breakdown.<br class=3D""></div></div></blockquote><div><br = class=3D""></div>Since that link is now expired, here=E2=80=99s an up to = date place to see all recent crash reports involving = `mdb_cursor_put`: <a = href=3D"https://crash-stats.mozilla.org/signature/?signature=3Dmdb_cursor_= put#reports" = class=3D"">https://crash-stats.mozilla.org/signature/?signature=3Dmdb_curs= or_put#reports</a> <div class=3D""><br class=3D""></div><div = class=3D"">Victor</div></div></body></html>= --Apple-Mail=_F8BCAB64-A19B-4B00-B41B-03BE06C12A1F--

1 0

Re: (ITS#9062) Please have /build/mkdep respect TMPDIR if set
by quanah＠symas.com 13 Aug '19

13 Aug '19

--On Sunday, August 11, 2019 7:34 PM +0000 zagarc(a)oclc.org wrote: > Would you please consider changing this to: > > TMP=${TMPDIR-/tmp}/mkdep$$ Thanks for the report, this is fixed in OpenLDAP master. Regards, Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

(ITS#9063) tls_reqcert in slapd.conf/slapd-config man pages should be bold
by quanah＠openldap.org 13 Aug '19

13 Aug '19

Full_Name: Quanah Gibson-Mount Version: 2.4.48 OS: N/A URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (47.208.128.44) When looking at the man pages, the tls_reqcert description in the section of the man page about syncrepl is missing a BOLD tag, unlike the other keywords. Trivial fix.

1 0

(ITS#9062) Please have /build/mkdep respect TMPDIR if set
by zagarc＠oclc.org 11 Aug '19

11 Aug '19

Full_Name: Chris Zagar Version: 2.4.48 OS: Linux URL: Submission from: (NULL) (68.98.212.84) /build/mkdep contains this line: TMP=/tmp/mkdep$$ that forces the use of the /tmp directory. The /tmp directory is vulnerable to race conditions. The rest of OpenLDAP obeys the TMPDIR environment variable if it exists as a mitigation to this risk. Would you please consider changing this to: TMP=${TMPDIR-/tmp}/mkdep$$ so this will obey TMPDIR as well? Thank you. Chris Zagar zagarc(a)oclc.org

1 0

Re: (ITS#9061) LDAP replication issue on a big network
by quanah＠symas.com 07 Aug '19

07 Aug '19

--On Wednesday, August 07, 2019 10:35 AM +0000 alex.s(a)wildix.com wrote: > Full_Name: Alex > Version: 2.4.44+dfsg-5+deb9u2 Hello, The ITS system is for bug reports, not usage questions. Additionally, if you're doing replication of any type, please upgrade to the latest release (2.4.48). Ensure you are using delta-syncrepl. For any other questions, please use the correct forum which is the openldap-technical(a)openldap.org mailing list: <https://www.openldap.org/lists/mm/listinfo/openldap-technical>. I believe there is a backport of 2.4.48 for Debian9 from the backports repo. This ITS will be closed. Regards, Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

(ITS#9061) LDAP replication issue on a big network
by alex.s＠wildix.com 07 Aug '19

07 Aug '19

Full_Name: Alex Version: 2.4.44+dfsg-5+deb9u2 OS: Debian 9 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (154.41.3.130) Hello I have some issues with replication between Master and Slave LDAP servers. Preconditions data: I have a Master server with a huge LDAP data And also I have about 200 Slave servers around the world which have a data replica in their databases. What happened: I have applied a bckup on Master server. (Actually data have not changed except entryCSN) entryCSN has been deleted from backup previously, before apply, for actualise data in LDAP database. As a result I had a situation when all Slave servers starts replication. The first question is: how I can avoid full replication after apply backup on Master? I understand that entryCSN changed and LDAP should sync some objects. But can I use another way to actualize the data in LDAP database instead of entryCSN? The second question is: after apply backup and restarts the Master server I had a problem with local LDAP because all Slave servers start connecting and start their replication. In this case LDAP on Master server not responding even on localhost via ldapsearch. May I change some parameters to increase concurrent connections? Because LDAP starts not responding if the quantity of simultaneous connected of Slave servers exceeds 10-15 Thank you in advance Best regards, Alex

1 0

Re: (ITS#9060) lmdb corruption after many memory map resizes
by ruan.declercq＠netronome.com 06 Aug '19

06 Aug '19

--00000000000075fbb5058f76dbf0 Content-Type: text/plain; charset="UTF-8" The documentation states: "It may be called at later times if no transactions are active in this process. Note that the library does not check for this condition, the caller must ensure it explicitly." Maybe I'm missing something, but my code aims to have no active transactions in this process when I do the resize. That's why I first do the txn_commit() and then the resize. On Tue, Aug 6, 2019 at 6:08 PM Howard Chu <hyc(a)symas.com> wrote: > ruan.declercq(a)netronome.com wrote: > > Full_Name: Ruan de Clercq > > Version: > > OS: Ubuntu 19.04 > > URL: ftp://ftp.openldap.org/incoming/ > > Submission from: (NULL) (155.93.214.171) > > > > > > Hi, > > > > I am using 0.9.23-0ubuntu1 on ubuntu 19.04. > > > > I have an application where the exact db size isn't known beforehand. > Therefore, > > I set the memory map size beforehand, and then increase the size when I > run out > > of space. > > This is incorrect usage of LMDB. The docs explicitly state to use a large > size at > the beginning and leave it alone. > > > However, when I use lmdb in concurrent processes, and frequently > > increase the size of the memory map, I eventually get a corrupt database > > (MDB_CORRUPTED). > > Your code is incorrect. The docs explicitly state that there must not be > any outstanding > activity when using the set_mapsize call. > > Closing this ITS. > > > Here's a minimal code example: > > > > #include <stdio.h> > > #include <stdlib.h> > > #include <inttypes.h> > > #include <time.h> > > #include "lmdb.h" > > #include <cstring> > > > > int resize(MDB_env *env) { > > // read map size and increase > > int val; > > MDB_envinfo stat; > > MDB_txn *txn; > > if ((val = mdb_env_info(env, &stat) != MDB_SUCCESS) || > > (val = mdb_env_set_mapsize(env, stat.me_mapsize + 4096) != > MDB_SUCCESS) > > || > > (val = mdb_env_info(env, &stat) != MDB_SUCCESS)) { > > printf("set new mapsize %d\n", val); > > return val; > > } > > printf("Resized to %lu\n", stat.me_mapsize); > > if (((val = mdb_txn_begin(env, NULL, 0, &txn) != MDB_SUCCESS)) || > > ((val = mdb_txn_commit(txn)))) { > > printf("txn %d\n", val); > > } > > return val; > > } > > > > int main(int argc,char * argv[]) > > { > > timespec tp; > > clock_gettime(CLOCK_MONOTONIC, &tp); > > srand(tp.tv_sec + tp.tv_nsec); > > > > MDB_env *env = NULL; > > MDB_txn *txn = NULL; > > MDB_dbi dbi; > > int val; > > if ((val = mdb_env_create(&env) != MDB_SUCCESS) || > > (val = mdb_env_set_maxreaders(env, 126) != MDB_SUCCESS)) { > > printf("Create error %d", val); > > goto out; > > } > > > > if ((val = mdb_env_set_mapsize(env, 4096 * 128) != MDB_SUCCESS) || > > (val = mdb_env_set_maxdbs(env, 1) != MDB_SUCCESS) || > > (val = mdb_env_open(env, "./data/", MDB_MAPASYNC | MDB_WRITEMAP, > 0664) != > > MDB_SUCCESS) || > > (val = mdb_txn_begin(env, NULL, 0, &txn) != MDB_SUCCESS) || > > (val = mdb_dbi_open(txn, "test", MDB_CREATE, &dbi) != MDB_SUCCESS) > || > > (val = mdb_txn_commit(txn) != MDB_SUCCESS)) { > > printf("create db %d\n", val); > > goto out; > > } > > > > for (int i = 0; i < 1000; i++) { > > char sval[32]; > > sprintf(sval, "%03x %d foo bar", 32, rand()); > > MDB_val mdb_key, mdb_val; > > mdb_key.mv_size = strlen(sval);; > > mdb_key.mv_data = sval; > > mdb_val.mv_size = strlen(sval); > > mdb_val.mv_data = sval; > > > > if ((val = mdb_txn_begin(env, NULL, 0, &txn) != MDB_SUCCESS)) { > > printf("mdb_txn_begin %d\n", val); > > goto out; > > } > > if ((val = mdb_put(txn, dbi, &mdb_key, &mdb_val, 0)) != MDB_SUCCESS) > { > > mdb_txn_commit(txn); > > if (val == MDB_MAP_FULL) { > > val = resize(env); > > continue; > > } > > printf("mdb_put %d\n", val); > > goto out; > > } > > if ((val = mdb_txn_commit(txn)) != MDB_SUCCESS) { > > if (val == MDB_MAP_FULL) { > > val = resize(env); > > continue; > > } > > printf("mdb_txn_commit %d\n", val); > > goto out; > > } > > } > > out: > > if (env) { > > mdb_dbi_close(env, dbi); > > mdb_env_close(env); > > } > > printf("OK\n"); > > } > > > > After compiling I run the following command a couple of times: > > for i in {1..20}; do echo $i; (lmdb_test &); done > > > > > > > -- > -- Howard Chu > CTO, Symas Corp. http://www.symas.com > Director, Highland Sun http://highlandsun.com/hyc/ > Chief Architect, OpenLDAP http://www.openldap.org/project/ > -- Regards, Ruan de Clercq --00000000000075fbb5058f76dbf0 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div dir=3D"ltr"><div>The documentation states:</div><div>= "It may be called at later times if no transactions are active in=C2= =A0this process. Note that the library does not check for this condition, t= he caller must ensure it explicitly."</div><div>Maybe I'm missing = something, but my code aims to have no active transactions in this process = when I do the resize. That's why I first do the txn_commit() and then t= he resize.</div><div><br></div><div>On Tue, Aug 6, 2019 at 6:08 PM Howard C= hu <<a href=3D"mailto:hyc@symas.com">hyc(a)symas.com</a>> wrote:</div><= div class=3D"gmail_quote"><blockquote class=3D"gmail_quote" style=3D"margin= :0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-lef= t-color:rgb(204,204,204);padding-left:1ex"><a href=3D"mailto:ruan.declercq@= netronome.com" target=3D"_blank">ruan.declercq(a)netronome.com</a> wrote:<br> > Full_Name: Ruan de Clercq<br> > Version: <br> > OS: Ubuntu 19.04<br> > URL: <a href=3D"ftp://ftp.openldap.org/incoming/" rel=3D"noreferrer" t= arget=3D"_blank">ftp://ftp.openldap.org/incoming/</a><br> > Submission from: (NULL) (155.93.214.171)<br> > <br> > <br> > Hi,<br> > <br> > I am using 0.9.23-0ubuntu1 on ubuntu 19.04. <br> > <br> > I have an application where the exact db size isn't known beforeha= nd. Therefore,<br> > I set the memory map size beforehand, and then increase the size when = I run out<br> > of space.<br> <br> This is incorrect usage of LMDB. The docs explicitly state to use a large s= ize at<br> the beginning and leave it alone.<br> <br> > However, when I use lmdb in concurrent processes, and frequently<br> > increase the size of the memory map, I eventually get a corrupt databa= se<br> > (MDB_CORRUPTED).<br> <br> Your code is incorrect. The docs explicitly state that there must not be an= y outstanding<br> activity when using the set_mapsize call.<br> <br> Closing this ITS.<br> ><br> =C2=A0Here's a minimal code example:<br> > <br> > #include <stdio.h><br> > #include <stdlib.h><br> > #include <inttypes.h><br> > #include <time.h><br> > #include "lmdb.h"<br> > #include <cstring><br> > <br> > int resize(MDB_env *env) {<br> >=C2=A0 =C2=A0// read map size and increase<br> >=C2=A0 =C2=A0int val;<br> >=C2=A0 =C2=A0MDB_envinfo stat;<br> >=C2=A0 =C2=A0MDB_txn *txn;<br> >=C2=A0 =C2=A0if ((val =3D mdb_env_info(env, &stat) !=3D MDB_SUCCESS= ) ||<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0(val =3D mdb_env_set_mapsize(env, stat.me_ma= psize + 4096) !=3D MDB_SUCCESS)<br> > ||<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0(val =3D mdb_env_info(env, &stat) !=3D M= DB_SUCCESS)) {<br> >=C2=A0 =C2=A0 =C2=A0printf("set new mapsize %d\n", val);<br> >=C2=A0 =C2=A0 =C2=A0return val;<br> >=C2=A0 =C2=A0}<br> >=C2=A0 =C2=A0printf("Resized to %lu\n", stat.me_mapsize);<br> >=C2=A0 =C2=A0if (((val =3D mdb_txn_begin(env, NULL, 0, &txn) !=3D M= DB_SUCCESS)) ||<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0((val =3D mdb_txn_commit(txn)))) {<br> >=C2=A0 =C2=A0 =C2=A0printf("txn %d\n", val);<br> >=C2=A0 =C2=A0}<br> >=C2=A0 =C2=A0return val;<br> > }<br> > <br> > int main(int argc,char * argv[])<br> > {<br> >=C2=A0 =C2=A0timespec tp;<br> >=C2=A0 =C2=A0clock_gettime(CLOCK_MONOTONIC, &tp);<br> >=C2=A0 =C2=A0srand(tp.tv_sec + tp.tv_nsec);<br> > <br> >=C2=A0 =C2=A0MDB_env *env =3D NULL;<br> >=C2=A0 =C2=A0MDB_txn *txn =3D NULL;<br> >=C2=A0 =C2=A0MDB_dbi dbi;<br> >=C2=A0 =C2=A0int val;<br> >=C2=A0 =C2=A0if ((val =3D mdb_env_create(&env) !=3D MDB_SUCCESS) ||= <br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0(val =3D mdb_env_set_maxreaders(env, 126) != =3D MDB_SUCCESS)) {<br> >=C2=A0 =C2=A0 =C2=A0printf("Create error %d", val);<br> >=C2=A0 =C2=A0 =C2=A0goto out;<br> >=C2=A0 =C2=A0}<br> > <br> >=C2=A0 =C2=A0if ((val =3D mdb_env_set_mapsize(env, 4096 * 128) !=3D MDB= _SUCCESS) ||<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0(val =3D mdb_env_set_maxdbs(env, 1) !=3D MDB= _SUCCESS) ||<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0(val =3D mdb_env_open(env, "./data/&quo= t;, MDB_MAPASYNC | MDB_WRITEMAP, 0664) !=3D<br> > MDB_SUCCESS) ||<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0(val =3D mdb_txn_begin(env, NULL, 0, &tx= n) !=3D MDB_SUCCESS) ||<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0(val =3D mdb_dbi_open(txn, "test",= MDB_CREATE, &dbi) !=3D MDB_SUCCESS) ||<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0(val =3D mdb_txn_commit(txn) !=3D MDB_SUCCES= S)) {<br> >=C2=A0 =C2=A0 =C2=A0printf("create db %d\n", val);<br> >=C2=A0 =C2=A0 =C2=A0goto out;<br> >=C2=A0 =C2=A0}<br> > <br> >=C2=A0 =C2=A0for (int i =3D 0; i < 1000; i++) {<br> >=C2=A0 =C2=A0 =C2=A0char sval[32];<br> >=C2=A0 =C2=A0 =C2=A0sprintf(sval, "%03x %d foo bar", 32, rand= ());<br> >=C2=A0 =C2=A0 =C2=A0MDB_val mdb_key, mdb_val;<br> >=C2=A0 =C2=A0 =C2=A0mdb_key.mv_size =3D strlen(sval);;<br> >=C2=A0 =C2=A0 =C2=A0mdb_key.mv_data =3D sval;<br> >=C2=A0 =C2=A0 =C2=A0mdb_val.mv_size =3D strlen(sval);<br> >=C2=A0 =C2=A0 =C2=A0mdb_val.mv_data =3D sval;<br> > <br> >=C2=A0 =C2=A0 =C2=A0if ((val =3D mdb_txn_begin(env, NULL, 0, &txn) = !=3D MDB_SUCCESS)) {<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0printf("mdb_txn_begin %d\n", val);= <br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0goto out;<br> >=C2=A0 =C2=A0 =C2=A0}<br> >=C2=A0 =C2=A0 =C2=A0if ((val =3D mdb_put(txn, dbi, &mdb_key, &m= db_val, 0)) !=3D MDB_SUCCESS) {<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0mdb_txn_commit(txn);<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0if (val =3D=3D MDB_MAP_FULL) {<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0val =3D resize(env);<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0continue;<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0}<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0printf("mdb_put %d\n", val);<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0goto out;<br> >=C2=A0 =C2=A0 =C2=A0}<br> >=C2=A0 =C2=A0 =C2=A0if ((val =3D mdb_txn_commit(txn)) !=3D MDB_SUCCESS)= {<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0if (val =3D=3D MDB_MAP_FULL) {<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0val =3D resize(env);<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0continue;<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0}<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0printf("mdb_txn_commit %d\n", val)= ;<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0goto out;<br> >=C2=A0 =C2=A0 =C2=A0}<br> >=C2=A0 =C2=A0}<br> > out:<br> >=C2=A0 =C2=A0if (env) {<br> >=C2=A0 =C2=A0 =C2=A0mdb_dbi_close(env, dbi);<br> >=C2=A0 =C2=A0 =C2=A0mdb_env_close(env);<br> >=C2=A0 =C2=A0}<br> >=C2=A0 =C2=A0printf("OK\n");<br> > }<br> > <br> > After compiling I run the following command a couple of times:<br> > for i in {1..20}; do echo $i; (lmdb_test &); done<br> > <br> > <br> <br> <br> -- <br> =C2=A0 -- Howard Chu<br> =C2=A0 CTO, Symas Corp.=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0<a href=3D"= http://www.symas.com" rel=3D"noreferrer" target=3D"_blank">http://www.symas= .com</a><br> =C2=A0 Director, Highland Sun=C2=A0 =C2=A0 =C2=A0<a href=3D"http://highland= sun.com/hyc/" rel=3D"noreferrer" target=3D"_blank">http://highlandsun.com/h= yc/</a><br> =C2=A0 Chief Architect, OpenLDAP=C2=A0 <a href=3D"http://www.openldap.org/p= roject/" rel=3D"noreferrer" target=3D"_blank">http://www.openldap.org/proje= ct/</a><br> </blockquote></div><br clear=3D"all"><div><br></div>-- <br><div dir=3D"ltr"= class=3D"gmail_signature"><div dir=3D"ltr"><div>Regards,</div><div>Ruan de= Clercq<br></div></div></div></div></div> --00000000000075fbb5058f76dbf0--

1 0

Re: (ITS#9060) lmdb corruption after many memory map resizes
by hyc＠symas.com 06 Aug '19

06 Aug '19

ruan.declercq(a)netronome.com wrote: > Full_Name: Ruan de Clercq > Version: > OS: Ubuntu 19.04 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (155.93.214.171) > > > Hi, > > I am using 0.9.23-0ubuntu1 on ubuntu 19.04. > > I have an application where the exact db size isn't known beforehand. Therefore, > I set the memory map size beforehand, and then increase the size when I run out > of space. This is incorrect usage of LMDB. The docs explicitly state to use a large size at the beginning and leave it alone. > However, when I use lmdb in concurrent processes, and frequently > increase the size of the memory map, I eventually get a corrupt database > (MDB_CORRUPTED). Your code is incorrect. The docs explicitly state that there must not be any outstanding activity when using the set_mapsize call. Closing this ITS. > Here's a minimal code example: > > #include <stdio.h> > #include <stdlib.h> > #include <inttypes.h> > #include <time.h> > #include "lmdb.h" > #include <cstring> > > int resize(MDB_env *env) { > // read map size and increase > int val; > MDB_envinfo stat; > MDB_txn *txn; > if ((val = mdb_env_info(env, &stat) != MDB_SUCCESS) || > (val = mdb_env_set_mapsize(env, stat.me_mapsize + 4096) != MDB_SUCCESS) > || > (val = mdb_env_info(env, &stat) != MDB_SUCCESS)) { > printf("set new mapsize %d\n", val); > return val; > } > printf("Resized to %lu\n", stat.me_mapsize); > if (((val = mdb_txn_begin(env, NULL, 0, &txn) != MDB_SUCCESS)) || > ((val = mdb_txn_commit(txn)))) { > printf("txn %d\n", val); > } > return val; > } > > int main(int argc,char * argv[]) > { > timespec tp; > clock_gettime(CLOCK_MONOTONIC, &tp); > srand(tp.tv_sec + tp.tv_nsec); > > MDB_env *env = NULL; > MDB_txn *txn = NULL; > MDB_dbi dbi; > int val; > if ((val = mdb_env_create(&env) != MDB_SUCCESS) || > (val = mdb_env_set_maxreaders(env, 126) != MDB_SUCCESS)) { > printf("Create error %d", val); > goto out; > } > > if ((val = mdb_env_set_mapsize(env, 4096 * 128) != MDB_SUCCESS) || > (val = mdb_env_set_maxdbs(env, 1) != MDB_SUCCESS) || > (val = mdb_env_open(env, "./data/", MDB_MAPASYNC | MDB_WRITEMAP, 0664) != > MDB_SUCCESS) || > (val = mdb_txn_begin(env, NULL, 0, &txn) != MDB_SUCCESS) || > (val = mdb_dbi_open(txn, "test", MDB_CREATE, &dbi) != MDB_SUCCESS) || > (val = mdb_txn_commit(txn) != MDB_SUCCESS)) { > printf("create db %d\n", val); > goto out; > } > > for (int i = 0; i < 1000; i++) { > char sval[32]; > sprintf(sval, "%03x %d foo bar", 32, rand()); > MDB_val mdb_key, mdb_val; > mdb_key.mv_size = strlen(sval);; > mdb_key.mv_data = sval; > mdb_val.mv_size = strlen(sval); > mdb_val.mv_data = sval; > > if ((val = mdb_txn_begin(env, NULL, 0, &txn) != MDB_SUCCESS)) { > printf("mdb_txn_begin %d\n", val); > goto out; > } > if ((val = mdb_put(txn, dbi, &mdb_key, &mdb_val, 0)) != MDB_SUCCESS) { > mdb_txn_commit(txn); > if (val == MDB_MAP_FULL) { > val = resize(env); > continue; > } > printf("mdb_put %d\n", val); > goto out; > } > if ((val = mdb_txn_commit(txn)) != MDB_SUCCESS) { > if (val == MDB_MAP_FULL) { > val = resize(env); > continue; > } > printf("mdb_txn_commit %d\n", val); > goto out; > } > } > out: > if (env) { > mdb_dbi_close(env, dbi); > mdb_env_close(env); > } > printf("OK\n"); > } > > After compiling I run the following command a couple of times: > for i in {1..20}; do echo $i; (lmdb_test &); done > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#9037) observing crash in mdb_cursor_put()
by vporof＠mozilla.com 06 Aug '19

06 Aug '19

> On 1. Aug 2019, at 19:47, Howard Chu <hyc(a)symas.com> wrote: >=20 > vporof(a)mozilla.com wrote: >> Hey folks. >>=20 >> =3D46rom Myk=3DE2=3D80=3D99s investigations in the previous followup, = it seems =3D >> that the suggested changes to `mdb_cursor_init` to avoid using an =3D >> invalid DBI might not be solving the actual issue, given the = behaviour =3D >> of `mdb_page_search`. >=20 > Agreed, that assert that I suggested isn't catching what we want. FWIW, here's my findings when attempting to look into what was happening = with regards to that test case: = https://gist.github.com/victorporof/d1f98b8a52f79c7ff99f361d3a2adc3e It's unclear how much overlap there is with the previous findings, and = whether or not calling `mdb_put` should assert with the DBI previously = opened via `mdb_dbi_open` with MDB_CREATE. Let me know if what I'm = observing here is expected. >=20 >> It=3DE2=3D80=3D99s also causing the seemingly correct test program to = assert =3D >> when it wasn=3DE2=3D80=3D99t before. It=3DE2=3D80=3D99s unclear = whether this should =3D >> be the case or not, perhaps Howard can confirm the expected = behaviour. >>=20 >> In any case, we=3DE2=3D80=3D99re wondering if there=3DE2=3D80=3D99s = been any other =3D >> progress, or if someone managed to reproduce this issue? Shipping new = =3D >> features built on top of LMDB in Firefox is currently blocked due to = =3D >> these failures, so any additional info would be helpful. >=20 > Sorry, still not seeing this over here. What else do you know about = the > systems where this is occurring? RAM size, storage on HDD / SSD / USB = ? Here's everything we know about: = https://crash-stats.mozilla.org/report/index/5d77bd19-41ce-459f-9c1c-7f9fb= 0190324 See the "details" and "telemetry environment" sections for a = breakdown. Victor >=20 > --=20 > -- Howard Chu > CTO, Symas Corp. http://www.symas.com > Director, Highland Sun http://highlandsun.com/hyc/ > Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#9060) lmdb corruption after many memory map resizes
by ruan.declercq＠netronome.com 06 Aug '19

06 Aug '19

Full_Name: Ruan de Clercq Version: OS: Ubuntu 19.04 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (155.93.214.171) Hi, I am using 0.9.23-0ubuntu1 on ubuntu 19.04. I have an application where the exact db size isn't known beforehand. Therefore, I set the memory map size beforehand, and then increase the size when I run out of space. However, when I use lmdb in concurrent processes, and frequently increase the size of the memory map, I eventually get a corrupt database (MDB_CORRUPTED). Here's a minimal code example: #include <stdio.h> #include <stdlib.h> #include <inttypes.h> #include <time.h> #include "lmdb.h" #include <cstring> int resize(MDB_env *env) { // read map size and increase int val; MDB_envinfo stat; MDB_txn *txn; if ((val = mdb_env_info(env, &stat) != MDB_SUCCESS) || (val = mdb_env_set_mapsize(env, stat.me_mapsize + 4096) != MDB_SUCCESS) || (val = mdb_env_info(env, &stat) != MDB_SUCCESS)) { printf("set new mapsize %d\n", val); return val; } printf("Resized to %lu\n", stat.me_mapsize); if (((val = mdb_txn_begin(env, NULL, 0, &txn) != MDB_SUCCESS)) || ((val = mdb_txn_commit(txn)))) { printf("txn %d\n", val); } return val; } int main(int argc,char * argv[]) { timespec tp; clock_gettime(CLOCK_MONOTONIC, &tp); srand(tp.tv_sec + tp.tv_nsec); MDB_env *env = NULL; MDB_txn *txn = NULL; MDB_dbi dbi; int val; if ((val = mdb_env_create(&env) != MDB_SUCCESS) || (val = mdb_env_set_maxreaders(env, 126) != MDB_SUCCESS)) { printf("Create error %d", val); goto out; } if ((val = mdb_env_set_mapsize(env, 4096 * 128) != MDB_SUCCESS) || (val = mdb_env_set_maxdbs(env, 1) != MDB_SUCCESS) || (val = mdb_env_open(env, "./data/", MDB_MAPASYNC | MDB_WRITEMAP, 0664) != MDB_SUCCESS) || (val = mdb_txn_begin(env, NULL, 0, &txn) != MDB_SUCCESS) || (val = mdb_dbi_open(txn, "test", MDB_CREATE, &dbi) != MDB_SUCCESS) || (val = mdb_txn_commit(txn) != MDB_SUCCESS)) { printf("create db %d\n", val); goto out; } for (int i = 0; i < 1000; i++) { char sval[32]; sprintf(sval, "%03x %d foo bar", 32, rand()); MDB_val mdb_key, mdb_val; mdb_key.mv_size = strlen(sval);; mdb_key.mv_data = sval; mdb_val.mv_size = strlen(sval); mdb_val.mv_data = sval; if ((val = mdb_txn_begin(env, NULL, 0, &txn) != MDB_SUCCESS)) { printf("mdb_txn_begin %d\n", val); goto out; } if ((val = mdb_put(txn, dbi, &mdb_key, &mdb_val, 0)) != MDB_SUCCESS) { mdb_txn_commit(txn); if (val == MDB_MAP_FULL) { val = resize(env); continue; } printf("mdb_put %d\n", val); goto out; } if ((val = mdb_txn_commit(txn)) != MDB_SUCCESS) { if (val == MDB_MAP_FULL) { val = resize(env); continue; } printf("mdb_txn_commit %d\n", val); goto out; } } out: if (env) { mdb_dbi_close(env, dbi); mdb_env_close(env); } printf("OK\n"); } After compiling I run the following command a couple of times: for i in {1..20}; do echo $i; (lmdb_test &); done

1 0

Re: (ITS#9037) observing crash in mdb_cursor_put()
by hyc＠symas.com 01 Aug '19

01 Aug '19

vporof(a)mozilla.com wrote: > Hey folks. > > =46rom Myk=E2=80=99s investigations in the previous followup, it seems = > that the suggested changes to `mdb_cursor_init` to avoid using an = > invalid DBI might not be solving the actual issue, given the behaviour = > of `mdb_page_search`. Agreed, that assert that I suggested isn't catching what we want. > It=E2=80=99s also causing the seemingly correct test program to assert = > when it wasn=E2=80=99t before. It=E2=80=99s unclear whether this should = > be the case or not, perhaps Howard can confirm the expected behaviour. > > In any case, we=E2=80=99re wondering if there=E2=80=99s been any other = > progress, or if someone managed to reproduce this issue? Shipping new = > features built on top of LMDB in Firefox is currently blocked due to = > these failures, so any additional info would be helpful. Sorry, still not seeing this over here. What else do you know about the systems where this is occurring? RAM size, storage on HDD / SSD / USB ? -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#9059) session log parsing triggers cascading REFRESH
by quanah＠symas.com 01 Aug '19

01 Aug '19

--On Thursday, August 01, 2019 4:16 PM +0000 quanah(a)openldap.org wrote: > Full_Name: Quanah Gibson-Mount > Version: 2.4.47 > OS: N/A > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (47.208.128.44) > > > In investigating why a particular consumer went into REFRESH mode, I find > that this was triggered by one of its providers currently parsing a > sessionlog. Sessionlog parsing should *not* cause a consumer of that > server to fall back to REFRESH. Some notes on this specific environment and situation. This is a 4-way MMR setup. Via the use of a load balancer, only iam-ed-ldp-p01 recieves write operations. Thus there should never be any REFRESHes occurring anywhere, so the fact that there is is already problematic. Servers are p01, p02, p03 (all in the same data center) and b01 (offsite DR system). At the time this issue hit: p01 was upgraded to a new release of OpenLDAP @ 21:26:48, thus we see: Jul 30 21:25:17 iam-ed-ldp-p01 slapd[30495]: slapd shutdown: waiting for 0 operations/tasks to finish Jul 30 21:26:48 iam-ed-ldp-p01 slapd[20978]: @(#) $OpenLDAP: slapd 2.4.47 (Mar 22 2019 09:12:20) $#012#011build@rhel7build.rb.symas.net:/home/build/sold-2.4.47.3/redhat7-x86_64/build/openldap.x86_64/servers/slapd Jul 30 21:26:48 iam-ed-ldp-p01 slapd[20979]: slapd starting Jul 30 21:26:48 iam-ed-ldp-p01 slapd[20979]: do_syncrep2: rid=002 LDAP_RES_INTERMEDIATE - REFRESH_DELETE Jul 30 21:26:48 iam-ed-ldp-p01 slapd[20979]: do_syncrep2: rid=001 LDAP_RES_SEARCH_RESULT Jul 30 21:26:48 iam-ed-ldp-p01 slapd[20979]: do_syncrep2: rid=001 delta-sync lost sync, switching to REFRESH Jul 30 21:26:48 iam-ed-ldp-p01 slapd[20979]: do_syncrep2: rid=001 (4096) Content Sync Refresh Required Jul 30 21:26:48 iam-ed-ldp-p01 slapd[20979]: do_syncrep2: rid=003 LDAP_RES_INTERMEDIATE - REFRESH_DELETE p03 was operating w/o issue. Interstingly enough, when it was restarted for the software update @ 21:37:16, it got an error (53) talking to p02 instead of an error (4096) like p01 did. This is likely because p02 was restarted for its software update @ 21:32:19, which wiped out its sessionlog. p02 had started doing session log parsing @ 21:26:17 with Jul 30 21:26:17 iam-ed-ldp-p02 slapd[3750]: conn=18205759 op=1 SEARCH RESULT tag=101 err=4096 duration=18.462ms nentries=0 text=sync cookie is stale We see the connection to p02 from p01 with a similar issue when it starts up: Jul 30 21:26:48 iam-ed-ldp-p02 slapd[3750]: conn=18205790 op=1 SRCH base="cn=accesslog" scope=2 deref=0 filter="(&(objectClass=auditWriteObject)(reqResult=0))" Jul 30 21:26:48 iam-ed-ldp-p02 slapd[3750]: conn=18205790 op=1 SRCH attr=reqDN reqType reqMod reqNewRDN reqDeleteOldRDN reqNewSuperior entryCSN Jul 30 21:26:48 iam-ed-ldp-p02 slapd[3750]: conn=18205790 op=1 SEARCH RESULT tag=101 err=4096 duration=0.431ms nentries=0 text=sync cookie is stale b01 had its software update @ 21:21:25 Generally it seems like restarting/wiping the sessionlog made things incredibly bad across the cluster. -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

(ITS#9059) session log parsing triggers cascading REFRESH
by quanah＠openldap.org 01 Aug '19

01 Aug '19

Full_Name: Quanah Gibson-Mount Version: 2.4.47 OS: N/A URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (47.208.128.44) In investigating why a particular consumer went into REFRESH mode, I find that this was triggered by one of its providers currently parsing a sessionlog. Sessionlog parsing should *not* cause a consumer of that server to fall back to REFRESH. Server that got dropped into REFRESH incorrectly: Jul 30 21:26:48 iam-ed-ldp-p01 slapd[20979]: do_syncrep2: rid=001 LDAP_RES_SEARCH_RESULT Jul 30 21:26:48 iam-ed-ldp-p01 slapd[20979]: do_syncrep2: rid=001 delta-sync lost sync, switching to REFRESH Jul 30 21:26:48 iam-ed-ldp-p01 slapd[20979]: do_syncrep2: rid=001 (4096) Content Sync Refresh Required RID001 server: .... Jul 30 21:26:48 iam-ed-ldp-p02 slapd[3750]: log csn 20181127055648.177968Z#000000#001#000000 Jul 30 21:26:48 iam-ed-ldp-p02 slapd[3750]: log csn 20181109151409.323374Z#000000#001#000000 Jul 30 21:26:48 iam-ed-ldp-p02 slapd[3750]: log csn 20181127122502.661263Z#000000#001#000000 Jul 30 21:26:48 iam-ed-ldp-p02 slapd[3750]: cmp -256, too old Jul 30 21:26:48 iam-ed-ldp-p02 slapd[3750]: cmp -256, too old Jul 30 21:26:48 iam-ed-ldp-p02 slapd[3750]: log csn 20181127061139.881198Z#000000#001#000000 Jul 30 21:26:48 iam-ed-ldp-p02 slapd[3750]: cmp -256, too old Jul 30 21:26:48 iam-ed-ldp-p02 slapd[3750]: cmp -256, too old Jul 30 21:26:48 iam-ed-ldp-p02 slapd[3750]: log csn 20181127062500.514773Z#000000#001#000000 Jul 30 21:26:48 iam-ed-ldp-p02 slapd[3750]: cmp -256, too old Jul 30 21:26:48 iam-ed-ldp-p02 slapd[3750]: cmp -256, too old Jul 30 21:26:48 iam-ed-ldp-p02 slapd[3750]: log csn 20181127061139.941766Z#000000#001#000000 Jul 30 21:26:48 iam-ed-ldp-p02 slapd[3750]: log csn 20181127062500.622874Z#000000#001#000000 Jul 30 21:26:48 iam-ed-ldp-p02 slapd[3750]: cmp -256, too old Jul 30 21:26:48 iam-ed-ldp-p02 slapd[3750]: cmp -256, too old Jul 30 21:26:48 iam-ed-ldp-p02 slapd[3750]: log csn 20181127061139.977196Z#000000#001#000000 Jul 30 21:26:48 iam-ed-ldp-p02 slapd[3750]: log csn 20181127062500.656726Z#000000#001#000000 Jul 30 21:26:48 iam-ed-ldp-p02 slapd[3750]: log csn 20181109151409.575116Z#000000#001#000000 Jul 30 21:26:48 iam-ed-ldp-p02 slapd[3750]: cmp -256, too old Jul 30 21:26:48 iam-ed-ldp-p02 slapd[3750]: cmp -256, too old Jul 30 21:26:48 iam-ed-ldp-p02 slapd[3750]: log csn 20181127062500.672371Z#000000#001#000000 Jul 30 21:26:48 iam-ed-ldp-p02 slapd[3750]: cmp -256, too old Jul 30 21:26:48 iam-ed-ldp-p02 slapd[3750]: cmp -256, too old Jul 30 21:26:48 iam-ed-ldp-p02 slapd[3750]: log csn 20181127062500.701608Z#000000#001#000000 Jul 30 21:26:48 iam-ed-ldp-p02 slapd[3750]: log csn 20181109151409.833435Z#000000#001#000000 Jul 30 21:26:48 iam-ed-ldp-p02 slapd[3750]: cmp -256, too old Jul 30 21:26:48 iam-ed-ldp-p02 slapd[3750]: cmp -256, too old Jul 30 21:26:48 iam-ed-ldp-p02 slapd[3750]: log csn 20181127062500.766277Z#000000#001#000000 Jul 30 21:26:48 iam-ed-ldp-p02 slapd[3750]: log csn 20181109151410.102670Z#000000#001#000000 Jul 30 21:26:48 iam-ed-ldp-p02 slapd[3750]: log csn 20181127055648.192922Z#000000#001#000000 Jul 30 21:26:48 iam-ed-ldp-p02 slapd[3750]: log csn 20181127061140.091604Z#000000#001#000000 Jul 30 21:26:48 iam-ed-ldp-p02 slapd[3750]: cmp -256, too old Jul 30 21:26:48 iam-ed-ldp-p02 slapd[3750]: cmp -256, too old Jul 30 21:26:48 iam-ed-ldp-p02 slapd[3750]: log csn 20181127062500.859669Z#000000#001#000000 Jul 30 21:26:48 iam-ed-ldp-p02 slapd[3750]: log csn 20181127061140.160929Z#000000#001#000000 Jul 30 21:26:48 iam-ed-ldp-p02 slapd[3750]: cmp -256, too old Jul 30 21:26:48 iam-ed-ldp-p02 slapd[3750]: log csn 20181127122502.664055Z#000000#001#000000 Jul 30 21:26:48 iam-ed-ldp-p02 slapd[3750]: log csn 20181127061140.213051Z#000000#001#000000 Jul 30 21:26:48 iam-ed-ldp-p02 slapd[3750]: cmp -256, too old Jul 30 21:26:48 iam-ed-ldp-p02 slapd[3750]: cmp -256, too old Jul 30 21:26:48 iam-ed-ldp-p02 slapd[3750]: cmp -256, too old Jul 30 21:26:48 iam-ed-ldp-p02 slapd[3750]: log csn 20181127122502.666873Z#000000#001#000000 Jul 30 21:26:48 iam-ed-ldp-p02 slapd[3750]: log csn 20181127062500.892859Z#000000#001#000000 ....

1 0

Re: (ITS#9055) contrib/slapd-modules/passwd/totp improvements
by gv＠members.scinet.supercomputing.org 31 Jul '19

31 Jul '19

On Wed, Jul 31, 2019 at 11:02:55AM -0400, Greg Veldman wrote: > On Wed, Jul 31, 2019 at 03:25:21PM +0100, Howard Chu wrote: > > I've been looking for a way to support the hashing callbacks. Without them, > > this won't be accepted. > > > > Easiest at this point is simply to define a character (maybe space, or tab) > > as a delimiter between seed and password. > > Thanks Howard. I was trying to stay away from that as it would > make it somewhat confusing to use that character, but if you > think it's OK to implement that way I'll give it a shot. I'll > just make sure it's well documented in the manpage as well... v3 of the patch is available, which includes hashing functions and documents the expected input format when using those functions. I don't have the updated module on any of my servers yet, but running slappasswd from my build directory does seem to yield the same results as the non-password versions: $ ../../../../servers/slapd/slappasswd -T passwd -o module-load=`pwd`/.libs/pw-totp.so -h "{TOTP1}" New password: Re-enter new password: {TOTP1}GAYA==== $ ../../../../servers/slapd/slappasswd -T passwd -o module-load=`pwd`/.libs/pw-totp.so -h "{TOTP1ANDPW}" New password: Re-enter new password: {TOTP1ANDPW}GAYA====|{SSHA}Qo6WiIWWsWohlwZSo9oQkImKvSNArGio This is using an OTP seed of 00 and a password of foo https://scinet.supercomputing.org/~gv/slapd-totp-v3.txt -- Greg Veldman

1 0

Re: (ITS#9055) contrib/slapd-modules/passwd/totp improvements
by gv＠members.scinet.supercomputing.org 31 Jul '19

31 Jul '19

On Wed, Jul 31, 2019 at 03:25:21PM +0100, Howard Chu wrote: > I've been looking for a way to support the hashing callbacks. Without them, > this won't be accepted. > > Easiest at this point is simply to define a character (maybe space, or tab) > as a delimiter between seed and password. Thanks Howard. I was trying to stay away from that as it would make it somewhat confusing to use that character, but if you think it's OK to implement that way I'll give it a shot. I'll just make sure it's well documented in the manpage as well... -- Greg Veldman

1 0

Re: (ITS#9055) contrib/slapd-modules/passwd/totp improvements
by hyc＠symas.com 31 Jul '19

31 Jul '19

gv(a)members.scinet.supercomputing.org wrote: > On Fri, Jul 19, 2019 at 04:08:13PM -0400, Greg Veldman wrote: >> Updated patch: >> >> https://scinet.supercomputing.org/~gv/slapd-totp-v2.txt > > I don't mean to push too hard on this because I know everyone's > busy, but it has been almost two weeks. If anyone sees any > outstanding issues with v2 of the patch, I'd be happy to implement > any further changes to get it to an acceptable state. If not, > can this change either be merged or rejected outright so the issue > can be closed out? I've been looking for a way to support the hashing callbacks. Without them, this won't be accepted. Easiest at this point is simply to define a character (maybe space, or tab) as a delimiter between seed and password. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#9055) contrib/slapd-modules/passwd/totp improvements
by gv＠members.scinet.supercomputing.org 31 Jul '19

31 Jul '19

On Fri, Jul 19, 2019 at 04:08:13PM -0400, Greg Veldman wrote: > Updated patch: > > https://scinet.supercomputing.org/~gv/slapd-totp-v2.txt I don't mean to push too hard on this because I know everyone's busy, but it has been almost two weeks. If anyone sees any outstanding issues with v2 of the patch, I'd be happy to implement any further changes to get it to an acceptable state. If not, can this change either be merged or rejected outright so the issue can be closed out? Thanks. -- Greg Veldman

1 0

Re: (ITS#9037) observing crash in mdb_cursor_put()
by vporof＠mozilla.com 31 Jul '19

31 Jul '19

Hey folks. =46rom Myk=E2=80=99s investigations in the previous followup, it seems = that the suggested changes to `mdb_cursor_init` to avoid using an = invalid DBI might not be solving the actual issue, given the behaviour = of `mdb_page_search`. It=E2=80=99s also causing the seemingly correct test program to assert = when it wasn=E2=80=99t before. It=E2=80=99s unclear whether this should = be the case or not, perhaps Howard can confirm the expected behaviour. In any case, we=E2=80=99re wondering if there=E2=80=99s been any other = progress, or if someone managed to reproduce this issue? Shipping new = features built on top of LMDB in Firefox is currently blocked due to = these failures, so any additional info would be helpful. Victor

1 0

(ITS#9058) [PATCH]Fix implicit function declaration
by rosenp＠gmail.com 31 Jul '19

31 Jul '19

Full_Name: Version: OS: URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (76.14.106.55) --- a/libraries/libldap/tls2.c +++ b/libraries/libldap/tls2.c @@ -41,6 +41,7 @@ static tls_impl *tls_imp = &ldap_int_tls_impl; #define HAS_TLS( sb ) ber_sockbuf_ctrl( sb, LBER_SB_OPT_HAS_IO, \ (void *)tls_imp->ti_sbio ) +static int ldap_pvt_tls_check_hostname( LDAP *ld, void *s, const char *name_in ); #endif /* HAVE_TLS */ #ifdef LDAP_DEVEL

1 0

(ITS#9057) Makefile:291: recipe for target 'mdb-yes' failed
by lzy110100＠aliyun.com 30 Jul '19

30 Jul '19

Full_Name: Yon Liu Version: git://git.openldap.org/openldap.git (clone at 20190729) OS: Ubuntu18.04 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (59.37.4.228) info: ubuntu:18.04 openldap: git://git.openldap.org/openldap.git(clone at 20190729) steps: (reference: http://www.openldap.org/software/release/install.html) ./configure make depend make make test issue: (partial log) >>>>> test067-tls completed OK for mdb. >>>>> Starting test068-sasl-tls-external for mdb... running defines.sh Running slapadd to build slapd database... Starting ldap:/// slapd on TCP/IP port 9011 and ldaps:/// slapd on 9012... Using ldapwhoami with SASL/EXTERNAL....ldapwhoami (startTLS) failed (1)! >>>>> test068-sasl-tls-external failed for mdb (exit 1) Makefile:291: recipe for target 'mdb-yes' failed make[2]: *** [mdb-yes] Error 1 make[2]: Leaving directory '/home/bl/openldap/tests' Makefile:279: recipe for target 'test' failed make[1]: *** [test] Error 2 make[1]: Leaving directory '/home/bl/openldap/tests' Makefile:293: recipe for target 'test' failed make: *** [test] Error 2

1 0

Re: (ITS#7657) Alias dereferencing with MDB slow compared with BDB
by hyc＠symas.com 26 Jul '19

26 Jul '19

Mark.Cairney(a)ed.ac.uk wrote: > Hi Howard, Hi Mark, thanks for the feedback. For completeness' sake, the tool I used to generate our test dataset is in http://www.openldap.org/its/index.cgi?findid=8875#followup14 > > Just to confirm that I'm seeing significantly better results with the > 2.4.48 RC release for querying OUs full of aliases and de-referencing them: > > Comparing 2 servers with the same data on them (one 2.4.47, the other > 2.4.48 with the new patch) and just after restarting slapd yield the > following: > > 2.4.48: 0.52s user 0.09s system 46% cpu 1.315 total (with a small spike > in CPU) > > 2.4.47: 0.46s user 0.94s system 0% cpu 3:39.89 total (with the CPU > spiking to 100% on the server for the duration of the query) > > > This is for an OU with 11932 entries in it (all aliasedObjects) > > On 15/07/2019 17:07, Howard Chu wrote: >> Fyi, we recently revisited this issue and are seeing improved results with >> commit e90e8c7d3c12d897bb0584ba04dc519d4f23acf9 in master. >> > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7657) Alias dereferencing with MDB slow compared with BDB
by Mark.Cairney＠ed.ac.uk 26 Jul '19

26 Jul '19

Hi Howard, Just to confirm that I'm seeing significantly better results with the 2.4.48 RC release for querying OUs full of aliases and de-referencing them: Comparing 2 servers with the same data on them (one 2.4.47, the other 2.4.48 with the new patch) and just after restarting slapd yield the following: 2.4.48: 0.52s user 0.09s system 46% cpu 1.315 total (with a small spike in CPU) 2.4.47: 0.46s user 0.94s system 0% cpu 3:39.89 total (with the CPU spiking to 100% on the server for the duration of the query) This is for an OU with 11932 entries in it (all aliasedObjects) On 15/07/2019 17:07, Howard Chu wrote: > Fyi, we recently revisited this issue and are seeing improved results with > commit e90e8c7d3c12d897bb0584ba04dc519d4f23acf9 in master. > -- /**************************** Mark Cairney ITI Enterprise Services Information Services University of Edinburgh Tel: 0131 650 6565 Email: Mark.Cairney(a)ed.ac.uk PGP: 0x435A9621 *******************************/ The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.

1 0

Re: (ITS#9052) ACL protections get lost if same identity uses different SSF levels
by quanah＠symas.com 25 Jul '19

25 Jul '19

--On Wednesday, July 24, 2019 3:45 PM -0700 Quanah Gibson-Mount <quanah(a)symas.com> wrote: > For informational purposes, here's additional detail as the subject and > original problem description do not fully capture the extend of the > problem. In all 2.x releases prior to 2.4.48 (I.e., 2.0.x, 2.1.x, 2.2.x, > 2.3.x, and 2.4.x up to 2.4.47), the SASL security factor layer was set > globally rather than per connection. So once a connection had been made > that sets a SASL SSF, any and all non SASL connections would inherit that > value. Correction -- sasl SSF was set per connection structure. Any new client connection that used the same connection structure as a previous connection would inherit the sasl_ssf value of the prior connection. In slapd, one can generally tell which connection structure is being used by looking at the file descriptor in use by a given connection (stats level logging will display this information, for example). On a busy server where connection structures are routinly being re-used then there is a high probability that this would apply to most connections as long as the majority of connections are setting SASL SSF values. --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

Re: (ITS#9052) ACL protections get lost if same identity uses different SSF levels
by quanah＠symas.com 24 Jul '19

24 Jul '19

For informational purposes, here's additional detail as the subject and original problem description do not fully capture the extend of the problem. In all 2.x releases prior to 2.4.48 (I.e., 2.0.x, 2.1.x, 2.2.x, 2.3.x, and 2.4.x up to 2.4.47), the SASL security factor layer was set globally rather than per connection. So once a connection had been made that sets a SASL SSF, any and all non SASL connections would inherit that value. If ACLs are used to limit access via setting restrictions with the sasl_ssf parameter, connections with no sasl_ssf could match those ACLs incorrectly. For example, access to * by users sasl_ssf=56 read by users tls_ssf=128 read by * none Would allow a user who bound without any encryption full access to the database as long as one SASL connection had been made that had a minimum sasl_ssf value of 56. Another contrived example: access to attrs=userPassword by self sasl_ssf=56 =xw by * auth Would allow a user to change their own password whether or not they had performed a SASL bind with a sasl_ssf of 56. --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

ITS#9052 published: SECURITY: ACL protections get lost if same identity uses different SSF levels
by openldap-its＠OpenLDAP.org 24 Jul '19

24 Jul '19

Private ITS#9052 is now publicly readable URL: http://www.OpenLDAP.org/its/?findid=9052

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs