- openldap-bugs - openldap.org

Re: (ITS#8916) MDB_VL32 in readonly causes MDB_CORRUPTED
by hyc＠symas.com 19 Sep '18

19 Sep '18

pierre.chambart(a)ocamlpro.com wrote: > Full_Name: Pierre Chambart > Version: > OS: > URL: > Submission from: (NULL) (185.194.234.133) > > > We have been trying LMDB compiled with the MDB_VL32 flag and encountered some > MDB_CORRUPTED error that didn't occur without MDB_VL32. > We could produce a minimal reproduction case: > https://raw.githubusercontent.com/chambart/lmdb/vl32_readonly_fix/libraries… > This is basically writing a key, reading it, writing something else then reading > it again. > > After some investigation, this is how I understand what is happening: > > * The chunk being mapped for the first read contains 3 pages instead of the 16 > pages that would be mapped without readonly. > * The second write extend the size of the base to 5 pages > * Finally the last read retrieve the mapped chunk from the first read, which has > size 3, but assumes that it has size 5, hence reading some random value in > memory past the end of the pages. > > It can be verified by asserting that id3.mcnt is larger than rem at the end of > mdb_rpage_get, around this line: > > ok: > p = (MDB_page *)((char *)id3.mptr + rem * env->me_psize); > > This does not seem to be the case. > > I made a dirty fix that seems to work. It's simply to remove those lines: > > /* don't map past last written page in read-only envs */ > if ((env->me_flags & MDB_RDONLY) && pgno + MDB_RPAGE_CHUNK-1 > > txn->mt_last_pgno) > id3.mcnt = txn->mt_last_pgno + 1 - pgno; > else > > This seems to be some optimization that avoids mapping the 16 pages of the > chunk. But given that those pages are > initialized, there doesn't seem to be much harm in mapping a few more pages in > that relatively rare situation. Please check git history before assuming the code is an optimization. These lines prevent a crash on Windows. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#8916) MDB_VL32 in readonly causes MDB_CORRUPTED
by pierre.chambart＠ocamlpro.com 19 Sep '18

19 Sep '18

Full_Name: Pierre Chambart Version: OS: URL: Submission from: (NULL) (185.194.234.133) We have been trying LMDB compiled with the MDB_VL32 flag and encountered some MDB_CORRUPTED error that didn't occur without MDB_VL32. We could produce a minimal reproduction case: https://raw.githubusercontent.com/chambart/lmdb/vl32_readonly_fix/libraries… This is basically writing a key, reading it, writing something else then reading it again. After some investigation, this is how I understand what is happening: * The chunk being mapped for the first read contains 3 pages instead of the 16 pages that would be mapped without readonly. * The second write extend the size of the base to 5 pages * Finally the last read retrieve the mapped chunk from the first read, which has size 3, but assumes that it has size 5, hence reading some random value in memory past the end of the pages. It can be verified by asserting that id3.mcnt is larger than rem at the end of mdb_rpage_get, around this line: ok: p = (MDB_page *)((char *)id3.mptr + rem * env->me_psize); This does not seem to be the case. I made a dirty fix that seems to work. It's simply to remove those lines: /* don't map past last written page in read-only envs */ if ((env->me_flags & MDB_RDONLY) && pgno + MDB_RPAGE_CHUNK-1 > txn->mt_last_pgno) id3.mcnt = txn->mt_last_pgno + 1 - pgno; else This seems to be some optimization that avoids mapping the 16 pages of the chunk. But given that those pages are initialized, there doesn't seem to be much harm in mapping a few more pages in that relatively rare situation. The patch is provided in https://github.com/chambart/lmdb/commit/52e57a392e91935efe6d57337da09f23e40… If you want to reproduce the bug you can use the https://github.com/chambart/lmdb/tree/vl32_readonly_fix which also contains the patch to the makefile to build in 32bits with MDB_VL32. Thanks a lot, hoping that this can be of some help.

1 0

Re: (ITS#8650) EAGAIN from gnutls_handshake not respected
by ryan＠nardis.ca 19 Sep '18

19 Sep '18

Made some good progress on this one this evening. The original issue this ITS is about is that gnutls_handshake() can, in some versions of GnuTLS, return GNUTLS_E_AGAIN even when the socket is blocking. Specifically, this happens in the case I described with a large CA list sent by the server. For slapd, the patch I committed is unfortunately completely wrong. It has been using non-blocking sockets forever, EAGAIN is expected and handled robustly -- or it was, until I introduced the busy-loop. For clients I'm still working on figuring out the right path forward. There is some EAGAIN handling conditional on LDAP_USE_NON_BLOCKING_TLS which itself is behind LDAP_DEVEL. However this code is meant for non-blocking sockets, and in my case it ends up stuck in poll() waiting for a notification that never arrives. In 2.4, ret == 1 simply falls into the success case and proceeds to send data without completing the handshake first. It's possible that what I actually want here is a (ret > 0) case in ldap_int_tls_start for when LDAP_USE_NON_BLOCKING_TLS is absent and ldap_int_tls_connect returns 1. (I'd also need to adapt the non-blocking path to be able to handle a blocking socket as well.) But it's also possible that gnutls_handshake() returning GNUTLS_E_AGAIN with a blocking socket is simply a GnuTLS bug that was introduced at some point. I still need to determine exactly when and why its behaviour changed. (It is still happening with 3.5.19.) In any case, my patch has to be reverted, as its impact (making slapd busy-loop) is obviously worse than the status quo (misbehaving clients in a specific case). I have pushed that revert now and will continue digging as time permits.

1 0

(ITS#8915) OpenLDAP and OpenSSL v1.1.1
by neal.brown＠threesl.com 13 Sep '18

13 Sep '18

Full_Name: Neal Brown Version: 2.4.46 OS: Linux URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (88.98.37.245) OpenLDAP will not configure for building against OpenSSL v1.1.1 due to SSLv3 being disabled.

1 0

(ITS#8914) OpenLDAP and OpenSSL v1.1.1
by nea.brown＠threesl.com 13 Sep '18

13 Sep '18

Full_Name: Neal Brown Version: 2.4.46 OS: Linux URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (88.98.37.245) OpenLDAP will not configure for building against OpenSSL v1.1.1 due to SSLv3 being disabled.

1 0

Re: (ITS#8898) mdb_get on corrupted LMDB file crash the process
by hyc＠symas.com 12 Sep '18

12 Sep '18

Irad K wrote: > Hi Howard and thanks for the info.=C2=A0 >=20 > I've read your insights about the corruption nature and I'd like to see= if I can write some sort of integrity check before I try to open the fil= e.=C2=A0 > Perhaps you could guide me where to look for the file format, where can= I find all the meta-pages and in which offset can I find the used pages = ? Read the source yourself if you want to know the underlying format. Other= wise, just use mdb_env_info(). =C2=A0 > Basically, I wish to check the latest meta-page and see if the number o= f pages correspond with the file actual size.=C2=A0 >=20 > Second, as I've stated, my target operating system is macOS.. are you f= amiliar of any known issues with file atomicity in this particular OS ?=C2= =A0 > Perhaps you could tell me how it's implemented under the hood to achiev= e data consistency if I'm using the database using MDB_NOMETASYNC, so I f= urther > investigate this issue and address it to Apple if necessary.=20 I've seen other reports of corruptions on MacOSX. Since they're in SQLite= I haven't investigated further. http://sqlite.1065341.n5.nabble.com/SQLITE-vs-OSX-mmap-inevitable-catalog= -corruption-td85620.html http://sqlite.1065341.n5.nabble.com/Re-Database-corruption-and-PRAGMA-ful= lfsync-on-macOS-td95366i20.html >=20 > Irad >=20 > On Mon, Sep 10, 2018 at 8:44 PM Howard Chu <hyc(a)symas.com <mailto:hyc@s= ymas.com>> wrote: >=20 > iradization(a)gmail.com <mailto:iradization@gmail.com> wrote: > > Full_Name: Irad.k > > Version: recent from GitHub. > > OS: macOS > > URL: ftp://ftp.openldap.org/incoming/ > > Submission from: (NULL) (82.81.84.130) > > > > > > Hi, > > > > I'm working with LMDB with the following flags :=C2=A0 MDB_NOMETA= SYNC | MDB_NOSUBDIR > > | MDB_NOTLS. > > > > Somehow, even though the DB should be ACI(without the D),=C2=A0 i= t got corrupted > > after recovering from kernel panic, and It crashes my process whe= n trying to > > access one of the records (see crash log below). > > > > Here's a link to the file : > > > > https://drive.google.com/file/d/12Q3KYYrapiJOgiaccnDL3tQACkqrM3C5= /view?usp=3Dsharing >=20 > This file is only 6 pages long, but the latest meta page claims tha= t it's using 11 pages. > Looking at the previous meta page, it claims that 9 pages were used= . >=20 > Clearly your OS failed to write the complete contents out before it= panicked. At the > very least the file should have been 9 pages long. Sounds like you = have an OS bug. >=20 > > According to the crash log from the process, It can clearly be se= en that the > > invalid address reside inside the mapped file region which is the= lmdb mapped > > file, but still I get KERN_MEMORY_ERROR on that address. > > > >>From what I know, an attempt to access address within the mapped = range can > > either retrieve the page contents directly from memory (if it's a= lready there), > > or trigger page fault trap that eventually lead to reading the mi= ssing data from > > disk and return it to process as well. > > > > One thing that raise some concerns is that the file size is only = 24k and the > > mapping spans over 256M. However, the file's meta data seems to b= e coherent to > > file contents. > > > > Any idea how did it happen, and what exactly in the file cause th= is corruption ? > > > > > > CRASH LOG : > > -----------------------------------------------------------------= ----- > > Exception Type:=C2=A0 =C2=A0 =C2=A0 =C2=A0 EXC_BAD_ACCESS (SIGBUS= ) > > Exception Codes:=C2=A0 =C2=A0 =C2=A0 =C2=A0KERN_MEMORY_ERROR at 0= x000000010648800a > > Exception Note:=C2=A0 =C2=A0 =C2=A0 =C2=A0 EXC_CORPSE_NOTIFY > > > > Termination Signal:=C2=A0 =C2=A0 Bus error: 10 > > Termination Reason:=C2=A0 =C2=A0 Namespace SIGNAL, Code 0xa > > Terminating Process:=C2=A0 =C2=A0exc handler [0] > > > > VM Regions Near 0x10648800a: > > > > __LINKEDIT=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00000000= 106464000-000000010647f000 [=C2=A0 108K] r--/rwx SM=3DCOW > >=C2=A0 ^Z^C [/usr/lib/dyld] > > --> mapped file=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 00000001= 0647f000-000000011647f000 [256.0M] r--/rwx > > SM=3DPRV=C2=A0 Object_id=3D9034edd9 > > STACK GUARD=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 000070000f2b= 3000-000070000f2b4000 [=C2=A0 =C2=A0 4K] ---/rwx SM=3DNUL > >=C2=A0 stack guard for thread 1 > > > > Thread 0 Crashed:: Dispatch queue: com.apple.main-thread > > 0=C2=A0 =C2=A0myprog=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00x0000000101666756 mdb_page_search_root + = 39 > > 1=C2=A0 =C2=A0myprog=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00x00000001016660f7 mdb_page_search + 182 > > 2=C2=A0 =C2=A0myprog=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00x00000001016614de mdb_cursor_set + 88 > > 3=C2=A0 =C2=A0myprog=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00x0000000101661476 mdb_get + 134 > > > > > > >=20 >=20 > --=20 > =C2=A0 -- Howard Chu > =C2=A0 CTO, Symas Corp.=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0htt= p://www.symas.com > =C2=A0 Director, Highland Sun=C2=A0 =C2=A0 =C2=A0http://highlandsun= .com/hyc/ > =C2=A0 Chief Architect, OpenLDAP=C2=A0 http://www.openldap.org/proj= ect/ >=20 --=20 -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8898) mdb_get on corrupted LMDB file crash the process
by iradization＠gmail.com 12 Sep '18

12 Sep '18

--000000000000d405310575a6de5e Content-Type: text/plain; charset="UTF-8" Hi Howard and thanks for the info. I've read your insights about the corruption nature and I'd like to see if I can write some sort of integrity check before I try to open the file. Perhaps you could guide me where to look for the file format, where can I find all the meta-pages and in which offset can I find the used pages ? Basically, I wish to check the latest meta-page and see if the number of pages correspond with the file actual size. Second, as I've stated, my target operating system is macOS.. are you familiar of any known issues with file atomicity in this particular OS ? Perhaps you could tell me how it's implemented under the hood to achieve data consistency if I'm using the database using MDB_NOMETASYNC, so I further investigate this issue and address it to Apple if necessary. Irad On Mon, Sep 10, 2018 at 8:44 PM Howard Chu <hyc(a)symas.com> wrote: > iradization(a)gmail.com wrote: > > Full_Name: Irad.k > > Version: recent from GitHub. > > OS: macOS > > URL: ftp://ftp.openldap.org/incoming/ > > Submission from: (NULL) (82.81.84.130) > > > > > > Hi, > > > > I'm working with LMDB with the following flags : MDB_NOMETASYNC | > MDB_NOSUBDIR > > | MDB_NOTLS. > > > > Somehow, even though the DB should be ACI(without the D), it got > corrupted > > after recovering from kernel panic, and It crashes my process when > trying to > > access one of the records (see crash log below). > > > > Here's a link to the file : > > > > > https://drive.google.com/file/d/12Q3KYYrapiJOgiaccnDL3tQACkqrM3C5/view?usp=… > > This file is only 6 pages long, but the latest meta page claims that it's > using 11 pages. > Looking at the previous meta page, it claims that 9 pages were used. > > Clearly your OS failed to write the complete contents out before it > panicked. At the > very least the file should have been 9 pages long. Sounds like you have an > OS bug. > > > According to the crash log from the process, It can clearly be seen that > the > > invalid address reside inside the mapped file region which is the lmdb > mapped > > file, but still I get KERN_MEMORY_ERROR on that address. > > > >>From what I know, an attempt to access address within the mapped range > can > > either retrieve the page contents directly from memory (if it's already > there), > > or trigger page fault trap that eventually lead to reading the missing > data from > > disk and return it to process as well. > > > > One thing that raise some concerns is that the file size is only 24k and > the > > mapping spans over 256M. However, the file's meta data seems to be > coherent to > > file contents. > > > > Any idea how did it happen, and what exactly in the file cause this > corruption ? > > > > > > CRASH LOG : > > ---------------------------------------------------------------------- > > Exception Type: EXC_BAD_ACCESS (SIGBUS) > > Exception Codes: KERN_MEMORY_ERROR at 0x000000010648800a > > Exception Note: EXC_CORPSE_NOTIFY > > > > Termination Signal: Bus error: 10 > > Termination Reason: Namespace SIGNAL, Code 0xa > > Terminating Process: exc handler [0] > > > > VM Regions Near 0x10648800a: > > > > __LINKEDIT 0000000106464000-000000010647f000 [ 108K] > r--/rwx SM=COW > > ^Z^C [/usr/lib/dyld] > > --> mapped file 000000010647f000-000000011647f000 [256.0M] > r--/rwx > > SM=PRV Object_id=9034edd9 > > STACK GUARD 000070000f2b3000-000070000f2b4000 [ 4K] > ---/rwx SM=NUL > > stack guard for thread 1 > > > > Thread 0 Crashed:: Dispatch queue: com.apple.main-thread > > 0 myprog 0x0000000101666756 mdb_page_search_root + > 39 > > 1 myprog 0x00000001016660f7 mdb_page_search + 182 > > 2 myprog 0x00000001016614de mdb_cursor_set + 88 > > 3 myprog 0x0000000101661476 mdb_get + 134 > > > > > > > > > -- > -- Howard Chu > CTO, Symas Corp. http://www.symas.com > Director, Highland Sun http://highlandsun.com/hyc/ > Chief Architect, OpenLDAP http://www.openldap.org/project/ > --000000000000d405310575a6de5e Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div dir=3D"ltr"><div dir=3D"ltr">Hi Howard and thanks for= the info.=C2=A0<div><br><div>I've read your insights about the corrupt= ion nature and I'd like to see if I can write some sort of integrity ch= eck before I try to open the file.=C2=A0</div><div>Perhaps you could guide = me where to look for the file format, where can I find all the meta-pages a= nd in which offset can I find the used pages ?=C2=A0</div><div>Basically, I= wish to check the latest meta-page and see if the number of pages correspo= nd with the file actual size.=C2=A0<br></div><div><br></div><div>Second, as= I've stated, my target operating system is macOS.. are you familiar of= any known issues with file atomicity in this particular OS ?=C2=A0</div></= div><div><div>Perhaps you could tell me how it's implemented under the = hood to achieve data consistency if I'm using the database using MDB_NO= METASYNC, so I further investigate this issue and address it to Apple if ne= cessary.=C2=A0</div></div><div><br></div><div>Irad</div></div></div></div><= br><div class=3D"gmail_quote"><div dir=3D"ltr">On Mon, Sep 10, 2018 at 8:44= PM Howard Chu <<a href=3D"mailto:hyc@symas.com">hyc(a)symas.com</a>> w= rote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex= ;border-left:1px #ccc solid;padding-left:1ex"><a href=3D"mailto:iradization= @gmail.com" target=3D"_blank">iradization(a)gmail.com</a> wrote:<br> > Full_Name: Irad.k<br> > Version: recent from GitHub.<br> > OS: macOS<br> > URL: <a href=3D"ftp://ftp.openldap.org/incoming/" rel=3D"noreferrer" t= arget=3D"_blank">ftp://ftp.openldap.org/incoming/</a><br> > Submission from: (NULL) (82.81.84.130)<br> > <br> > <br> > Hi, <br> > <br> > I'm working with LMDB with the following flags :=C2=A0 MDB_NOMETAS= YNC | MDB_NOSUBDIR<br> > | MDB_NOTLS.<br> > <br> > Somehow, even though the DB should be ACI(without the D),=C2=A0 it got= corrupted<br> > after recovering from kernel panic, and It crashes my process when try= ing to<br> > access one of the records (see crash log below). <br> > <br> > Here's a link to the file : <br> > <br> > <a href=3D"https://drive.google.com/file/d/12Q3KYYrapiJOgiaccnDL3tQACk= qrM3C5/view?usp=3Dsharing" rel=3D"noreferrer" target=3D"_blank">https://dri= ve.google.com/file/d/12Q3KYYrapiJOgiaccnDL3tQACkqrM3C5/view?usp=3Dsharing</= a><br> <br> This file is only 6 pages long, but the latest meta page claims that it&#39= ;s using 11 pages.<br> Looking at the previous meta page, it claims that 9 pages were used.<br> <br> Clearly your OS failed to write the complete contents out before it panicke= d. At the<br> very least the file should have been 9 pages long. Sounds like you have an = OS bug.<br> <br> > According to the crash log from the process, It can clearly be seen th= at the<br> > invalid address reside inside the mapped file region which is the lmdb= mapped<br> > file, but still I get KERN_MEMORY_ERROR on that address.<br> > <br> >>From what I know, an attempt to access address within the mapped ra= nge can<br> > either retrieve the page contents directly from memory (if it's al= ready there),<br> > or trigger page fault trap that eventually lead to reading the missing= data from<br> > disk and return it to process as well.<br> > <br> > One thing that raise some concerns is that the file size is only 24k a= nd the<br> > mapping spans over 256M. However, the file's meta data seems to be= coherent to<br> > file contents.<br> > <br> > Any idea how did it happen, and what exactly in the file cause this co= rruption ?<br> > <br> > <br> > CRASH LOG :<br> > ----------------------------------------------------------------------= <br> > Exception Type:=C2=A0 =C2=A0 =C2=A0 =C2=A0 EXC_BAD_ACCESS (SIGBUS)<br> > Exception Codes:=C2=A0 =C2=A0 =C2=A0 =C2=A0KERN_MEMORY_ERROR at 0x0000= 00010648800a<br> > Exception Note:=C2=A0 =C2=A0 =C2=A0 =C2=A0 EXC_CORPSE_NOTIFY<br> > <br> > Termination Signal:=C2=A0 =C2=A0 Bus error: 10<br> > Termination Reason:=C2=A0 =C2=A0 Namespace SIGNAL, Code 0xa<br> > Terminating Process:=C2=A0 =C2=A0exc handler [0]<br> > <br> > VM Regions Near 0x10648800a:<br> > <br> > __LINKEDIT=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0000000010646= 4000-000000010647f000 [=C2=A0 108K] r--/rwx SM=3DCOW<br> >=C2=A0 ^Z^C [/usr/lib/dyld]<br> > --> mapped file=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 0000000106= 47f000-000000011647f000 [256.0M] r--/rwx<br> > SM=3DPRV=C2=A0 Object_id=3D9034edd9<br> > STACK GUARD=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 000070000f2b3000-= 000070000f2b4000 [=C2=A0 =C2=A0 4K] ---/rwx SM=3DNUL<br> >=C2=A0 stack guard for thread 1<br> > <br> > Thread 0 Crashed:: Dispatch queue: com.apple.main-thread<br> > 0=C2=A0 =C2=A0myprog=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A00x0000000101666756 mdb_page_search_root + 39<br> > 1=C2=A0 =C2=A0myprog=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A00x00000001016660f7 mdb_page_search + 182<br> > 2=C2=A0 =C2=A0myprog=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A00x00000001016614de mdb_cursor_set + 88<br> > 3=C2=A0 =C2=A0myprog=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A00x0000000101661476 mdb_get + 134<br> > <br> > <br> > <br> <br> <br> -- <br> =C2=A0 -- Howard Chu<br> =C2=A0 CTO, Symas Corp.=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0<a href=3D"= http://www.symas.com" rel=3D"noreferrer" target=3D"_blank">http://www.symas= .com</a><br> =C2=A0 Director, Highland Sun=C2=A0 =C2=A0 =C2=A0<a href=3D"http://highland= sun.com/hyc/" rel=3D"noreferrer" target=3D"_blank">http://highlandsun.com/h= yc/</a><br> =C2=A0 Chief Architect, OpenLDAP=C2=A0 <a href=3D"http://www.openldap.org/p= roject/" rel=3D"noreferrer" target=3D"_blank">http://www.openldap.org/proje= ct/</a><br> </blockquote></div> --000000000000d405310575a6de5e--

1 0

Re: (ITS#8898) mdb_get on corrupted LMDB file crash the process
by hyc＠symas.com 10 Sep '18

10 Sep '18

iradization(a)gmail.com wrote: > Full_Name: Irad.k > Version: recent from GitHub. > OS: macOS > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (82.81.84.130) > > > Hi, > > I'm working with LMDB with the following flags : MDB_NOMETASYNC | MDB_NOSUBDIR > | MDB_NOTLS. > > Somehow, even though the DB should be ACI(without the D), it got corrupted > after recovering from kernel panic, and It crashes my process when trying to > access one of the records (see crash log below). > > Here's a link to the file : > > https://drive.google.com/file/d/12Q3KYYrapiJOgiaccnDL3tQACkqrM3C5/view?usp=… This file is only 6 pages long, but the latest meta page claims that it's using 11 pages. Looking at the previous meta page, it claims that 9 pages were used. Clearly your OS failed to write the complete contents out before it panicked. At the very least the file should have been 9 pages long. Sounds like you have an OS bug. > According to the crash log from the process, It can clearly be seen that the > invalid address reside inside the mapped file region which is the lmdb mapped > file, but still I get KERN_MEMORY_ERROR on that address. > >>From what I know, an attempt to access address within the mapped range can > either retrieve the page contents directly from memory (if it's already there), > or trigger page fault trap that eventually lead to reading the missing data from > disk and return it to process as well. > > One thing that raise some concerns is that the file size is only 24k and the > mapping spans over 256M. However, the file's meta data seems to be coherent to > file contents. > > Any idea how did it happen, and what exactly in the file cause this corruption ? > > > CRASH LOG : > ---------------------------------------------------------------------- > Exception Type: EXC_BAD_ACCESS (SIGBUS) > Exception Codes: KERN_MEMORY_ERROR at 0x000000010648800a > Exception Note: EXC_CORPSE_NOTIFY > > Termination Signal: Bus error: 10 > Termination Reason: Namespace SIGNAL, Code 0xa > Terminating Process: exc handler [0] > > VM Regions Near 0x10648800a: > > __LINKEDIT 0000000106464000-000000010647f000 [ 108K] r--/rwx SM=COW > ^Z^C [/usr/lib/dyld] > --> mapped file 000000010647f000-000000011647f000 [256.0M] r--/rwx > SM=PRV Object_id=9034edd9 > STACK GUARD 000070000f2b3000-000070000f2b4000 [ 4K] ---/rwx SM=NUL > stack guard for thread 1 > > Thread 0 Crashed:: Dispatch queue: com.apple.main-thread > 0 myprog 0x0000000101666756 mdb_page_search_root + 39 > 1 myprog 0x00000001016660f7 mdb_page_search + 182 > 2 myprog 0x00000001016614de mdb_cursor_set + 88 > 3 myprog 0x0000000101661476 mdb_get + 134 > > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8901) slapd segfaults in mdb_env_reader_dest
by hyc＠symas.com 10 Sep '18

10 Sep '18

ckowalczyk(a)suse.com wrote: > Full_Name: Chris Kowalczyk > Version: 2.4.41 > OS: SLE12 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (2620:113:80c0:5::2222) > > > Hello, > I have been experiencing segfaults in mdb_env_reader_dest happening randomly, > only from time to time, during shutting off openldap. > > After some investigation I narrowed the problem down to mdb_env_reader_dest > destructor function being called after its object had been deleted while > powering off the application. mdb_env_reader_dest was called automatically, > because it was agginned to the object in pthread_key_create call: > https://github.com/openldap/openldap/blob/master/libraries/liblmdb/mdb.c#L4… > > The problem was not happening often, and always during powering off, so no data > was lost etc. However, due to its annoyance, I prepared a simple patch, which > prevents mdb_env_reader_dest function being called if the object was already > deleted. The patch works fine, I haven't seen the problem since it was > introduced. > > I am using OpenLDAP 2.4.41, but the code seems to be the same in other versions > as well. Would you like me to create a pull request for it? We've discussed this extensively in private emails. There is no documented reason why the provided patch would have the claimed effect. Deferring this until a valid explanation for why this patch is correct can be shown. > > ========================== > === The backtrace: > ========================== > > Program terminated with signal 11, Segmentation fault. > #0 mdb_env_reader_dest (ptr=0x7f4a9c71c080) at > ../../../libraries/liblmdb/mdb.c:4050 > 4050 reader->mr_pid = 0; > > #thread apply all bt > > Thread 2 (LWP 20920): > #0 0x00007f4a9ac31d9d in close () at ../sysdeps/unix/syscall-template.S:84 > #1 0x000055902eaeae8c in mdb_env_close1 (env=env@entry=0x55902fce50e0) > at ../../../libraries/liblmdb/mdb.c:4756 > #2 0x000055902eaeb578 in mdb_env_close1 (env=0x55902fce50e0) at > ../../../libraries/liblmdb/mdb.c:4779 > #3 mdb_env_close (env=0x55902fce50e0) at ../../../libraries/liblmdb/mdb.c:4778 > #4 0x000055902ea69114 in mdb_db_close (be=0x55902faeb300, cr=<optimized out>) > at init.c:340 > #5 0x000055902ea3815b in over_db_close (be=0x55902faeb300, cr=0x0) at > backover.c:182 > #6 0x000055902e9d8c0b in backend_shutdown (be=0x55902faeb300, be@entry=0x0) at > backend.c:376 > #7 0x000055902e9fa258 in slap_shutdown (be=be@entry=0x0) at init.c:232 > #8 0x000055902e9af12f in main (argc=<optimized out>, argv=<optimized out>) at > main.c:1027 > > Thread 1 (LWP 20928): > #0 mdb_env_reader_dest (ptr=0x7f4a9c71c080) at > ../../../libraries/liblmdb/mdb.c:4050 > #1 0x00007f4a9ac2a512 in __nptl_deallocate_tsd () at pthread_create.c:176 > #2 0x00007f4a9ac2a767 in start_thread (arg=0x7f4a97acc700) at > pthread_create.c:347 > #3 0x00007f4a9a968aad in clone () at > ../sysdeps/unix/sysv/linux/x86_64/clone.S:109 > > # frame 0 > #0 mdb_env_reader_dest (ptr=0x7f4a9c71c080) at > ../../../libraries/liblmdb/mdb.c:4050 > 4050 reader->mr_pid = 0; > #p reader > $1 = (MDB_reader *) 0x7f4a9c71c080 > #p *reader > Cannot access memory at address 0x7f4a9c71c080 > > ========================== > === The proposed patch: > ========================== > > diff --git a/libraries/liblmdb/mdb.c b/libraries/liblmdb/mdb.c > index 6bdf3151d..56212151b 100644 > --- a/libraries/liblmdb/mdb.c > +++ b/libraries/liblmdb/mdb.c > @@ -4692,6 +4692,11 @@ mdb_env_close0(MDB_env *env, int excl) > > if (env->me_flags & MDB_ENV_TXKEY) { > pthread_key_delete(env->me_txkey); > + > + // No need to call desctructor anymore, as all pid > + // values are cleared below. > + env->me_txkey = NULL; > + > #ifdef _WIN32 > /* Delete our key from the global list */ > for (i=0; i<mdb_tls_nkeys; i++) > > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8913) Add meson build system to lmdb
by nacho.resa＠gmail.com 05 Sep '18

05 Sep '18

--0000000000006c1a190575262da9 Content-Type: text/plain; charset="UTF-8" not super important since it is anyway quite fast but takes half of the time running meson + ninja than running make -j6... :) real 0m3.070s user 0m7.031s sys 0m0.493s vs real 0m1.501s user 0m2.138s sys 0m0.446s In any case it is not the end of the world to keep a fork for gvsbuild and maybe fedora, at the end of the day this library does not change that often to make the build system painful to maintain out of upstream. On Wed, Sep 5, 2018 at 7:15 PM Howard Chu <hyc(a)symas.com> wrote: > Ignacio Casal Quinteiro wrote: > > FWIW any chance to reconsider having both build systems? > > I have no desire to support multiple build systems. The Project does not > support > any tools that we don't use ourselves. > > The files you add significantly increase the line count, and don't even > contain any comments. > In contrast, the current Makefile is mostly comments. > > > with meson it is easy to build on windows. Plus I'd like you > > to see at these numbers: > > > > time make: > > real 0m5.629s > > user 0m5.197s > > sys 0m0.399s > > > > time meson build > > real 0m0.452s > > user 0m0.335s > > sys 0m0.131s > > > > time ninja -C build > > real 0m0.992s > > user 0m1.812s > > sys 0m0.362s > > > > You can see that configuring and building with meson is faster than just > using make > > On my system the majority of build time is in gcc itself. The proportion > of time that > make itself uses is trivial. I would guess the only reason you're seeing a > speedup is > because meson parallelizes the build. You would get the same effect using > "make -jXX". > > > > > On Wed, Sep 5, 2018 at 6:31 PM Ignacio Casal Quinteiro < > nacho.resa(a)gmail.com <mailto:nacho.resa@gmail.com>> wrote: > > > > yeah... knowing my previous tries with lmdb I did not have much > hope... > > > > On Wed, Sep 5, 2018 at 6:21 PM Howard Chu <hyc(a)symas.com <mailto: > hyc(a)symas.com>> wrote: > > > > nacho.resa(a)gmail.com <mailto:nacho.resa@gmail.com> wrote: > > > Full_Name: Ignacio Casal Quinteiro > > > Version: > > > OS: > > > URL: ftp://ftp.openldap.org/incoming/ > > > Submission from: (NULL) (54.239.6.177) > > > > > > > > > Meson is the new thing out there, > > > > No. > > -- > -- Howard Chu > CTO, Symas Corp. http://www.symas.com > Director, Highland Sun http://highlandsun.com/hyc/ > Chief Architect, OpenLDAP http://www.openldap.org/project/ > -- Ignacio Casal Quinteiro --0000000000006c1a190575262da9 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div dir=3D"ltr"><div dir=3D"ltr"><div>not super important= since it is anyway quite fast but takes half of the time running meson + n= inja than running make -j6... :)</div><div>real=C2=A0=C2=A0=C2=A0 0m3.070s<= br>user=C2=A0=C2=A0=C2=A0 0m7.031s<br>sys=C2=A0=C2=A0=C2=A0 0m0.493s<br></d= iv><div><br></div><div>vs</div><div><br></div><div>real=C2=A0=C2=A0=C2=A0 0= m1.501s<br>user=C2=A0=C2=A0=C2=A0 0m2.138s<br>sys=C2=A0=C2=A0=C2=A0 0m0.446= s<br></div><div><br></div><div>In any case it is not the end of the world t= o keep a fork for gvsbuild and maybe fedora, at the end of the day this lib= rary</div><div>does not change that often to make the build system painful = to maintain out of upstream.<br></div></div></div></div><br><div class=3D"g= mail_quote"><div dir=3D"ltr">On Wed, Sep 5, 2018 at 7:15 PM Howard Chu <= <a href=3D"mailto:hyc@symas.com">hyc(a)symas.com</a>> wrote:<br></div><blo= ckquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #c= cc solid;padding-left:1ex">Ignacio Casal Quinteiro wrote:<br> > FWIW any chance to reconsider having both build systems?<br> <br> I have no desire to support multiple build systems. The Project does not su= pport<br> any tools that we don't use ourselves.<br> <br> The files you add significantly increase the line count, and don't even= contain any comments.<br> In contrast, the current Makefile is mostly comments.<br> <br> > with meson it is easy to build on windows. Plus I'd like you<br> > to see at these numbers:<br> > <br> > time make:<br> > real=C2=A0=C2=A0=C2=A0 0m5.629s<br> > user=C2=A0=C2=A0=C2=A0 0m5.197s<br> > sys=C2=A0=C2=A0=C2=A0 0m0.399s<br> > <br> > time meson build<br> > real=C2=A0=C2=A0=C2=A0 0m0.452s<br> > user=C2=A0=C2=A0=C2=A0 0m0.335s<br> > sys=C2=A0=C2=A0=C2=A0 0m0.131s<br> > <br> > time ninja -C build<br> > real=C2=A0=C2=A0=C2=A0 0m0.992s<br> > user=C2=A0=C2=A0=C2=A0 0m1.812s<br> > sys=C2=A0=C2=A0=C2=A0 0m0.362s<br> > <br> > You can see that configuring and building with meson is faster than ju= st using make<br> <br> On my system the majority of build time is in gcc itself. The proportion of= time that<br> make itself uses is trivial. I would guess the only reason you're seein= g a speedup is<br> because meson parallelizes the build. You would get the same effect using &= quot;make -jXX".<br> <br> > <br> > On Wed, Sep 5, 2018 at 6:31 PM Ignacio Casal Quinteiro <<a href=3D"= mailto:nacho.resa@gmail.com" target=3D"_blank">nacho.resa(a)gmail.com</a> &lt= ;mailto:<a href=3D"mailto:nacho.resa@gmail.com" target=3D"_blank">nacho.res= a(a)gmail.com</a>>> wrote:<br> > <br> >=C2=A0 =C2=A0 =C2=A0yeah... knowing my previous tries with lmdb I did n= ot have much hope...<br> > <br> >=C2=A0 =C2=A0 =C2=A0On Wed, Sep 5, 2018 at 6:21 PM Howard Chu <<a hr= ef=3D"mailto:hyc@symas.com" target=3D"_blank">hyc(a)symas.com</a> <mailto:= <a href=3D"mailto:hyc@symas.com" target=3D"_blank">hyc(a)symas.com</a>>&gt= ; wrote:<br> > <br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0<a href=3D"mailto:nacho.resa@gmail.co= m" target=3D"_blank">nacho.resa(a)gmail.com</a> <mailto:<a href=3D"mailto:= nacho.resa(a)gmail.com" target=3D"_blank">nacho.resa(a)gmail.com</a>> wrote:= <br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0> Full_Name: Ignacio Casal Quintei= ro<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0> Version:<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0> OS:<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0> URL: <a href=3D"ftp://ftp.openld= ap.org/incoming/" rel=3D"noreferrer" target=3D"_blank">ftp://ftp.openldap.o= rg/incoming/</a><br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0> Submission from: (NULL) (54.239.= 6.177)<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0><br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0><br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0> Meson is the new thing out there= ,<br> > <br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0No.<br> <br> -- <br> =C2=A0 -- Howard Chu<br> =C2=A0 CTO, Symas Corp.=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0<a href=3D"= http://www.symas.com" rel=3D"noreferrer" target=3D"_blank">http://www.symas= .com</a><br> =C2=A0 Director, Highland Sun=C2=A0 =C2=A0 =C2=A0<a href=3D"http://highland= sun.com/hyc/" rel=3D"noreferrer" target=3D"_blank">http://highlandsun.com/h= yc/</a><br> =C2=A0 Chief Architect, OpenLDAP=C2=A0 <a href=3D"http://www.openldap.org/p= roject/" rel=3D"noreferrer" target=3D"_blank">http://www.openldap.org/proje= ct/</a><br> </blockquote></div><br clear=3D"all"><br>-- <br><div dir=3D"ltr" class=3D"g= mail_signature" data-smartmail=3D"gmail_signature">Ignacio Casal Quinteiro<= /div> --0000000000006c1a190575262da9--

1 0

Re: (ITS#8913) Add meson build system to lmdb
by hyc＠symas.com 05 Sep '18

05 Sep '18

Ignacio Casal Quinteiro wrote: > FWIW any chance to reconsider having both build systems? I have no desire to support multiple build systems. The Project does not = support any tools that we don't use ourselves. The files you add significantly increase the line count, and don't even c= ontain any comments. In contrast, the current Makefile is mostly comments. > with meson it is easy to build on windows. Plus I'd like you > to see at these numbers: >=20 > time make: > real=C2=A0=C2=A0=C2=A0 0m5.629s > user=C2=A0=C2=A0=C2=A0 0m5.197s > sys=C2=A0=C2=A0=C2=A0 0m0.399s >=20 > time meson build > real=C2=A0=C2=A0=C2=A0 0m0.452s > user=C2=A0=C2=A0=C2=A0 0m0.335s > sys=C2=A0=C2=A0=C2=A0 0m0.131s >=20 > time ninja -C build > real=C2=A0=C2=A0=C2=A0 0m0.992s > user=C2=A0=C2=A0=C2=A0 0m1.812s > sys=C2=A0=C2=A0=C2=A0 0m0.362s >=20 > You can see that configuring and building with meson is faster than jus= t using make On my system the majority of build time is in gcc itself. The proportion = of time that make itself uses is trivial. I would guess the only reason you're seeing = a speedup is because meson parallelizes the build. You would get the same effect using= "make -jXX". >=20 > On Wed, Sep 5, 2018 at 6:31 PM Ignacio Casal Quinteiro <nacho.resa@gmai= l.com <mailto:nacho.resa@gmail.com>> wrote: >=20 > yeah... knowing my previous tries with lmdb I did not have much hop= e... >=20 > On Wed, Sep 5, 2018 at 6:21 PM Howard Chu <hyc(a)symas.com <mailto:hy= c(a)symas.com>> wrote: >=20 > nacho.resa(a)gmail.com <mailto:nacho.resa@gmail.com> wrote: > > Full_Name: Ignacio Casal Quinteiro > > Version: > > OS: > > URL: ftp://ftp.openldap.org/incoming/ > > Submission from: (NULL) (54.239.6.177) > > > > > > Meson is the new thing out there, >=20 > No. --=20 -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8913) Add meson build system to lmdb
by nacho.resa＠gmail.com 05 Sep '18

05 Sep '18

--0000000000007aaf1005752283b9 Content-Type: text/plain; charset="UTF-8" FWIW any chance to reconsider having both build systems? with meson it is easy to build on windows. Plus I'd like you to see at these numbers: time make: real 0m5.629s user 0m5.197s sys 0m0.399s time meson build real 0m0.452s user 0m0.335s sys 0m0.131s time ninja -C build real 0m0.992s user 0m1.812s sys 0m0.362s You can see that configuring and building with meson is faster than just using make On Wed, Sep 5, 2018 at 6:31 PM Ignacio Casal Quinteiro <nacho.resa(a)gmail.com> wrote: > yeah... knowing my previous tries with lmdb I did not have much hope... > > On Wed, Sep 5, 2018 at 6:21 PM Howard Chu <hyc(a)symas.com> wrote: > >> nacho.resa(a)gmail.com wrote: >> > Full_Name: Ignacio Casal Quinteiro >> > Version: >> > OS: >> > URL: ftp://ftp.openldap.org/incoming/ >> > Submission from: (NULL) (54.239.6.177) >> > >> > >> > Meson is the new thing out there, >> >> No. >> >> >> -- >> -- Howard Chu >> CTO, Symas Corp. http://www.symas.com >> Director, Highland Sun http://highlandsun.com/hyc/ >> Chief Architect, OpenLDAP http://www.openldap.org/project/ >> > > > -- > Ignacio Casal Quinteiro > -- Ignacio Casal Quinteiro --0000000000007aaf1005752283b9 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div dir=3D"ltr"><div dir=3D"ltr"><div dir=3D"ltr"><div di= r=3D"ltr"><div dir=3D"ltr"><div dir=3D"ltr"><div>FWIW any chance to reconsi= der having both build systems?</div><div>with meson it is easy to build on = windows. Plus I'd like you</div><div>to see at these numbers:</div><div= ><br></div><div>time make:</div><div>real=C2=A0=C2=A0=C2=A0 0m5.629s<br>use= r=C2=A0=C2=A0=C2=A0 0m5.197s<br>sys=C2=A0=C2=A0=C2=A0 0m0.399s<br></div><di= v><br></div><div>time meson build</div><div>real=C2=A0=C2=A0=C2=A0 0m0.452s= <br>user=C2=A0=C2=A0=C2=A0 0m0.335s<br>sys=C2=A0=C2=A0=C2=A0 0m0.131s<br></= div><div><br></div><div>time ninja -C build</div><div>real=C2=A0=C2=A0=C2= =A0 0m0.992s<br>user=C2=A0=C2=A0=C2=A0 0m1.812s<br>sys=C2=A0=C2=A0=C2=A0 0m= 0.362s<br></div><div><br></div><div>You can see that configuring and buildi= ng with meson is faster than just using make<br></div></div></div></div></d= iv></div></div></div><br><div class=3D"gmail_quote"><div dir=3D"ltr">On Wed= , Sep 5, 2018 at 6:31 PM Ignacio Casal Quinteiro <<a href=3D"mailto:nach= o.resa(a)gmail.com">nacho.resa(a)gmail.com</a>> wrote:<br></div><blockquote = class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid= ;padding-left:1ex"><div dir=3D"ltr">yeah... knowing my previous tries with = lmdb I did not have much hope...<br></div><br><div class=3D"gmail_quote"><d= iv dir=3D"ltr">On Wed, Sep 5, 2018 at 6:21 PM Howard Chu <<a href=3D"mai= lto:hyc@symas.com" target=3D"_blank">hyc(a)symas.com</a>> wrote:<br></div>= <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"><a href=3D"mailto:nacho.resa@gmail.com" targ= et=3D"_blank">nacho.resa(a)gmail.com</a> wrote:<br> > Full_Name: Ignacio Casal Quinteiro<br> > Version: <br> > OS: <br> > URL: <a href=3D"ftp://ftp.openldap.org/incoming/" rel=3D"noreferrer" t= arget=3D"_blank">ftp://ftp.openldap.org/incoming/</a><br> > Submission from: (NULL) (54.239.6.177)<br> > <br> > <br> > Meson is the new thing out there,<br> <br> No.<br> <br> <br> -- <br> =C2=A0 -- Howard Chu<br> =C2=A0 CTO, Symas Corp.=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0<a href=3D"= http://www.symas.com" rel=3D"noreferrer" target=3D"_blank">http://www.symas= .com</a><br> =C2=A0 Director, Highland Sun=C2=A0 =C2=A0 =C2=A0<a href=3D"http://highland= sun.com/hyc/" rel=3D"noreferrer" target=3D"_blank">http://highlandsun.com/h= yc/</a><br> =C2=A0 Chief Architect, OpenLDAP=C2=A0 <a href=3D"http://www.openldap.org/p= roject/" rel=3D"noreferrer" target=3D"_blank">http://www.openldap.org/proje= ct/</a><br> </blockquote></div><br clear=3D"all"><br>-- <br><div dir=3D"ltr" class=3D"m= _-8944057145512108354gmail_signature" data-smartmail=3D"gmail_signature">Ig= nacio Casal Quinteiro</div> </blockquote></div><br clear=3D"all"><br>-- <br><div dir=3D"ltr" class=3D"g= mail_signature" data-smartmail=3D"gmail_signature">Ignacio Casal Quinteiro<= /div> --0000000000007aaf1005752283b9--

1 0

Re: (ITS#8913) Add meson build system to lmdb
by nacho.resa＠gmail.com 05 Sep '18

05 Sep '18

--000000000000a46a9a05752250a1 Content-Type: text/plain; charset="UTF-8" yeah... knowing my previous tries with lmdb I did not have much hope... On Wed, Sep 5, 2018 at 6:21 PM Howard Chu <hyc(a)symas.com> wrote: > nacho.resa(a)gmail.com wrote: > > Full_Name: Ignacio Casal Quinteiro > > Version: > > OS: > > URL: ftp://ftp.openldap.org/incoming/ > > Submission from: (NULL) (54.239.6.177) > > > > > > Meson is the new thing out there, > > No. > > > -- > -- Howard Chu > CTO, Symas Corp. http://www.symas.com > Director, Highland Sun http://highlandsun.com/hyc/ > Chief Architect, OpenLDAP http://www.openldap.org/project/ > -- Ignacio Casal Quinteiro --000000000000a46a9a05752250a1 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr">yeah... knowing my previous tries with lmdb I did not have= much hope...<br></div><br><div class=3D"gmail_quote"><div dir=3D"ltr">On W= ed, Sep 5, 2018 at 6:21 PM Howard Chu <<a href=3D"mailto:hyc@symas.com">= hyc(a)symas.com</a>> wrote:<br></div><blockquote class=3D"gmail_quote" sty= le=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><a hre= f=3D"mailto:nacho.resa@gmail.com" target=3D"_blank">nacho.resa(a)gmail.com</a= > wrote:<br> > Full_Name: Ignacio Casal Quinteiro<br> > Version: <br> > OS: <br> > URL: <a href=3D"ftp://ftp.openldap.org/incoming/" rel=3D"noreferrer" t= arget=3D"_blank">ftp://ftp.openldap.org/incoming/</a><br> > Submission from: (NULL) (54.239.6.177)<br> > <br> > <br> > Meson is the new thing out there,<br> <br> No.<br> <br> <br> -- <br> =C2=A0 -- Howard Chu<br> =C2=A0 CTO, Symas Corp.=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0<a href=3D"= http://www.symas.com" rel=3D"noreferrer" target=3D"_blank">http://www.symas= .com</a><br> =C2=A0 Director, Highland Sun=C2=A0 =C2=A0 =C2=A0<a href=3D"http://highland= sun.com/hyc/" rel=3D"noreferrer" target=3D"_blank">http://highlandsun.com/h= yc/</a><br> =C2=A0 Chief Architect, OpenLDAP=C2=A0 <a href=3D"http://www.openldap.org/p= roject/" rel=3D"noreferrer" target=3D"_blank">http://www.openldap.org/proje= ct/</a><br> </blockquote></div><br clear=3D"all"><br>-- <br><div dir=3D"ltr" class=3D"g= mail_signature" data-smartmail=3D"gmail_signature">Ignacio Casal Quinteiro<= /div> --000000000000a46a9a05752250a1--

1 0

Re: (ITS#8913) Add meson build system to lmdb
by hyc＠symas.com 05 Sep '18

05 Sep '18

nacho.resa(a)gmail.com wrote: > Full_Name: Ignacio Casal Quinteiro > Version: > OS: > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (54.239.6.177) > > > Meson is the new thing out there, No. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#8913) Add meson build system to lmdb
by nacho.resa＠gmail.com 05 Sep '18

05 Sep '18

Full_Name: Ignacio Casal Quinteiro Version: OS: URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (54.239.6.177) Meson is the new thing out there, maybe projects are already using it and allows embedding in other projects apart from being fast, efficient and easier to maintain than a Makefile. On the following patch you can find a working meson build system which builds lmdb: https://github.com/wingtk/lmdb/commit/24b1b1ce2055b6a612603700bdcfc7ba3859f…

1 0

Re: (ITS#8910) Open Ldap back sql installation Query
by Howard Chu 05 Sep '18

05 Sep '18

The OpenLDAP-ITS address is for reporting bugs in OpenLDAP software. Nothing in your report is indicative of a bug in OpenLDAP. If you have questions about use of OpenLDAP Software which are not answered in the documentation, FAQ, and archives, use the OpenLDAP-technical list (subscription required; instructions here <http://www.openldap.org/lists/mm/listinfo/openldap-technical>). This issue report will be closed. p.

1 0

Re: (ITS#8912) incorrect rootDSE (namingContexts)
by hyc＠symas.com 05 Sep '18

05 Sep '18

jochen(a)keutel.de wrote: > Full_Name: Jochen Keutel > Version: 2.4.46 > OS: Debian 9 > URL: > Submission from: (NULL) (80.146.191.218) > > > With certain configurations it happens that the attribute namingContexts of the > rootDSE contains the same value twice (which is not correct). It seems to be > related to the fact that the naming context of a hidden backend is not ignored > (servers/slapd/root_dse.c). Thanks for the report. Fixed now in git master. > > To reproduce it: I started to configure replication: szenario syncrepl proxy > (push based replication, see 18.3.5 in OpenLDAP Admin Guide - "primary directory > also contains back-ldap databases"). Configuring the LDAP backend leads > unfortunately to a root DSE showing the same name context twice: > > namingContexts: dc=keutel,dc=de > namingContexts: dc=keutel,dc=de > > Is this a known problem? Esp. this stops PHPLDAPAdmin from working: It prints a > lot of PHP arrays in this case. > > I've set "hidden on" for this backend but the problem remains. > > My configuration: > > 1. slapd.conf on server1 (master): > > database mdb > suffix "dc=keutel,dc=de" > ... > > database ldap > hidden on > suffix "dc=keutel,dc=de" > rootdn "cn=admin,dc=keutel,dc=de" > uri ldaps://server2/ > > lastmod on > restrict all > > acl-bind bindmethod=simple > binddn="cn=replication,dc=keutel,dc=de" > credentials=secret > > syncrepl rid=001 > provider=ldaps://server1/ > binddn="cn=replication,dc=keutel,dc=de" > bindmethod=simple > credentials=secret > searchbase="dc=keutel,dc=de" > type=refreshAndPersist > retry="5 5 300 5" > > 2. converting this to dynamic config using slaptest gives the following entry: > > > dn: olcDatabase={2}ldap > objectClass: olcDatabaseConfig > objectClass: olcLDAPConfig > olcDatabase: {2}ldap > olcHidden: TRUE > olcSuffix: dc=keutel,dc=de > ... > > 3. starting slapd with this dynamic configuration > > 4. reading rootDSE: attribute namingContexts occurs twice with the same value. > > > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#8912) incorrect rootDSE (namingContexts)
by jochen＠keutel.de 04 Sep '18

04 Sep '18

Full_Name: Jochen Keutel Version: 2.4.46 OS: Debian 9 URL: Submission from: (NULL) (80.146.191.218) With certain configurations it happens that the attribute namingContexts of the rootDSE contains the same value twice (which is not correct). It seems to be related to the fact that the naming context of a hidden backend is not ignored (servers/slapd/root_dse.c). To reproduce it: I started to configure replication: szenario syncrepl proxy (push based replication, see 18.3.5 in OpenLDAP Admin Guide - "primary directory also contains back-ldap databases"). Configuring the LDAP backend leads unfortunately to a root DSE showing the same name context twice: namingContexts: dc=keutel,dc=de namingContexts: dc=keutel,dc=de Is this a known problem? Esp. this stops PHPLDAPAdmin from working: It prints a lot of PHP arrays in this case. I've set "hidden on" for this backend but the problem remains. My configuration: 1. slapd.conf on server1 (master): database mdb suffix "dc=keutel,dc=de" ... database ldap hidden on suffix "dc=keutel,dc=de" rootdn "cn=admin,dc=keutel,dc=de" uri ldaps://server2/ lastmod on restrict all acl-bind bindmethod=simple binddn="cn=replication,dc=keutel,dc=de" credentials=secret syncrepl rid=001 provider=ldaps://server1/ binddn="cn=replication,dc=keutel,dc=de" bindmethod=simple credentials=secret searchbase="dc=keutel,dc=de" type=refreshAndPersist retry="5 5 300 5" 2. converting this to dynamic config using slaptest gives the following entry: dn: olcDatabase={2}ldap objectClass: olcDatabaseConfig objectClass: olcLDAPConfig olcDatabase: {2}ldap olcHidden: TRUE olcSuffix: dc=keutel,dc=de ... 3. starting slapd with this dynamic configuration 4. reading rootDSE: attribute namingContexts occurs twice with the same value.

1 0

Re: (ITS#8911) Open LDAP 2.4 MMR with delta-sync replication
by hyc＠symas.com 30 Aug '18

30 Aug '18

The ITS is for bug reports not help requests. Use the openldap-technical mailing list. sakshi.mittal(a)india.nec.com wrote: > Full_Name: Sakshi Mittal > Version: 2.4.44 > OS: Centos7 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (14.140.114.207) > > > We are trying to setup MMR with delta sync replication in OpenLDAP 2.4.44. > > We have two provider servers and 2 consumer servers. We have setup delta sync > replication between the provider and consumer servers. > Now we want to configure MMR between the two provider servers. > But it does not seem to work together with delta sync. > > I have tried setting up MMR separately and it works also delta sync replication > also works separately. > > Can you please help in configuring the two together also please let me know if > this is even possible. As on some sites it is written that OpenLDAP 2.4 does not > support MMR together with delta-sync replication. > > Please confirm. > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#8911) Open LDAP 2.4 MMR with delta-sync replication
by sakshi.mittal＠india.nec.com 30 Aug '18

30 Aug '18

Full_Name: Sakshi Mittal Version: 2.4.44 OS: Centos7 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (14.140.114.207) We are trying to setup MMR with delta sync replication in OpenLDAP 2.4.44. We have two provider servers and 2 consumer servers. We have setup delta sync replication between the provider and consumer servers. Now we want to configure MMR between the two provider servers. But it does not seem to work together with delta sync. I have tried setting up MMR separately and it works also delta sync replication also works separately. Can you please help in configuring the two together also please let me know if this is even possible. As on some sites it is written that OpenLDAP 2.4 does not support MMR together with delta-sync replication. Please confirm.

1 0

Re: (ITS#8908) LMDB Documentation for MDB_XXX_MULTIPLE
by karl＠waclawek.net 29 Aug '18

29 Aug '18

--000000000000a517d40574956804 Content-Type: text/plain; charset="UTF-8" Thanks! On Wed, Aug 29, 2018, 12:05 Howard Chu, <hyc(a)symas.com> wrote: > karl(a)waclawek.net wrote: > > --00000000000018b43a057492d87f > > Content-Type: text/plain; charset="UTF-8" > > > > As far as point 2 is concerned I was basing it on this piece of code, > where > > it looks like MDB_GET_MULTIPLE just checks some flags and then goes to > the > > fetchm label where the code does nothing but retrieve data: > > You're not reading the code correctly. See below. > > > > case MDB_GET_MULTIPLE: > > if (data == NULL || !(mc->mc_flags & C_INITIALIZED)) { > > rc = EINVAL; > > break; > > } > > if (!(mc->mc_db->md_flags & MDB_DUPFIXED)) { > > rc = MDB_INCOMPATIBLE; > > break; > > } > > rc = MDB_SUCCESS; > > if (!(mc->mc_xcursor->mx_cursor.mc_flags & C_INITIALIZED) || > > (mc->mc_xcursor->mx_cursor.mc_flags & C_EOF)) > > break; > > goto fetchm; > > case MDB_NEXT_MULTIPLE: > > if (data == NULL) { > > rc = EINVAL; > > break; > > } > > if (!(mc->mc_db->md_flags & MDB_DUPFIXED)) { > > rc = MDB_INCOMPATIBLE; > > break; > > } > > rc = mdb_cursor_next(mc, key, data, MDB_NEXT_DUP); > > if (rc == MDB_SUCCESS) { > > if (mc->mc_xcursor->mx_cursor.mc_flags & C_INITIALIZED) { > > MDB_cursor *mx; > > fetchm: > > mx = &mc->mc_xcursor->mx_cursor; > > data->mv_size = NUMKEYS(mx->mc_pg[mx->mc_top]) * > > mx->mc_db->md_pad; > > data->mv_data = METADATA(mx->mc_pg[mx->mc_top]); > > mx->mc_ki[mx->mc_top] = NUMKEYS(mx->mc_pg[mx->mc_top])-1; > > ^^^ this line advances the cursor. > > > } else { > > rc = MDB_NOTFOUND; > > } > > } > > break; > > > > > > On Tue, Aug 28, 2018 at 8:26 PM Howard Chu <openldap-its(a)openldap.org> > > wrote: > > > >>> Full_Name: Karl Waclawek > >>> Version: LMDB 0.9.22 > >>> OS: Windows 10 > >>> URL: ftp://ftp.openldap.org/incoming/ > >>> Submission from: (NULL) (2607:fea8:7a80:814:3ca9:e8f6:801c:d6f8) > >>> > >>> > >>> The documentation in lmdb.h for MDB_GET_MULTIPLE, MDB_NEXT_MULTIPLE and > >>> MDB_PREV_MULTIPLE seems to have two issues: > >>> > >>> 1) The documentation indicates that both, the key and data, are > >> returned, but > >> in > >>> reality only the data is returned. > >> > >> Looks like you're right. Since these are DUPs the key would be the same > >> every time so there's no need to return it on each call. > >> Doc fixed in git mdb.master. > >> > >>> 2) The documentation states that MDB_GET_MULTIPLE moves the cursor. > This > >> does > >>> not seem to be true. > >> > >> The doc is correct here. The cursor is advanced far enough that a > >> MDB_NEXT_MULTIPLE will return the following data, as opposed to just > >> returning the same data again. > >> > > > > > > -- > -- Howard Chu > CTO, Symas Corp. http://www.symas.com > Director, Highland Sun http://highlandsun.com/hyc/ > Chief Architect, OpenLDAP http://www.openldap.org/project/ > --000000000000a517d40574956804 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"auto">Thanks!</div><br><div class=3D"gmail_quote"><div dir=3D"l= tr">On Wed, Aug 29, 2018, 12:05 Howard Chu, <<a href=3D"mailto:hyc@symas= .com">hyc(a)symas.com</a>> wrote:<br></div><blockquote class=3D"gmail_quot= e" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">= <a href=3D"mailto:karl@waclawek.net" target=3D"_blank" rel=3D"noreferrer">k= arl(a)waclawek.net</a> wrote:<br> > --00000000000018b43a057492d87f<br> > Content-Type: text/plain; charset=3D"UTF-8"<br> > <br> > As far as point 2 is concerned I was basing it on this piece of code, = where<br> > it looks like MDB_GET_MULTIPLE just checks some flags and then goes to= the<br> > fetchm label where the code does nothing but retrieve data:<br> <br> You're not reading the code correctly. See below.<br> > <br> > case MDB_GET_MULTIPLE:<br> > if (data =3D=3D NULL || !(mc->mc_flags & C_INITIALIZED)) {<br> > rc =3D EINVAL;<br> > break;<br> > }<br> > if (!(mc->mc_db->md_flags & MDB_DUPFIXED)) {<br> > rc =3D MDB_INCOMPATIBLE;<br> > break;<br> > }<br> > rc =3D MDB_SUCCESS;<br> > if (!(mc->mc_xcursor->mx_cursor.mc_flags & C_INITIALIZED) ||= <br> > (mc->mc_xcursor->mx_cursor.mc_flags & C_EOF))<br> > break;<br> > goto fetchm;<br> > case MDB_NEXT_MULTIPLE:<br> > if (data =3D=3D NULL) {<br> > rc =3D EINVAL;<br> > break;<br> > }<br> > if (!(mc->mc_db->md_flags & MDB_DUPFIXED)) {<br> > rc =3D MDB_INCOMPATIBLE;<br> > break;<br> > }<br> > rc =3D mdb_cursor_next(mc, key, data, MDB_NEXT_DUP);<br> > if (rc =3D=3D MDB_SUCCESS) {<br> > if (mc->mc_xcursor->mx_cursor.mc_flags & C_INITIALIZED) {<br= > > MDB_cursor *mx;<br> > fetchm:<br> > mx =3D &mc->mc_xcursor->mx_cursor;<br> > data->mv_size =3D NUMKEYS(mx->mc_pg[mx->mc_top]) *<br> > mx->mc_db->md_pad;<br> > data->mv_data =3D METADATA(mx->mc_pg[mx->mc_top]);<br> > mx->mc_ki[mx->mc_top] =3D NUMKEYS(mx->mc_pg[mx->mc_top])-1= ;<br> <br> ^^^ this line advances the cursor.<br> <br> > } else {<br> > rc =3D MDB_NOTFOUND;<br> > }<br> > }<br> > break;<br> > <br> > <br> > On Tue, Aug 28, 2018 at 8:26 PM Howard Chu <<a href=3D"mailto:openl= dap-its(a)openldap.org" target=3D"_blank" rel=3D"noreferrer">openldap-its@ope= nldap.org</a>><br> > wrote:<br> > <br> >>> Full_Name: Karl Waclawek<br> >>> Version: LMDB 0.9.22<br> >>> OS: Windows 10<br> >>> URL: <a href=3D"ftp://ftp.openldap.org/incoming/" rel=3D"noref= errer noreferrer" target=3D"_blank">ftp://ftp.openldap.org/incoming/</a><br= > >>> Submission from: (NULL) (2607:fea8:7a80:814:3ca9:e8f6:801c:d6f= 8)<br> >>><br> >>><br> >>> The documentation in lmdb.h for MDB_GET_MULTIPLE, MDB_NEXT_MUL= TIPLE and<br> >>> MDB_PREV_MULTIPLE seems to have two issues:<br> >>><br> >>> 1) The documentation indicates that both, the key and data, ar= e<br> >> returned, but<br> >> in<br> >>> reality only the data is returned.<br> >><br> >> Looks like you're right. Since these are DUPs the key would be= the same<br> >> every time so there's no need to return it on each call.<br> >> Doc fixed in git mdb.master.<br> >><br> >>> 2) The documentation states that MDB_GET_MULTIPLE moves the cu= rsor. This<br> >> does<br> >>> not seem to be true.<br> >><br> >> The doc is correct here. The cursor is advanced far enough that a<= br> >> MDB_NEXT_MULTIPLE will return the following data, as opposed to ju= st<br> >> returning the same data again.<br> >><br> > <br> <br> <br> <br> -- <br> =C2=A0 =C2=A0-- Howard Chu<br> =C2=A0 =C2=A0CTO, Symas Corp.=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0<a hr= ef=3D"http://www.symas.com" rel=3D"noreferrer noreferrer" target=3D"_blank"= >http://www.symas.com</a><br> =C2=A0 =C2=A0Director, Highland Sun=C2=A0 =C2=A0 =C2=A0<a href=3D"http://hi= ghlandsun.com/hyc/" rel=3D"noreferrer noreferrer" target=3D"_blank">http://= highlandsun.com/hyc/</a><br> =C2=A0 =C2=A0Chief Architect, OpenLDAP=C2=A0 <a href=3D"http://www.openldap= .org/project/" rel=3D"noreferrer noreferrer" target=3D"_blank">http://www.o= penldap.org/project/</a><br> </blockquote></div> --000000000000a517d40574956804--

1 0

Re: (ITS#8908) LMDB Documentation for MDB_XXX_MULTIPLE
by hyc＠symas.com 29 Aug '18

29 Aug '18

karl(a)waclawek.net wrote: > --00000000000018b43a057492d87f > Content-Type: text/plain; charset="UTF-8" > > As far as point 2 is concerned I was basing it on this piece of code, where > it looks like MDB_GET_MULTIPLE just checks some flags and then goes to the > fetchm label where the code does nothing but retrieve data: You're not reading the code correctly. See below. > > case MDB_GET_MULTIPLE: > if (data == NULL || !(mc->mc_flags & C_INITIALIZED)) { > rc = EINVAL; > break; > } > if (!(mc->mc_db->md_flags & MDB_DUPFIXED)) { > rc = MDB_INCOMPATIBLE; > break; > } > rc = MDB_SUCCESS; > if (!(mc->mc_xcursor->mx_cursor.mc_flags & C_INITIALIZED) || > (mc->mc_xcursor->mx_cursor.mc_flags & C_EOF)) > break; > goto fetchm; > case MDB_NEXT_MULTIPLE: > if (data == NULL) { > rc = EINVAL; > break; > } > if (!(mc->mc_db->md_flags & MDB_DUPFIXED)) { > rc = MDB_INCOMPATIBLE; > break; > } > rc = mdb_cursor_next(mc, key, data, MDB_NEXT_DUP); > if (rc == MDB_SUCCESS) { > if (mc->mc_xcursor->mx_cursor.mc_flags & C_INITIALIZED) { > MDB_cursor *mx; > fetchm: > mx = &mc->mc_xcursor->mx_cursor; > data->mv_size = NUMKEYS(mx->mc_pg[mx->mc_top]) * > mx->mc_db->md_pad; > data->mv_data = METADATA(mx->mc_pg[mx->mc_top]); > mx->mc_ki[mx->mc_top] = NUMKEYS(mx->mc_pg[mx->mc_top])-1; ^^^ this line advances the cursor. > } else { > rc = MDB_NOTFOUND; > } > } > break; > > > On Tue, Aug 28, 2018 at 8:26 PM Howard Chu <openldap-its(a)openldap.org> > wrote: > >>> Full_Name: Karl Waclawek >>> Version: LMDB 0.9.22 >>> OS: Windows 10 >>> URL: ftp://ftp.openldap.org/incoming/ >>> Submission from: (NULL) (2607:fea8:7a80:814:3ca9:e8f6:801c:d6f8) >>> >>> >>> The documentation in lmdb.h for MDB_GET_MULTIPLE, MDB_NEXT_MULTIPLE and >>> MDB_PREV_MULTIPLE seems to have two issues: >>> >>> 1) The documentation indicates that both, the key and data, are >> returned, but >> in >>> reality only the data is returned. >> >> Looks like you're right. Since these are DUPs the key would be the same >> every time so there's no need to return it on each call. >> Doc fixed in git mdb.master. >> >>> 2) The documentation states that MDB_GET_MULTIPLE moves the cursor. This >> does >>> not seem to be true. >> >> The doc is correct here. The cursor is advanced far enough that a >> MDB_NEXT_MULTIPLE will return the following data, as opposed to just >> returning the same data again. >> > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8908) LMDB Documentation for MDB_XXX_MULTIPLE
by karl＠waclawek.net 29 Aug '18

29 Aug '18

--00000000000018b43a057492d87f Content-Type: text/plain; charset="UTF-8" As far as point 2 is concerned I was basing it on this piece of code, where it looks like MDB_GET_MULTIPLE just checks some flags and then goes to the fetchm label where the code does nothing but retrieve data: case MDB_GET_MULTIPLE: if (data == NULL || !(mc->mc_flags & C_INITIALIZED)) { rc = EINVAL; break; } if (!(mc->mc_db->md_flags & MDB_DUPFIXED)) { rc = MDB_INCOMPATIBLE; break; } rc = MDB_SUCCESS; if (!(mc->mc_xcursor->mx_cursor.mc_flags & C_INITIALIZED) || (mc->mc_xcursor->mx_cursor.mc_flags & C_EOF)) break; goto fetchm; case MDB_NEXT_MULTIPLE: if (data == NULL) { rc = EINVAL; break; } if (!(mc->mc_db->md_flags & MDB_DUPFIXED)) { rc = MDB_INCOMPATIBLE; break; } rc = mdb_cursor_next(mc, key, data, MDB_NEXT_DUP); if (rc == MDB_SUCCESS) { if (mc->mc_xcursor->mx_cursor.mc_flags & C_INITIALIZED) { MDB_cursor *mx; fetchm: mx = &mc->mc_xcursor->mx_cursor; data->mv_size = NUMKEYS(mx->mc_pg[mx->mc_top]) * mx->mc_db->md_pad; data->mv_data = METADATA(mx->mc_pg[mx->mc_top]); mx->mc_ki[mx->mc_top] = NUMKEYS(mx->mc_pg[mx->mc_top])-1; } else { rc = MDB_NOTFOUND; } } break; On Tue, Aug 28, 2018 at 8:26 PM Howard Chu <openldap-its(a)openldap.org> wrote: > > Full_Name: Karl Waclawek > > Version: LMDB 0.9.22 > > OS: Windows 10 > > URL: ftp://ftp.openldap.org/incoming/ > > Submission from: (NULL) (2607:fea8:7a80:814:3ca9:e8f6:801c:d6f8) > > > > > > The documentation in lmdb.h for MDB_GET_MULTIPLE, MDB_NEXT_MULTIPLE and > > MDB_PREV_MULTIPLE seems to have two issues: > > > > 1) The documentation indicates that both, the key and data, are > returned, but > in > > reality only the data is returned. > > Looks like you're right. Since these are DUPs the key would be the same > every time so there's no need to return it on each call. > Doc fixed in git mdb.master. > > > 2) The documentation states that MDB_GET_MULTIPLE moves the cursor. This > does > > not seem to be true. > > The doc is correct here. The cursor is advanced far enough that a > MDB_NEXT_MULTIPLE will return the following data, as opposed to just > returning the same data again. > --00000000000018b43a057492d87f Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div dir=3D"ltr">As far as point 2 is concerned I was basi= ng it on this piece of code, where it looks like MDB_GET_MULTIPLE just chec= ks some flags and then goes to the fetchm label where the code does nothing= but retrieve data:<div><br></div><div><div><font face=3D"monospace, monosp= ace"><span style=3D"white-space:pre"> </span>case MDB_GET_MULTIPLE:</font><= /div><div><font face=3D"monospace, monospace"><span style=3D"white-space:pr= e"> </span>if (data =3D=3D NULL || !(mc->mc_flags & C_INITIALIZED))= {</font></div><div><font face=3D"monospace, monospace"><span style=3D"whit= e-space:pre"> </span>rc =3D EINVAL;</font></div><div><font face=3D"monosp= ace, monospace"><span style=3D"white-space:pre"> </span>break;</font></di= v><div><font face=3D"monospace, monospace"><span style=3D"white-space:pre">= </span>}</font></div><div><font face=3D"monospace, monospace"><span style= =3D"white-space:pre"> </span>if (!(mc->mc_db->md_flags & MDB_DUP= FIXED)) {</font></div><div><font face=3D"monospace, monospace"><span style= =3D"white-space:pre"> </span>rc =3D MDB_INCOMPATIBLE;</font></div><div><f= ont face=3D"monospace, monospace"><span style=3D"white-space:pre"> </span= >break;</font></div><div><font face=3D"monospace, monospace"><span style=3D= "white-space:pre"> </span>}</font></div><div><font face=3D"monospace, mono= space"><span style=3D"white-space:pre"> </span>rc =3D MDB_SUCCESS;</font><= /div><div><font face=3D"monospace, monospace"><span style=3D"white-space:pr= e"> </span>if (!(mc->mc_xcursor->mx_cursor.mc_flags & C_INITIALI= ZED) ||</font></div><div><font face=3D"monospace, monospace"><span style=3D= "white-space:pre"> </span>(mc->mc_xcursor->mx_cursor.mc_flags &= C_EOF))</font></div><div><font face=3D"monospace, monospace"><span style= =3D"white-space:pre"> </span>break;</font></div><div><font face=3D"monosp= ace, monospace"><span style=3D"white-space:pre"> </span>goto fetchm;</font= ></div><div><font face=3D"monospace, monospace"><span style=3D"white-space:= pre"> </span>case MDB_NEXT_MULTIPLE:</font></div><div><font face=3D"monospa= ce, monospace"><span style=3D"white-space:pre"> </span>if (data =3D=3D NUL= L) {</font></div><div><font face=3D"monospace, monospace"><span style=3D"wh= ite-space:pre"> </span>rc =3D EINVAL;</font></div><div><font face=3D"mono= space, monospace"><span style=3D"white-space:pre"> </span>break;</font></= div><div><font face=3D"monospace, monospace"><span style=3D"white-space:pre= "> </span>}</font></div><div><font face=3D"monospace, monospace"><span sty= le=3D"white-space:pre"> </span>if (!(mc->mc_db->md_flags & MDB_D= UPFIXED)) {</font></div><div><font face=3D"monospace, monospace"><span styl= e=3D"white-space:pre"> </span>rc =3D MDB_INCOMPATIBLE;</font></div><div><= font face=3D"monospace, monospace"><span style=3D"white-space:pre"> </spa= n>break;</font></div><div><font face=3D"monospace, monospace"><span style= =3D"white-space:pre"> </span>}</font></div><div><font face=3D"monospace, m= onospace"><span style=3D"white-space:pre"> </span>rc =3D mdb_cursor_next(m= c, key, data, MDB_NEXT_DUP);</font></div><div><font face=3D"monospace, mono= space"><span style=3D"white-space:pre"> </span>if (rc =3D=3D MDB_SUCCESS) = {</font></div><div><font face=3D"monospace, monospace"><span style=3D"white= -space:pre"> </span>if (mc->mc_xcursor->mx_cursor.mc_flags & C_= INITIALIZED) {</font></div><div><font face=3D"monospace, monospace"><span s= tyle=3D"white-space:pre"> </span>MDB_cursor *mx;</font></div><div><font = face=3D"monospace, monospace">fetchm:</font></div><div><font face=3D"monosp= ace, monospace"><span style=3D"white-space:pre"> </span>mx =3D &mc-&= gt;mc_xcursor->mx_cursor;</font></div><div><font face=3D"monospace, mono= space"><span style=3D"white-space:pre"> </span>data->mv_size =3D NUMK= EYS(mx->mc_pg[mx->mc_top]) *</font></div><div><font face=3D"monospace= , monospace"><span style=3D"white-space:pre"> </span>mx->mc_db->m= d_pad;</font></div><div><font face=3D"monospace, monospace"><span style=3D"= white-space:pre"> </span>data->mv_data =3D METADATA(mx->mc_pg[mx-&= gt;mc_top]);</font></div><div><font face=3D"monospace, monospace"><span sty= le=3D"white-space:pre"> </span>mx->mc_ki[mx->mc_top] =3D NUMKEYS(m= x->mc_pg[mx->mc_top])-1;</font></div><div><font face=3D"monospace, mo= nospace"><span style=3D"white-space:pre"> </span>} else {</font></div><di= v><font face=3D"monospace, monospace"><span style=3D"white-space:pre"> <= /span>rc =3D MDB_NOTFOUND;</font></div><div><font face=3D"monospace, monosp= ace"><span style=3D"white-space:pre"> </span>}</font></div><div><font fac= e=3D"monospace, monospace"><span style=3D"white-space:pre"> </span>}</font= ></div><div><font face=3D"monospace, monospace"><span style=3D"white-space:= pre"> </span>break;</font></div></div><div><br></div></div></div><br><div = class=3D"gmail_quote"><div dir=3D"ltr">On Tue, Aug 28, 2018 at 8:26 PM Howa= rd Chu <<a href=3D"mailto:openldap-its@openldap.org">openldap-its@openld= ap.org</a>> wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"m= argin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">> Full_Nam= e: Karl Waclawek<br> > Version: LMDB 0.9.22<br> > OS: Windows 10<br> > URL: <a href=3D"ftp://ftp.openldap.org/incoming/" rel=3D"noreferrer" t= arget=3D"_blank">ftp://ftp.openldap.org/incoming/</a><br> > Submission from: (NULL) (2607:fea8:7a80:814:3ca9:e8f6:801c:d6f8)<br> > <br> > <br> > The documentation in lmdb.h for MDB_GET_MULTIPLE, MDB_NEXT_MULTIPLE an= d<br> > MDB_PREV_MULTIPLE seems to have two issues:<br> > <br> > 1) The documentation indicates that both, the key and data, are return= ed, but<br> in<br> > reality only the data is returned.<br> <br> Looks like you're right. Since these are DUPs the key would be the same= <br> every time so there's no need to return it on each call.<br> Doc fixed in git mdb.master.<br> <br> > 2) The documentation states that MDB_GET_MULTIPLE moves the cursor. Th= is does<br> > not seem to be true.<br> <br> The doc is correct here. The cursor is advanced far enough that a<br> MDB_NEXT_MULTIPLE will return the following data, as opposed to just<br> returning the same data again.<br> </blockquote></div> --00000000000018b43a057492d87f--

1 0

(ITS#8910) Open Ldap back sql installation Query
by nagalakshmibatchu345＠gmail.com 29 Aug '18

29 Aug '18

Full_Name: Nagalakshmi Batchu Version: 2.4.44 OS: Windows URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (183.82.124.72) Hi, I have doubt in Back sql Configuration.In the document /etc/odbc.ini mentioned like this. Here where i need to find the above mentioned file path.

1 0

Re: (ITS#8909) "authz-policy all" works as "authz-policy any", possibly yielding unauthorized access
by hyc＠symas.com 29 Aug '18

29 Aug '18

Guilhem Moulin wrote: > On Wed, 29 Aug 2018 at 01:14:51 +0100, Howard Chu wrote: >> Thanks for the report. Looks like this has been present since commit >> 113727ba. Fixed now in git master > > Thanks for the quick fix! Not sure why rc's value is preserved here but > set to LDAP_INAPPROPRIATE_AUTH in all other failing cases, though. But > that doesn't seem to matter beside debug logs now showing a return value > other than 48, disclosing the actual reason of the failure; for instance > > <== slap_sasl_authorized: return 16 > SASL Proxy Authorize [conn=1022]: proxy authorization disallowed (16) > > for a missing authTo under authz-policy "all". > Probably not a bad thing overall, but for consistency it's now patched to set INAPPROPRIATE_AUTH as with the other cases. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8909) "authz-policy all" works as "authz-policy any", possibly yielding unauthorized access
by guilhem＠fripost.org 29 Aug '18

29 Aug '18

On Wed, 29 Aug 2018 at 01:14:51 +0100, Howard Chu wrote: > Thanks for the report. Looks like this has been present since commit > 113727ba. Fixed now in git master Thanks for the quick fix! Not sure why rc's value is preserved here but set to LDAP_INAPPROPRIATE_AUTH in all other failing cases, though. But that doesn't seem to matter beside debug logs now showing a return value other than 48, disclosing the actual reason of the failure; for instance <== slap_sasl_authorized: return 16 SASL Proxy Authorize [conn=1022]: proxy authorization disallowed (16) for a missing authTo under authz-policy "all". -- Guilhem.

1 0

Re: (ITS#8906) Fail to download package in yocto project
by Howard Chu 29 Aug '18

29 Aug '18

> Full_Name: Yong Zhang > Version: 2.4.43 > OS: linux > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (180.166.53.43) > > > ftp://ftp.openldap.org/pub/OpenLDAP/openldap-release/${BP}.tgz is not available ftp://ftp.openldap.org/pub/OpenLDAP/openldap-release/openldap-2.4.43.tgz Looks fine to me.

1 0

Re: (ITS#8908) LMDB Documentation for MDB_XXX_MULTIPLE
by Howard Chu 29 Aug '18

29 Aug '18

> Full_Name: Karl Waclawek > Version: LMDB 0.9.22 > OS: Windows 10 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (2607:fea8:7a80:814:3ca9:e8f6:801c:d6f8) > > > The documentation in lmdb.h for MDB_GET_MULTIPLE, MDB_NEXT_MULTIPLE and > MDB_PREV_MULTIPLE seems to have two issues: > > 1) The documentation indicates that both, the key and data, are returned, but in > reality only the data is returned. Looks like you're right. Since these are DUPs the key would be the same every time so there's no need to return it on each call. Doc fixed in git mdb.master. > 2) The documentation states that MDB_GET_MULTIPLE moves the cursor. This does > not seem to be true. The doc is correct here. The cursor is advanced far enough that a MDB_NEXT_MULTIPLE will return the following data, as opposed to just returning the same data again.

1 0

Re: (ITS#8909) "authz-policy all" works as "authz-policy any", possibly yielding unauthorized access
by hyc＠symas.com 29 Aug '18

29 Aug '18

guilhem(a)fripost.org wrote: > Full_Name: Guilhem Moulin > Version: 2.4.44 > OS: Debian GNU/Linux (Stretch) > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (109.225.112.70) >=20 >=20 > slapd.conf(5) manpage (in both 2.4.44 and in current =E2=80=94 0f320b3 = =E2=80=94 master) > mentions that authz-policy's "all" flag requires both source and destin= ations > authorizations rules to succeed. However if the source rule (the authe= ntication > identity's "authzTo" attribute) fails but the destination rule (the > authorization identity's "authzFrom" attribute) succeeds, then the auth= orization > is granted, violating the intended semantics and possibly yielding unau= thorized > access. See the following log excerpt: Thanks for the report. Looks like this has been present since commit 1137= 27ba. Fixed now in git master >=20 > SASL proxy authorize [conn=3D1019]: authcid=3D"authcid" > authzid=3D"dn:uid=3Dauthzid,dc=3Dexample,dc=3Dnet" > =3D=3D>slap_sasl_authorized: can uid=3Dauthcid,dc=3Dexample,dc=3Dnet be= come > uid=3Dauthzid,dc=3Dexample,dc=3Dnet? > =3D=3D>slap_sasl_check_authz: does uid=3Dauthzid,dc=3Dexample,dc=3Dnet = match authzTo rule > in uid=3Dauthcid,dc=3Dexample,dc=3Dnet? > <=3D=3Dslap_sasl_check_authz: authzTo check returning 50 > =3D=3D>slap_sasl_check_authz: does uid=3Dauthcid,dc=3Dexample,dc=3Dnet = match authzFrom > rule in uid=3Dauthzid,dc=3Dexample,dc=3Dnet? > <=3D=3D=3Dslap_sasl_match: comparison returned 0 > <=3D=3Dslap_sasl_check_authz: authzFrom check returning 0 > <=3D=3D slap_sasl_authorized: return 0 > conn=3D1019 op=3D1 BIND authcid=3D"authcid" > authzid=3D"dn:uid=3Dauthzid,dc=3Dexample,dc=3Dnet" > SASL Authorize [conn=3D1019]: proxy authorization allowed > authzDN=3D"uid=3Dauthzid,dc=3Dexample,dc=3Dnet" >=20 > AFAICT the problem is in servers/slapd/saslauthz.c:slap_sasl_authorized= (), and > is also present in master. Here is a naive patch that fails the author= ization > if the source rules doesn't verify and SASL_AUTHZ_AND is set in authz_p= olicy. >=20 > --- a/servers/slapd/saslauthz.c > +++ b/servers/slapd/saslauthz.c > @@ -2077,6 +2077,10 @@ int slap_sasl_authorized( Operation *op, > if( rc =3D=3D LDAP_SUCCESS && !(authz_policy & SASL_AUTHZ_AND)= ) { > goto DONE; > } > + else if( rc !=3D LDAP_SUCCESS && (authz_policy & SASL_AUTHZ_AN= D) ) { > + rc =3D LDAP_INAPPROPRIATE_AUTH; > + goto DONE; > + } > } > =20 > /* Check destination rules */ >=20 >=20 --=20 -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#8909) "authz-policy all" works as "authz-policy any", possibly yielding unauthorized access
by guilhem＠fripost.org 29 Aug '18

29 Aug '18

Full_Name: Guilhem Moulin Version: 2.4.44 OS: Debian GNU/Linux (Stretch) URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (109.225.112.70) slapd.conf(5) manpage (in both 2.4.44 and in current 0f320b3 master) mentions that authz-policy's "all" flag requires both source and destinations authorizations rules to succeed. However if the source rule (the authentication identity's "authzTo" attribute) fails but the destination rule (the authorization identity's "authzFrom" attribute) succeeds, then the authorization is granted, violating the intended semantics and possibly yielding unauthorized access. See the following log excerpt: SASL proxy authorize [conn=1019]: authcid="authcid" authzid="dn:uid=authzid,dc=example,dc=net" ==>slap_sasl_authorized: can uid=authcid,dc=example,dc=net become uid=authzid,dc=example,dc=net? ==>slap_sasl_check_authz: does uid=authzid,dc=example,dc=net match authzTo rule in uid=authcid,dc=example,dc=net? <==slap_sasl_check_authz: authzTo check returning 50 ==>slap_sasl_check_authz: does uid=authcid,dc=example,dc=net match authzFrom rule in uid=authzid,dc=example,dc=net? <===slap_sasl_match: comparison returned 0 <==slap_sasl_check_authz: authzFrom check returning 0 <== slap_sasl_authorized: return 0 conn=1019 op=1 BIND authcid="authcid" authzid="dn:uid=authzid,dc=example,dc=net" SASL Authorize [conn=1019]: proxy authorization allowed authzDN="uid=authzid,dc=example,dc=net" AFAICT the problem is in servers/slapd/saslauthz.c:slap_sasl_authorized(), and is also present in master. Here is a naive patch that fails the authorization if the source rules doesn't verify and SASL_AUTHZ_AND is set in authz_policy. --- a/servers/slapd/saslauthz.c +++ b/servers/slapd/saslauthz.c @@ -2077,6 +2077,10 @@ int slap_sasl_authorized( Operation *op, if( rc == LDAP_SUCCESS && !(authz_policy & SASL_AUTHZ_AND) ) { goto DONE; } + else if( rc != LDAP_SUCCESS && (authz_policy & SASL_AUTHZ_AND) ) { + rc = LDAP_INAPPROPRIATE_AUTH; + goto DONE; + } } /* Check destination rules */

1 0

Re: (ITS#8650) EAGAIN from gnutls_handshake not respected
by subbarao＠computer.org 27 Aug '18

27 Aug '18

On 08/26/2018 03:07 PM, Ryan Tandy wrote: > If I create a patch to log some additional debug info about the GnuTLS > setup, would you be willing to run it in your environment? For example > I'm curious whether EAGAIN is being returned from the read side or > write side (guessing read, but would be nice to confirm). Sure, send me the patch. If the patch just does some additional GnuTLS logging, then I don't see a problem with running it in the production environment and sending you the results. Regards, -Kartik

1 0

(ITS#8908) LMDB Documentation for MDB_XXX_MULTIPLE
by karl＠waclawek.net 27 Aug '18

27 Aug '18

Full_Name: Karl Waclawek Version: LMDB 0.9.22 OS: Windows 10 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (2607:fea8:7a80:814:3ca9:e8f6:801c:d6f8) The documentation in lmdb.h for MDB_GET_MULTIPLE, MDB_NEXT_MULTIPLE and MDB_PREV_MULTIPLE seems to have two issues: 1) The documentation indicates that both, the key and data, are returned, but in reality only the data is returned. 2) The documentation states that MDB_GET_MULTIPLE moves the cursor. This does not seem to be true.

1 0

Re: (ITS#8650) EAGAIN from gnutls_handshake not respected
by ryan＠nardis.ca 26 Aug '18

26 Aug '18

Hi Karthik, Sorry about the lack of updates on this one. It looks clear that my patch for this ITS was wrong and needs to be reverted. Looking again at the original issue, after reverting the patch, I've found that the behaviour varies with GnuTLS version. I need to figure out why this is, which probably means spending some time bisecting GnuTLS changes. If I create a patch to log some additional debug info about the GnuTLS setup, would you be willing to run it in your environment? For example I'm curious whether EAGAIN is being returned from the read side or write side (guessing read, but would be nice to confirm). thanks Ryan

1 0

(ITS#8907) Memory leak on mass adding users to ldap
by nowhereman＠nowhereman.ru 24 Aug '18

24 Aug '18

Full_Name: Alexey Karpov Version: 2.4.44 OS: RHEL 7.5 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (87.245.195.105) We have two OpenLDAP servers version 2.4.44, running on RHEL 7.5, replicated in mirror mode and working as active-standby. OpenLDAP works as userstore for WSO2 Identity Server. Servers configured with tls security and valid certificates. All requests come to first server (ldap1), second server (ldap2) working as reserve. When we start batch adding users (about 3-5 inserts per second, mixed with queries and changing values), we observe a memory leak on server ldap1, load-dependent, up to gigabyte per minute, and the lag in sync between servers. If we stop server ldap2, leaking stops and server ldap1 working properly. Memory is not freeing until server is restarted. Also we try this setup with version 2.4.44 on Centos 7 and custom builded version 2.4.46 on Centos 7. Results are the same - on batch adding users, server, receiving requests has memory leak and leaking stops when second server stopped.

1 0

(ITS#8906) Fail to download package in yocto project
by zyong＠qti.qualcomm.com 24 Aug '18

24 Aug '18

Full_Name: Yong Zhang Version: 2.4.43 OS: linux URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (180.166.53.43) ftp://ftp.openldap.org/pub/OpenLDAP/openldap-release/${BP}.tgz is not available

1 0

(ITS#8905) Client side debug log should include timestamps
by quanah＠openldap.org 21 Aug '18

21 Aug '18

Full_Name: Quanah Gibson-Mount Version: 2.4.46 OS: N/A URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (47.208.148.239) When specifying full debug on the client side (-d -1), there are no timestamps provided. This is problematic when trying to diagnose where the client side is having issues when connecting to the slapd server. I.e., an ldapwhoami command that takes 19 seconds to go from start to finish, while the slapd side shows everything taking 1 second or less.

1 0

(ITS#8904) Cannot enable SSL3 when disabled by default in OpenSSL
by mhonek＠redhat.com 21 Aug '18

21 Aug '18

Full_Name: Matus Honek Version: 2.4.46 OS: Fedora 28 URL: ftp://ftp.openldap.org/incoming/Matus-Honek-180821.patch Submission from: (NULL) (213.175.37.10) When in OpenSSL one disables SSL3 by default (the SSL_OP_NO_SSLv3 is set by default, like in recent Fedora distributions) then with the current code in OpenLDAP it is not possible to have it re-enabled using TLS_PROTOCOL_MIN configuration option. The attached patch explicitly clears the SSL_OP_NO_SSLv3 option when TLS_PROTOCOL_MIN is set so that SSL3 should be enabled. Feel free to use it; I believe IPR should not be necessary for a one liner. However, in the future when more protocols will be disabled by default (possibly soon for TLS1.0 and TLS1.1), similar fixes will be needed for those as well. Or, it may be decided to not support the protocols that are disabled by default but in that case probably a log message should be issued once user tries to enable a by default disabled protocol.

1 0

RE: (ITS#8848) New LDAP URL syntax to support binding to specific IP address at client side
by sudhir.singam＠nokia.com 21 Aug '18

21 Aug '18

Hi, Any comments ?? Regards, Sudhir Singam DELIVERING BEST-IN-CLASS PLATFORM is our vision -----Original Message----- From: openldap-its(a)OpenLDAP.org <openldap-its(a)OpenLDAP.org>=20 Sent: Monday, May 07, 2018 12:02 PM To: Singam, Sudhir (Nokia - IN/Bangalore) <sudhir.singam(a)nokia.com> Subject: Re: (ITS#8848) New LDAP URL syntax to support binding to specific = IP address at client side *** THIS IS AN AUTOMATICALLY GENERATED REPLY *** Thanks for your report to the OpenLDAP Issue Tracking System. Your report has been assigned the tracking number ITS#8848. One of our support engineers will look at your report in due course. Note that this may take some time because our support engineers are volunteers. They only work on OpenLDAP when they have spare time. If you need to provide additional information in regards to your issue report, you may do so by replying to this message. Note that any mail sent to openldap-its(a)openldap.org with (ITS#8848) in the subject will automatically be attached to the issue report. mailto:openldap-its@openldap.org?subject=3D(ITS#8848) You may follow the progress of this report by loading the following URL in a web browser: http://www.OpenLDAP.org/its/index.cgi?findid=3D8848 Please remember to retain your issue tracking number (ITS#8848) on any further messages you send to us regarding this report. If you don't then you'll just waste our time and yours because we won't be able to properly track the report. Please note that the Issue Tracking System is not intended to be used to seek help in the proper use of OpenLDAP Software. Such requests will be closed. OpenLDAP Software is user supported. http://www.OpenLDAP.org/support/ -------------- Copyright 1998-2007 The OpenLDAP Foundation, All Rights Reserved.

1 0

(ITS#8903) Add Bind Early Option to ldappasswd
by randall＠mason.ch 21 Aug '18

21 Aug '18

Full_Name: Randall Mason Version: HEAD OS: Debian Linux URL: https://gist.githubusercontent.com/ClashTheBunny/a9d2b8d0119964a0eb8a5e2ed7… Submission from: (NULL) (67.165.132.233) ldappasswd is slightly different from a standard passwd workflow in that it requests an old password, then a new password, then the old password again. This confuses people who are used to the unix passwd tool as well as people who use password manager. I've seen quite a few people who have generated a new password, overwriting the old one, and then need a password reset because they still need to bind to modify their password. This patch adds an option to bind at the beginning of the process so that you can pass '-E' to ldappasswd and it will bind early in the process so that the process is the same as the standard passwd. All it does is run the bind towards the beginning of the process instead of the end. The attached patch file is derived from OpenLDAP Software. All of the modifications to OpenLDAP Software represented in the following patch(es) were developed by Randall Mason randall(a)mason.ch. I have not assigned rights and/or interest in this work to any party. I, Randall Mason, hereby place the following modifications to OpenLDAP Software (and only these modifications) into the public domain. Hence, these modifications may be freely used and/or redistributed for any purpose with or without attribution and/or other notice.

1 0

(ITS#8902) man page update for slapadd(5)
by quanah＠openldap.org 20 Aug '18

20 Aug '18

Full_Name: Quanah Gibson-Mount Version: 2.4.46 OS: N/A URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (47.208.148.239) The man page for slapadd's -w option needs to be updated to note that it's generally only safe to use with a cold slapcat.

1 0

(ITS#8901) slapd segfaults in mdb_env_reader_dest
by ckowalczyk＠suse.com 20 Aug '18

20 Aug '18

Full_Name: Chris Kowalczyk Version: 2.4.41 OS: SLE12 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (2620:113:80c0:5::2222) Hello, I have been experiencing segfaults in mdb_env_reader_dest happening randomly, only from time to time, during shutting off openldap. After some investigation I narrowed the problem down to mdb_env_reader_dest destructor function being called after its object had been deleted while powering off the application. mdb_env_reader_dest was called automatically, because it was agginned to the object in pthread_key_create call: https://github.com/openldap/openldap/blob/master/libraries/liblmdb/mdb.c#L4… The problem was not happening often, and always during powering off, so no data was lost etc. However, due to its annoyance, I prepared a simple patch, which prevents mdb_env_reader_dest function being called if the object was already deleted. The patch works fine, I haven't seen the problem since it was introduced. I am using OpenLDAP 2.4.41, but the code seems to be the same in other versions as well. Would you like me to create a pull request for it? ========================== === The backtrace: ========================== Program terminated with signal 11, Segmentation fault. #0 mdb_env_reader_dest (ptr=0x7f4a9c71c080) at ../../../libraries/liblmdb/mdb.c:4050 4050 reader->mr_pid = 0; #thread apply all bt Thread 2 (LWP 20920): #0 0x00007f4a9ac31d9d in close () at ../sysdeps/unix/syscall-template.S:84 #1 0x000055902eaeae8c in mdb_env_close1 (env=env@entry=0x55902fce50e0) at ../../../libraries/liblmdb/mdb.c:4756 #2 0x000055902eaeb578 in mdb_env_close1 (env=0x55902fce50e0) at ../../../libraries/liblmdb/mdb.c:4779 #3 mdb_env_close (env=0x55902fce50e0) at ../../../libraries/liblmdb/mdb.c:4778 #4 0x000055902ea69114 in mdb_db_close (be=0x55902faeb300, cr=<optimized out>) at init.c:340 #5 0x000055902ea3815b in over_db_close (be=0x55902faeb300, cr=0x0) at backover.c:182 #6 0x000055902e9d8c0b in backend_shutdown (be=0x55902faeb300, be@entry=0x0) at backend.c:376 #7 0x000055902e9fa258 in slap_shutdown (be=be@entry=0x0) at init.c:232 #8 0x000055902e9af12f in main (argc=<optimized out>, argv=<optimized out>) at main.c:1027 Thread 1 (LWP 20928): #0 mdb_env_reader_dest (ptr=0x7f4a9c71c080) at ../../../libraries/liblmdb/mdb.c:4050 #1 0x00007f4a9ac2a512 in __nptl_deallocate_tsd () at pthread_create.c:176 #2 0x00007f4a9ac2a767 in start_thread (arg=0x7f4a97acc700) at pthread_create.c:347 #3 0x00007f4a9a968aad in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109 # frame 0 #0 mdb_env_reader_dest (ptr=0x7f4a9c71c080) at ../../../libraries/liblmdb/mdb.c:4050 4050 reader->mr_pid = 0; #p reader $1 = (MDB_reader *) 0x7f4a9c71c080 #p *reader Cannot access memory at address 0x7f4a9c71c080 ========================== === The proposed patch: ========================== diff --git a/libraries/liblmdb/mdb.c b/libraries/liblmdb/mdb.c index 6bdf3151d..56212151b 100644 --- a/libraries/liblmdb/mdb.c +++ b/libraries/liblmdb/mdb.c @@ -4692,6 +4692,11 @@ mdb_env_close0(MDB_env *env, int excl) if (env->me_flags & MDB_ENV_TXKEY) { pthread_key_delete(env->me_txkey); + + // No need to call desctructor anymore, as all pid + // values are cleared below. + env->me_txkey = NULL; + #ifdef _WIN32 /* Delete our key from the global list */ for (i=0; i<mdb_tls_nkeys; i++)

1 0

(ITS#8900) contextCSN issue for export if last op was to delete an entry
by quanah＠openldap.org 17 Aug '18

17 Aug '18

Full_Name: Quanah Gibson-Mount Version: 2.4.46 OS: N/A URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (47.208.148.239) When making an export for backup using slapcat there could be issues on restore if the last operation logged was for a delete entry operation, as the contextCSN would then point to an entry that no longer exists. If that backup was then used at restoration time in an replicated environment and the master gets a write op before any replicas are online, the replicas will fallback to REFRESH mode since they can't find entry the contextCSN points to.

1 0

(ITS#8899) SLAPD failed to start after updating openldap 2.4.46
by muthaiyan1991＠gmail.com 17 Aug '18

17 Aug '18

Full_Name: Muthaiyan Vel Version: 2.4.46 OS: Solaris 10 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (116.197.184.12) I have recently updated openldap to version 2.4.46. After upgrading SLAPD is failed to start. Its showing an error "slap_sasl_init: auxprop add plugin failed". I have compiled the openldap with mdb. Dependents used for openldap 2.4.46: cyrus-sasl-2.1.26 openssl-1.0.2o krb5-1.16.1 Steps to Compile OpenLDAP and Dependency: sasl Compiled without LDAP: -------------------------- export LDFLAGS=" -m64 -L/tmp/CODE_BASE/Openldap_Upgrade/openssl-1.0.2o -L/tmp/CODE_BASE/Openldap_Upgrade/krb5-1.16.1/src/lib -L/tmp/CODE_BASE/Openldap_Upgrade/krb5-1.16.1/src/lib/gssapi/" ./configure --enable-auth-sasldb --disable-ldapdb --with-openssl=/tmp/CODE_BASE/Openldap_Upgrade/openssl-1.0.2o --enable-gssapi=/tmp/CODE_BASE/Openldap_Upgrade/krb5-1.16.1/src/lib/gssapi --with-krb5-lib=/tmp/CODE_BASE/Openldap_Upgrade/krb5-1.16.1/src/lib --with-krb5-include=/tmp/CODE_BASE/Openldap_Upgrade/krb5-1.16.1/src/include make openldap: --------- export LDFLAGS=" -m64 -L/tmp/CODE_BASE/Openldap_Upgrade/openssl-1.0.2o/ -L/tmp/CODE_BASE/Openldap_Upgrade/krb5-1.16.1/src/lib/ -L/tmp/CODE_BASE/Openldap_Upgrade/krb5-1.16.1/src/lib/gssapi/ -L/tmp/CODE_BASE/Openldap_Upgrade/cyrus-sasl-2.1.26/lib/.libs/ -L/tmp/CODE_BASE/Openldap_Upgrade/cyrus-sasl-2.1.26/plugins/.libs/" ./configure --enable-debug --with-tls=openssl --with-cyrus-sasl --enable-slapd --enable-bdb=no --enable-hdb=no --enable-mdb=yes --enable-spasswd make sasl Compiled with LDAP: ------------------------ export LDFLAGS=" -m64 -L/tmp/CODE_BASE/Openldap_Upgrade/openssl-1.0.2o/ -L/tmp/CODE_BASE/Openldap_Upgrade/krb5-1.16.1/src/lib/ -L/tmp/CODE_BASE/Openldap_Upgrade/krb5-1.16.1/src/lib/gssapi/ -L/tmp/CODE_BASE/Openldap_Upgrade/openldap-2.4.46/libraries/libldap_r/.libs/ -L/tmp/CODE_BASE/Openldap_Upgrade/openldap-2.4.46/libraries/liblber/.libs/ -L/tmp/CODE_BASE/Openldap_Upgrade/openldap-2.4.46/libraries/libldap/.libs/" make distclean ./configure --enable-auth-sasldb --disable-ldapdb --with-openssl=/tmp/CODE_BASE/Openldap_Upgrade/openssl-1.0.2o/ --enable-gssapi=/tmp/CODE_BASE/Openldap_Upgrade/krb5-1.16.1/src/lib/gssapi/ --with-krb5-lib=/tmp/CODE_BASE/Openldap_Upgrade/krb5-1.16.1/src/lib/ --with-krb5-include=/tmp/CODE_BASE/Openldap_Upgrade/krb5-1.16.1/src/include/ make ./configure --enable-auth-sasldb --with-ldap=/tmp/CODE_BASE/Openldap_Upgrade/openldap-2.4.46/libraries/ --enable-ldapdb --with-openssl=/tmp/CODE_BASE/Openldap_Upgrade/openssl-1.0.2o/ --enable-gssapi=/tmp/CODE_BASE/Openldap_Upgrade/krb5-1.16.1/src/lib/gssapi/ --with-krb5-lib=/tmp/CODE_BASE/Openldap_Upgrade/krb5-1.16.1/src/lib/ --with-krb5-include=/tmp/CODE_BASE/Openldap_Upgrade/krb5-1.16.1/src/include/ make Used these libraries from the above compiled source code: libldap_r-2.4.so.2 liblber-2.4.so.2 libsasl2.so.3 slapd Full Debug Log: ldap_url_parse_ext(ldap://localhost/) ldap_init: trying /usr/local/etc/openldap/ldap.conf ldap_init: using /usr/local/etc/openldap/ldap.conf ldap_init: HOME env is / ldap_init: trying //ldaprc ldap_init: trying //.ldaprc ldap_init: trying ldaprc ldap_init: LDAPCONF env is NULL ldap_init: LDAPRC env is NULL 5b770206 @(#) $OpenLDAP: slapd 2.4.46 (Aug 16 2018 21:16:43) $ root@bng-sbr-perf1:/export/local/jai/CODE_BASE/Openldap_Upgrade/openldap-2.4.46/servers/slapd ldap_pvt_gethostbyname_a: host=bng-sbr-perf1, r=0 5b770206 daemon_init: ldap://127.0.0.1:389 5b770206 daemon_init: listen on ldap://127.0.0.1:389 5b770206 daemon_init: 1 listeners to open... ldap_url_parse_ext(ldap://127.0.0.1:389) 5b770206 daemon: listener initialized ldap://127.0.0.1:389 5b770206 daemon_init: 1 listeners opened ldap_create 5b770206 slapd init: initiated server. 5b770206 slap_sasl_init: auxprop add plugin failed 5b770206 slapd destroy: freeing system resources. 5b770206 slapd stopped. 5b770206 connections_destroy: nothing to destroy. Please help to solve this problem.

1 0

(ITS#8898) mdb_get on corrupted LMDB file crash the process
by iradization＠gmail.com 15 Aug '18

15 Aug '18

Full_Name: Irad.k Version: recent from GitHub. OS: macOS URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (82.81.84.130) Hi, I'm working with LMDB with the following flags : MDB_NOMETASYNC | MDB_NOSUBDIR | MDB_NOTLS. Somehow, even though the DB should be ACI(without the D), it got corrupted after recovering from kernel panic, and It crashes my process when trying to access one of the records (see crash log below). Here's a link to the file : https://drive.google.com/file/d/12Q3KYYrapiJOgiaccnDL3tQACkqrM3C5/view?usp=… According to the crash log from the process, It can clearly be seen that the invalid address reside inside the mapped file region which is the lmdb mapped file, but still I get KERN_MEMORY_ERROR on that address. >From what I know, an attempt to access address within the mapped range can either retrieve the page contents directly from memory (if it's already there), or trigger page fault trap that eventually lead to reading the missing data from disk and return it to process as well. One thing that raise some concerns is that the file size is only 24k and the mapping spans over 256M. However, the file's meta data seems to be coherent to file contents. Any idea how did it happen, and what exactly in the file cause this corruption ? CRASH LOG : ---------------------------------------------------------------------- Exception Type: EXC_BAD_ACCESS (SIGBUS) Exception Codes: KERN_MEMORY_ERROR at 0x000000010648800a Exception Note: EXC_CORPSE_NOTIFY Termination Signal: Bus error: 10 Termination Reason: Namespace SIGNAL, Code 0xa Terminating Process: exc handler [0] VM Regions Near 0x10648800a: __LINKEDIT 0000000106464000-000000010647f000 [ 108K] r--/rwx SM=COW ^Z^C [/usr/lib/dyld] --> mapped file 000000010647f000-000000011647f000 [256.0M] r--/rwx SM=PRV Object_id=9034edd9 STACK GUARD 000070000f2b3000-000070000f2b4000 [ 4K] ---/rwx SM=NUL stack guard for thread 1 Thread 0 Crashed:: Dispatch queue: com.apple.main-thread 0 myprog 0x0000000101666756 mdb_page_search_root + 39 1 myprog 0x00000001016660f7 mdb_page_search + 182 2 myprog 0x00000001016614de mdb_cursor_set + 88 3 myprog 0x0000000101661476 mdb_get + 134

1 0

Re: (ITS#8897) Delta syncrepl + refreshOnly segfault if 2 servers receive same group modification operation within interval
by Mark.Cairney＠ed.ac.uk 13 Aug '18

13 Aug '18

Hi Quanah, It certainly sounds very similar from the description. I've applied the patch on our Dev systems and tried to recreate the crash but this time they don't crash and the change appears to make it to all 4 servers. Based on that I'm happy to close it as a duplicate. For completeness here's the log snippet from one of the 2 servers not to get the change directly: 2018-08-13T15:51:41.309019+01:00 bonsai.authorise-dev.is.ed.ac.uk slapd[33415]: conn=2891 op=1 SEARCH RESULT tag=101 err=0 nentries=0 text= 2018-08-13T15:51:41.310958+01:00 bonsai.authorise-dev.is.ed.ac.uk slapd[33415]: conn=2891 op=2 UNBIND 2018-08-13T15:51:41.311320+01:00 bonsai.authorise-dev.is.ed.ac.uk slapd[33415]: conn=2891 fd=17 closed 2018-08-13T15:52:18.347529+01:00 bonsai.authorise-dev.is.ed.ac.uk slapd[33415]: do_syncrep2: rid=003 cookie=rid=003,sid=00a,csn=20180810152745.105428Z#000000#009#000000 2018-08-13T15:52:18.347925+01:00 bonsai.authorise-dev.is.ed.ac.uk slapd[33415]: do_syncrep2: rid=003 CSN too old, ignoring 20180810152745.105428Z#000000#009#000000 (reqStart=20180810152801.000001Z,cn=accesslog) 2018-08-13T15:52:19.361612+01:00 bonsai.authorise-dev.is.ed.ac.uk slapd[33415]: do_syncrep2: rid=004 cookie=rid=004,sid=00c,csn=20180813145040.294725Z#000000#00c#000000 2018-08-13T15:52:19.361936+01:00 bonsai.authorise-dev.is.ed.ac.uk slapd[33415]: slap_queue_csn: queueing 0x4279380 20180813145040.294725Z#000000#00c#000000 2018-08-13T15:52:19.362187+01:00 bonsai.authorise-dev.is.ed.ac.uk slapd[33415]: syncrepl_message_to_op: rid=004 tid 6f1f6700 2018-08-13T15:52:19.378677+01:00 bonsai.authorise-dev.is.ed.ac.uk slapd[33415]: slap_queue_csn: queueing 0x3e067c0 20180813145040.294725Z#000000#00c#000000 2018-08-13T15:52:19.380316+01:00 bonsai.authorise-dev.is.ed.ac.uk slapd[33415]: slap_graduate_commit_csn: removing 0x3e067c0 20180813145040.294725Z#000000#00c#000000 2018-08-13T15:52:19.380570+01:00 bonsai.authorise-dev.is.ed.ac.uk slapd[33415]: slap_graduate_commit_csn: removing 0x4279380 20180813145040.294725Z#000000#00c#000000 2018-08-13T15:52:19.380819+01:00 bonsai.authorise-dev.is.ed.ac.uk slapd[33415]: syncrepl_message_to_op: rid=004 be_modify uid=jaw,ou=people,ou=central,dc=authorise-dev,dc=ed,dc=ac,dc=uk (0) 2018-08-13T15:52:19.381044+01:00 bonsai.authorise-dev.is.ed.ac.uk slapd[33415]: slap_queue_csn: queueing 0x419d5c0 20180813145040.294725Z#000000#00c#000000 2018-08-13T15:52:19.385145+01:00 bonsai.authorise-dev.is.ed.ac.uk slapd[33415]: slap_graduate_commit_csn: removing 0x419d5c0 20180813145040.294725Z#000000#00c#000000 2018-08-13T15:52:19.385403+01:00 bonsai.authorise-dev.is.ed.ac.uk slapd[33415]: do_syncrep2: rid=004 LDAP_RES_SEARCH_RESULT 2018-08-13T15:52:19.385654+01:00 bonsai.authorise-dev.is.ed.ac.uk slapd[33415]: do_syncrep2: rid=004 cookie=rid=004,sid=00c,csn=20120217162731.749366Z#000000#000#000000;20180701051000.119854Z#000000#003#000000;20180711011500.114848Z#000000#004#000000;20180710075708.402194Z#000000#005#000000;20180703112408.905121Z#000000#006#000000;20180813144557.086593Z#000000#009#000000;20180813145034.034818Z#000000#00b#000000;20180813145040.294725Z#000000#00c#000000 Kind regards, Mark On 10/08/18 17:00, Quanah Gibson-Mount wrote: > --On Friday, August 10, 2018 4:41 PM +0000 Mark.Cairney(a)ed.ac.uk wrote: > >> Full_Name: Mark Cairney >> Version: 2.4.46 >> OS: Centos 7 >> URL: >> Submission from: (NULL) (129.215.149.98) >> >> >> In a MMR setup of >=3 servers (4 in this case) you can cause a segfault >> if you issue the same modification to 2 separate servers within your >> replication interval. >> It seems like it's the last server to receive the change falls over when >> it receives the change from one of the other servers during replication: > > Hi Mark, > > I'm pretty sure you're reporting ITS#8843. I suggest applying this patch: > > <https://www.openldap.org/devel/gitweb.cgi?p=openldap.git;a=commit;h=cc24cf6…> > > > --Quanah > > > > -- > > Quanah Gibson-Mount > Product Architect > Symas Corporation > Packaged, certified, and supported LDAP solutions powered by OpenLDAP: > <http://www.symas.com> > > -- /**************************** Mark Cairney ITI Enterprise Services Information Services University of Edinburgh Tel: 0131 650 6565 Email: Mark.Cairney(a)ed.ac.uk PGP: 0x435A9621 *******************************/ The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.

1 0

Re: (ITS#8897) Delta syncrepl + refreshOnly segfault if 2 servers receive same group modification operation within interval
by quanah＠symas.com 10 Aug '18

10 Aug '18

--On Friday, August 10, 2018 4:41 PM +0000 Mark.Cairney(a)ed.ac.uk wrote: > Full_Name: Mark Cairney > Version: 2.4.46 > OS: Centos 7 > URL: > Submission from: (NULL) (129.215.149.98) > > > In a MMR setup of >=3 servers (4 in this case) you can cause a segfault > if you issue the same modification to 2 separate servers within your > replication interval. > It seems like it's the last server to receive the change falls over when > it receives the change from one of the other servers during replication: Hi Mark, I'm pretty sure you're reporting ITS#8843. I suggest applying this patch: <https://www.openldap.org/devel/gitweb.cgi?p=openldap.git;a=commit;h=cc24cf6…> --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

(ITS#8897) Delta syncrepl + refreshOnly segfault if 2 servers receive same group modification operation within interval
by Mark.Cairney＠ed.ac.uk 10 Aug '18

10 Aug '18

Full_Name: Mark Cairney Version: 2.4.46 OS: Centos 7 URL: Submission from: (NULL) (129.215.149.98) In a MMR setup of >=3 servers (4 in this case) you can cause a segfault if you issue the same modification to 2 separate servers within your replication interval. It seems like it's the last server to receive the change falls over when it receives the change from one of the other servers during replication: 2018-08-10T16:01:39.461367+01:00 spruce.authorise-dev.is.ed.ac.uk slapd[11286]: conn=1004 op=3 BIND authcid="mcairney(a)EASE.ED.AC.UK" authzid="mcairney(a)EASE.ED.AC.UK" 2018-08-10T16:01:39.461843+01:00 spruce.authorise-dev.is.ed.ac.uk slapd[11286]: conn=1004 op=3 BIND dn="uid=mcairney,ou=people,ou=central,dc=authorise-dev,dc=ed,dc=ac,dc=uk" mech=GSSAPI sasl_ssf=256 ssf=256 2018-08-10T16:01:39.462205+01:00 spruce.authorise-dev.is.ed.ac.uk slapd[11286]: conn=1004 op=3 RESULT tag=97 err=0 text= 2018-08-10T16:01:39.462858+01:00 spruce.authorise-dev.is.ed.ac.uk slapd[11286]: conn=1004 op=4 MOD dn="cn=marksgroup,ou=ug,ou=iti,ou=is,dc=authorise-dev,dc=ed,dc=ac,dc=uk" 2018-08-10T16:01:39.463268+01:00 spruce.authorise-dev.is.ed.ac.uk slapd[11286]: conn=1004 op=4 MOD attr=member 2018-08-10T16:01:39.463703+01:00 spruce.authorise-dev.is.ed.ac.uk slapd[11286]: slap_queue_csn: queueing 0x46a1a00 20180810150139.461113Z#000000#00c#000000 2018-08-10T16:01:39.464026+01:00 spruce.authorise-dev.is.ed.ac.uk slapd[11286]: slap_queue_csn: queueing 0x46a0540 20180810150139.461113Z#000000#00c#000000 2018-08-10T16:01:39.464554+01:00 spruce.authorise-dev.is.ed.ac.uk slapd[11286]: slap_graduate_commit_csn: removing 0x46a0540 20180810150139.461113Z#000000#00c#000000 2018-08-10T16:01:39.464922+01:00 spruce.authorise-dev.is.ed.ac.uk slapd[11286]: conn=1004 op=4 RESULT tag=103 err=0 text= 2018-08-10T16:01:39.465271+01:00 spruce.authorise-dev.is.ed.ac.uk slapd[11286]: slap_graduate_commit_csn: removing 0x46a1a00 20180810150139.461113Z#000000#00c#000000 2018-08-10T16:01:39.465647+01:00 spruce.authorise-dev.is.ed.ac.uk slapd[11286]: conn=1004 op=5 UNBIND 2018-08-10T16:01:39.465962+01:00 spruce.authorise-dev.is.ed.ac.uk slapd[11286]: conn=1004 fd=12 closed 2018-08-10T16:01:59.042478+01:00 spruce.authorise-dev.is.ed.ac.uk slapd[11286]: conn=1005 fd=12 ACCEPT from IP=129.215.17.49:59638 (IP=129.215.17.52:636) 2018-08-10T16:01:59.045948+01:00 spruce.authorise-dev.is.ed.ac.uk slapd[11286]: conn=1005 fd=12 TLS established tls_ssf=256 ssf=256 2018-08-10T16:01:59.046828+01:00 spruce.authorise-dev.is.ed.ac.uk slapd[11286]: conn=1005 op=0 BIND dn="cn=manager,dc=authorise-dev,dc=ed,dc=ac,dc=uk" method=128 2018-08-10T16:01:59.052303+01:00 spruce.authorise-dev.is.ed.ac.uk slapd[11286]: conn=1005 op=0 BIND dn="cn=manager,dc=authorise-dev,dc=ed,dc=ac,dc=uk" mech=SIMPLE ssf=0 2018-08-10T16:01:59.052924+01:00 spruce.authorise-dev.is.ed.ac.uk slapd[11286]: conn=1005 op=0 RESULT tag=97 err=0 text= 2018-08-10T16:01:59.053419+01:00 spruce.authorise-dev.is.ed.ac.uk slapd[11286]: conn=1005 op=1 SRCH base="cn=accesslog" scope=2 deref=0 filter="(&(objectClass=auditWriteObject)(reqResult=0))" 2018-08-10T16:01:59.053912+01:00 spruce.authorise-dev.is.ed.ac.uk slapd[11286]: conn=1005 op=1 SRCH attr=reqDN reqType reqMod reqNewRDN reqDeleteOldRDN reqNewSuperior entryCSN 2018-08-10T16:01:59.054366+01:00 spruce.authorise-dev.is.ed.ac.uk slapd[11286]: Entry reqStart=20180810144019.000000Z,cn=accesslog CSN 20180810144019.929847Z#000000#00c#000000 older or equal to ctx 20180810144019.929847Z#000000#00c#000000 2018-08-10T16:01:59.054902+01:00 spruce.authorise-dev.is.ed.ac.uk slapd[11286]: Entry reqStart=20180810144922.000002Z,cn=accesslog CSN 20180810144822.605802Z#000000#009#000000 older or equal to ctx 20180810150125.942413Z#000000#009#000000 2018-08-10T16:01:59.055330+01:00 spruce.authorise-dev.is.ed.ac.uk slapd[11286]: syncprov_search_response: cookie=rid=004,sid=00c,csn=20120217162731.749366Z#000000#000#000000;20180701051000.119854Z#000000#003#000000;20180711011500.114848Z#000000#004#000000;20180710075708.402194Z#000000#005#000000;20180703112408.905121Z#000000#006#000000;20180810144822.605802Z#000000#009#000000;20180711161057.835548Z#000000#00b#000000;20180810150139.461113Z#000000#00c#000000 2018-08-10T16:01:59.055753+01:00 spruce.authorise-dev.is.ed.ac.uk slapd[11286]: conn=1005 op=1 SEARCH RESULT tag=101 err=0 nentries=1 text= 2018-08-10T16:01:59.056308+01:00 spruce.authorise-dev.is.ed.ac.uk slapd[11286]: conn=1005 op=2 ABANDON msg=2 2018-08-10T16:01:59.056735+01:00 spruce.authorise-dev.is.ed.ac.uk slapd[11286]: conn=1005 op=3 SRCH base="dc=authorise-dev,dc=ed,dc=ac,dc=uk" scope=2 deref=0 filter="(objectClass=*)" 2018-08-10T16:01:59.057148+01:00 spruce.authorise-dev.is.ed.ac.uk slapd[11286]: conn=1005 op=3 SRCH attr=* + 2018-08-10T16:02:06.066075+01:00 spruce.authorise-dev.is.ed.ac.uk slapd[11286]: do_syncrep2: rid=002 LDAP_RES_SEARCH_RESULT 2018-08-10T16:02:06.075530+01:00 spruce.authorise-dev.is.ed.ac.uk slapd[11286]: do_syncrep2: rid=001 cookie=rid=001,sid=009,csn=20180810150125.942413Z#000000#009#000000 2018-08-10T16:02:06.076055+01:00 spruce.authorise-dev.is.ed.ac.uk slapd[11286]: slap_queue_csn: queueing 0x46a0380 20180810150125.942413Z#000000#009#000000 2018-08-10T16:02:06.076275+01:00 spruce.authorise-dev.is.ed.ac.uk slapd[11286]: syncrepl_message_to_op: rid=001 tid f8504700 The following gets logged in /var/log/messages: 2018-08-10T15:22:25.582679+01:00 spruce.authorise-dev.is.ed.ac.uk su: (to ldap) root on pts/0 2018-08-10T15:59:50.381730+01:00 spruce.authorise-dev.is.ed.ac.uk su: (to ldap) root on pts/0 2018-08-10T16:02:06.076563+01:00 spruce.authorise-dev.is.ed.ac.uk kernel: slapd[11289]: segfault at 1c ip 000000000051c6d4 sp 00007fcbf85022b0 error 4 in slapd[400000+21e000] Even worse is that because the change is stuck in it's accesslog if you try and restart slapd it just falls over again: 2018-08-10T16:12:06.020868+01:00 spruce.authorise-dev.is.ed.ac.uk slapd[11451]: @(#) $OpenLDAP: slapd 2.4.46 (Mar 27 2018 09:23:46) $#012#011mockbuild@harmon.is.ed.ac.uk:/builddir/build/BUILD/openldap-2.4.46/servers/slapd 2018-08-10T16:12:06.052521+01:00 spruce.authorise-dev.is.ed.ac.uk slapd[11453]: slapd starting 2018-08-10T16:12:06.067909+01:00 spruce.authorise-dev.is.ed.ac.uk slapd[11453]: do_syncrep2: rid=002 cookie=rid=002,sid=00b,csn=20180810150125.942413Z#000000#009#000000 2018-08-10T16:12:06.068382+01:00 spruce.authorise-dev.is.ed.ac.uk slapd[11453]: slap_queue_csn: queueing 0x4252a40 20180810150125.942413Z#000000#009#000000 2018-08-10T16:12:06.068652+01:00 spruce.authorise-dev.is.ed.ac.uk slapd[11453]: syncrepl_message_to_op: rid=002 tid ce43f700 2018-08-10T16:12:06.068871+01:00 spruce.authorise-dev.is.ed.ac.uk slapd[11453]: do_syncrep2: rid=001 cookie=rid=001,sid=009,csn=20180810150125.942413Z#000000#009#000000 Finally you can bring the server back up if you clear out the accesslog and restart again. The ldif to trigger it was simply: dn: cn=marksgroup,ou=ug,ou=iti,ou=is,dc=authorise-dev,dc=ed,dc=ac,dc=uk changetype: modify add: member member: uid=mcairney,ou=people,ou=central,dc=authorise-dev,dc=ed,dc=ac,dc=uk - Applied using ldapmodify -H ldaps://<server>.authorise-dev.is.ed.ac.uk:636 -f file.ldif Deleting a member also triggers the same behaviour Changing user attributes e.g. mail in the same manner also triggers the issue. Adding/deleting an object doesn't cause a segfault but simply triggers a resync on each server.

1 0

Re: (ITS#8896) scale() uninitialized j
by hyc＠symas.com 09 Aug '18

09 Aug '18

ethanms(a)gmail.com wrote: > Full_Name: Ethan Schorer > Version: since 2.4 > OS: Linux > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (194.29.45.108) > > > In libraries/liblutil/utils.c in the function scale() the code defines "int i, > j;". > > Then, i gets set in a for loop, but j only gets set if prev->len > 1. Wrong. j gets set if prev->len >= 1. > And then, j is used even if it wasn't initialized. Isn't this a bug? No. > Shouldn't j be set to 0 when defined? Or don't use it? There is no bug here. Closing this ITS. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#8896) scale() uninitialized j
by ethanms＠gmail.com 09 Aug '18

09 Aug '18

Full_Name: Ethan Schorer Version: since 2.4 OS: Linux URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (194.29.45.108) In libraries/liblutil/utils.c in the function scale() the code defines "int i, j;". Then, i gets set in a for loop, but j only gets set if prev->len > 1. And then, j is used even if it wasn't initialized. Isn't this a bug? Shouldn't j be set to 0 when defined? Or don't use it?

1 0

Re: (ITS#8895) LMDB bug on Windows 64 when built by MSVC
by hyc＠symas.com 09 Aug '18

09 Aug '18

hrissan(a)mail.ru wrote: > Full_Name: Hrissan > Version: lmdb commit 26c7df88e44e31623d0802a564f24781acdefde3 > OS: Windows 7 64-bit > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (108.61.166.173) > > > Dear Howard, lots of thanks for quick fix of ITS#8891, now another issue... > > I know your position that Microsoft Visual Studio is crap (it may be, but not > everyone has a choice), but here is the problem: > > When building LMDB targeting Win64 using 64-bit compiler in Microsoft Visual > Studio, the database will still be limited to just 2 gigabytes, with mdb_put > returning error as soon as database attempts to grow beyond that. > > The problem is "off_t" is incorrectly defined as "long" in MSVC posix > compatibility layer, and long is 32-bit on windows 64-bit build. As a result > mdb_page_flush (and other functions using off_t) become broken. > > I understand this is Microsoft's, not LMDB issue, but still LMDB source contains > lots of workarounds, so probably adding one will save the day for many ppl. > Microsoft has this issue since at least 2008 (And it is still there in latest > Visual Studio Community 2017 version 15.7.6), so no hope for fix from them. > I propose the following fix (as narrow as possible - for example no fix for > MinGW on Windows or MSVC on Linux) There's no logic in this proposal. The bug is clearly in the MS header files; the correct fix is to edit the MS header files and use the proper definition. Fix the actual bugs. Report the actual bugs to the people that are actually responsible for them. > > #if defined(_WIN32) && defined(_MSC_VER) > typedef long long int lmdb_off_t; // 64-bit on both 32- and 64- bit Windows > #else > typedef off_t lmdb_off_t; > #endif > > And using lmdb_off_t instead of off_t on 4 occasions where it is used. > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs