openldap-bugs

openldap-bugs@openldap.org

1 participants
21001 discussions

(ITS#8914) OpenLDAP and OpenSSL v1.1.1
by nea.brown＠threesl.com 13 Sep '18

13 Sep '18

Full_Name: Neal Brown Version: 2.4.46 OS: Linux URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (88.98.37.245) OpenLDAP will not configure for building against OpenSSL v1.1.1 due to SSLv3 being disabled.

1 0

Re: (ITS#8898) mdb_get on corrupted LMDB file crash the process
by hyc＠symas.com 12 Sep '18

12 Sep '18

Irad K wrote: > Hi Howard and thanks for the info.=C2=A0 >=20 > I've read your insights about the corruption nature and I'd like to see= if I can write some sort of integrity check before I try to open the fil= e.=C2=A0 > Perhaps you could guide me where to look for the file format, where can= I find all the meta-pages and in which offset can I find the used pages = ? Read the source yourself if you want to know the underlying format. Other= wise, just use mdb_env_info(). =C2=A0 > Basically, I wish to check the latest meta-page and see if the number o= f pages correspond with the file actual size.=C2=A0 >=20 > Second, as I've stated, my target operating system is macOS.. are you f= amiliar of any known issues with file atomicity in this particular OS ?=C2= =A0 > Perhaps you could tell me how it's implemented under the hood to achiev= e data consistency if I'm using the database using MDB_NOMETASYNC, so I f= urther > investigate this issue and address it to Apple if necessary.=20 I've seen other reports of corruptions on MacOSX. Since they're in SQLite= I haven't investigated further. http://sqlite.1065341.n5.nabble.com/SQLITE-vs-OSX-mmap-inevitable-catalog= -corruption-td85620.html http://sqlite.1065341.n5.nabble.com/Re-Database-corruption-and-PRAGMA-ful= lfsync-on-macOS-td95366i20.html >=20 > Irad >=20 > On Mon, Sep 10, 2018 at 8:44 PM Howard Chu <hyc(a)symas.com <mailto:hyc@s= ymas.com>> wrote: >=20 > iradization(a)gmail.com <mailto:iradization@gmail.com> wrote: > > Full_Name: Irad.k > > Version: recent from GitHub. > > OS: macOS > > URL: ftp://ftp.openldap.org/incoming/ > > Submission from: (NULL) (82.81.84.130) > > > > > > Hi, > > > > I'm working with LMDB with the following flags :=C2=A0 MDB_NOMETA= SYNC | MDB_NOSUBDIR > > | MDB_NOTLS. > > > > Somehow, even though the DB should be ACI(without the D),=C2=A0 i= t got corrupted > > after recovering from kernel panic, and It crashes my process whe= n trying to > > access one of the records (see crash log below). > > > > Here's a link to the file : > > > > https://drive.google.com/file/d/12Q3KYYrapiJOgiaccnDL3tQACkqrM3C5= /view?usp=3Dsharing >=20 > This file is only 6 pages long, but the latest meta page claims tha= t it's using 11 pages. > Looking at the previous meta page, it claims that 9 pages were used= . >=20 > Clearly your OS failed to write the complete contents out before it= panicked. At the > very least the file should have been 9 pages long. Sounds like you = have an OS bug. >=20 > > According to the crash log from the process, It can clearly be se= en that the > > invalid address reside inside the mapped file region which is the= lmdb mapped > > file, but still I get KERN_MEMORY_ERROR on that address. > > > >>From what I know, an attempt to access address within the mapped = range can > > either retrieve the page contents directly from memory (if it's a= lready there), > > or trigger page fault trap that eventually lead to reading the mi= ssing data from > > disk and return it to process as well. > > > > One thing that raise some concerns is that the file size is only = 24k and the > > mapping spans over 256M. However, the file's meta data seems to b= e coherent to > > file contents. > > > > Any idea how did it happen, and what exactly in the file cause th= is corruption ? > > > > > > CRASH LOG : > > -----------------------------------------------------------------= ----- > > Exception Type:=C2=A0 =C2=A0 =C2=A0 =C2=A0 EXC_BAD_ACCESS (SIGBUS= ) > > Exception Codes:=C2=A0 =C2=A0 =C2=A0 =C2=A0KERN_MEMORY_ERROR at 0= x000000010648800a > > Exception Note:=C2=A0 =C2=A0 =C2=A0 =C2=A0 EXC_CORPSE_NOTIFY > > > > Termination Signal:=C2=A0 =C2=A0 Bus error: 10 > > Termination Reason:=C2=A0 =C2=A0 Namespace SIGNAL, Code 0xa > > Terminating Process:=C2=A0 =C2=A0exc handler [0] > > > > VM Regions Near 0x10648800a: > > > > __LINKEDIT=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00000000= 106464000-000000010647f000 [=C2=A0 108K] r--/rwx SM=3DCOW > >=C2=A0 ^Z^C [/usr/lib/dyld] > > --> mapped file=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 00000001= 0647f000-000000011647f000 [256.0M] r--/rwx > > SM=3DPRV=C2=A0 Object_id=3D9034edd9 > > STACK GUARD=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 000070000f2b= 3000-000070000f2b4000 [=C2=A0 =C2=A0 4K] ---/rwx SM=3DNUL > >=C2=A0 stack guard for thread 1 > > > > Thread 0 Crashed:: Dispatch queue: com.apple.main-thread > > 0=C2=A0 =C2=A0myprog=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00x0000000101666756 mdb_page_search_root + = 39 > > 1=C2=A0 =C2=A0myprog=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00x00000001016660f7 mdb_page_search + 182 > > 2=C2=A0 =C2=A0myprog=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00x00000001016614de mdb_cursor_set + 88 > > 3=C2=A0 =C2=A0myprog=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00x0000000101661476 mdb_get + 134 > > > > > > >=20 >=20 > --=20 > =C2=A0 -- Howard Chu > =C2=A0 CTO, Symas Corp.=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0htt= p://www.symas.com > =C2=A0 Director, Highland Sun=C2=A0 =C2=A0 =C2=A0http://highlandsun= .com/hyc/ > =C2=A0 Chief Architect, OpenLDAP=C2=A0 http://www.openldap.org/proj= ect/ >=20 --=20 -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8898) mdb_get on corrupted LMDB file crash the process
by iradization＠gmail.com 11 Sep '18

11 Sep '18

--000000000000d405310575a6de5e Content-Type: text/plain; charset="UTF-8" Hi Howard and thanks for the info. I've read your insights about the corruption nature and I'd like to see if I can write some sort of integrity check before I try to open the file. Perhaps you could guide me where to look for the file format, where can I find all the meta-pages and in which offset can I find the used pages ? Basically, I wish to check the latest meta-page and see if the number of pages correspond with the file actual size. Second, as I've stated, my target operating system is macOS.. are you familiar of any known issues with file atomicity in this particular OS ? Perhaps you could tell me how it's implemented under the hood to achieve data consistency if I'm using the database using MDB_NOMETASYNC, so I further investigate this issue and address it to Apple if necessary. Irad On Mon, Sep 10, 2018 at 8:44 PM Howard Chu <hyc(a)symas.com> wrote: > iradization(a)gmail.com wrote: > > Full_Name: Irad.k > > Version: recent from GitHub. > > OS: macOS > > URL: ftp://ftp.openldap.org/incoming/ > > Submission from: (NULL) (82.81.84.130) > > > > > > Hi, > > > > I'm working with LMDB with the following flags : MDB_NOMETASYNC | > MDB_NOSUBDIR > > | MDB_NOTLS. > > > > Somehow, even though the DB should be ACI(without the D), it got > corrupted > > after recovering from kernel panic, and It crashes my process when > trying to > > access one of the records (see crash log below). > > > > Here's a link to the file : > > > > > https://drive.google.com/file/d/12Q3KYYrapiJOgiaccnDL3tQACkqrM3C5/view?usp=… > > This file is only 6 pages long, but the latest meta page claims that it's > using 11 pages. > Looking at the previous meta page, it claims that 9 pages were used. > > Clearly your OS failed to write the complete contents out before it > panicked. At the > very least the file should have been 9 pages long. Sounds like you have an > OS bug. > > > According to the crash log from the process, It can clearly be seen that > the > > invalid address reside inside the mapped file region which is the lmdb > mapped > > file, but still I get KERN_MEMORY_ERROR on that address. > > > >>From what I know, an attempt to access address within the mapped range > can > > either retrieve the page contents directly from memory (if it's already > there), > > or trigger page fault trap that eventually lead to reading the missing > data from > > disk and return it to process as well. > > > > One thing that raise some concerns is that the file size is only 24k and > the > > mapping spans over 256M. However, the file's meta data seems to be > coherent to > > file contents. > > > > Any idea how did it happen, and what exactly in the file cause this > corruption ? > > > > > > CRASH LOG : > > ---------------------------------------------------------------------- > > Exception Type: EXC_BAD_ACCESS (SIGBUS) > > Exception Codes: KERN_MEMORY_ERROR at 0x000000010648800a > > Exception Note: EXC_CORPSE_NOTIFY > > > > Termination Signal: Bus error: 10 > > Termination Reason: Namespace SIGNAL, Code 0xa > > Terminating Process: exc handler [0] > > > > VM Regions Near 0x10648800a: > > > > __LINKEDIT 0000000106464000-000000010647f000 [ 108K] > r--/rwx SM=COW > > ^Z^C [/usr/lib/dyld] > > --> mapped file 000000010647f000-000000011647f000 [256.0M] > r--/rwx > > SM=PRV Object_id=9034edd9 > > STACK GUARD 000070000f2b3000-000070000f2b4000 [ 4K] > ---/rwx SM=NUL > > stack guard for thread 1 > > > > Thread 0 Crashed:: Dispatch queue: com.apple.main-thread > > 0 myprog 0x0000000101666756 mdb_page_search_root + > 39 > > 1 myprog 0x00000001016660f7 mdb_page_search + 182 > > 2 myprog 0x00000001016614de mdb_cursor_set + 88 > > 3 myprog 0x0000000101661476 mdb_get + 134 > > > > > > > > > -- > -- Howard Chu > CTO, Symas Corp. http://www.symas.com > Director, Highland Sun http://highlandsun.com/hyc/ > Chief Architect, OpenLDAP http://www.openldap.org/project/ > --000000000000d405310575a6de5e Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div dir=3D"ltr"><div dir=3D"ltr">Hi Howard and thanks for= the info.=C2=A0<div><br><div>I've read your insights about the corrupt= ion nature and I'd like to see if I can write some sort of integrity ch= eck before I try to open the file.=C2=A0</div><div>Perhaps you could guide = me where to look for the file format, where can I find all the meta-pages a= nd in which offset can I find the used pages ?=C2=A0</div><div>Basically, I= wish to check the latest meta-page and see if the number of pages correspo= nd with the file actual size.=C2=A0<br></div><div><br></div><div>Second, as= I've stated, my target operating system is macOS.. are you familiar of= any known issues with file atomicity in this particular OS ?=C2=A0</div></= div><div><div>Perhaps you could tell me how it's implemented under the = hood to achieve data consistency if I'm using the database using MDB_NO= METASYNC, so I further investigate this issue and address it to Apple if ne= cessary.=C2=A0</div></div><div><br></div><div>Irad</div></div></div></div><= br><div class=3D"gmail_quote"><div dir=3D"ltr">On Mon, Sep 10, 2018 at 8:44= PM Howard Chu <<a href=3D"mailto:hyc@symas.com">hyc(a)symas.com</a>> w= rote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex= ;border-left:1px #ccc solid;padding-left:1ex"><a href=3D"mailto:iradization= @gmail.com" target=3D"_blank">iradization(a)gmail.com</a> wrote:<br> > Full_Name: Irad.k<br> > Version: recent from GitHub.<br> > OS: macOS<br> > URL: <a href=3D"ftp://ftp.openldap.org/incoming/" rel=3D"noreferrer" t= arget=3D"_blank">ftp://ftp.openldap.org/incoming/</a><br> > Submission from: (NULL) (82.81.84.130)<br> > <br> > <br> > Hi, <br> > <br> > I'm working with LMDB with the following flags :=C2=A0 MDB_NOMETAS= YNC | MDB_NOSUBDIR<br> > | MDB_NOTLS.<br> > <br> > Somehow, even though the DB should be ACI(without the D),=C2=A0 it got= corrupted<br> > after recovering from kernel panic, and It crashes my process when try= ing to<br> > access one of the records (see crash log below). <br> > <br> > Here's a link to the file : <br> > <br> > <a href=3D"https://drive.google.com/file/d/12Q3KYYrapiJOgiaccnDL3tQACk= qrM3C5/view?usp=3Dsharing" rel=3D"noreferrer" target=3D"_blank">https://dri= ve.google.com/file/d/12Q3KYYrapiJOgiaccnDL3tQACkqrM3C5/view?usp=3Dsharing</= a><br> <br> This file is only 6 pages long, but the latest meta page claims that it&#39= ;s using 11 pages.<br> Looking at the previous meta page, it claims that 9 pages were used.<br> <br> Clearly your OS failed to write the complete contents out before it panicke= d. At the<br> very least the file should have been 9 pages long. Sounds like you have an = OS bug.<br> <br> > According to the crash log from the process, It can clearly be seen th= at the<br> > invalid address reside inside the mapped file region which is the lmdb= mapped<br> > file, but still I get KERN_MEMORY_ERROR on that address.<br> > <br> >>From what I know, an attempt to access address within the mapped ra= nge can<br> > either retrieve the page contents directly from memory (if it's al= ready there),<br> > or trigger page fault trap that eventually lead to reading the missing= data from<br> > disk and return it to process as well.<br> > <br> > One thing that raise some concerns is that the file size is only 24k a= nd the<br> > mapping spans over 256M. However, the file's meta data seems to be= coherent to<br> > file contents.<br> > <br> > Any idea how did it happen, and what exactly in the file cause this co= rruption ?<br> > <br> > <br> > CRASH LOG :<br> > ----------------------------------------------------------------------= <br> > Exception Type:=C2=A0 =C2=A0 =C2=A0 =C2=A0 EXC_BAD_ACCESS (SIGBUS)<br> > Exception Codes:=C2=A0 =C2=A0 =C2=A0 =C2=A0KERN_MEMORY_ERROR at 0x0000= 00010648800a<br> > Exception Note:=C2=A0 =C2=A0 =C2=A0 =C2=A0 EXC_CORPSE_NOTIFY<br> > <br> > Termination Signal:=C2=A0 =C2=A0 Bus error: 10<br> > Termination Reason:=C2=A0 =C2=A0 Namespace SIGNAL, Code 0xa<br> > Terminating Process:=C2=A0 =C2=A0exc handler [0]<br> > <br> > VM Regions Near 0x10648800a:<br> > <br> > __LINKEDIT=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0000000010646= 4000-000000010647f000 [=C2=A0 108K] r--/rwx SM=3DCOW<br> >=C2=A0 ^Z^C [/usr/lib/dyld]<br> > --> mapped file=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 0000000106= 47f000-000000011647f000 [256.0M] r--/rwx<br> > SM=3DPRV=C2=A0 Object_id=3D9034edd9<br> > STACK GUARD=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 000070000f2b3000-= 000070000f2b4000 [=C2=A0 =C2=A0 4K] ---/rwx SM=3DNUL<br> >=C2=A0 stack guard for thread 1<br> > <br> > Thread 0 Crashed:: Dispatch queue: com.apple.main-thread<br> > 0=C2=A0 =C2=A0myprog=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A00x0000000101666756 mdb_page_search_root + 39<br> > 1=C2=A0 =C2=A0myprog=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A00x00000001016660f7 mdb_page_search + 182<br> > 2=C2=A0 =C2=A0myprog=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A00x00000001016614de mdb_cursor_set + 88<br> > 3=C2=A0 =C2=A0myprog=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A00x0000000101661476 mdb_get + 134<br> > <br> > <br> > <br> <br> <br> -- <br> =C2=A0 -- Howard Chu<br> =C2=A0 CTO, Symas Corp.=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0<a href=3D"= http://www.symas.com" rel=3D"noreferrer" target=3D"_blank">http://www.symas= .com</a><br> =C2=A0 Director, Highland Sun=C2=A0 =C2=A0 =C2=A0<a href=3D"http://highland= sun.com/hyc/" rel=3D"noreferrer" target=3D"_blank">http://highlandsun.com/h= yc/</a><br> =C2=A0 Chief Architect, OpenLDAP=C2=A0 <a href=3D"http://www.openldap.org/p= roject/" rel=3D"noreferrer" target=3D"_blank">http://www.openldap.org/proje= ct/</a><br> </blockquote></div> --000000000000d405310575a6de5e--

1 0

Re: (ITS#8898) mdb_get on corrupted LMDB file crash the process
by hyc＠symas.com 10 Sep '18

10 Sep '18

iradization(a)gmail.com wrote: > Full_Name: Irad.k > Version: recent from GitHub. > OS: macOS > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (82.81.84.130) > > > Hi, > > I'm working with LMDB with the following flags : MDB_NOMETASYNC | MDB_NOSUBDIR > | MDB_NOTLS. > > Somehow, even though the DB should be ACI(without the D), it got corrupted > after recovering from kernel panic, and It crashes my process when trying to > access one of the records (see crash log below). > > Here's a link to the file : > > https://drive.google.com/file/d/12Q3KYYrapiJOgiaccnDL3tQACkqrM3C5/view?usp=… This file is only 6 pages long, but the latest meta page claims that it's using 11 pages. Looking at the previous meta page, it claims that 9 pages were used. Clearly your OS failed to write the complete contents out before it panicked. At the very least the file should have been 9 pages long. Sounds like you have an OS bug. > According to the crash log from the process, It can clearly be seen that the > invalid address reside inside the mapped file region which is the lmdb mapped > file, but still I get KERN_MEMORY_ERROR on that address. > >>From what I know, an attempt to access address within the mapped range can > either retrieve the page contents directly from memory (if it's already there), > or trigger page fault trap that eventually lead to reading the missing data from > disk and return it to process as well. > > One thing that raise some concerns is that the file size is only 24k and the > mapping spans over 256M. However, the file's meta data seems to be coherent to > file contents. > > Any idea how did it happen, and what exactly in the file cause this corruption ? > > > CRASH LOG : > ---------------------------------------------------------------------- > Exception Type: EXC_BAD_ACCESS (SIGBUS) > Exception Codes: KERN_MEMORY_ERROR at 0x000000010648800a > Exception Note: EXC_CORPSE_NOTIFY > > Termination Signal: Bus error: 10 > Termination Reason: Namespace SIGNAL, Code 0xa > Terminating Process: exc handler [0] > > VM Regions Near 0x10648800a: > > __LINKEDIT 0000000106464000-000000010647f000 [ 108K] r--/rwx SM=COW > ^Z^C [/usr/lib/dyld] > --> mapped file 000000010647f000-000000011647f000 [256.0M] r--/rwx > SM=PRV Object_id=9034edd9 > STACK GUARD 000070000f2b3000-000070000f2b4000 [ 4K] ---/rwx SM=NUL > stack guard for thread 1 > > Thread 0 Crashed:: Dispatch queue: com.apple.main-thread > 0 myprog 0x0000000101666756 mdb_page_search_root + 39 > 1 myprog 0x00000001016660f7 mdb_page_search + 182 > 2 myprog 0x00000001016614de mdb_cursor_set + 88 > 3 myprog 0x0000000101661476 mdb_get + 134 > > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8901) slapd segfaults in mdb_env_reader_dest
by hyc＠symas.com 10 Sep '18

10 Sep '18

ckowalczyk(a)suse.com wrote: > Full_Name: Chris Kowalczyk > Version: 2.4.41 > OS: SLE12 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (2620:113:80c0:5::2222) > > > Hello, > I have been experiencing segfaults in mdb_env_reader_dest happening randomly, > only from time to time, during shutting off openldap. > > After some investigation I narrowed the problem down to mdb_env_reader_dest > destructor function being called after its object had been deleted while > powering off the application. mdb_env_reader_dest was called automatically, > because it was agginned to the object in pthread_key_create call: > https://github.com/openldap/openldap/blob/master/libraries/liblmdb/mdb.c#L4… > > The problem was not happening often, and always during powering off, so no data > was lost etc. However, due to its annoyance, I prepared a simple patch, which > prevents mdb_env_reader_dest function being called if the object was already > deleted. The patch works fine, I haven't seen the problem since it was > introduced. > > I am using OpenLDAP 2.4.41, but the code seems to be the same in other versions > as well. Would you like me to create a pull request for it? We've discussed this extensively in private emails. There is no documented reason why the provided patch would have the claimed effect. Deferring this until a valid explanation for why this patch is correct can be shown. > > ========================== > === The backtrace: > ========================== > > Program terminated with signal 11, Segmentation fault. > #0 mdb_env_reader_dest (ptr=0x7f4a9c71c080) at > ../../../libraries/liblmdb/mdb.c:4050 > 4050 reader->mr_pid = 0; > > #thread apply all bt > > Thread 2 (LWP 20920): > #0 0x00007f4a9ac31d9d in close () at ../sysdeps/unix/syscall-template.S:84 > #1 0x000055902eaeae8c in mdb_env_close1 (env=env@entry=0x55902fce50e0) > at ../../../libraries/liblmdb/mdb.c:4756 > #2 0x000055902eaeb578 in mdb_env_close1 (env=0x55902fce50e0) at > ../../../libraries/liblmdb/mdb.c:4779 > #3 mdb_env_close (env=0x55902fce50e0) at ../../../libraries/liblmdb/mdb.c:4778 > #4 0x000055902ea69114 in mdb_db_close (be=0x55902faeb300, cr=<optimized out>) > at init.c:340 > #5 0x000055902ea3815b in over_db_close (be=0x55902faeb300, cr=0x0) at > backover.c:182 > #6 0x000055902e9d8c0b in backend_shutdown (be=0x55902faeb300, be@entry=0x0) at > backend.c:376 > #7 0x000055902e9fa258 in slap_shutdown (be=be@entry=0x0) at init.c:232 > #8 0x000055902e9af12f in main (argc=<optimized out>, argv=<optimized out>) at > main.c:1027 > > Thread 1 (LWP 20928): > #0 mdb_env_reader_dest (ptr=0x7f4a9c71c080) at > ../../../libraries/liblmdb/mdb.c:4050 > #1 0x00007f4a9ac2a512 in __nptl_deallocate_tsd () at pthread_create.c:176 > #2 0x00007f4a9ac2a767 in start_thread (arg=0x7f4a97acc700) at > pthread_create.c:347 > #3 0x00007f4a9a968aad in clone () at > ../sysdeps/unix/sysv/linux/x86_64/clone.S:109 > > # frame 0 > #0 mdb_env_reader_dest (ptr=0x7f4a9c71c080) at > ../../../libraries/liblmdb/mdb.c:4050 > 4050 reader->mr_pid = 0; > #p reader > $1 = (MDB_reader *) 0x7f4a9c71c080 > #p *reader > Cannot access memory at address 0x7f4a9c71c080 > > ========================== > === The proposed patch: > ========================== > > diff --git a/libraries/liblmdb/mdb.c b/libraries/liblmdb/mdb.c > index 6bdf3151d..56212151b 100644 > --- a/libraries/liblmdb/mdb.c > +++ b/libraries/liblmdb/mdb.c > @@ -4692,6 +4692,11 @@ mdb_env_close0(MDB_env *env, int excl) > > if (env->me_flags & MDB_ENV_TXKEY) { > pthread_key_delete(env->me_txkey); > + > + // No need to call desctructor anymore, as all pid > + // values are cleared below. > + env->me_txkey = NULL; > + > #ifdef _WIN32 > /* Delete our key from the global list */ > for (i=0; i<mdb_tls_nkeys; i++) > > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8913) Add meson build system to lmdb
by nacho.resa＠gmail.com 05 Sep '18

05 Sep '18

--0000000000006c1a190575262da9 Content-Type: text/plain; charset="UTF-8" not super important since it is anyway quite fast but takes half of the time running meson + ninja than running make -j6... :) real 0m3.070s user 0m7.031s sys 0m0.493s vs real 0m1.501s user 0m2.138s sys 0m0.446s In any case it is not the end of the world to keep a fork for gvsbuild and maybe fedora, at the end of the day this library does not change that often to make the build system painful to maintain out of upstream. On Wed, Sep 5, 2018 at 7:15 PM Howard Chu <hyc(a)symas.com> wrote: > Ignacio Casal Quinteiro wrote: > > FWIW any chance to reconsider having both build systems? > > I have no desire to support multiple build systems. The Project does not > support > any tools that we don't use ourselves. > > The files you add significantly increase the line count, and don't even > contain any comments. > In contrast, the current Makefile is mostly comments. > > > with meson it is easy to build on windows. Plus I'd like you > > to see at these numbers: > > > > time make: > > real 0m5.629s > > user 0m5.197s > > sys 0m0.399s > > > > time meson build > > real 0m0.452s > > user 0m0.335s > > sys 0m0.131s > > > > time ninja -C build > > real 0m0.992s > > user 0m1.812s > > sys 0m0.362s > > > > You can see that configuring and building with meson is faster than just > using make > > On my system the majority of build time is in gcc itself. The proportion > of time that > make itself uses is trivial. I would guess the only reason you're seeing a > speedup is > because meson parallelizes the build. You would get the same effect using > "make -jXX". > > > > > On Wed, Sep 5, 2018 at 6:31 PM Ignacio Casal Quinteiro < > nacho.resa(a)gmail.com <mailto:nacho.resa@gmail.com>> wrote: > > > > yeah... knowing my previous tries with lmdb I did not have much > hope... > > > > On Wed, Sep 5, 2018 at 6:21 PM Howard Chu <hyc(a)symas.com <mailto: > hyc(a)symas.com>> wrote: > > > > nacho.resa(a)gmail.com <mailto:nacho.resa@gmail.com> wrote: > > > Full_Name: Ignacio Casal Quinteiro > > > Version: > > > OS: > > > URL: ftp://ftp.openldap.org/incoming/ > > > Submission from: (NULL) (54.239.6.177) > > > > > > > > > Meson is the new thing out there, > > > > No. > > -- > -- Howard Chu > CTO, Symas Corp. http://www.symas.com > Director, Highland Sun http://highlandsun.com/hyc/ > Chief Architect, OpenLDAP http://www.openldap.org/project/ > -- Ignacio Casal Quinteiro --0000000000006c1a190575262da9 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div dir=3D"ltr"><div dir=3D"ltr"><div>not super important= since it is anyway quite fast but takes half of the time running meson + n= inja than running make -j6... :)</div><div>real=C2=A0=C2=A0=C2=A0 0m3.070s<= br>user=C2=A0=C2=A0=C2=A0 0m7.031s<br>sys=C2=A0=C2=A0=C2=A0 0m0.493s<br></d= iv><div><br></div><div>vs</div><div><br></div><div>real=C2=A0=C2=A0=C2=A0 0= m1.501s<br>user=C2=A0=C2=A0=C2=A0 0m2.138s<br>sys=C2=A0=C2=A0=C2=A0 0m0.446= s<br></div><div><br></div><div>In any case it is not the end of the world t= o keep a fork for gvsbuild and maybe fedora, at the end of the day this lib= rary</div><div>does not change that often to make the build system painful = to maintain out of upstream.<br></div></div></div></div><br><div class=3D"g= mail_quote"><div dir=3D"ltr">On Wed, Sep 5, 2018 at 7:15 PM Howard Chu <= <a href=3D"mailto:hyc@symas.com">hyc(a)symas.com</a>> wrote:<br></div><blo= ckquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #c= cc solid;padding-left:1ex">Ignacio Casal Quinteiro wrote:<br> > FWIW any chance to reconsider having both build systems?<br> <br> I have no desire to support multiple build systems. The Project does not su= pport<br> any tools that we don't use ourselves.<br> <br> The files you add significantly increase the line count, and don't even= contain any comments.<br> In contrast, the current Makefile is mostly comments.<br> <br> > with meson it is easy to build on windows. Plus I'd like you<br> > to see at these numbers:<br> > <br> > time make:<br> > real=C2=A0=C2=A0=C2=A0 0m5.629s<br> > user=C2=A0=C2=A0=C2=A0 0m5.197s<br> > sys=C2=A0=C2=A0=C2=A0 0m0.399s<br> > <br> > time meson build<br> > real=C2=A0=C2=A0=C2=A0 0m0.452s<br> > user=C2=A0=C2=A0=C2=A0 0m0.335s<br> > sys=C2=A0=C2=A0=C2=A0 0m0.131s<br> > <br> > time ninja -C build<br> > real=C2=A0=C2=A0=C2=A0 0m0.992s<br> > user=C2=A0=C2=A0=C2=A0 0m1.812s<br> > sys=C2=A0=C2=A0=C2=A0 0m0.362s<br> > <br> > You can see that configuring and building with meson is faster than ju= st using make<br> <br> On my system the majority of build time is in gcc itself. The proportion of= time that<br> make itself uses is trivial. I would guess the only reason you're seein= g a speedup is<br> because meson parallelizes the build. You would get the same effect using &= quot;make -jXX".<br> <br> > <br> > On Wed, Sep 5, 2018 at 6:31 PM Ignacio Casal Quinteiro <<a href=3D"= mailto:nacho.resa@gmail.com" target=3D"_blank">nacho.resa(a)gmail.com</a> &lt= ;mailto:<a href=3D"mailto:nacho.resa@gmail.com" target=3D"_blank">nacho.res= a(a)gmail.com</a>>> wrote:<br> > <br> >=C2=A0 =C2=A0 =C2=A0yeah... knowing my previous tries with lmdb I did n= ot have much hope...<br> > <br> >=C2=A0 =C2=A0 =C2=A0On Wed, Sep 5, 2018 at 6:21 PM Howard Chu <<a hr= ef=3D"mailto:hyc@symas.com" target=3D"_blank">hyc(a)symas.com</a> <mailto:= <a href=3D"mailto:hyc@symas.com" target=3D"_blank">hyc(a)symas.com</a>>&gt= ; wrote:<br> > <br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0<a href=3D"mailto:nacho.resa@gmail.co= m" target=3D"_blank">nacho.resa(a)gmail.com</a> <mailto:<a href=3D"mailto:= nacho.resa(a)gmail.com" target=3D"_blank">nacho.resa(a)gmail.com</a>> wrote:= <br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0> Full_Name: Ignacio Casal Quintei= ro<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0> Version:<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0> OS:<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0> URL: <a href=3D"ftp://ftp.openld= ap.org/incoming/" rel=3D"noreferrer" target=3D"_blank">ftp://ftp.openldap.o= rg/incoming/</a><br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0> Submission from: (NULL) (54.239.= 6.177)<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0><br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0><br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0> Meson is the new thing out there= ,<br> > <br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0No.<br> <br> -- <br> =C2=A0 -- Howard Chu<br> =C2=A0 CTO, Symas Corp.=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0<a href=3D"= http://www.symas.com" rel=3D"noreferrer" target=3D"_blank">http://www.symas= .com</a><br> =C2=A0 Director, Highland Sun=C2=A0 =C2=A0 =C2=A0<a href=3D"http://highland= sun.com/hyc/" rel=3D"noreferrer" target=3D"_blank">http://highlandsun.com/h= yc/</a><br> =C2=A0 Chief Architect, OpenLDAP=C2=A0 <a href=3D"http://www.openldap.org/p= roject/" rel=3D"noreferrer" target=3D"_blank">http://www.openldap.org/proje= ct/</a><br> </blockquote></div><br clear=3D"all"><br>-- <br><div dir=3D"ltr" class=3D"g= mail_signature" data-smartmail=3D"gmail_signature">Ignacio Casal Quinteiro<= /div> --0000000000006c1a190575262da9--

1 0

Re: (ITS#8913) Add meson build system to lmdb
by hyc＠symas.com 05 Sep '18

05 Sep '18

Ignacio Casal Quinteiro wrote: > FWIW any chance to reconsider having both build systems? I have no desire to support multiple build systems. The Project does not = support any tools that we don't use ourselves. The files you add significantly increase the line count, and don't even c= ontain any comments. In contrast, the current Makefile is mostly comments. > with meson it is easy to build on windows. Plus I'd like you > to see at these numbers: >=20 > time make: > real=C2=A0=C2=A0=C2=A0 0m5.629s > user=C2=A0=C2=A0=C2=A0 0m5.197s > sys=C2=A0=C2=A0=C2=A0 0m0.399s >=20 > time meson build > real=C2=A0=C2=A0=C2=A0 0m0.452s > user=C2=A0=C2=A0=C2=A0 0m0.335s > sys=C2=A0=C2=A0=C2=A0 0m0.131s >=20 > time ninja -C build > real=C2=A0=C2=A0=C2=A0 0m0.992s > user=C2=A0=C2=A0=C2=A0 0m1.812s > sys=C2=A0=C2=A0=C2=A0 0m0.362s >=20 > You can see that configuring and building with meson is faster than jus= t using make On my system the majority of build time is in gcc itself. The proportion = of time that make itself uses is trivial. I would guess the only reason you're seeing = a speedup is because meson parallelizes the build. You would get the same effect using= "make -jXX". >=20 > On Wed, Sep 5, 2018 at 6:31 PM Ignacio Casal Quinteiro <nacho.resa@gmai= l.com <mailto:nacho.resa@gmail.com>> wrote: >=20 > yeah... knowing my previous tries with lmdb I did not have much hop= e... >=20 > On Wed, Sep 5, 2018 at 6:21 PM Howard Chu <hyc(a)symas.com <mailto:hy= c(a)symas.com>> wrote: >=20 > nacho.resa(a)gmail.com <mailto:nacho.resa@gmail.com> wrote: > > Full_Name: Ignacio Casal Quinteiro > > Version: > > OS: > > URL: ftp://ftp.openldap.org/incoming/ > > Submission from: (NULL) (54.239.6.177) > > > > > > Meson is the new thing out there, >=20 > No. --=20 -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8913) Add meson build system to lmdb
by nacho.resa＠gmail.com 05 Sep '18

05 Sep '18

--0000000000007aaf1005752283b9 Content-Type: text/plain; charset="UTF-8" FWIW any chance to reconsider having both build systems? with meson it is easy to build on windows. Plus I'd like you to see at these numbers: time make: real 0m5.629s user 0m5.197s sys 0m0.399s time meson build real 0m0.452s user 0m0.335s sys 0m0.131s time ninja -C build real 0m0.992s user 0m1.812s sys 0m0.362s You can see that configuring and building with meson is faster than just using make On Wed, Sep 5, 2018 at 6:31 PM Ignacio Casal Quinteiro <nacho.resa(a)gmail.com> wrote: > yeah... knowing my previous tries with lmdb I did not have much hope... > > On Wed, Sep 5, 2018 at 6:21 PM Howard Chu <hyc(a)symas.com> wrote: > >> nacho.resa(a)gmail.com wrote: >> > Full_Name: Ignacio Casal Quinteiro >> > Version: >> > OS: >> > URL: ftp://ftp.openldap.org/incoming/ >> > Submission from: (NULL) (54.239.6.177) >> > >> > >> > Meson is the new thing out there, >> >> No. >> >> >> -- >> -- Howard Chu >> CTO, Symas Corp. http://www.symas.com >> Director, Highland Sun http://highlandsun.com/hyc/ >> Chief Architect, OpenLDAP http://www.openldap.org/project/ >> > > > -- > Ignacio Casal Quinteiro > -- Ignacio Casal Quinteiro --0000000000007aaf1005752283b9 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div dir=3D"ltr"><div dir=3D"ltr"><div dir=3D"ltr"><div di= r=3D"ltr"><div dir=3D"ltr"><div dir=3D"ltr"><div>FWIW any chance to reconsi= der having both build systems?</div><div>with meson it is easy to build on = windows. Plus I'd like you</div><div>to see at these numbers:</div><div= ><br></div><div>time make:</div><div>real=C2=A0=C2=A0=C2=A0 0m5.629s<br>use= r=C2=A0=C2=A0=C2=A0 0m5.197s<br>sys=C2=A0=C2=A0=C2=A0 0m0.399s<br></div><di= v><br></div><div>time meson build</div><div>real=C2=A0=C2=A0=C2=A0 0m0.452s= <br>user=C2=A0=C2=A0=C2=A0 0m0.335s<br>sys=C2=A0=C2=A0=C2=A0 0m0.131s<br></= div><div><br></div><div>time ninja -C build</div><div>real=C2=A0=C2=A0=C2= =A0 0m0.992s<br>user=C2=A0=C2=A0=C2=A0 0m1.812s<br>sys=C2=A0=C2=A0=C2=A0 0m= 0.362s<br></div><div><br></div><div>You can see that configuring and buildi= ng with meson is faster than just using make<br></div></div></div></div></d= iv></div></div></div><br><div class=3D"gmail_quote"><div dir=3D"ltr">On Wed= , Sep 5, 2018 at 6:31 PM Ignacio Casal Quinteiro <<a href=3D"mailto:nach= o.resa(a)gmail.com">nacho.resa(a)gmail.com</a>> wrote:<br></div><blockquote = class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid= ;padding-left:1ex"><div dir=3D"ltr">yeah... knowing my previous tries with = lmdb I did not have much hope...<br></div><br><div class=3D"gmail_quote"><d= iv dir=3D"ltr">On Wed, Sep 5, 2018 at 6:21 PM Howard Chu <<a href=3D"mai= lto:hyc@symas.com" target=3D"_blank">hyc(a)symas.com</a>> wrote:<br></div>= <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"><a href=3D"mailto:nacho.resa@gmail.com" targ= et=3D"_blank">nacho.resa(a)gmail.com</a> wrote:<br> > Full_Name: Ignacio Casal Quinteiro<br> > Version: <br> > OS: <br> > URL: <a href=3D"ftp://ftp.openldap.org/incoming/" rel=3D"noreferrer" t= arget=3D"_blank">ftp://ftp.openldap.org/incoming/</a><br> > Submission from: (NULL) (54.239.6.177)<br> > <br> > <br> > Meson is the new thing out there,<br> <br> No.<br> <br> <br> -- <br> =C2=A0 -- Howard Chu<br> =C2=A0 CTO, Symas Corp.=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0<a href=3D"= http://www.symas.com" rel=3D"noreferrer" target=3D"_blank">http://www.symas= .com</a><br> =C2=A0 Director, Highland Sun=C2=A0 =C2=A0 =C2=A0<a href=3D"http://highland= sun.com/hyc/" rel=3D"noreferrer" target=3D"_blank">http://highlandsun.com/h= yc/</a><br> =C2=A0 Chief Architect, OpenLDAP=C2=A0 <a href=3D"http://www.openldap.org/p= roject/" rel=3D"noreferrer" target=3D"_blank">http://www.openldap.org/proje= ct/</a><br> </blockquote></div><br clear=3D"all"><br>-- <br><div dir=3D"ltr" class=3D"m= _-8944057145512108354gmail_signature" data-smartmail=3D"gmail_signature">Ig= nacio Casal Quinteiro</div> </blockquote></div><br clear=3D"all"><br>-- <br><div dir=3D"ltr" class=3D"g= mail_signature" data-smartmail=3D"gmail_signature">Ignacio Casal Quinteiro<= /div> --0000000000007aaf1005752283b9--

1 0

Re: (ITS#8913) Add meson build system to lmdb
by nacho.resa＠gmail.com 05 Sep '18

05 Sep '18

--000000000000a46a9a05752250a1 Content-Type: text/plain; charset="UTF-8" yeah... knowing my previous tries with lmdb I did not have much hope... On Wed, Sep 5, 2018 at 6:21 PM Howard Chu <hyc(a)symas.com> wrote: > nacho.resa(a)gmail.com wrote: > > Full_Name: Ignacio Casal Quinteiro > > Version: > > OS: > > URL: ftp://ftp.openldap.org/incoming/ > > Submission from: (NULL) (54.239.6.177) > > > > > > Meson is the new thing out there, > > No. > > > -- > -- Howard Chu > CTO, Symas Corp. http://www.symas.com > Director, Highland Sun http://highlandsun.com/hyc/ > Chief Architect, OpenLDAP http://www.openldap.org/project/ > -- Ignacio Casal Quinteiro --000000000000a46a9a05752250a1 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr">yeah... knowing my previous tries with lmdb I did not have= much hope...<br></div><br><div class=3D"gmail_quote"><div dir=3D"ltr">On W= ed, Sep 5, 2018 at 6:21 PM Howard Chu <<a href=3D"mailto:hyc@symas.com">= hyc(a)symas.com</a>> wrote:<br></div><blockquote class=3D"gmail_quote" sty= le=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><a hre= f=3D"mailto:nacho.resa@gmail.com" target=3D"_blank">nacho.resa(a)gmail.com</a= > wrote:<br> > Full_Name: Ignacio Casal Quinteiro<br> > Version: <br> > OS: <br> > URL: <a href=3D"ftp://ftp.openldap.org/incoming/" rel=3D"noreferrer" t= arget=3D"_blank">ftp://ftp.openldap.org/incoming/</a><br> > Submission from: (NULL) (54.239.6.177)<br> > <br> > <br> > Meson is the new thing out there,<br> <br> No.<br> <br> <br> -- <br> =C2=A0 -- Howard Chu<br> =C2=A0 CTO, Symas Corp.=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0<a href=3D"= http://www.symas.com" rel=3D"noreferrer" target=3D"_blank">http://www.symas= .com</a><br> =C2=A0 Director, Highland Sun=C2=A0 =C2=A0 =C2=A0<a href=3D"http://highland= sun.com/hyc/" rel=3D"noreferrer" target=3D"_blank">http://highlandsun.com/h= yc/</a><br> =C2=A0 Chief Architect, OpenLDAP=C2=A0 <a href=3D"http://www.openldap.org/p= roject/" rel=3D"noreferrer" target=3D"_blank">http://www.openldap.org/proje= ct/</a><br> </blockquote></div><br clear=3D"all"><br>-- <br><div dir=3D"ltr" class=3D"g= mail_signature" data-smartmail=3D"gmail_signature">Ignacio Casal Quinteiro<= /div> --000000000000a46a9a05752250a1--

1 0

Re: (ITS#8913) Add meson build system to lmdb
by hyc＠symas.com 05 Sep '18

05 Sep '18

nacho.resa(a)gmail.com wrote: > Full_Name: Ignacio Casal Quinteiro > Version: > OS: > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (54.239.6.177) > > > Meson is the new thing out there, No. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs