openldap-bugs

openldap-bugs@openldap.org

1 participants
20965 discussions

Re: (ITS#8973) Error with domainScope control when using C# LDAP API
by clement.oudot＠worteks.com 05 Mar '19

05 Mar '19

Le 06/02/2019 à 14:16, clement.oudot(a)worteks.com a écrit : > Le 06/02/2019 Ã 12:57, hyc(a)symas.com a Ã©critÂ : >> hyc(a)symas.com wrote: >>> clement.oudot(a)worteks.com wrote: >>>> Full_Name: Clement OUDOT >>>> Version: 2.4.47 >>>> OS: GNU/Linux >>>> URL: ftp://ftp.openldap.org/incoming/ >>>> Submission from: (NULL) (81.250.130.213) >>>> >>>> >>>> Hello, >>>> >>>> after upgrading from 2.4.46 to 2.4.47, this C# code does not work anymore : >>> The code in controls.c is correct. The DomainScope control does not take a value, >>> so the value is required to be NULL. >> Microsoft spec https://msdn.microsoft.com/en-us/library/cc223348.aspx >> >> "When sending this control to the DC, the controlValue field of the Control structure is omitted." >> > Thanks a lot for your feedback, I will try to understand with the C# > code is sending a value inside the control. For the record, the bug seems to be inside powershell and the API "system.DirectoryServices.DirectorySearcher" which is sending the control. To solve this, you need to use another API: "System.DirectoryServices.Protocols" See https://docs.microsoft.com/fr-fr/dotnet/api/system.directoryservices.protoc… -- Clément Oudot | Identity Solutions Manager clement.oudot(a)worteks.com Worteks | https://www.worteks.com

1 0

Re: (ITS#8980) Async mode and TLS Non-Blocking Issue
by quanah＠symas.com 28 Feb '19

28 Feb '19

--On Wednesday, February 20, 2019 2:30 PM -0500 Vern Smith <vsmith(a)interlinknetworks.com> wrote: > > The attached files are derived from OpenLDAP Software. All of the > modifications to OpenLDAP Software represented in the following patch(es) > were developed by Interlink Networks LLC. Interlink Networks LLC has not > assigned rights and/or interest in this work to any party. I, Vernon > Smith am authorized by Interlink Networks LLC, my employer, to release > this work under the following terms. Thanks! This has been applied to git master. Regards, Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

Re: (ITS#8985) Fix of an assertion error in back-asyncmeta during restart of remote target
by quanah＠symas.com 28 Feb '19

28 Feb '19

--On Thursday, February 28, 2019 5:00 PM +0000 nivanova(a)symas.com wrote: > Full_Name: Nadezhda Ivanova > Version: 2.5 > OS: Ubuntu 18.04 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (93.152.142.241) > > > The attached patches provide a fix for a problem, reported by a customer: Thanks for the report, applied to git master. Warm regards, Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

(ITS#8985) Fix of an assertion error in back-asyncmeta during restart of remote target
by nivanova＠symas.com 28 Feb '19

28 Feb '19

Full_Name: Nadezhda Ivanova Version: 2.5 OS: Ubuntu 18.04 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (93.152.142.241) The attached patches provide a fix for a problem, reported by a customer: Occasionally, when the remote target of an asyncmeta database is restarted, slapd crashes with assertion error in connection_init: assert( c->c_struct_state == SLAP_C_UNUSED ); The issue occurs very rarely, but I managed to reproduce it. It was caused by the fact that on socket error, try_read1msg would close the LDAP connection (ld) to that target, making the fd available to be used again for another connection. Asyncmeta is not aware of that, so it doesn't stop the listener on the corresponding Connection. If a new LDAP connection is opened to a different, active target using the same fd, asyncmeta would call connection_init on an already used Connection, which causes the assertion failure. The fix consists of three separate patches: ftp://ftp.openldap.org/incoming/nadezhda-ivanova-190228_2.patch This moves initizalization of c_mutex to connections_init, and therefore any further access to a Connection can be synchronized, eliminating some race conditions. ftp://ftp.openldap.org/incoming/nadezhda-ivanova-190228_3.patch This adds the LDAP_OPT_KEEPCONN option, which instructs try_read1msg that the connection will be freed by the client. ftp://ftp.openldap.org/incoming/nadezhda-ivanova-190228_4.patch This updates asyncmeta to make use of LDAP_OPT_KEEPCONN. IPR: The attached files are derived from OpenLDAP Software. All of the modifications to OpenLDAP Software represented in the following patch were developed by Symas Corporation. Symas Corporation has not assigned rights and/or interest in this work to any party. I, Nadezhda Ivanova, am authorized by Symas Corporation, my employer, to release this work under the following terms. The attached modifications to OpenLDAP Software are subject to the following notice: Copyright 2019 Symas Corporation Redistribution and use in source and binary forms, with or without modification, are permitted only as authorized by the OpenLDAP Public License.

1 0

Re: (ITS#8734) Fixes for multiple back-asyncmeta issues
by quanah＠symas.com 28 Feb '19

28 Feb '19

--On Thursday, February 28, 2019 4:06 PM +0000 nivanova(a)symas.com wrote: > A new version of the patch has been uploaded at > ftp://ftp.openldap.org/incoming/nadezhda-ivanova-190228.patch Thanks! Applied to git master. --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

Re: (ITS#8734) Fixes for multiple back-asyncmeta issues
by nivanova＠symas.com 28 Feb '19

28 Feb '19

A new version of the patch has been uploaded at ftp://ftp.openldap.org/incoming/nadezhda-ivanova-190228.patch The patch has been integrated with master and tested. It includes all the changes necessary to fix back-asyncmeta issues discovered during on-site testing since the start of 2016. These include: Issues with stability - crashes and assertion failures Incorrect behavior during unstable network conditions, such as inability to reset connections or process responses, or "hanging" to wait for a response that would never be received. Memory leaks and memory management fixes - major redesign of the way back-asyncmeta works with memory contexts. Rewrite was replaced with suffix-massage in configuration, and the network-timeout value was changed to milliseconds. Incorrect behavior when SASL is used to bind to a target. Many problems caused by race conditions Fixes for compiler warnings, and tests. Cleanup of unused code. Nadezhda Ivanova Software Engineer Symas Corporation http://www.symas.com

1 0

(ITS#8984) Issue with socket buffer assertion failed
by gandhali.atre＠plantronics.com 25 Feb '19

25 Feb '19

Full_Name: Gandhali Atre Version: 2.4.44 OS: Android URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (140.242.26.200) Hi, I have a product that allows to configure LDAP servers. Looks like when changing the configurations or doing some search 1/10 times there is a crash and below are the crash details. messages 2019-02-22 22:11:38.543 logcat:DEBUG(6046):Abort message '../../../libraries/liblber/sockbuf.c:89: ber_sockbuf_ctrl: assertion "SOCKBUF_VALID( sb )" failed' messages 2019-02-22 22:11:38.543 logcat:DEBUG(6046) r0 00000000 r1 000011bb r2 00000006 r3 00000008 messages 2019-02-22 22:11:38.543 logcat:DEBUG(6046) r4 00001176 r5 000011bb r6 ed27df1c r7 0000010c messages 2019-02-22 22:11:38.543 logcat:DEBUG(6046) r8 00000000 r9 ed27f290 sl 025b6410 fp ed27ef64 messages 2019-02-22 22:11:38.543 logcat:DEBUG(6046) ip 00000000 sp ed27df08 lr ef37c7f3 pc ef376774 cpsr 20000030 messages 2019-02-22 22:11:38.545 logcat:DEBUG(6046) messages 2019-02-22 22:11:38.545 backtrace messages 2019-02-22 22:11:38.545 logcat:DEBUG(6046) #00 pc 0001a774 /system/lib/libc.so (abort+63) messages 2019-02-22 22:11:38.548 logcat:DEBUG(6046) #01 pc 0001a963 /system/lib/libc.so (__assert2+22) messages 2019-02-22 22:11:38.548 logcat:DEBUG(6046) #02 pc 0000a860 /system/lib/liblber-2.4.so.2 (ber_sockbuf_ctrl+188) I read about some fixes already made for this assertion failures. Looks like it isn't working or this is something new. Any help with this would be appreciated., Thanks.

1 0

(ITS#8983) Implement old Persistent Search draft in libldap
by hyc＠openldap.org 25 Feb '19

25 Feb '19

Full_Name: Howard Chu Version: 2.5 OS: URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (80.233.42.124) Submitted by: hyc Sun/Netscape-based servers provide this https://tools.ietf.org/html/draft-ietf-ldapext-psearch-03 Definitions have been present in <ldap.h> for years, but no implementation in libldap was ever provided.

1 0

RE: (ITS#8975) Calling mdb_env_set_mapsize on a shared DB failssometimes
by kriszyp＠gmail.com 21 Feb '19

21 Feb '19

--_36998F79-0E7E-483F-8FFD-6401D532832A_ Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" I see it now! I really did check even earlier this morning and wasn=E2=80= =99t there until recently! Anyway, thank you, sorry for the noise. Thanks, Kris From: Quanah Gibson-Mount Sent: February 21, 2019 10:12 AM To: kriszyp(a)gmail.com; openldap-its(a)OpenLDAP.org Subject: Re: (ITS#8975) Calling mdb_env_set_mapsize on a shared DB failssom= etimes Hi, It is literally the first set of commits listed at that URL, so it's=20 clearly already there. Hope that helps! --Quanah --On Thursday, February 21, 2019 4:45 PM +0000 kriszyp(a)gmail.com wrote: > --0000000000000dacee05826a34a9 > Content-Type: text/plain; charset=3D"UTF-8" > > Thank you, I appreciate it! Do you know how often or when this will get > into the github mirror https://github.com/LMDB/lmdb/commits/mdb.RE/0.9 (o= r > if I am misunderstanding how the branches are synced)? > > On Sun, Feb 10, 2019 at 3:14 AM Howard Chu <hyc(a)symas.com> wrote: > >> kriszyp(a)gmail.com wrote: >> > Full_Name: Kristopher William Zyp >> > Version: LMDB 0.9.23 >> > OS: Windows Server 2012 R2, Windows 10 >> > URL: >> https://github.com/kriszyp/node-lmdb/commit/6df903907f5516320e9a8afce45b >> d32ab4e8e1f2.patch >> > Submission from: (NULL) (71.199.6.148) >> > >> >> Thanks for the report and patch, added to mdb.RE/0.9 >> >> > Calling mdb_env_set_mapsize to increase the map size, when a DB is als= o >> in use >> > by other processes, when MDB_WRITEMAP is enabled (and the db file has >> been >> > opened with PAGE_READWRITE access), on Windows, will occasionally >> > (seems >> like >> > about 1/100 attempts fails) produce an error "The requested operation >> cannot be >> > performed on a file with a user-mapped section open", or segfaults. Th= e >> error >> > occurs in the SetFilePointer (or SetEndOfFile) call in mdb_env_map tha= t >> is >> > performed to increase the allocated file size to the map size, prior t= o >> > CreateFileMapping. >> > >> > As it turns out this is pretty easy to solve, because manually >> > expanding >> the >> > file size is not necessary when calling CreateFileMapping with >> PAGE_READWRITE >> > access, as Windows will automatically expand the file size for us, whe= n >> opened >> > with the page write access enabled. Of course, this means all processe= s >> must be >> > consistent in use of MDB_WRITEMAP, but the documentation already makes >> this >> > explicit and clear. >> > >> > I believe this can be fixed by simply adding a check for MDB_WRITEMAP >> > in >> the if >> > statement that calls SetFilePointer: >> > >> > if (!(flags & MDB_WRITEMAP) && >> > (SetFilePointer(env->me_fd, >> sizelo, &sizehi, 0) >> > !=3D (DWORD)sizelo >> > || !SetEndOfFile(env->me_fd) >> > || SetFilePointer(env->me_fd, 0, NULL, 0) !=3D 0= )) >> > return ErrCode(); >> > >> > The attached URL has the change as a patch/diff as applied to our node >> package. >> > >> > I am certainly happy to just keep this change on our own branches. >> > There >> may be >> > nuances of this that I might not be aware of, but it seems to be >> > working >> great >> > for us and I have tested this with MDB_WRITEMAP enabled and disabled. >> > So >> I >> > thought I would offer/suggest this change, as it seems like it is >> > straightforward change to improve stability. Thank you! >> > >> > >> >> >> -- >> -- Howard Chu >> CTO, Symas Corp. http://www.symas.com >> Director, Highland Sun http://highlandsun.com/hyc/ >> Chief Architect, OpenLDAP http://www.openldap.org/project/ >> > > --0000000000000dacee05826a34a9 > Content-Type: text/html; charset=3D"UTF-8" > Content-Transfer-Encoding: quoted-printable > > <div dir=3D3D"ltr"><div dir=3D3D"ltr">Thank you, I appreciate it! Do you = know > h=3D ow often or when this will get into the github mirror <a > href=3D3D"https://gi=3D > thub.com/LMDB/lmdb/commits/mdb.RE/0.9">https://github.com/LMDB/lmdb/commi > ts=3D /mdb.RE/0.9</a> (or if I am misunderstanding how the branches are > synced)?<=3D /div></div> <div class=3D3D"gmail_quote"><div dir=3D3D"lt= r" > class=3D3D"gmail_a=3D ttr">On Sun, Feb 10, 2019 at 3:14 AM Howard Chu &lt= ;<a > href=3D3D"mailto:hyc@s=3D ymas.com">hyc(a)symas.com</a>> > wrote: </div><blockquote class=3D3D"gmail_=3D quote" style=3D3D"margin= :0px > 0px 0px 0.8ex;border-left:1px solid rgb(204,204,=3D > 204);padding-left:1ex"><a href=3D3D"mailto:kriszyp@gmail.com" > target=3D3D"_blan=3D k">kriszyp(a)gmail.com</a> wrote: > > Full_Name: Kristopher William Zyp > > Version: LMDB 0.9.23 > > OS: Windows Server 2012 R2, Windows 10 > > URL: <a > href=3D3D"https://github.com/kriszyp/node-lmdb/commit/6df903907f=3D > 5516320e9a8afce45bd32ab4e8e1f2.patch" rel=3D3D"noreferrer" > target=3D3D"_blank">=3D > https://github.com/kriszyp/node-lmdb/commit/6df903907f5516320e9a8afce45bd > 32=3D ab4e8e1f2.patch</a> > > Submission from: (NULL) (71.199.6.148) > > > > Thanks for the report and patch, added to mdb.RE/0.9 > > > Calling mdb_env_set_mapsize to increase the map size, when a DB is > als=3D o in use > > by other processes, when MDB_WRITEMAP is enabled (and the db file > has =3D been > > opened with PAGE_READWRITE access), on Windows, will occasionally > (see=3D ms like > > about 1/100 attempts fails) produce an error "The requested > opera=3D tion cannot be > > performed on a file with a user-mapped section open", or > segfault=3D s. The error > > occurs in the SetFilePointer (or SetEndOfFile) call in mdb_env_map > tha=3D t is > > performed to increase the allocated file size to the map size, prior > t=3D o > > CreateFileMapping. > > > > As it turns out this is pretty easy to solve, because manually > expandi=3D ng the > > file size is not necessary when calling CreateFileMapping with > PAGE_RE=3D ADWRITE > > access, as Windows will automatically expand the file size for us, > whe=3D n opened > > with the page write access enabled. Of course, this means all > processe=3D s must be > > consistent in use of MDB_WRITEMAP, but the documentation already > makes=3D this > > explicit and clear. > > > > I believe this can be fixed by simply adding a check for > MDB_WRITEMAP =3D in the if > > statement that calls SetFilePointer: > > > >=3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3D= C2=3DA0 =3DC2=3DA0if (!(flags > &amp=3D ; MDB_WRITEMAP) && (SetFilePointer(env->me_fd, sizelo, > &size=3D hi, 0) > > !=3D3D (DWORD)sizelo > >=3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3D= C2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 > =3D =3DC2=3DA0 =3DC2=3DA0|| !SetEndOfFile(env->me_fd) > >=3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3D= C2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 > =3D =3DC2=3DA0 =3DC2=3DA0|| SetFilePointer(env->me_fd, 0, NULL, 0) != =3D3D 0)) > >=3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3D= C2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 > =3D =3DC2=3DA0 =3DC2=3DA0return ErrCode(); > > > > The attached URL has the change as a patch/diff as applied to our > node=3D package. > > > > I am certainly happy to just keep this change on our own branches. > The=3D re may be > > nuances of this that I might not be aware of, but it seems to be > worki=3D ng great > > for us and I have tested this with MDB_WRITEMAP enabled and > disabled. =3D So I > > thought I would offer/suggest this change, as it seems like it is<br= > > > straightforward change to improve stability. Thank you! > > > > > > > -- > =3DC2=3DA0 -- Howard Chu > =3DC2=3DA0 CTO, Symas Corp.=3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 = =3DC2=3DA0 =3DC2=3DA0<a > href=3D3D"=3D http://www.symas.com" rel=3D3D"noreferrer" > target=3D3D"_blank">http://www.symas=3D .com</a> > =3DC2=3DA0 Director, Highland Sun=3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0<a > href=3D3D"http://highland=3D sun.com/hyc/" rel=3D3D"noreferrer" > target=3D3D"_blank">http://highlandsun.com/h=3D yc/</a> > =3DC2=3DA0 Chief Architect, OpenLDAP=3DC2=3DA0 <a > href=3D3D"http://www.openldap.org/p=3D roject/" rel=3D3D"noreferrer" > target=3D3D"_blank">http://www.openldap.org/proje=3D ct/</a> > </blockquote></div> > > --0000000000000dacee05826a34a9-- > > > -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com> --_36998F79-0E7E-483F-8FFD-6401D532832A_ Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset="utf-8" <html xmlns:o=3D"urn:schemas-microsoft-com:office:office" xmlns:w=3D"urn:sc= hemas-microsoft-com:office:word" xmlns:m=3D"http://schemas.microsoft.com/of= fice/2004/12/omml" xmlns=3D"http://www.w3.org/TR/REC-html40"><head><meta ht= tp-equiv=3DContent-Type content=3D"text/html; charset=3Dutf-8"><meta name= =3DGenerator content=3D"Microsoft Word 15 (filtered medium)"><style></style></head><body lang=3DEN-CA link=3Dblue vlink=3D"#954F72"><div cla= ss=3DWordSection1>I see it now! I really did check eve= n earlier this morning and wasn=E2=80=99t there until recently! Anyway, tha= nk you, sorry for the noise.<o:p> </o:p><= p class=3DMsoNormal>Thanks, Kris<o:p> </o:= p><div style=3D'mso-element:para-border-div;border:none;border-top:soli= d #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm'>From: <a href=3D"mailto:quanah@symas.com">Qua= nah Gibson-Mount</a> Sent: February 21, 2019 10:12 AM To: <= /b><a href=3D"mailto:kriszyp@gmail.com">kriszyp(a)gmail.com</a>; <a href=3D"m= ailto:openldap-its@OpenLDAP.org">openldap-its(a)OpenLDAP.org</a> Subjec= t: Re: (ITS#8975) Calling mdb_env_set_mapsize on a shared DB failssomet= imes</div><o:p> </o:p>Hi,<o:p> </o:p>It = is literally the first set of commits listed at that URL, so it's clearly already there.<o:p> <= /o:p>Hope that helps!<o:p= > </o:p>--Quanah<o:p= > </o:p>--On Thursday, February 21, 2019 4:45= PM +0000 kriszyp(a)gmail.com wrote:<o:p> </o:p= >> --0000000000000dacee05826a34a9> Content-Type: text/plain; charset=3D"UTF-8"= ><o:p> </o:p>> Tha= nk you, I appreciate it! Do you know how often or when this will get> into the github mirror https://github.com/LMDB/lmdb/= commits/mdb.RE/0.9 (or> if I am misunderstandin= g how the branches are synced)?><o:p> </o:= p>> On Sun, Feb 10, 2019 at 3:14 AM Howard Chu = <hyc(a)symas.com> wrote:><o:p> </o:p><= /p>>> kriszyp(a)gmail.com wrote:>> > Full_Name: Kristopher William Zyp>> > Version: LMDB 0.9.23>> &gt= ; OS: Windows Server 2012 R2, Windows 10>> &= gt; URL:>> https://github.com/kriszyp/node-l= mdb/commit/6df903907f5516320e9a8afce45b>> d3= 2ab4e8e1f2.patch>> > Submission from: (NU= LL) (71.199.6.148)>> >>><o:p> </o:p>>> Thanks for= the report and patch, added to mdb.RE/0.9>>= <o:p> </o:p>>> > Calling mdb_env_set= _mapsize to increase the map size, when a DB is also>> in use>> > by other processes,= when MDB_WRITEMAP is enabled (and the db file has= >> been>> > opened with PAGE_READWR= ITE access), on Windows, will occasionally>>= > (seems>> like= >> > about 1/100 attempts fails) produce an error "The reques= ted operation>> cannot be>> > performed on a file with a user-mapped section open&quo= t;, or segfaults. The>> error>> > occurs in the SetFilePointer (or SetEndOfFile) call= in mdb_env_map that>> is>> > performed to increase the allocated file size to the ma= p size, prior to>> > CreateFileMapping.</= p>>> >>> > = As it turns out this is pretty easy to solve, because manually>> > expanding>> the</= p>>> > file size is not necessary when callin= g CreateFileMapping with>> PAGE_READWRITE</p= >>> > access, as Windows will automatically e= xpand the file size for us, when>> opened</p= >>> > with the page write access enabled. Of = course, this means all processes>> must be</= p>>> > consistent in use of MDB_WRITEMAP, but= the documentation already makes>> this<= p class=3DMsoNormal>>> > explicit and clear.>> >>> > I believe this can b= e fixed by simply adding a check for MDB_WRITEMAP&= gt;> > in>> the if>> > statement that calls SetFilePointer:>> >>> >=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 if (!(fl= ags & MDB_WRITEMAP) &&>> >=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 (SetFilePointer(env->me_fd,>> size= lo, &sizehi, 0)>> > !=3D (DWORD)sizel= o>> >=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 || !SetEndOfFile(env->me_fd)>> >=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= || SetFilePointer(env->me_fd, 0, NULL, 0) !=3D 0))>> >=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= return ErrCode();>> >>> > The attached URL has the change as a patch/diff as appl= ied to our node>> package.>> >>> > I am certainly h= appy to just keep this change on our own branches.= >> > There>> may be>> > nuances of this that I might not be aware of, but it= seems to be>> > working>> great>> > for us and I h= ave tested this with MDB_WRITEMAP enabled and disabled.>> > So>> I>> > thought I would offer/suggest this change, as it seems = like it is>> > straightforward change to = improve stability. Thank you!>> >>> >>><o:p> <= /o:p>>><o:p> </o:p>>> -->>=C2=A0=C2=A0 -- Howard Chu= >>=C2=A0=C2=A0 CTO, Symas Corp.=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 http://www.symas.com<p= class=3DMsoNormal>>>=C2=A0=C2=A0 Director, Highland Sun=C2=A0=C2=A0= =C2=A0=C2=A0 http://highlandsun.com/hyc/>>= =C2=A0=C2=A0 Chief Architect, OpenLDAP=C2=A0 http://www.openldap.org/projec= t/>><o:p> </o:p>><o:p> </o:p>> --0000000000000dacee05= 826a34a9> Content-Type: text/html; charset=3D&q= uot;UTF-8"> Content-Transfer-Encoding: quo= ted-printable><o:p> </o:p>> <div dir=3D3D"ltr"><div dir=3D3D"ltr&= quot;>Thank you, I appreciate it! Do you know&g= t; h=3D ow often or when this will get into the github mirror <a> href=3D3D"https://gi=3D= > thub.com/LMDB/lmdb/commits/mdb.RE/0.9">https://github.com/LMDB= /lmdb/commi> ts=3D /mdb.RE/0.9</a> (or if= I am misunderstanding how the branches are> sy= nced)?<=3D /div></div> <div class=3D3D"gmail_q= uote"><div dir=3D3D"ltr"> = class=3D3D"gmail_a=3D ttr">On Sun, Feb 10, 2019 at 3:14 AM How= ard Chu &lt;<a> href=3D3D"mailto:hy= c@s=3D ymas.com">hyc(a)symas.com</a>&gtm</a>&gt;> wrote: </div><blockquote class=3D3D"gmail= _=3D quote" style=3D3D"margin:0px> 0p= x 0px 0.8ex;border-left:1px solid rgb(204,204,=3D&= gt; 204);padding-left:1ex"><a href=3D3D"mailto:kriszyp@gmai= l.com"> target=3D3D"_blan=3D k"&= gt;kriszyp(a)gmail.com</a> wrote: &gt= ; &gt; Full_Name: Kristopher William Zyp > &gt; Version: LMDB 0.9.23 &g= t; &gt; OS: Windows Server 2012 R2, Windows 10 > &gt; URL: <a> href=3D3D&q= uot;https://github.com/kriszyp/node-lmdb/commit/6df903907f=3D> 5516320e9a8afce45bd32ab4e8e1f2.patch" rel=3D3D"= noreferrer"> target=3D3D"_blank"= >=3D> https://github.com/kriszyp/node-lmdb/c= ommit/6df903907f5516320e9a8afce45bd> 32=3D ab4e= 8e1f2.patch</a> > &gt; Submi= ssion from: (NULL) (71.199.6.148) > &a= mp;gt; > > Thanks for the report and patch, added to mdb.RE/0.9 <= /p>> > &g= t; Calling mdb_env_set_mapsize to increase the map size, when a DB is<p= class=3DMsoNormal>> als=3D o in use &= gt; &gt; by other processes, when MDB_WRITEMAP is enabled (and the db f= ile> has =3D been > &gt; opened with PAGE_READWRITE access), on Windows, will occ= asionally> (see=3D ms like > &gt; about 1/100 attempts fails) produce an error &a= mp;quot;The requested> opera=3D tion cannot be&= lt;br>> &gt; performed on a file with a = user-mapped section open&quot;, or> segfaul= t=3D s. The error > &gt; occurs in= the SetFilePointer (or SetEndOfFile) call in mdb_env_map> tha=3D t is > &gt; per= formed to increase the allocated file size to the map size, prior> t=3D o > &gt; = CreateFileMapping. > &gt; <br&g= t;> &gt; As it turns out this is pretty eas= y to solve, because manually> expandi=3D ng the= > &gt; file size is not necessary= when calling CreateFileMapping with> PAGE_RE= =3D ADWRITE > &gt; access, as Wind= ows will automatically expand the file size for us,> whe=3D n opened > &gt; with = the page write access enabled. Of course, this means all> processe=3D s must be > &am= p;gt; consistent in use of MDB_WRITEMAP, but the documentation already<= p class=3DMsoNormal>> makes=3D=C2=A0 this > &gt; explicit and clear. &gt= ; &gt; > &gt; I believe this = can be fixed by simply adding a check for> MDB_= WRITEMAP =3D in the if > &gt; stat= ement that calls SetFilePointer: > &am= p;gt; > &gt;=3DC2=3DA0 =3DC2=3DA0= =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0if (!(fla= gs> &amp=3D ; MDB_WRITEMAP) &amp;&a= mp; (SetFilePointer(env-&gt;me_fd, sizelo,>= &amp;size=3D hi, 0) > &gt; != =3D3D (DWORD)sizelo > &gt;=3DC2=3D= A0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2= =3DA0 =3DC2=3DA0 =3DC2=3DA0> =3D =3DC2=3DA0 =3D= C2=3DA0|| !SetEndOfFile(env-&gt;me_fd) > &gt;=3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2= =3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0&= gt; =3D =3DC2=3DA0 =3DC2=3DA0|| SetFilePointer(env-&gt;me_fd, 0, NULL, = 0) !=3D3D 0)) > &gt;=3DC2=3DA0 =3D= C2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 = =3DC2=3DA0 =3DC2=3DA0> =3D =3DC2=3DA0 =3DC2=3DA= 0return ErrCode(); > &gt; <br&g= t;> &gt; The attached URL has the change as= a patch/diff as applied to our> node=3D=C2=A0 = package. > &gt; > &gt; I am certainly happy to just keep this chang= e on our own branches.> The=3D re may be<br&= gt;> &gt; nuances of this that I might not = be aware of, but it seems to be> worki=3D ng gr= eat > &gt; for us and I have teste= d this with MDB_WRITEMAP enabled and> disabled.= =3D So I > &gt; thought I would o= ffer/suggest this change, as it seems like it is > &gt; straightforward change to improve stability. Thank yo= u! > &gt; > &gt; > <br&g= t;> > -- = > =3DC2=3DA0 -- Howard Chu <= /p>> =3DC2=3DA0 CTO, Symas Corp.=3DC2=3DA0 =3DC2=3D= A0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0<a> href=3D3D"=3D http://www.symas.com" rel=3D3D"noreferre= r"> target=3D3D"_blank">http:= //www.symas=3D .com</a> > =3DC2= =3DA0 Director, Highland Sun=3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0<a> href=3D3D"http://highland=3D sun.com/hyc/" re= l=3D3D"noreferrer"> target=3D3D"= _blank">http://highlandsun.com/h=3D yc/</a> > =3DC2=3DA0 Chief Architect, OpenLDAP=3DC2=3DA0 <a<= /p>> href=3D3D"http://www.openldap.org/p=3D ro= ject/" rel=3D3D"noreferrer"> tar= get=3D3D"_blank">http://www.openldap.org/proje=3D ct/</a&gt= ; > </blockquote></div></p= >><o:p> </o:p>> --= 0000000000000dacee05826a34a9--><o:p> </o:p= >><o:p> </o:p>&gt= ;<o:p> </o:p><o:p> </o:p><o:p> </o:p><o:p> </o:p><p= class=3DMsoNormal>--<o:p> </o:p>Quanah Gibson-MountProduct Architect<= /p>Symas CorporationPackaged,= certified, and supported LDAP solutions powered by OpenLDAP:<http://www.symas.com><o:p>&nbs= p;</o:p><o:p> </o:p></div></body></html>= --_36998F79-0E7E-483F-8FFD-6401D532832A_--

1 0

Re: (ITS#8975) Calling mdb_env_set_mapsize on a shared DB fails sometimes
by quanah＠symas.com 21 Feb '19

21 Feb '19

Hi, It is literally the first set of commits listed at that URL, so it's clearly already there. Hope that helps! --Quanah --On Thursday, February 21, 2019 4:45 PM +0000 kriszyp(a)gmail.com wrote: > --0000000000000dacee05826a34a9 > Content-Type: text/plain; charset="UTF-8" > > Thank you, I appreciate it! Do you know how often or when this will get > into the github mirror https://github.com/LMDB/lmdb/commits/mdb.RE/0.9 (or > if I am misunderstanding how the branches are synced)? > > On Sun, Feb 10, 2019 at 3:14 AM Howard Chu <hyc(a)symas.com> wrote: > >> kriszyp(a)gmail.com wrote: >> > Full_Name: Kristopher William Zyp >> > Version: LMDB 0.9.23 >> > OS: Windows Server 2012 R2, Windows 10 >> > URL: >> https://github.com/kriszyp/node-lmdb/commit/6df903907f5516320e9a8afce45b >> d32ab4e8e1f2.patch >> > Submission from: (NULL) (71.199.6.148) >> > >> >> Thanks for the report and patch, added to mdb.RE/0.9 >> >> > Calling mdb_env_set_mapsize to increase the map size, when a DB is also >> in use >> > by other processes, when MDB_WRITEMAP is enabled (and the db file has >> been >> > opened with PAGE_READWRITE access), on Windows, will occasionally >> > (seems >> like >> > about 1/100 attempts fails) produce an error "The requested operation >> cannot be >> > performed on a file with a user-mapped section open", or segfaults. The >> error >> > occurs in the SetFilePointer (or SetEndOfFile) call in mdb_env_map that >> is >> > performed to increase the allocated file size to the map size, prior to >> > CreateFileMapping. >> > >> > As it turns out this is pretty easy to solve, because manually >> > expanding >> the >> > file size is not necessary when calling CreateFileMapping with >> PAGE_READWRITE >> > access, as Windows will automatically expand the file size for us, when >> opened >> > with the page write access enabled. Of course, this means all processes >> must be >> > consistent in use of MDB_WRITEMAP, but the documentation already makes >> this >> > explicit and clear. >> > >> > I believe this can be fixed by simply adding a check for MDB_WRITEMAP >> > in >> the if >> > statement that calls SetFilePointer: >> > >> > if (!(flags & MDB_WRITEMAP) && >> > (SetFilePointer(env->me_fd, >> sizelo, &sizehi, 0) >> > != (DWORD)sizelo >> > || !SetEndOfFile(env->me_fd) >> > || SetFilePointer(env->me_fd, 0, NULL, 0) != 0)) >> > return ErrCode(); >> > >> > The attached URL has the change as a patch/diff as applied to our node >> package. >> > >> > I am certainly happy to just keep this change on our own branches. >> > There >> may be >> > nuances of this that I might not be aware of, but it seems to be >> > working >> great >> > for us and I have tested this with MDB_WRITEMAP enabled and disabled. >> > So >> I >> > thought I would offer/suggest this change, as it seems like it is >> > straightforward change to improve stability. Thank you! >> > >> > >> >> >> -- >> -- Howard Chu >> CTO, Symas Corp. http://www.symas.com >> Director, Highland Sun http://highlandsun.com/hyc/ >> Chief Architect, OpenLDAP http://www.openldap.org/project/ >> > > --0000000000000dacee05826a34a9 > Content-Type: text/html; charset="UTF-8" > Content-Transfer-Encoding: quoted-printable > > <div dir=3D"ltr"><div dir=3D"ltr">Thank you, I appreciate it! Do you know > h= ow often or when this will get into the github mirror <a > href=3D"https://gi= > thub.com/LMDB/lmdb/commits/mdb.RE/0.9">https://github.com/LMDB/lmdb/commi > ts= /mdb.RE/0.9</a> (or if I am misunderstanding how the branches are > synced)?<= /div></div> <div class=3D"gmail_quote"><div dir=3D"ltr" > class=3D"gmail_a= ttr">On Sun, Feb 10, 2019 at 3:14 AM Howard Chu <<a > href=3D"mailto:hyc@s= ymas.com">hyc(a)symas.com</a>> > wrote: </div><blockquote class=3D"gmail_= quote" style=3D"margin:0px > 0px 0px 0.8ex;border-left:1px solid rgb(204,204,= > 204);padding-left:1ex"><a href=3D"mailto:kriszyp@gmail.com" > target=3D"_blan= k">kriszyp(a)gmail.com</a> wrote: > > Full_Name: Kristopher William Zyp > > Version: LMDB 0.9.23 > > OS: Windows Server 2012 R2, Windows 10 > > URL: <a > href=3D"https://github.com/kriszyp/node-lmdb/commit/6df903907f= > 5516320e9a8afce45bd32ab4e8e1f2.patch" rel=3D"noreferrer" > target=3D"_blank">= > https://github.com/kriszyp/node-lmdb/commit/6df903907f5516320e9a8afce45bd > 32= ab4e8e1f2.patch</a> > > Submission from: (NULL) (71.199.6.148) > > > > Thanks for the report and patch, added to mdb.RE/0.9 > > > Calling mdb_env_set_mapsize to increase the map size, when a DB is > als= o in use > > by other processes, when MDB_WRITEMAP is enabled (and the db file > has = been > > opened with PAGE_READWRITE access), on Windows, will occasionally > (see= ms like > > about 1/100 attempts fails) produce an error "The requested > opera= tion cannot be > > performed on a file with a user-mapped section open", or > segfault= s. The error > > occurs in the SetFilePointer (or SetEndOfFile) call in mdb_env_map > tha= t is > > performed to increase the allocated file size to the map size, prior > t= o > > CreateFileMapping. > > > > As it turns out this is pretty easy to solve, because manually > expandi= ng the > > file size is not necessary when calling CreateFileMapping with > PAGE_RE= ADWRITE > > access, as Windows will automatically expand the file size for us, > whe= n opened > > with the page write access enabled. Of course, this means all > processe= s must be > > consistent in use of MDB_WRITEMAP, but the documentation already > makes= this > > explicit and clear. > > > > I believe this can be fixed by simply adding a check for > MDB_WRITEMAP = in the if > > statement that calls SetFilePointer: > > > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if (!(flags > &amp= ; MDB_WRITEMAP) && (SetFilePointer(env->me_fd, sizelo, > &size= hi, 0) > > !=3D (DWORD)sizelo > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 > = =C2=A0 =C2=A0|| !SetEndOfFile(env->me_fd) > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 > = =C2=A0 =C2=A0|| SetFilePointer(env->me_fd, 0, NULL, 0) !=3D 0)) > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 > = =C2=A0 =C2=A0return ErrCode(); > > > > The attached URL has the change as a patch/diff as applied to our > node= package. > > > > I am certainly happy to just keep this change on our own branches. > The= re may be > > nuances of this that I might not be aware of, but it seems to be > worki= ng great > > for us and I have tested this with MDB_WRITEMAP enabled and > disabled. = So I > > thought I would offer/suggest this change, as it seems like it is > > straightforward change to improve stability. Thank you! > > > > > > > -- > =C2=A0 -- Howard Chu > =C2=A0 CTO, Symas Corp.=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0<a > href=3D"= http://www.symas.com" rel=3D"noreferrer" > target=3D"_blank">http://www.symas= .com</a> > =C2=A0 Director, Highland Sun=C2=A0 =C2=A0 =C2=A0<a > href=3D"http://highland= sun.com/hyc/" rel=3D"noreferrer" > target=3D"_blank">http://highlandsun.com/h= yc/</a> > =C2=A0 Chief Architect, OpenLDAP=C2=A0 <a > href=3D"http://www.openldap.org/p= roject/" rel=3D"noreferrer" > target=3D"_blank">http://www.openldap.org/proje= ct/</a> > </blockquote></div> > > --0000000000000dacee05826a34a9-- > > > -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs