--_36998F79-0E7E-483F-8FFD-6401D532832A_
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain; charset="utf-8"
I see it now! I really did check even earlier this morning and wasn=E2=80=
=99t there until recently! Anyway, thank you, sorry for the noise.
Thanks,
Kris
From: Quanah Gibson-Mount
Sent: February 21, 2019 10:12 AM
To: kriszyp(a)gmail.com; openldap-its(a)OpenLDAP.org
Subject: Re: (ITS#8975) Calling mdb_env_set_mapsize on a shared DB failssom=
etimes
Hi,
It is literally the first set of commits listed at that URL, so it's=20
clearly already there.
Hope that helps!
--Quanah
--On Thursday, February 21, 2019 4:45 PM +0000 kriszyp(a)gmail.com wrote:
> --0000000000000dacee05826a34a9
> Content-Type: text/plain; charset=3D"UTF-8"
>
> Thank you, I appreciate it! Do you know how often or when this will get
> into the github mirror https://github.com/LMDB/lmdb/commits/mdb.RE/0.9 (o=
r
> if I am misunderstanding how the branches are synced)?
>
> On Sun, Feb 10, 2019 at 3:14 AM Howard Chu <hyc(a)symas.com> wrote:
>
>> kriszyp(a)gmail.com wrote:
>> > Full_Name: Kristopher William Zyp
>> > Version: LMDB 0.9.23
>> > OS: Windows Server 2012 R2, Windows 10
>> > URL:
>> https://github.com/kriszyp/node-lmdb/commit/6df903907f5516320e9a8afce45b
>> d32ab4e8e1f2.patch
>> > Submission from: (NULL) (71.199.6.148)
>> >
>>
>> Thanks for the report and patch, added to mdb.RE/0.9
>>
>> > Calling mdb_env_set_mapsize to increase the map size, when a DB is als=
o
>> in use
>> > by other processes, when MDB_WRITEMAP is enabled (and the db file has
>> been
>> > opened with PAGE_READWRITE access), on Windows, will occasionally
>> > (seems
>> like
>> > about 1/100 attempts fails) produce an error "The requested operation
>> cannot be
>> > performed on a file with a user-mapped section open", or segfaults. Th=
e
>> error
>> > occurs in the SetFilePointer (or SetEndOfFile) call in mdb_env_map tha=
t
>> is
>> > performed to increase the allocated file size to the map size, prior t=
o
>> > CreateFileMapping.
>> >
>> > As it turns out this is pretty easy to solve, because manually
>> > expanding
>> the
>> > file size is not necessary when calling CreateFileMapping with
>> PAGE_READWRITE
>> > access, as Windows will automatically expand the file size for us, whe=
n
>> opened
>> > with the page write access enabled. Of course, this means all processe=
s
>> must be
>> > consistent in use of MDB_WRITEMAP, but the documentation already makes
>> this
>> > explicit and clear.
>> >
>> > I believe this can be fixed by simply adding a check for MDB_WRITEMAP
>> > in
>> the if
>> > statement that calls SetFilePointer:
>> >
>> > if (!(flags & MDB_WRITEMAP) &&
>> > (SetFilePointer(env->me_fd,
>> sizelo, &sizehi, 0)
>> > !=3D (DWORD)sizelo
>> > || !SetEndOfFile(env->me_fd)
>> > || SetFilePointer(env->me_fd, 0, NULL, 0) !=3D 0=
))
>> > return ErrCode();
>> >
>> > The attached URL has the change as a patch/diff as applied to our node
>> package.
>> >
>> > I am certainly happy to just keep this change on our own branches.
>> > There
>> may be
>> > nuances of this that I might not be aware of, but it seems to be
>> > working
>> great
>> > for us and I have tested this with MDB_WRITEMAP enabled and disabled.
>> > So
>> I
>> > thought I would offer/suggest this change, as it seems like it is
>> > straightforward change to improve stability. Thank you!
>> >
>> >
>>
>>
>> --
>> -- Howard Chu
>> CTO, Symas Corp. http://www.symas.com
>> Director, Highland Sun http://highlandsun.com/hyc/
>> Chief Architect, OpenLDAP http://www.openldap.org/project/
>>
>
> --0000000000000dacee05826a34a9
> Content-Type: text/html; charset=3D"UTF-8"
> Content-Transfer-Encoding: quoted-printable
>
> <div dir=3D3D"ltr"><div dir=3D3D"ltr">Thank you, I appreciate it! Do you =
know
> h=3D ow often or when this will get into the github mirror <a
> href=3D3D"https://gi=3D
> thub.com/LMDB/lmdb/commits/mdb.RE/0.9">https://github.com/LMDB/lmdb/commi
> ts=3D /mdb.RE/0.9</a> (or if I am misunderstanding how the branches are
> synced)?<=3D /div></div><br><div class=3D3D"gmail_quote"><div dir=3D3D"lt=
r"
> class=3D3D"gmail_a=3D ttr">On Sun, Feb 10, 2019 at 3:14 AM Howard Chu <=
;<a
> href=3D3D"mailto:hyc@s=3D ymas.com">hyc(a)symas.com</a>>
> wrote:<br></div><blockquote class=3D3D"gmail_=3D quote" style=3D3D"margin=
:0px
> 0px 0px 0.8ex;border-left:1px solid rgb(204,204,=3D
> 204);padding-left:1ex"><a href=3D3D"mailto:kriszyp@gmail.com"
> target=3D3D"_blan=3D k">kriszyp(a)gmail.com</a> wrote:<br>
> > Full_Name: Kristopher William Zyp<br>
> > Version: LMDB 0.9.23<br>
> > OS: Windows Server 2012 R2, Windows 10<br>
> > URL: <a
> href=3D3D"https://github.com/kriszyp/node-lmdb/commit/6df903907f=3D
> 5516320e9a8afce45bd32ab4e8e1f2.patch" rel=3D3D"noreferrer"
> target=3D3D"_blank">=3D
> https://github.com/kriszyp/node-lmdb/commit/6df903907f5516320e9a8afce45bd
> 32=3D ab4e8e1f2.patch</a><br>
> > Submission from: (NULL) (71.199.6.148)<br>
> > <br>
> <br>
> Thanks for the report and patch, added to mdb.RE/0.9<br>
> <br>
> > Calling mdb_env_set_mapsize to increase the map size, when a DB is
> als=3D o in use<br>
> > by other processes, when MDB_WRITEMAP is enabled (and the db file
> has =3D been<br>
> > opened with PAGE_READWRITE access), on Windows, will occasionally
> (see=3D ms like<br>
> > about 1/100 attempts fails) produce an error "The requested
> opera=3D tion cannot be<br>
> > performed on a file with a user-mapped section open", or
> segfault=3D s. The error<br>
> > occurs in the SetFilePointer (or SetEndOfFile) call in mdb_env_map
> tha=3D t is<br>
> > performed to increase the allocated file size to the map size, prior
> t=3D o<br>
> > CreateFileMapping.<br>
> > <br>
> > As it turns out this is pretty easy to solve, because manually
> expandi=3D ng the<br>
> > file size is not necessary when calling CreateFileMapping with
> PAGE_RE=3D ADWRITE<br>
> > access, as Windows will automatically expand the file size for us,
> whe=3D n opened<br>
> > with the page write access enabled. Of course, this means all
> processe=3D s must be<br>
> > consistent in use of MDB_WRITEMAP, but the documentation already
> makes=3D this<br>
> > explicit and clear.<br>
> > <br>
> > I believe this can be fixed by simply adding a check for
> MDB_WRITEMAP =3D in the if<br>
> > statement that calls SetFilePointer:<br>
> > <br>
> >=3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3D=
C2=3DA0 =3DC2=3DA0if (!(flags
> &=3D ; MDB_WRITEMAP) && (SetFilePointer(env->me_fd, sizelo,
> &size=3D hi, 0)<br>
> > !=3D3D (DWORD)sizelo<br>
> >=3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3D=
C2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0
> =3D =3DC2=3DA0 =3DC2=3DA0|| !SetEndOfFile(env->me_fd)<br>
> >=3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3D=
C2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0
> =3D =3DC2=3DA0 =3DC2=3DA0|| SetFilePointer(env->me_fd, 0, NULL, 0) !=
=3D3D 0))<br>
> >=3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3D=
C2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0
> =3D =3DC2=3DA0 =3DC2=3DA0return ErrCode();<br>
> > <br>
> > The attached URL has the change as a patch/diff as applied to our
> node=3D package.<br>
> > <br>
> > I am certainly happy to just keep this change on our own branches.
> The=3D re may be<br>
> > nuances of this that I might not be aware of, but it seems to be
> worki=3D ng great<br>
> > for us and I have tested this with MDB_WRITEMAP enabled and
> disabled. =3D So I<br>
> > thought I would offer/suggest this change, as it seems like it is<br=
>
> > straightforward change to improve stability. Thank you!<br>
> > <br>
> > <br>
> <br>
> <br>
> -- <br>
> =3DC2=3DA0 -- Howard Chu<br>
> =3DC2=3DA0 CTO, Symas Corp.=3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =
=3DC2=3DA0 =3DC2=3DA0<a
> href=3D3D"=3D http://www.symas.com" rel=3D3D"noreferrer"
> target=3D3D"_blank">http://www.symas=3D .com</a><br>
> =3DC2=3DA0 Director, Highland Sun=3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0<a
> href=3D3D"http://highland=3Dsun.com/hyc/" rel=3D3D"noreferrer"
> target=3D3D"_blank">http://highlandsun.com/h=3D yc/</a><br>
> =3DC2=3DA0 Chief Architect, OpenLDAP=3DC2=3DA0 <a
> href=3D3D"http://www.openldap.org/p=3D roject/" rel=3D3D"noreferrer"
> target=3D3D"_blank">http://www.openldap.org/proje=3D ct/</a><br>
> </blockquote></div>
>
> --0000000000000dacee05826a34a9--
>
>
>
--
Quanah Gibson-Mount
Product Architect
Symas Corporation
Packaged, certified, and supported LDAP solutions powered by OpenLDAP:
<http://www.symas.com>
--_36998F79-0E7E-483F-8FFD-6401D532832A_
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html; charset="utf-8"
<html xmlns:o=3D"urn:schemas-microsoft-com:office:office" xmlns:w=3D"urn:sc=
hemas-microsoft-com:office:word" xmlns:m=3D"http://schemas.microsoft.com/of=
fice/2004/12/omml" xmlns=3D"http://www.w3.org/TR/REC-html40"><head><meta ht=
tp-equiv=3DContent-Type content=3D"text/html; charset=3Dutf-8"><meta name=
=3DGenerator content=3D"Microsoft Word 15 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
.MsoChpDefault
{mso-style-type:export-only;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style></head><body lang=3DEN-CA link=3Dblue vlink=3D"#954F72"><div cla=
ss=3DWordSection1><p class=3DMsoNormal>I see it now! I really did check eve=
n earlier this morning and wasn=E2=80=99t there until recently! Anyway, tha=
nk you, sorry for the noise.</p><p class=3DMsoNormal><o:p> </o:p></p><=
p class=3DMsoNormal>Thanks,<br>Kris</p><p class=3DMsoNormal><o:p> </o:=
p></p><div style=3D'mso-element:para-border-div;border:none;border-top:soli=
d #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm'><p class=3DMsoNormal style=3D'bo=
rder:none;padding:0cm'><b>From: </b><a href=3D"mailto:quanah@symas.com">Qua=
nah Gibson-Mount</a><br><b>Sent: </b>February 21, 2019 10:12 AM<br><b>To: <=
/b><a href=3D"mailto:kriszyp@gmail.com">kriszyp(a)gmail.com</a>; <a href=3D"m=
ailto:openldap-its@OpenLDAP.org">openldap-its(a)OpenLDAP.org</a><br><b>Subjec=
t: </b>Re: (ITS#8975) Calling mdb_env_set_mapsize on a shared DB failssomet=
imes</p></div><p class=3DMsoNormal><o:p> </o:p></p><p class=3DMsoNorma=
l>Hi,</p><p class=3DMsoNormal><o:p> </o:p></p><p class=3DMsoNormal>It =
is literally the first set of commits listed at that URL, so it's </p><p cl=
ass=3DMsoNormal>clearly already there.</p><p class=3DMsoNormal><o:p> <=
/o:p></p><p class=3DMsoNormal>Hope that helps!</p><p class=3DMsoNormal><o:p=
> </o:p></p><p class=3DMsoNormal>--Quanah</p><p class=3DMsoNormal><o:p=
> </o:p></p><p class=3DMsoNormal>--On Thursday, February 21, 2019 4:45=
PM +0000 kriszyp(a)gmail.com wrote:</p><p class=3DMsoNormal><o:p> </o:p=
></p><p class=3DMsoNormal>> --0000000000000dacee05826a34a9</p><p class=
=3DMsoNormal>> Content-Type: text/plain; charset=3D"UTF-8"</p>=
<p class=3DMsoNormal>><o:p> </o:p></p><p class=3DMsoNormal>> Tha=
nk you, I appreciate it! Do you know how often or when this will get</p><p =
class=3DMsoNormal>> into the github mirror https://github.com/LMDB/lmdb/=
commits/mdb.RE/0.9 (or</p><p class=3DMsoNormal>> if I am misunderstandin=
g how the branches are synced)?</p><p class=3DMsoNormal>><o:p> </o:=
p></p><p class=3DMsoNormal>> On Sun, Feb 10, 2019 at 3:14 AM Howard Chu =
<hyc(a)symas.com> wrote:</p><p class=3DMsoNormal>><o:p> </o:p><=
/p><p class=3DMsoNormal>>> kriszyp(a)gmail.com wrote:</p><p class=3DMso=
Normal>>> > Full_Name: Kristopher William Zyp</p><p class=3DMsoNor=
mal>>> > Version: LMDB 0.9.23</p><p class=3DMsoNormal>>> >=
; OS: Windows Server 2012 R2, Windows 10</p><p class=3DMsoNormal>>> &=
gt; URL:</p><p class=3DMsoNormal>>> https://github.com/kriszyp/node-l=
mdb/commit/6df903907f5516320e9a8afce45b</p><p class=3DMsoNormal>>> d3=
2ab4e8e1f2.patch</p><p class=3DMsoNormal>>> > Submission from: (NU=
LL) (71.199.6.148)</p><p class=3DMsoNormal>>> ></p><p class=3DMsoN=
ormal>>><o:p> </o:p></p><p class=3DMsoNormal>>> Thanks for=
the report and patch, added to mdb.RE/0.9</p><p class=3DMsoNormal>>>=
<o:p> </o:p></p><p class=3DMsoNormal>>> > Calling mdb_env_set=
_mapsize to increase the map size, when a DB is also</p><p class=3DMsoNorma=
l>>> in use</p><p class=3DMsoNormal>>> > by other processes,=
when MDB_WRITEMAP is enabled (and the db file has</p><p class=3DMsoNormal>=
>> been</p><p class=3DMsoNormal>>> > opened with PAGE_READWR=
ITE access), on Windows, will occasionally</p><p class=3DMsoNormal>>>=
> (seems</p><p class=3DMsoNormal>>> like</p><p class=3DMsoNormal>=
>> > about 1/100 attempts fails) produce an error "The reques=
ted operation</p><p class=3DMsoNormal>>> cannot be</p><p class=3DMsoN=
ormal>>> > performed on a file with a user-mapped section open&quo=
t;, or segfaults. The</p><p class=3DMsoNormal>>> error</p><p class=3D=
MsoNormal>>> > occurs in the SetFilePointer (or SetEndOfFile) call=
in mdb_env_map that</p><p class=3DMsoNormal>>> is</p><p class=3DMsoN=
ormal>>> > performed to increase the allocated file size to the ma=
p size, prior to</p><p class=3DMsoNormal>>> > CreateFileMapping.</=
p><p class=3DMsoNormal>>> ></p><p class=3DMsoNormal>>> > =
As it turns out this is pretty easy to solve, because manually</p><p class=
=3DMsoNormal>>> > expanding</p><p class=3DMsoNormal>>> the</=
p><p class=3DMsoNormal>>> > file size is not necessary when callin=
g CreateFileMapping with</p><p class=3DMsoNormal>>> PAGE_READWRITE</p=
><p class=3DMsoNormal>>> > access, as Windows will automatically e=
xpand the file size for us, when</p><p class=3DMsoNormal>>> opened</p=
><p class=3DMsoNormal>>> > with the page write access enabled. Of =
course, this means all processes</p><p class=3DMsoNormal>>> must be</=
p><p class=3DMsoNormal>>> > consistent in use of MDB_WRITEMAP, but=
the documentation already makes</p><p class=3DMsoNormal>>> this</p><=
p class=3DMsoNormal>>> > explicit and clear.</p><p class=3DMsoNorm=
al>>> ></p><p class=3DMsoNormal>>> > I believe this can b=
e fixed by simply adding a check for MDB_WRITEMAP</p><p class=3DMsoNormal>&=
gt;> > in</p><p class=3DMsoNormal>>> the if</p><p class=3DMsoNo=
rmal>>> > statement that calls SetFilePointer:</p><p class=3DMsoNo=
rmal>>> ></p><p class=3DMsoNormal>>> >=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 if (!(fl=
ags & MDB_WRITEMAP) &&</p><p class=3DMsoNormal>>> >=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0 (SetFilePointer(env->me_fd,</p><p class=3DMsoNormal>>> size=
lo, &sizehi, 0)</p><p class=3DMsoNormal>>> > !=3D (DWORD)sizel=
o</p><p class=3DMsoNormal>>> >=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0 || !SetEndOfFile(env->me_fd)</p><p class=3DMsoNorm=
al>>> >=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
|| SetFilePointer(env->me_fd, 0, NULL, 0) !=3D 0))</p><p class=3DMsoNor=
mal>>> >=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
return ErrCode();</p><p class=3DMsoNormal>>> ></p><p class=3DMsoN=
ormal>>> > The attached URL has the change as a patch/diff as appl=
ied to our node</p><p class=3DMsoNormal>>> package.</p><p class=3DMso=
Normal>>> ></p><p class=3DMsoNormal>>> > I am certainly h=
appy to just keep this change on our own branches.</p><p class=3DMsoNormal>=
>> > There</p><p class=3DMsoNormal>>> may be</p><p class=3DM=
soNormal>>> > nuances of this that I might not be aware of, but it=
seems to be</p><p class=3DMsoNormal>>> > working</p><p class=3DMs=
oNormal>>> great</p><p class=3DMsoNormal>>> > for us and I h=
ave tested this with MDB_WRITEMAP enabled and disabled.</p><p class=3DMsoNo=
rmal>>> > So</p><p class=3DMsoNormal>>> I</p><p class=3DMsoN=
ormal>>> > thought I would offer/suggest this change, as it seems =
like it is</p><p class=3DMsoNormal>>> > straightforward change to =
improve stability. Thank you!</p><p class=3DMsoNormal>>> ></p><p c=
lass=3DMsoNormal>>> ></p><p class=3DMsoNormal>>><o:p> <=
/o:p></p><p class=3DMsoNormal>>><o:p> </o:p></p><p class=3DMsoNo=
rmal>>> --</p><p class=3DMsoNormal>>>=C2=A0=C2=A0 -- Howard Chu=
</p><p class=3DMsoNormal>>>=C2=A0=C2=A0 CTO, Symas Corp.=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 http://www.symas.com</p><p=
class=3DMsoNormal>>>=C2=A0=C2=A0 Director, Highland Sun=C2=A0=C2=A0=
=C2=A0=C2=A0 http://highlandsun.com/hyc/</p><p class=3DMsoNormal>>>=
=C2=A0=C2=A0 Chief Architect, OpenLDAP=C2=A0 http://www.openldap.org/projec=
t/</p><p class=3DMsoNormal>>><o:p> </o:p></p><p class=3DMsoNorma=
l>><o:p> </o:p></p><p class=3DMsoNormal>> --0000000000000dacee05=
826a34a9</p><p class=3DMsoNormal>> Content-Type: text/html; charset=3D&q=
uot;UTF-8"</p><p class=3DMsoNormal>> Content-Transfer-Encoding: quo=
ted-printable</p><p class=3DMsoNormal>><o:p> </o:p></p><p class=3DM=
soNormal>> <div dir=3D3D"ltr"><div dir=3D3D"ltr&=
quot;>Thank you, I appreciate it! Do you know</p><p class=3DMsoNormal>&g=
t; h=3D ow often or when this will get into the github mirror <a</p><p c=
lass=3DMsoNormal>> href=3D3D"https://gi=3D</p><p class=3DMsoNormal>=
> thub.com/LMDB/lmdb/commits/mdb.RE/0.9">https://github.com/LMDB=
/lmdb/commi</p><p class=3DMsoNormal>> ts=3D /mdb.RE/0.9</a> (or if=
I am misunderstanding how the branches are</p><p class=3DMsoNormal>> sy=
nced)?<=3D /div></div><br><div class=3D3D"gmail_q=
uote"><div dir=3D3D"ltr"</p><p class=3DMsoNormal>> =
class=3D3D"gmail_a=3D ttr">On Sun, Feb 10, 2019 at 3:14 AM How=
ard Chu &lt;<a</p><p class=3DMsoNormal>> href=3D3D"mailto:hy=
c@s=3D ymas.com">hyc(a)symas.com</a>&gtm</a>&gt;</p><p class=3DMsoN=
ormal>> wrote:<br></div><blockquote class=3D3D"gmail=
_=3D quote" style=3D3D"margin:0px</p><p class=3DMsoNormal>> 0p=
x 0px 0.8ex;border-left:1px solid rgb(204,204,=3D</p><p class=3DMsoNormal>&=
gt; 204);padding-left:1ex"><a href=3D3D"mailto:kriszyp@gmai=
l.com"</p><p class=3DMsoNormal>> target=3D3D"_blan=3D k"&=
gt;kriszyp(a)gmail.com</a> wrote:<br></p><p class=3DMsoNormal>>=
; &gt; Full_Name: Kristopher William Zyp<br></p><p class=3DMsoNor=
mal>> &gt; Version: LMDB 0.9.23<br></p><p class=3DMsoNormal>&g=
t; &gt; OS: Windows Server 2012 R2, Windows 10<br></p><p class=3D=
MsoNormal>> &gt; URL: <a</p><p class=3DMsoNormal>> href=3D3D&q=
uot;https://github.com/kriszyp/node-lmdb/commit/6df903907f=3D</p><p class=
=3DMsoNormal>> 5516320e9a8afce45bd32ab4e8e1f2.patch" rel=3D3D"=
noreferrer"</p><p class=3DMsoNormal>> target=3D3D"_blank"=
>=3D</p><p class=3DMsoNormal>> https://github.com/kriszyp/node-lmdb/c=
ommit/6df903907f5516320e9a8afce45bd</p><p class=3DMsoNormal>> 32=3D ab4e=
8e1f2.patch</a><br></p><p class=3DMsoNormal>> &gt; Submi=
ssion from: (NULL) (71.199.6.148)<br></p><p class=3DMsoNormal>> &a=
mp;gt; <br></p><p class=3DMsoNormal>> <br></p><p class=3DMso=
Normal>> Thanks for the report and patch, added to mdb.RE/0.9<br><=
/p><p class=3DMsoNormal>> <br></p><p class=3DMsoNormal>> &g=
t; Calling mdb_env_set_mapsize to increase the map size, when a DB is</p><p=
class=3DMsoNormal>> als=3D o in use<br></p><p class=3DMsoNormal>&=
gt; &gt; by other processes, when MDB_WRITEMAP is enabled (and the db f=
ile</p><p class=3DMsoNormal>> has =3D been<br></p><p class=3DMsoNo=
rmal>> &gt; opened with PAGE_READWRITE access), on Windows, will occ=
asionally</p><p class=3DMsoNormal>> (see=3D ms like<br></p><p clas=
s=3DMsoNormal>> &gt; about 1/100 attempts fails) produce an error &a=
mp;quot;The requested</p><p class=3DMsoNormal>> opera=3D tion cannot be&=
lt;br></p><p class=3DMsoNormal>> &gt; performed on a file with a =
user-mapped section open&quot;, or</p><p class=3DMsoNormal>> segfaul=
t=3D s. The error<br></p><p class=3DMsoNormal>> &gt; occurs in=
the SetFilePointer (or SetEndOfFile) call in mdb_env_map</p><p class=3DMso=
Normal>> tha=3D t is<br></p><p class=3DMsoNormal>> &gt; per=
formed to increase the allocated file size to the map size, prior</p><p cla=
ss=3DMsoNormal>> t=3D o<br></p><p class=3DMsoNormal>> &gt; =
CreateFileMapping.<br></p><p class=3DMsoNormal>> &gt; <br&g=
t;</p><p class=3DMsoNormal>> &gt; As it turns out this is pretty eas=
y to solve, because manually</p><p class=3DMsoNormal>> expandi=3D ng the=
<br></p><p class=3DMsoNormal>> &gt; file size is not necessary=
when calling CreateFileMapping with</p><p class=3DMsoNormal>> PAGE_RE=
=3D ADWRITE<br></p><p class=3DMsoNormal>> &gt; access, as Wind=
ows will automatically expand the file size for us,</p><p class=3DMsoNormal=
>> whe=3D n opened<br></p><p class=3DMsoNormal>> &gt; with =
the page write access enabled. Of course, this means all</p><p class=3DMsoN=
ormal>> processe=3D s must be<br></p><p class=3DMsoNormal>> &am=
p;gt; consistent in use of MDB_WRITEMAP, but the documentation already</p><=
p class=3DMsoNormal>> makes=3D=C2=A0 this<br></p><p class=3DMsoNor=
mal>> &gt; explicit and clear.<br></p><p class=3DMsoNormal>>=
; &gt; <br></p><p class=3DMsoNormal>> &gt; I believe this =
can be fixed by simply adding a check for</p><p class=3DMsoNormal>> MDB_=
WRITEMAP =3D in the if<br></p><p class=3DMsoNormal>> &gt; stat=
ement that calls SetFilePointer:<br></p><p class=3DMsoNormal>> &am=
p;gt; <br></p><p class=3DMsoNormal>> &gt;=3DC2=3DA0 =3DC2=3DA0=
=3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0if (!(fla=
gs</p><p class=3DMsoNormal>> &amp=3D ; MDB_WRITEMAP) &amp;&a=
mp; (SetFilePointer(env-&gt;me_fd, sizelo,</p><p class=3DMsoNormal>>=
&amp;size=3D hi, 0)<br></p><p class=3DMsoNormal>> &gt; !=
=3D3D (DWORD)sizelo<br></p><p class=3DMsoNormal>> &gt;=3DC2=3D=
A0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=
=3DA0 =3DC2=3DA0 =3DC2=3DA0</p><p class=3DMsoNormal>> =3D =3DC2=3DA0 =3D=
C2=3DA0|| !SetEndOfFile(env-&gt;me_fd)<br></p><p class=3DMsoNorma=
l>> &gt;=3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=
=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0</p><p class=3DMsoNormal>&=
gt; =3D =3DC2=3DA0 =3DC2=3DA0|| SetFilePointer(env-&gt;me_fd, 0, NULL, =
0) !=3D3D 0))<br></p><p class=3DMsoNormal>> &gt;=3DC2=3DA0 =3D=
C2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =
=3DC2=3DA0 =3DC2=3DA0</p><p class=3DMsoNormal>> =3D =3DC2=3DA0 =3DC2=3DA=
0return ErrCode();<br></p><p class=3DMsoNormal>> &gt; <br&g=
t;</p><p class=3DMsoNormal>> &gt; The attached URL has the change as=
a patch/diff as applied to our</p><p class=3DMsoNormal>> node=3D=C2=A0 =
package.<br></p><p class=3DMsoNormal>> &gt; <br></p><p c=
lass=3DMsoNormal>> &gt; I am certainly happy to just keep this chang=
e on our own branches.</p><p class=3DMsoNormal>> The=3D re may be<br&=
gt;</p><p class=3DMsoNormal>> &gt; nuances of this that I might not =
be aware of, but it seems to be</p><p class=3DMsoNormal>> worki=3D ng gr=
eat<br></p><p class=3DMsoNormal>> &gt; for us and I have teste=
d this with MDB_WRITEMAP enabled and</p><p class=3DMsoNormal>> disabled.=
=3D So I<br></p><p class=3DMsoNormal>> &gt; thought I would o=
ffer/suggest this change, as it seems like it is<br></p><p class=3DMs=
oNormal>> &gt; straightforward change to improve stability. Thank yo=
u!<br></p><p class=3DMsoNormal>> &gt; <br></p><p class=
=3DMsoNormal>> &gt; <br></p><p class=3DMsoNormal>> <br&g=
t;</p><p class=3DMsoNormal>> <br></p><p class=3DMsoNormal>> -- =
<br></p><p class=3DMsoNormal>> =3DC2=3DA0 -- Howard Chu<br><=
/p><p class=3DMsoNormal>> =3DC2=3DA0 CTO, Symas Corp.=3DC2=3DA0 =3DC2=3D=
A0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0<a</p><p class=3DMsoNormal=
>> href=3D3D"=3D http://www.symas.com" rel=3D3D"noreferre=
r"</p><p class=3DMsoNormal>> target=3D3D"_blank">http:=
//www.symas=3D .com</a><br></p><p class=3DMsoNormal>> =3DC2=
=3DA0 Director, Highland Sun=3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0<a</p><p cla=
ss=3DMsoNormal>> href=3D3D"http://highland=3D sun.com/hyc/" re=
l=3D3D"noreferrer"</p><p class=3DMsoNormal>> target=3D3D"=
_blank">http://highlandsun.com/h=3D yc/</a><br></p><p c=
lass=3DMsoNormal>> =3DC2=3DA0 Chief Architect, OpenLDAP=3DC2=3DA0 <a<=
/p><p class=3DMsoNormal>> href=3D3D"http://www.openldap.org/p=3D ro=
ject/" rel=3D3D"noreferrer"</p><p class=3DMsoNormal>> tar=
get=3D3D"_blank">http://www.openldap.org/proje=3D ct/</a>=
;<br></p><p class=3DMsoNormal>> </blockquote></div></p=
><p class=3DMsoNormal>><o:p> </o:p></p><p class=3DMsoNormal>> --=
0000000000000dacee05826a34a9--</p><p class=3DMsoNormal>><o:p> </o:p=
></p><p class=3DMsoNormal>><o:p> </o:p></p><p class=3DMsoNormal>>=
;<o:p> </o:p></p><p class=3DMsoNormal><o:p> </o:p></p><p class=3D=
MsoNormal><o:p> </o:p></p><p class=3DMsoNormal><o:p> </o:p></p><p=
class=3DMsoNormal>--</p><p class=3DMsoNormal><o:p> </o:p></p><p class=
=3DMsoNormal>Quanah Gibson-Mount</p><p class=3DMsoNormal>Product Architect<=
/p><p class=3DMsoNormal>Symas Corporation</p><p class=3DMsoNormal>Packaged,=
certified, and supported LDAP solutions powered by OpenLDAP:</p><p class=
=3DMsoNormal><http://www.symas.com></p><p class=3DMsoNormal><o:p>&nbs=
p;</o:p></p><p class=3DMsoNormal><o:p> </o:p></p></div></body></html>=
--_36998F79-0E7E-483F-8FFD-6401D532832A_--
Hi,
It is literally the first set of commits listed at that URL, so it's
clearly already there.
Hope that helps!
--Quanah
--On Thursday, February 21, 2019 4:45 PM +0000 kriszyp(a)gmail.com wrote:
> --0000000000000dacee05826a34a9
> Content-Type: text/plain; charset="UTF-8"
>
> Thank you, I appreciate it! Do you know how often or when this will get
> into the github mirror https://github.com/LMDB/lmdb/commits/mdb.RE/0.9 (or
> if I am misunderstanding how the branches are synced)?
>
> On Sun, Feb 10, 2019 at 3:14 AM Howard Chu <hyc(a)symas.com> wrote:
>
>> kriszyp(a)gmail.com wrote:
>> > Full_Name: Kristopher William Zyp
>> > Version: LMDB 0.9.23
>> > OS: Windows Server 2012 R2, Windows 10
>> > URL:
>> https://github.com/kriszyp/node-lmdb/commit/6df903907f5516320e9a8afce45b
>> d32ab4e8e1f2.patch
>> > Submission from: (NULL) (71.199.6.148)
>> >
>>
>> Thanks for the report and patch, added to mdb.RE/0.9
>>
>> > Calling mdb_env_set_mapsize to increase the map size, when a DB is also
>> in use
>> > by other processes, when MDB_WRITEMAP is enabled (and the db file has
>> been
>> > opened with PAGE_READWRITE access), on Windows, will occasionally
>> > (seems
>> like
>> > about 1/100 attempts fails) produce an error "The requested operation
>> cannot be
>> > performed on a file with a user-mapped section open", or segfaults. The
>> error
>> > occurs in the SetFilePointer (or SetEndOfFile) call in mdb_env_map that
>> is
>> > performed to increase the allocated file size to the map size, prior to
>> > CreateFileMapping.
>> >
>> > As it turns out this is pretty easy to solve, because manually
>> > expanding
>> the
>> > file size is not necessary when calling CreateFileMapping with
>> PAGE_READWRITE
>> > access, as Windows will automatically expand the file size for us, when
>> opened
>> > with the page write access enabled. Of course, this means all processes
>> must be
>> > consistent in use of MDB_WRITEMAP, but the documentation already makes
>> this
>> > explicit and clear.
>> >
>> > I believe this can be fixed by simply adding a check for MDB_WRITEMAP
>> > in
>> the if
>> > statement that calls SetFilePointer:
>> >
>> > if (!(flags & MDB_WRITEMAP) &&
>> > (SetFilePointer(env->me_fd,
>> sizelo, &sizehi, 0)
>> > != (DWORD)sizelo
>> > || !SetEndOfFile(env->me_fd)
>> > || SetFilePointer(env->me_fd, 0, NULL, 0) != 0))
>> > return ErrCode();
>> >
>> > The attached URL has the change as a patch/diff as applied to our node
>> package.
>> >
>> > I am certainly happy to just keep this change on our own branches.
>> > There
>> may be
>> > nuances of this that I might not be aware of, but it seems to be
>> > working
>> great
>> > for us and I have tested this with MDB_WRITEMAP enabled and disabled.
>> > So
>> I
>> > thought I would offer/suggest this change, as it seems like it is
>> > straightforward change to improve stability. Thank you!
>> >
>> >
>>
>>
>> --
>> -- Howard Chu
>> CTO, Symas Corp. http://www.symas.com
>> Director, Highland Sun http://highlandsun.com/hyc/
>> Chief Architect, OpenLDAP http://www.openldap.org/project/
>>
>
> --0000000000000dacee05826a34a9
> Content-Type: text/html; charset="UTF-8"
> Content-Transfer-Encoding: quoted-printable
>
> <div dir=3D"ltr"><div dir=3D"ltr">Thank you, I appreciate it! Do you know
> h= ow often or when this will get into the github mirror <a
> href=3D"https://gi=
> thub.com/LMDB/lmdb/commits/mdb.RE/0.9">https://github.com/LMDB/lmdb/commi
> ts= /mdb.RE/0.9</a> (or if I am misunderstanding how the branches are
> synced)?<= /div></div><br><div class=3D"gmail_quote"><div dir=3D"ltr"
> class=3D"gmail_a= ttr">On Sun, Feb 10, 2019 at 3:14 AM Howard Chu <<a
> href=3D"mailto:hyc@s= ymas.com">hyc(a)symas.com</a>>
> wrote:<br></div><blockquote class=3D"gmail_= quote" style=3D"margin:0px
> 0px 0px 0.8ex;border-left:1px solid rgb(204,204,=
> 204);padding-left:1ex"><a href=3D"mailto:kriszyp@gmail.com"
> target=3D"_blan= k">kriszyp(a)gmail.com</a> wrote:<br>
> > Full_Name: Kristopher William Zyp<br>
> > Version: LMDB 0.9.23<br>
> > OS: Windows Server 2012 R2, Windows 10<br>
> > URL: <a
> href=3D"https://github.com/kriszyp/node-lmdb/commit/6df903907f=
> 5516320e9a8afce45bd32ab4e8e1f2.patch" rel=3D"noreferrer"
> target=3D"_blank">=
> https://github.com/kriszyp/node-lmdb/commit/6df903907f5516320e9a8afce45bd
> 32= ab4e8e1f2.patch</a><br>
> > Submission from: (NULL) (71.199.6.148)<br>
> > <br>
> <br>
> Thanks for the report and patch, added to mdb.RE/0.9<br>
> <br>
> > Calling mdb_env_set_mapsize to increase the map size, when a DB is
> als= o in use<br>
> > by other processes, when MDB_WRITEMAP is enabled (and the db file
> has = been<br>
> > opened with PAGE_READWRITE access), on Windows, will occasionally
> (see= ms like<br>
> > about 1/100 attempts fails) produce an error "The requested
> opera= tion cannot be<br>
> > performed on a file with a user-mapped section open", or
> segfault= s. The error<br>
> > occurs in the SetFilePointer (or SetEndOfFile) call in mdb_env_map
> tha= t is<br>
> > performed to increase the allocated file size to the map size, prior
> t= o<br>
> > CreateFileMapping.<br>
> > <br>
> > As it turns out this is pretty easy to solve, because manually
> expandi= ng the<br>
> > file size is not necessary when calling CreateFileMapping with
> PAGE_RE= ADWRITE<br>
> > access, as Windows will automatically expand the file size for us,
> whe= n opened<br>
> > with the page write access enabled. Of course, this means all
> processe= s must be<br>
> > consistent in use of MDB_WRITEMAP, but the documentation already
> makes= this<br>
> > explicit and clear.<br>
> > <br>
> > I believe this can be fixed by simply adding a check for
> MDB_WRITEMAP = in the if<br>
> > statement that calls SetFilePointer:<br>
> > <br>
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if (!(flags
> &= ; MDB_WRITEMAP) && (SetFilePointer(env->me_fd, sizelo,
> &size= hi, 0)<br>
> > !=3D (DWORD)sizelo<br>
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0
> = =C2=A0 =C2=A0|| !SetEndOfFile(env->me_fd)<br>
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0
> = =C2=A0 =C2=A0|| SetFilePointer(env->me_fd, 0, NULL, 0) !=3D 0))<br>
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0
> = =C2=A0 =C2=A0return ErrCode();<br>
> > <br>
> > The attached URL has the change as a patch/diff as applied to our
> node= package.<br>
> > <br>
> > I am certainly happy to just keep this change on our own branches.
> The= re may be<br>
> > nuances of this that I might not be aware of, but it seems to be
> worki= ng great<br>
> > for us and I have tested this with MDB_WRITEMAP enabled and
> disabled. = So I<br>
> > thought I would offer/suggest this change, as it seems like it is<br>
> > straightforward change to improve stability. Thank you!<br>
> > <br>
> > <br>
> <br>
> <br>
> -- <br>
> =C2=A0 -- Howard Chu<br>
> =C2=A0 CTO, Symas Corp.=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0<a
> href=3D"= http://www.symas.com" rel=3D"noreferrer"
> target=3D"_blank">http://www.symas= .com</a><br>
> =C2=A0 Director, Highland Sun=C2=A0 =C2=A0 =C2=A0<a
> href=3D"http://highland=sun.com/hyc/" rel=3D"noreferrer"
> target=3D"_blank">http://highlandsun.com/h= yc/</a><br>
> =C2=A0 Chief Architect, OpenLDAP=C2=A0 <a
> href=3D"http://www.openldap.org/p= roject/" rel=3D"noreferrer"
> target=3D"_blank">http://www.openldap.org/proje= ct/</a><br>
> </blockquote></div>
>
> --0000000000000dacee05826a34a9--
>
>
>
--
Quanah Gibson-Mount
Product Architect
Symas Corporation
Packaged, certified, and supported LDAP solutions powered by OpenLDAP:
<http://www.symas.com>
--0000000000000dacee05826a34a9
Content-Type: text/plain; charset="UTF-8"
Thank you, I appreciate it! Do you know how often or when this will get
into the github mirror https://github.com/LMDB/lmdb/commits/mdb.RE/0.9 (or
if I am misunderstanding how the branches are synced)?
On Sun, Feb 10, 2019 at 3:14 AM Howard Chu <hyc(a)symas.com> wrote:
> kriszyp(a)gmail.com wrote:
> > Full_Name: Kristopher William Zyp
> > Version: LMDB 0.9.23
> > OS: Windows Server 2012 R2, Windows 10
> > URL:
> https://github.com/kriszyp/node-lmdb/commit/6df903907f5516320e9a8afce45bd32…
> > Submission from: (NULL) (71.199.6.148)
> >
>
> Thanks for the report and patch, added to mdb.RE/0.9
>
> > Calling mdb_env_set_mapsize to increase the map size, when a DB is also
> in use
> > by other processes, when MDB_WRITEMAP is enabled (and the db file has
> been
> > opened with PAGE_READWRITE access), on Windows, will occasionally (seems
> like
> > about 1/100 attempts fails) produce an error "The requested operation
> cannot be
> > performed on a file with a user-mapped section open", or segfaults. The
> error
> > occurs in the SetFilePointer (or SetEndOfFile) call in mdb_env_map that
> is
> > performed to increase the allocated file size to the map size, prior to
> > CreateFileMapping.
> >
> > As it turns out this is pretty easy to solve, because manually expanding
> the
> > file size is not necessary when calling CreateFileMapping with
> PAGE_READWRITE
> > access, as Windows will automatically expand the file size for us, when
> opened
> > with the page write access enabled. Of course, this means all processes
> must be
> > consistent in use of MDB_WRITEMAP, but the documentation already makes
> this
> > explicit and clear.
> >
> > I believe this can be fixed by simply adding a check for MDB_WRITEMAP in
> the if
> > statement that calls SetFilePointer:
> >
> > if (!(flags & MDB_WRITEMAP) && (SetFilePointer(env->me_fd,
> sizelo, &sizehi, 0)
> > != (DWORD)sizelo
> > || !SetEndOfFile(env->me_fd)
> > || SetFilePointer(env->me_fd, 0, NULL, 0) != 0))
> > return ErrCode();
> >
> > The attached URL has the change as a patch/diff as applied to our node
> package.
> >
> > I am certainly happy to just keep this change on our own branches. There
> may be
> > nuances of this that I might not be aware of, but it seems to be working
> great
> > for us and I have tested this with MDB_WRITEMAP enabled and disabled. So
> I
> > thought I would offer/suggest this change, as it seems like it is
> > straightforward change to improve stability. Thank you!
> >
> >
>
>
> --
> -- Howard Chu
> CTO, Symas Corp. http://www.symas.com
> Director, Highland Sun http://highlandsun.com/hyc/
> Chief Architect, OpenLDAP http://www.openldap.org/project/
>
--0000000000000dacee05826a34a9
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr"><div dir=3D"ltr">Thank you, I appreciate it! Do you know h=
ow often or when this will get into the github mirror <a href=3D"https://gi=thub.com/LMDB/lmdb/commits/mdb.RE/0.9">https://github.com/LMDB/lmdb/commits=
/mdb.RE/0.9</a> (or if I am misunderstanding how the branches are synced)?<=
/div></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_a=
ttr">On Sun, Feb 10, 2019 at 3:14 AM Howard Chu <<a href=3D"mailto:hyc@s=
ymas.com">hyc(a)symas.com</a>> wrote:<br></div><blockquote class=3D"gmail_=
quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,=
204);padding-left:1ex"><a href=3D"mailto:kriszyp@gmail.com" target=3D"_blan=
k">kriszyp(a)gmail.com</a> wrote:<br>
> Full_Name: Kristopher William Zyp<br>
> Version: LMDB 0.9.23<br>
> OS: Windows Server 2012 R2, Windows 10<br>
> URL: <a href=3D"https://github.com/kriszyp/node-lmdb/commit/6df903907f=
5516320e9a8afce45bd32ab4e8e1f2.patch" rel=3D"noreferrer" target=3D"_blank">=
https://github.com/kriszyp/node-lmdb/commit/6df903907f5516320e9a8afce45bd32=
ab4e8e1f2.patch</a><br>
> Submission from: (NULL) (71.199.6.148)<br>
> <br>
<br>
Thanks for the report and patch, added to mdb.RE/0.9<br>
<br>
> Calling mdb_env_set_mapsize to increase the map size, when a DB is als=
o in use<br>
> by other processes, when MDB_WRITEMAP is enabled (and the db file has =
been<br>
> opened with PAGE_READWRITE access), on Windows, will occasionally (see=
ms like<br>
> about 1/100 attempts fails) produce an error "The requested opera=
tion cannot be<br>
> performed on a file with a user-mapped section open", or segfault=
s. The error<br>
> occurs in the SetFilePointer (or SetEndOfFile) call in mdb_env_map tha=
t is<br>
> performed to increase the allocated file size to the map size, prior t=
o<br>
> CreateFileMapping.<br>
> <br>
> As it turns out this is pretty easy to solve, because manually expandi=
ng the<br>
> file size is not necessary when calling CreateFileMapping with PAGE_RE=
ADWRITE<br>
> access, as Windows will automatically expand the file size for us, whe=
n opened<br>
> with the page write access enabled. Of course, this means all processe=
s must be<br>
> consistent in use of MDB_WRITEMAP, but the documentation already makes=
this<br>
> explicit and clear.<br>
> <br>
> I believe this can be fixed by simply adding a check for MDB_WRITEMAP =
in the if<br>
> statement that calls SetFilePointer:<br>
> <br>
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if (!(flags &=
; MDB_WRITEMAP) && (SetFilePointer(env->me_fd, sizelo, &size=
hi, 0)<br>
> !=3D (DWORD)sizelo<br>
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0|| !SetEndOfFile(env->me_fd)<br>
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0|| SetFilePointer(env->me_fd, 0, NULL, 0) !=3D 0))<br>
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0return ErrCode();<br>
> <br>
> The attached URL has the change as a patch/diff as applied to our node=
package.<br>
> <br>
> I am certainly happy to just keep this change on our own branches. The=
re may be<br>
> nuances of this that I might not be aware of, but it seems to be worki=
ng great<br>
> for us and I have tested this with MDB_WRITEMAP enabled and disabled. =
So I<br>
> thought I would offer/suggest this change, as it seems like it is<br>
> straightforward change to improve stability. Thank you!<br>
> <br>
> <br>
<br>
<br>
-- <br>
=C2=A0 -- Howard Chu<br>
=C2=A0 CTO, Symas Corp.=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0<a href=3D"=
http://www.symas.com" rel=3D"noreferrer" target=3D"_blank">http://www.symas=
.com</a><br>
=C2=A0 Director, Highland Sun=C2=A0 =C2=A0 =C2=A0<a href=3D"http://highland=sun.com/hyc/" rel=3D"noreferrer" target=3D"_blank">http://highlandsun.com/h=
yc/</a><br>
=C2=A0 Chief Architect, OpenLDAP=C2=A0 <a href=3D"http://www.openldap.org/p=
roject/" rel=3D"noreferrer" target=3D"_blank">http://www.openldap.org/proje=
ct/</a><br>
</blockquote></div>
--0000000000000dacee05826a34a9--
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p><font face="Arial,Verdana,Helvetica">The attached files are
derived from OpenLDAP Software.
All of the modifications to OpenLDAP Software represented in the
following patch(es) were developed by Interlink Networks LLC. </font><font
face="Arial,Verdana,Helvetica"><font
face="Arial,Verdana,Helvetica">Interlink Networks LLC</font>
has not assigned rights and/or interest in
this work to any party. I, Vernon Smith am authorized by </font><font
face="Arial,Verdana,Helvetica"><font
face="Arial,Verdana,Helvetica">Interlink Networks LLC</font>,
my employer, to release this work under
the following terms.</font></p>
<p><font face="Arial,Verdana,Helvetica"><font
face="Arial,Verdana,Helvetica">Interlink Networks LLC </font></font><font
face="Arial,Verdana,Helvetica">hereby place the following
modifications to
OpenLDAP Software (and only these modifications) into the public
domain. Hence, these modifications may be freely used and/or
redistributed for any purpose with or without attribution and/or
other notice.</font></p>
<p><font face="Arial,Verdana,Helvetica">Thanks, Vern<br>
</font></p>
<div class="moz-cite-prefix">On 2/20/2019 9:42 AM, Quanah
Gibson-Mount wrote:<br>
</div>
<blockquote type="cite"
cite="mid:8A8B819A395EEAE290F20248@[192.168.1.39]">--On Tuesday,
February 19, 2019 2:41 PM +0000 <a class="moz-txt-link-abbreviated" href="mailto:quanah@symas.com">quanah(a)symas.com</a> wrote:
<br>
<br>
<blockquote type="cite">--On Tuesday, February 19, 2019 2:12 AM
-0500 Vern Smith
<br>
<a class="moz-txt-link-rfc2396E" href="mailto:vsmith@interlinknetworks.com"><vsmith(a)interlinknetworks.com></a> wrote:
<br>
<br>
<blockquote type="cite">ITS#8957 and ITS#8968 are my submits and
I was using those changes.
<br>
ITS#8963 is for OpenLDAP servers and I only an using the
libraries so it
<br>
does not apply to my updates.
<br>
</blockquote>
<br>
Please keep replies to the ITS system.
<br>
</blockquote>
<br>
Hi,
<br>
<br>
Although each change is individually tiny, the sum of changes is
not. Can you please add an IPR to this ITS, as documented at
<a class="moz-txt-link-rfc2396E" href="https://www.openldap.org/devel/contributing.html#notice"><https://www.openldap.org/devel/contributing.html#notice></a>
<br>
<br>
Thanks!
<br>
<br>
Regards,
<br>
Quanah
<br>
<br>
<br>
--
<br>
<br>
Quanah Gibson-Mount
<br>
Product Architect
<br>
Symas Corporation
<br>
Packaged, certified, and supported LDAP solutions powered by
OpenLDAP:
<br>
<a class="moz-txt-link-rfc2396E" href="http://www.symas.com"><http://www.symas.com></a>
<br>
<br>
<br>
</blockquote>
</body>
</html>
--On Tuesday, February 19, 2019 2:41 PM +0000 quanah(a)symas.com wrote:
> --On Tuesday, February 19, 2019 2:12 AM -0500 Vern Smith
> <vsmith(a)interlinknetworks.com> wrote:
>
>> ITS#8957 and ITS#8968 are my submits and I was using those changes.
>> ITS#8963 is for OpenLDAP servers and I only an using the libraries so it
>> does not apply to my updates.
>
> Please keep replies to the ITS system.
Hi,
Although each change is individually tiny, the sum of changes is not. Can
you please add an IPR to this ITS, as documented at
<https://www.openldap.org/devel/contributing.html#notice>
Thanks!
Regards,
Quanah
--
Quanah Gibson-Mount
Product Architect
Symas Corporation
Packaged, certified, and supported LDAP solutions powered by OpenLDAP:
<http://www.symas.com>
quanah(a)openldap.org wrote:
> Full_Name: Quanah Gibson-Mount
> Version: RE25
> OS: N/A
> URL: ftp://ftp.openldap.org/incoming/
> Submission from: (NULL) (47.208.148.239)
>
>
> It's currently difficult for downstream packagers to create a working
> development package as some items required to build external modules are only
> included what are considered "private" headers, such as slap.h (See ITS#7283).
> SLAP_CB_{BYPASS,CONTINUE} are examples of items that are contained in slap.h
> that are potentially required for building external modules.
>
> IIRC, it's also been discussed previously to rename "ldap_pvt.h" to "openldap.h"
> to indicate it's ok to package this header file in a -devel package (I could
> have the initial header name incorrect, it's been a while).
This discussion is a bit awkward. OpenLDAP is an open source project. What we provide
is a distribution of source code. It was never intended to be sliced and diced into
individual binary packages. If you're developing modules for OpenLDAP, you're expected
to have a complete OpenLDAP build tree in which your modules reside. Modules meant to
run inside slapd are by definition *internal* - they are running inside slapd after all.
The fact that they're written and maintained by 3rd parties doesn't change this.
Header files marked "private" mean they're only for internal use within an OpenLDAP
build tree. Applications that are generic LDAP clients should only use the public APIs.
We can still discuss formally making some OpenLDAP-specific APIs public, but that
should probably be done on a case by case basis. LDAP is supposed to be interoperable;
we should not encourage broader use of OpenLDAP-specific APIs that rely on details
of OpenLDAP's internal implementation. If they have no particular dependency on
internals, then we may safely expose them.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
Full_Name: Quanah Gibson-Mount
Version: RE25
OS: N/A
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (47.208.148.239)
It's currently difficult for downstream packagers to create a working
development package as some items required to build external modules are only
included what are considered "private" headers, such as slap.h (See ITS#7283).
SLAP_CB_{BYPASS,CONTINUE} are examples of items that are contained in slap.h
that are potentially required for building external modules.
IIRC, it's also been discussed previously to rename "ldap_pvt.h" to "openldap.h"
to indicate it's ok to package this header file in a -devel package (I could
have the initial header name incorrect, it's been a while).
--On Tuesday, February 19, 2019 2:12 AM -0500 Vern Smith
<vsmith(a)interlinknetworks.com> wrote:
> ITS#8957 and ITS#8968 are my submits and I was using those changes.
> ITS#8963 is for OpenLDAP servers and I only an using the libraries so it
> does not apply to my updates.
Please keep replies to the ITS system.
When filing an ITS, please note specifically which commits past a release
you've applied, so the ITS is a clear representation of the issue you're
reporting (and thus avoiding questions such as those I posited). Thanks!
--Quanah
--
Quanah Gibson-Mount
Product Architect
Symas Corporation
Packaged, certified, and supported LDAP solutions powered by OpenLDAP:
<http://www.symas.com>