--0000000000000dacee05826a34a9 Content-Type: text/plain; charset="UTF-8"
Thank you, I appreciate it! Do you know how often or when this will get into the github mirror https://github.com/LMDB/lmdb/commits/mdb.RE/0.9 (or if I am misunderstanding how the branches are synced)?
On Sun, Feb 10, 2019 at 3:14 AM Howard Chu hyc@symas.com wrote:
kriszyp@gmail.com wrote:
Full_Name: Kristopher William Zyp Version: LMDB 0.9.23 OS: Windows Server 2012 R2, Windows 10 URL:
https://github.com/kriszyp/node-lmdb/commit/6df903907f5516320e9a8afce45bd32a...
Submission from: (NULL) (71.199.6.148)
Thanks for the report and patch, added to mdb.RE/0.9
Calling mdb_env_set_mapsize to increase the map size, when a DB is also
in use
by other processes, when MDB_WRITEMAP is enabled (and the db file has
been
opened with PAGE_READWRITE access), on Windows, will occasionally (seems
like
about 1/100 attempts fails) produce an error "The requested operation
cannot be
performed on a file with a user-mapped section open", or segfaults. The
error
occurs in the SetFilePointer (or SetEndOfFile) call in mdb_env_map that
is
performed to increase the allocated file size to the map size, prior to CreateFileMapping.
As it turns out this is pretty easy to solve, because manually expanding
the
file size is not necessary when calling CreateFileMapping with
PAGE_READWRITE
access, as Windows will automatically expand the file size for us, when
opened
with the page write access enabled. Of course, this means all processes
must be
consistent in use of MDB_WRITEMAP, but the documentation already makes
this
explicit and clear.
I believe this can be fixed by simply adding a check for MDB_WRITEMAP in
the if
statement that calls SetFilePointer:
if (!(flags & MDB_WRITEMAP) && (SetFilePointer(env->me_fd,sizelo, &sizehi, 0)
!= (DWORD)sizelo || !SetEndOfFile(env->me_fd) || SetFilePointer(env->me_fd, 0, NULL, 0) != 0)) return ErrCode();
The attached URL has the change as a patch/diff as applied to our node
package.
I am certainly happy to just keep this change on our own branches. There
may be
nuances of this that I might not be aware of, but it seems to be working
great
for us and I have tested this with MDB_WRITEMAP enabled and disabled. So
I
thought I would offer/suggest this change, as it seems like it is straightforward change to improve stability. Thank you!
-- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
--0000000000000dacee05826a34a9 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr"><div dir=3D"ltr">Thank you, I appreciate it! Do you know h= ow often or when this will get into the github mirror <a href=3D"https://gi= thub.com/LMDB/lmdb/commits/mdb.RE/0.9">https://github.com/LMDB/lmdb/commits= /mdb.RE/0.9</a> (or if I am misunderstanding how the branches are synced)?<= /div></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_a= ttr">On Sun, Feb 10, 2019 at 3:14 AM Howard Chu <<a href=3D"mailto:hyc@s= ymas.com">hyc@symas.com</a>> wrote:<br></div><blockquote class=3D"gmail_= quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,= 204);padding-left:1ex"><a href=3D"mailto:kriszyp@gmail.com" target=3D"_blan= k">kriszyp@gmail.com</a> wrote:<br> > Full_Name: Kristopher William Zyp<br> > Version: LMDB 0.9.23<br> > OS: Windows Server 2012 R2, Windows 10<br> > URL: <a href=3D"https://github.com/kriszyp/node-lmdb/commit/6df903907f= 5516320e9a8afce45bd32ab4e8e1f2.patch" rel=3D"noreferrer" target=3D"_blank">= https://github.com/kriszyp/node-lmdb/commit/6df903907f5516320e9a8afce45bd32= ab4e8e1f2.patch</a><br> > Submission from: (NULL) (71.199.6.148)<br> > <br> <br> Thanks for the report and patch, added to mdb.RE/0.9<br> <br> > Calling mdb_env_set_mapsize to increase the map size, when a DB is als= o in use<br> > by other processes, when MDB_WRITEMAP is enabled (and the db file has = been<br> > opened with PAGE_READWRITE access), on Windows, will occasionally (see= ms like<br> > about 1/100 attempts fails) produce an error "The requested opera= tion cannot be<br> > performed on a file with a user-mapped section open", or segfault= s. The error<br> > occurs in the SetFilePointer (or SetEndOfFile) call in mdb_env_map tha= t is<br> > performed to increase the allocated file size to the map size, prior t= o<br> > CreateFileMapping.<br> > <br> > As it turns out this is pretty easy to solve, because manually expandi= ng the<br> > file size is not necessary when calling CreateFileMapping with PAGE_RE= ADWRITE<br> > access, as Windows will automatically expand the file size for us, whe= n opened<br> > with the page write access enabled. Of course, this means all processe= s must be<br> > consistent in use of MDB_WRITEMAP, but the documentation already makes= this<br> > explicit and clear.<br> > <br> > I believe this can be fixed by simply adding a check for MDB_WRITEMAP = in the if<br> > statement that calls SetFilePointer:<br> > <br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if (!(flags &= ; MDB_WRITEMAP) && (SetFilePointer(env->me_fd, sizelo, &size= hi, 0)<br> > !=3D (DWORD)sizelo<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0|| !SetEndOfFile(env->me_fd)<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0|| SetFilePointer(env->me_fd, 0, NULL, 0) !=3D 0))<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0return ErrCode();<br> > <br> > The attached URL has the change as a patch/diff as applied to our node= package.<br> > <br> > I am certainly happy to just keep this change on our own branches. The= re may be<br> > nuances of this that I might not be aware of, but it seems to be worki= ng great<br> > for us and I have tested this with MDB_WRITEMAP enabled and disabled. = So I<br> > thought I would offer/suggest this change, as it seems like it is<br> > straightforward change to improve stability. Thank you!<br> > <br> > <br> <br> <br> -- <br> =C2=A0 -- Howard Chu<br> =C2=A0 CTO, Symas Corp.=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0<a href=3D"= http://www.symas.com" rel=3D"noreferrer" target=3D"_blank">http://www.symas= .com</a><br> =C2=A0 Director, Highland Sun=C2=A0 =C2=A0 =C2=A0<a href=3D"http://highland= sun.com/hyc/" rel=3D"noreferrer" target=3D"_blank">http://highlandsun.com/h= yc/</a><br> =C2=A0 Chief Architect, OpenLDAP=C2=A0 <a href=3D"http://www.openldap.org/p= roject/" rel=3D"noreferrer" target=3D"_blank">http://www.openldap.org/proje= ct/</a><br> </blockquote></div>
--0000000000000dacee05826a34a9--