https://bugs.openldap.org/show_bug.cgi?id=8125
Quanah Gibson-Mount <quanah(a)openldap.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|UNCONFIRMED |CONFIRMED
Ever confirmed|0 |1
--
You are receiving this mail because:
You are on the CC list for the bug.
https://bugs.openldap.org/show_bug.cgi?id=9017
--- Comment #19 from Howard Chu <hyc(a)openldap.org> ---
openldap-its(a)openldap.org wrote:
> https://bugs.openldap.org/show_bug.cgi?id=9017
>
> --- Comment #18 from kriszyp(a)gmail.com <kriszyp(a)gmail.com> ---
>> I've committed your first two patches as-is,
> Thank you so much, appreciate the review and including this code. I
> hope it helps other Windows users leverage the awesome LMDB
> performance and integrity!
>
>> and taken a different approach for the off_t issue.
> Unfortunately, this commit didn't work for me. LMDB no longer compiles
> when using LARGE_INTEGER for offsets. Perhaps ULONG would work better?
> I get the following compilation errors (or is there possibly something
> I need to define to address this?):
>
> \dependencies\lmdb\libraries\liblmdb\mdb.c(3739): error C2440: '=':
> cannot convert from 'pgno_t' to 'LA
> RGE_INTEGER' [\build\node-lmdb.vcxproj]
> \dependencies\lmdb\libraries\liblmdb\mdb.c(3744): error C2088: '!=':
> illegal for union [C:\dev\node-lmd
> b\build\node-lmdb.vcxproj]
> \dependencies\lmdb\libraries\liblmdb\mdb.c(3767): error C2088: '&':
> illegal for union [
> \build\node-lmdb.vcxproj]
Apparently LARGE_INTEGER isn't an integer at all, it appears to be a
structure.
You ought to be able to use int64_t instead, or any other explicitly
64-bit signed integer type.
Since offsets can be forward or backward from a designated reference
point, off_t and MDB_OFF_T must be a signed type. size_t is explicitly
unsigned, so it is an error to use that for this purpose. It's also only
32 bits wide on a 32bit machine, so it'd be unsuitable for larger DBs.
The use of MDB_OFF_T/off_t for env->me_size is a bit bogus, since a
file size should be unsigned. But we don't have address spaces large
enough yet for the difference between 2^63 and 2^64 to matter. (And
also, the me_size field has actually never been used. Probably should
just be deleted...)
>> Do you still want the corresponding patch in 0.9? And can you please remind
>> me where the 0.9 patch is? Thanks.
> I could if you want, but you convinced me that using mdb.master and
> focusing 1.0 was the way to go, so that's what we have been using for
> the last few months (with my patches applied locally), and it has been
> working great on our servers. I don't need an 0.9 patch, but I could
> work on creating one if you think it is important.
No, that's fine.
--
You are receiving this mail because:
You are on the CC list for the bug.
https://bugs.openldap.org/show_bug.cgi?id=9017
--- Comment #18 from kriszyp(a)gmail.com <kriszyp(a)gmail.com> ---
> I've committed your first two patches as-is,
Thank you so much, appreciate the review and including this code. I
hope it helps other Windows users leverage the awesome LMDB
performance and integrity!
> and taken a different approach for the off_t issue.
Unfortunately, this commit didn't work for me. LMDB no longer compiles
when using LARGE_INTEGER for offsets. Perhaps ULONG would work better?
I get the following compilation errors (or is there possibly something
I need to define to address this?):
\dependencies\lmdb\libraries\liblmdb\mdb.c(3739): error C2440: '=':
cannot convert from 'pgno_t' to 'LA
RGE_INTEGER' [\build\node-lmdb.vcxproj]
\dependencies\lmdb\libraries\liblmdb\mdb.c(3744): error C2088: '!=':
illegal for union [C:\dev\node-lmd
b\build\node-lmdb.vcxproj]
\dependencies\lmdb\libraries\liblmdb\mdb.c(3767): error C2088: '&':
illegal for union [
\build\node-lmdb.vcxproj]
\dependencies\lmdb\libraries\liblmdb\mdb.c(3768): error C2088: '>>':
illegal for union [C:\dev\node-lmd
b\build\node-lmdb.vcxproj]
\dependencies\lmdb\libraries\liblmdb\mdb.c(3832): error C2088: '+':
illegal for union [
\build\node-lmdb.vcxproj]
\dependencies\lmdb\libraries\liblmdb\mdb.c(3667): error C2440:
'initializing': cannot convert from 'int
' to 'LARGE_INTEGER' [\build\node-lmdb.vcxproj]
\dependencies\lmdb\libraries\liblmdb\mdb.c(3680): error C2440:
'initializing': cannot convert from 'int
' to 'LARGE_INTEGER' [\build\node-lmdb.vcxproj]
\dependencies\lmdb\libraries\liblmdb\mdb.c(4333): error C2440: '=':
cannot convert from 'size_t' to 'LA
RGE_INTEGER' [\build\node-lmdb.vcxproj]
\dependencies\lmdb\libraries\liblmdb\mdb.c(4334): error C2088: '+':
illegal for union [
\build\node-lmdb.vcxproj]
\dependencies\lmdb\libraries\liblmdb\mdb.c(4335): error C2088: '-':
illegal for union [
\build\node-lmdb.vcxproj]
\dependencies\lmdb\libraries\liblmdb\mdb.c(4336): error C2088: '+=':
illegal for union [C:\dev\node-lmd
b\build\node-lmdb.vcxproj]
\dependencies\lmdb\libraries\liblmdb\mdb.c(4346): error C2440: '=':
cannot convert from 'LARGE_INTEGER'
to 'DWORD' [\build\node-lmdb.vcxproj]
\dependencies\lmdb\libraries\liblmdb\mdb.c(4369): error C2440: '=':
cannot convert from 'LARGE_INTEGER'
to 'DWORD' [\build\node-lmdb.vcxproj]
\dependencies\lmdb\libraries\liblmdb\mdb.c(5319): error C2440: '=':
cannot convert from 'DWORD' to 'LAR
GE_INTEGER' [\build\node-lmdb.vcxproj]
\dependencies\lmdb\libraries\liblmdb\mdb.c(5324): error C2440: '=':
cannot convert from 'size_t' to 'LA
RGE_INTEGER' [\build\node-lmdb.vcxproj]
\dependencies\lmdb\libraries\liblmdb\mdb.c(5325): error C2088: '<':
illegal for union [
\build\node-lmdb.vcxproj]
\dependencies\lmdb\libraries\liblmdb\mdb.c(5327): error C2440:
'function': cannot convert from 'LARGE_I
NTEGER' to 'LONG' [\build\node-lmdb.vcxproj]
\dependencies\lmdb\libraries\liblmdb\mdb.c(5327): warning C4024:
'SetFilePointer': different types for
formal and actual parameter 2 [\build\node-lmdb.vcxproj]
\dependencies\lmdb\libraries\liblmdb\mdb.c(5328): error C2440: 'type
cast': cannot convert from 'LARGE_
INTEGER' to 'DWORD' [\build\node-lmdb.vcxproj]
\dependencies\lmdb\libraries\liblmdb\mdb.c(5335): error C2088: '-':
illegal for union [
\build\node-lmdb.vcxproj]
\dependencies\lmdb\libraries\liblmdb\mdb.c(5336): error C2088: '/':
illegal for union [
\build\node-lmdb.vcxproj]
\dependencies\lmdb\libraries\liblmdb\mdb.c(5344): error C2440:
'function': cannot convert from 'LARGE_I
NTEGER' to 'SIZE_T' [\build\node-lmdb.vcxproj]
\dependencies\lmdb\libraries\liblmdb\mdb.c(5344): warning C4024:
'MapViewOfFileEx': different types for
formal and actual parameter 5 [\build\node-lmdb.vcxproj]
(and it all compiles fine when I use size_t
> Do you still want the corresponding patch in 0.9? And can you please remind
> me where the 0.9 patch is? Thanks.
I could if you want, but you convinced me that using mdb.master and
focusing 1.0 was the way to go, so that's what we have been using for
the last few months (with my patches applied locally), and it has been
working great on our servers. I don't need an 0.9 patch, but I could
work on creating one if you think it is important.
> And please don't send HTML emails, plaintext only.
My apologies. Missed some settings, I despise that HTML cruft as well.
Thanks,
Kris
--
You are receiving this mail because:
You are on the CC list for the bug.
https://bugs.openldap.org/show_bug.cgi?id=8847
Quanah Gibson-Mount <quanah(a)openldap.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|IN_PROGRESS |CONFIRMED
--
You are receiving this mail because:
You are on the CC list for the bug.
https://bugs.openldap.org/show_bug.cgi?id=8783
Quanah Gibson-Mount <quanah(a)openldap.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|RESOLVED |VERIFIED
--
You are receiving this mail because:
You are on the CC list for the bug.
https://bugs.openldap.org/show_bug.cgi?id=9017
Quanah Gibson-Mount <quanah(a)openldap.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|UNCONFIRMED |IN_PROGRESS
Ever confirmed|0 |1
--
You are receiving this mail because:
You are on the CC list for the bug.
https://bugs.openldap.org/show_bug.cgi?id=9017
--- Comment #17 from Quanah Gibson-Mount <quanah(a)openldap.org> ---
Commits:
• da0527ac
by Kris Zyp at 2020-04-24T14:44:33+01:00
ITS#9017 LMDB: improve Windows sync commit perf
• 147582b5
by Kris Zyp at 2020-04-24T14:47:32+01:00
ITS#9017 LMDB: allow using fixed file size on Windows
• b77c2ba7
by Howard Chu at 2020-04-24T15:03:33+01:00
ITS#9017 LMDB: fix off_t usage on Windows
Commits:
• 1f026685
by Howard Chu at 2020-04-24T15:07:33+01:00
ITS#9017 doxygen comment for MDB_FIXEDSIZE
--
You are receiving this mail because:
You are on the CC list for the bug.
https://bugs.openldap.org/show_bug.cgi?id=9017
--- Comment #16 from Howard Chu <hyc(a)openldap.org> ---
kriszyp(a)gmail.com wrote:
> Sorry, I should have added a little more detail. Specifically, I was seeing the `pos` variable in `mdb_page_flush` going over 2^31, and this being manifested in
> failed shift right here (resulting in 2^32 -1):
I've committed your first two patches as-is, and taken a different approach
for the off_t issue. (size_t is only 32bit on a 32bit Windows system, so it's
inappropriate.)
Do you still want the corresponding patch in 0.9? And can you please remind
me where the 0.9 patch is? Thanks.
And please don't send HTML emails, plaintext only.
>
> https://github.com/LMDB/lmdb/blob/mdb.master/libraries/liblmdb/mdb.c#L3711
>
>
>
> Thanks,
> Kris
>
>
>
> *From: *Howard Chu <mailto:hyc@symas.com>
> *Sent: *February 20, 2020 9:08 PM
> *To: *kriszyp(a)gmail.com <mailto:kriszyp@gmail.com>
> *Cc: *openldap-its(a)OpenLDAP.org <mailto:openldap-its@openldap.org>
> *Subject: *Re: (ITS#9017) Improving performance of commit sync in Windows
>
>
>
> Howard Chu wrote:
>
>> There should be an off64_t type instead.
>
>
>
>
>
>> But it looks to me like only the env->me_size field could ever overflow,
>
>> and using size_t for that should be fine. The other uses are for the meta page, which is always either offset 0 or
>
>> offset 4096 (with 4KB pagesize), and the lockfile, which is always smaller than 2GB.
>
>
>
> Ignore that, I was looking at the 0.9 branch when I wrote that.
--
You are receiving this mail because:
You are on the CC list for the bug.
https://bugs.openldap.org/show_bug.cgi?id=8783
Howard Chu <hyc(a)openldap.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Resolution|--- |INVALID
Status|UNCONFIRMED |RESOLVED
--
You are receiving this mail because:
You are on the CC list for the bug.
https://bugs.openldap.org/show_bug.cgi?id=8591
Ryan Tandy <ryan(a)openldap.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|UNCONFIRMED |CONFIRMED
Ever confirmed|0 |1
--
You are receiving this mail because:
You are on the CC list for the bug.