Hi I'm trying to build LMDB with Java/JNI bindings with Visual C++ Project Builder 9.00.30729 (vcbuild). Unfortunately, vcbuild don't ship with inttypes.h, stdint.h, or a sane ssize_t. So I searched around and found a few candidates [1] of inttypes.h and stdint.h that seems to be working for py-lmdb. However, unistd.h seems broken for windows - and the python guys use python.h instead. Any ideas where I might find a good unistd.h?

Cheers, -Kristoffer [1] https://github.com/deephacks/lmdbjni/tree/master/lmdbjni-win64/headers

That's the thing, the build doesn't complain about anything missing. But the binary seems broken because tests fail randomly. A user tried the generated binary on his machine and it worked but also said... "Hm, almost, the database file is no longer put to the requested directory, and its name is totally garbled. It finds it consistently though, and the contents are correct." https://github.com/deephacks/lmdbjni/issues/48 On Thu, Nov 19, 2015 at 9:57 PM, Howard Chu <hyc(a)symas.com&gt; wrote: > Kristoffer Sjögren wrote: >> >> Hi >> >> I'm trying to build LMDB with Java/JNI bindings with Visual C++ >> Project Builder 9.00.30729 (vcbuild). >> >> Unfortunately, vcbuild don't ship with inttypes.h, stdint.h, or a sane >> ssize_t. So I searched around and found a few candidates [1] of >> inttypes.h and stdint.h that seems to be working for py-lmdb. >> >> However, unistd.h seems broken for windows - and the python guys use >> python.h instead. >> >> Any ideas where I might find a good unistd.h? > > > I see you have one already - what is wrong with it, what definitions are > missing? > >> >> Cheers, >> -Kristoffer >> >> >> [1] https://github.com/deephacks/lmdbjni/tree/master/lmdbjni-win64/headers >> >> > > > -- > -- Howard Chu > CTO, Symas Corp. http://www.symas.com > Director, Highland Sun http://highlandsun.com/hyc/ > Chief Architect, OpenLDAP http://www.openldap.org/project/

The py-lmdb project have a comment [1] about reusing Python.h headers which seems to work for them. # Microsoft Visual Studio 9 ships with neither inttypes.h, stdint.h, or a sane # definition for ssize_t, so here we add lib/win32 to the search path, which # contains emulation header files provided by a third party. We force-include # Python.h everywhere since it has a portable definition of ssize_t, which # inttypes.h and stdint.h lack, and to avoid having to modify the LMDB source # code. Advapi32 is needed for LMDB's use of Windows security APIs.

[1] https://github.com/dw/py-lmdb/blob/master/setup.py On Thu, Nov 19, 2015 at 10:13 PM, Kristoffer Sjögren <stoffe(a)gmail.com&gt; wrote: > The actual build command: > > $ cmd.exe /X /C "vcbuild /platform:x64 vs2008.vcproj release" > > On Thu, Nov 19, 2015 at 10:11 PM, Kristoffer Sjögren <stoffe(a)gmail.com&gt; wrote: >> That's the thing, the build doesn't complain about anything missing. >> But the binary seems broken because tests fail randomly. >> >> A user tried the generated binary on his machine and it worked but also said... >> >> "Hm, almost, the database file is no longer put to the requested >> directory, and its name is totally garbled. It finds it consistently >> though, and the contents are correct." >> >> https://github.com/deephacks/lmdbjni/issues/48 >> >> On Thu, Nov 19, 2015 at 9:57 PM, Howard Chu <hyc(a)symas.com&gt; wrote: >>> Kristoffer Sjögren wrote: >>>> >>>> Hi >>>> >>>> I'm trying to build LMDB with Java/JNI bindings with Visual C++ >>>> Project Builder 9.00.30729 (vcbuild). >>>> >>>> Unfortunately, vcbuild don't ship with inttypes.h, stdint.h, or a sane >>>> ssize_t. So I searched around and found a few candidates [1] of >>>> inttypes.h and stdint.h that seems to be working for py-lmdb. >>>> >>>> However, unistd.h seems broken for windows - and the python guys use >>>> python.h instead. >>>> >>>> Any ideas where I might find a good unistd.h? >>> >>> >>> I see you have one already - what is wrong with it, what definitions are >>> missing? >>> >>>> >>>> Cheers, >>>> -Kristoffer >>>> >>>> >>>> [1] https://github.com/deephacks/lmdbjni/tree/master/lmdbjni-win64/headers >>>> >>>> >>> >>> >>> -- >>> -- Howard Chu >>> CTO, Symas Corp. http://www.symas.com >>> Director, Highland Sun http://highlandsun.com/hyc/ >>> Chief Architect, OpenLDAP http://www.openldap.org/project/

I can see the problem now from a local test. Hmm i'm a bit confused. The JNI code uses GetStringUTFChars calls [1] for all char * arguments going from Java to C, which is an array of bytes representing the string in modified UTF-8 encoding. Is this OK? The other option is to use GetStringChars which is unicode... [1] http://docs.oracle.com/javase/7/docs/technotes/guides/jni/spec/functions.... On Thu, Nov 19, 2015 at 10:32 PM, Howard Chu <hyc(a)symas.com&gt; wrote: > Kristoffer Sjögren wrote: >> >> The py-lmdb project have a comment [1] about reusing Python.h headers >> which seems to work for them. >> >> # Microsoft Visual Studio 9 ships with neither inttypes.h, stdint.h, or a >> sane >> # definition for ssize_t, so here we add lib/win32 to the search path, >> which >> # contains emulation header files provided by a third party. We >> force-include >> # Python.h everywhere since it has a portable definition of ssize_t, which >> # inttypes.h and stdint.h lack, and to avoid having to modify the LMDB >> source >> # code. Advapi32 is needed for LMDB's use of Windows security APIs. > > > Note in ITS#8067 we added an ssize_t typedef for MSVC. Also in #8069 we > changed to explicitly use the ANSI APIs, so if you're using Unicode file > paths that will certainly be garbled now. > >> >> [1] https://github.com/dw/py-lmdb/blob/master/setup.py >> >> On Thu, Nov 19, 2015 at 10:13 PM, Kristoffer Sjögren <stoffe(a)gmail.com&gt; >> wrote: >>> >>> The actual build command: >>> >>> $ cmd.exe /X /C "vcbuild /platform:x64 vs2008.vcproj release" >>> >>> On Thu, Nov 19, 2015 at 10:11 PM, Kristoffer Sjögren <stoffe(a)gmail.com&gt; >>> wrote: >>>> >>>> That's the thing, the build doesn't complain about anything missing. >>>> But the binary seems broken because tests fail randomly. >>>> >>>> A user tried the generated binary on his machine and it worked but also >>>> said... >>>> >>>> "Hm, almost, the database file is no longer put to the requested >>>> directory, and its name is totally garbled. It finds it consistently >>>> though, and the contents are correct." >>>> >>>> https://github.com/deephacks/lmdbjni/issues/48 >>>> >>>> On Thu, Nov 19, 2015 at 9:57 PM, Howard Chu <hyc(a)symas.com&gt; wrote: >>>>> >>>>> Kristoffer Sjögren wrote: >>>>>> >>>>>> >>>>>> Hi >>>>>> >>>>>> I'm trying to build LMDB with Java/JNI bindings with Visual C++ >>>>>> Project Builder 9.00.30729 (vcbuild). >>>>>> >>>>>> Unfortunately, vcbuild don't ship with inttypes.h, stdint.h, or a sane >>>>>> ssize_t. So I searched around and found a few candidates [1] of >>>>>> inttypes.h and stdint.h that seems to be working for py-lmdb. >>>>>> >>>>>> However, unistd.h seems broken for windows - and the python guys use >>>>>> python.h instead. >>>>>> >>>>>> Any ideas where I might find a good unistd.h? >>>>> >>>>> >>>>> >>>>> I see you have one already - what is wrong with it, what definitions >>>>> are >>>>> missing? >>>>> >>>>>> >>>>>> Cheers, >>>>>> -Kristoffer >>>>>> >>>>>> >>>>>> [1] >>>>>> https://github.com/deephacks/lmdbjni/tree/master/lmdbjni-win64/headers >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> -- Howard Chu >>>>> CTO, Symas Corp. http://www.symas.com >>>>> Director, Highland Sun http://highlandsun.com/hyc/ >>>>> Chief Architect, OpenLDAP http://www.openldap.org/project/ >> >> > > > -- > -- Howard Chu > CTO, Symas Corp. http://www.symas.com > Director, Highland Sun http://highlandsun.com/hyc/ > Chief Architect, OpenLDAP http://www.openldap.org/project/

Changing CharacterSet in the build file doesn't affect the garbled path. I tried to use explicit GetStringChars from the JNI code but with same result. Any pointers? I'm running short of ideas..

On Thu, Nov 19, 2015 at 11:55 PM, Kristoffer Sjögren <stoffe(a)gmail.com&gt; wrote: > Oh, but the build file [2] have a CharacterSet element set to Unicode. > Let me see if I can change this. > > [2] https://github.com/deephacks/lmdbjni/blob/master/lmdbjni/src/main/native-... > > On Thu, Nov 19, 2015 at 11:45 PM, Kristoffer Sjögren <stoffe(a)gmail.com&gt; wrote: >> I can see the problem now from a local test. >> >> Hmm i'm a bit confused. The JNI code uses GetStringUTFChars calls [1] >> for all char * arguments going from Java to C, which is an array of >> bytes representing the string in modified UTF-8 encoding. Is this OK? >> The other option is to use GetStringChars which is unicode... >> >> [1] http://docs.oracle.com/javase/7/docs/technotes/guides/jni/spec/functions.... >> >> On Thu, Nov 19, 2015 at 10:32 PM, Howard Chu <hyc(a)symas.com&gt; wrote: >>> Kristoffer Sjögren wrote: >>>> >>>> The py-lmdb project have a comment [1] about reusing Python.h headers >>>> which seems to work for them. >>>> >>>> # Microsoft Visual Studio 9 ships with neither inttypes.h, stdint.h, or a >>>> sane >>>> # definition for ssize_t, so here we add lib/win32 to the search path, >>>> which >>>> # contains emulation header files provided by a third party. We >>>> force-include >>>> # Python.h everywhere since it has a portable definition of ssize_t, which >>>> # inttypes.h and stdint.h lack, and to avoid having to modify the LMDB >>>> source >>>> # code. Advapi32 is needed for LMDB's use of Windows security APIs. >>> >>> >>> Note in ITS#8067 we added an ssize_t typedef for MSVC. Also in #8069 we >>> changed to explicitly use the ANSI APIs, so if you're using Unicode file >>> paths that will certainly be garbled now. >>> >>>> >>>> [1] https://github.com/dw/py-lmdb/blob/master/setup.py >>>> >>>> On Thu, Nov 19, 2015 at 10:13 PM, Kristoffer Sjögren <stoffe(a)gmail.com&gt; >>>> wrote: >>>>> >>>>> The actual build command: >>>>> >>>>> $ cmd.exe /X /C "vcbuild /platform:x64 vs2008.vcproj release" >>>>> >>>>> On Thu, Nov 19, 2015 at 10:11 PM, Kristoffer Sjögren <stoffe(a)gmail.com&gt; >>>>> wrote: >>>>>> >>>>>> That's the thing, the build doesn't complain about anything missing. >>>>>> But the binary seems broken because tests fail randomly. >>>>>> >>>>>> A user tried the generated binary on his machine and it worked but also >>>>>> said... >>>>>> >>>>>> "Hm, almost, the database file is no longer put to the requested >>>>>> directory, and its name is totally garbled. It finds it consistently >>>>>> though, and the contents are correct." >>>>>> >>>>>> https://github.com/deephacks/lmdbjni/issues/48 >>>>>> >>>>>> On Thu, Nov 19, 2015 at 9:57 PM, Howard Chu <hyc(a)symas.com&gt; wrote: >>>>>>> >>>>>>> Kristoffer Sjögren wrote: >>>>>>>> >>>>>>>> >>>>>>>> Hi >>>>>>>> >>>>>>>> I'm trying to build LMDB with Java/JNI bindings with Visual C++ >>>>>>>> Project Builder 9.00.30729 (vcbuild). >>>>>>>> >>>>>>>> Unfortunately, vcbuild don't ship with inttypes.h, stdint.h, or a sane >>>>>>>> ssize_t. So I searched around and found a few candidates [1] of >>>>>>>> inttypes.h and stdint.h that seems to be working for py-lmdb. >>>>>>>> >>>>>>>> However, unistd.h seems broken for windows - and the python guys use >>>>>>>> python.h instead. >>>>>>>> >>>>>>>> Any ideas where I might find a good unistd.h? >>>>>>> >>>>>>> >>>>>>> >>>>>>> I see you have one already - what is wrong with it, what definitions >>>>>>> are >>>>>>> missing? >>>>>>> >>>>>>>> >>>>>>>> Cheers, >>>>>>>> -Kristoffer >>>>>>>> >>>>>>>> >>>>>>>> [1] >>>>>>>> https://github.com/deephacks/lmdbjni/tree/master/lmdbjni-win64/headers >>>>>>>>

Yep, the commit fixes the problem. Thanks Howard!

On Fri, Nov 20, 2015 at 2:14 AM, Howard Chu <hyc(a)symas.com&gt; wrote: > Kristoffer Sjögren wrote: >> >> Changing CharacterSet in the build file doesn't affect the garbled path. >> >> I tried to use explicit GetStringChars from the JNI code but with same >> result. >> >> Any pointers? I'm running short of ideas.. > > > Looks like we need to use UTF-8, as in ITS#7992. I've just merged that patch > into mdb.master, see if that works better. You should use GetStringUTFChars. > >> >> On Thu, Nov 19, 2015 at 11:55 PM, Kristoffer Sjögren <stoffe(a)gmail.com&gt; >> wrote: >>> >>> Oh, but the build file [2] have a CharacterSet element set to Unicode. >>> Let me see if I can change this. >>> >>> [2] >>> https://github.com/deephacks/lmdbjni/blob/master/lmdbjni/src/main/native-... >>> >>> On Thu, Nov 19, 2015 at 11:45 PM, Kristoffer Sjögren <stoffe(a)gmail.com&gt; >>> wrote: >>>> >>>> I can see the problem now from a local test. >>>> >>>> Hmm i'm a bit confused. The JNI code uses GetStringUTFChars calls [1] >>>> for all char * arguments going from Java to C, which is an array of >>>> bytes representing the string in modified UTF-8 encoding. Is this OK? >>>> The other option is to use GetStringChars which is unicode... >>>> >>>> [1] >>>> http://docs.oracle.com/javase/7/docs/technotes/guides/jni/spec/functions.... >>>> >>>> On Thu, Nov 19, 2015 at 10:32 PM, Howard Chu <hyc(a)symas.com&gt; wrote: >>>>> >>>>> Kristoffer Sjögren wrote: >>>>>> >>>>>> >>>>>> The py-lmdb project have a comment [1] about reusing Python.h headers >>>>>> which seems to work for them. >>>>>> >>>>>> # Microsoft Visual Studio 9 ships with neither inttypes.h, stdint.h, >>>>>> or a >>>>>> sane >>>>>> # definition for ssize_t, so here we add lib/win32 to the search path, >>>>>> which >>>>>> # contains emulation header files provided by a third party. We >>>>>> force-include >>>>>> # Python.h everywhere since it has a portable definition of ssize_t, >>>>>> which >>>>>> # inttypes.h and stdint.h lack, and to avoid having to modify the LMDB >>>>>> source >>>>>> # code. Advapi32 is needed for LMDB's use of Windows security APIs. >>>>> >>>>> >>>>> >>>>> Note in ITS#8067 we added an ssize_t typedef for MSVC. Also in #8069 we >>>>> changed to explicitly use the ANSI APIs, so if you're using Unicode >>>>> file >>>>> paths that will certainly be garbled now. >>>>> >>>>>> >>>>>> [1] https://github.com/dw/py-lmdb/blob/master/setup.py >>>>>> >>>>>> On Thu, Nov 19, 2015 at 10:13 PM, Kristoffer Sjögren >>>>>> <stoffe(a)gmail.com&gt; >>>>>> wrote: >>>>>>> >>>>>>> >>>>>>> The actual build command: >>>>>>> >>>>>>> $ cmd.exe /X /C "vcbuild /platform:x64 vs2008.vcproj release" >>>>>>> >>>>>>> On Thu, Nov 19, 2015 at 10:11 PM, Kristoffer Sjögren >>>>>>> <stoffe(a)gmail.com&gt; >>>>>>> wrote: >>>>>>>> >>>>>>>> >>>>>>>> That's the thing, the build doesn't complain about anything missing. >>>>>>>> But the binary seems broken because tests fail randomly. >>>>>>>> >>>>>>>> A user tried the generated binary on his machine and it worked but >>>>>>>> also >>>>>>>> said... >>>>>>>> >>>>>>>> "Hm, almost, the database file is no longer put to the requested >>>>>>>> directory, and its name is totally garbled. It finds it consistently >>>>>>>> though, and the contents are correct." >>>>>>>> >>>>>>>> https://github.com/deephacks/lmdbjni/issues/48 >>>>>>>> >>>>>>>> On Thu, Nov 19, 2015 at 9:57 PM, Howard Chu <hyc(a)symas.com&gt; wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> Kristoffer Sjögren wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Hi >>>>>>>>>> >>>>>>>>>> I'm trying to build LMDB with Java/JNI bindings with Visual C++ >>>>>>>>>> Project Builder 9.00.30729 (vcbuild). >>>>>>>>>> >>>>>>>>>> Unfortunately, vcbuild don't ship with inttypes.h, stdint.h, or a >>>>>>>>>> sane >>>>>>>>>> ssize_t. So I searched around and found a few candidates [1] of >>>>>>>>>> inttypes.h and stdint.h that seems to be working for py-lmdb. >>>>>>>>>> >>>>>>>>>> However, unistd.h seems broken for windows - and the python guys >>>>>>>>>> use >>>>>>>>>> python.h instead. >>>>>>>>>> >>>>>>>>>> Any ideas where I might find a good unistd.h? >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> I see you have one already - what is wrong with it, what >>>>>>>>> definitions >>>>>>>>> are >>>>>>>>> missing? >>>>>>>>> >>>>>>>>>> >>>>>>>>>> Cheers, >>>>>>>>>> -Kristoffer >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> [1] >>>>>>>>>> >>>>>>>>>> https://github.com/deephacks/lmdbjni/tree/master/lmdbjni-win64/headers >>>>>>>>>>

The last build I know worked for lmdbjni on windows used LMDB_0.9.14. I was going to try a patch release on LMDB_0.9.16 but the fix doesn't merge cleanly. On Fri, Nov 20, 2015 at 10:26 AM, Howard Chu <hyc(a)symas.com&gt; wrote: > Kristoffer Sjögren wrote: >> >> Yep, the commit fixes the problem. Thanks Howard! > > > The only question now - what is "modified" in the Java UTF-8 encoding? > >> >> On Fri, Nov 20, 2015 at 2:14 AM, Howard Chu <hyc(a)symas.com&gt; wrote: >>> >>> Kristoffer Sjögren wrote: >>>> >>>> >>>> Changing CharacterSet in the build file doesn't affect the garbled path. >>>> >>>> I tried to use explicit GetStringChars from the JNI code but with same >>>> result. >>>> >>>> Any pointers? I'm running short of ideas.. >>> >>> >>> >>> Looks like we need to use UTF-8, as in ITS#7992. I've just merged that >>> patch >>> into mdb.master, see if that works better. You should use >>> GetStringUTFChars. >>> >>>> >>>> On Thu, Nov 19, 2015 at 11:55 PM, Kristoffer Sjögren <stoffe(a)gmail.com&gt; >>>> wrote: >>>>> >>>>> >>>>> Oh, but the build file [2] have a CharacterSet element set to Unicode. >>>>> Let me see if I can change this. >>>>> >>>>> [2] >>>>> >>>>> https://github.com/deephacks/lmdbjni/blob/master/lmdbjni/src/main/native-... >>>>> >>>>> On Thu, Nov 19, 2015 at 11:45 PM, Kristoffer Sjögren <stoffe(a)gmail.com&gt; >>>>> wrote: >>>>>> >>>>>> >>>>>> I can see the problem now from a local test. >>>>>> >>>>>> Hmm i'm a bit confused. The JNI code uses GetStringUTFChars calls [1] >>>>>> for all char * arguments going from Java to C, which is an array of >>>>>> bytes representing the string in modified UTF-8 encoding. Is this OK? >>>>>> The other option is to use GetStringChars which is unicode... >>>>>> >>>>>> [1] >>>>>> >>>>>> http://docs.oracle.com/javase/7/docs/technotes/guides/jni/spec/functions.... >>>>>> >>>>>> On Thu, Nov 19, 2015 at 10:32 PM, Howard Chu <hyc(a)symas.com&gt; wrote: >>>>>>> >>>>>>> >>>>>>> Kristoffer Sjögren wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> The py-lmdb project have a comment [1] about reusing Python.h >>>>>>>> headers >>>>>>>> which seems to work for them. >>>>>>>> >>>>>>>> # Microsoft Visual Studio 9 ships with neither inttypes.h, stdint.h, >>>>>>>> or a >>>>>>>> sane >>>>>>>> # definition for ssize_t, so here we add lib/win32 to the search >>>>>>>> path, >>>>>>>> which >>>>>>>> # contains emulation header files provided by a third party. We >>>>>>>> force-include >>>>>>>> # Python.h everywhere since it has a portable definition of ssize_t, >>>>>>>> which >>>>>>>> # inttypes.h and stdint.h lack, and to avoid having to modify the >>>>>>>> LMDB >>>>>>>> source >>>>>>>> # code. Advapi32 is needed for LMDB's use of Windows security APIs. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> Note in ITS#8067 we added an ssize_t typedef for MSVC. Also in #8069 >>>>>>> we >>>>>>> changed to explicitly use the ANSI APIs, so if you're using Unicode >>>>>>> file >>>>>>> paths that will certainly be garbled now. >>>>>>> >>>>>>>> >>>>>>>> [1] https://github.com/dw/py-lmdb/blob/master/setup.py >>>>>>>> >>>>>>>> On Thu, Nov 19, 2015 at 10:13 PM, Kristoffer Sjögren >>>>>>>> <stoffe(a)gmail.com&gt; >>>>>>>> wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> The actual build command: >>>>>>>>> >>>>>>>>> $ cmd.exe /X /C "vcbuild /platform:x64 vs2008.vcproj release" >>>>>>>>> >>>>>>>>> On Thu, Nov 19, 2015 at 10:11 PM, Kristoffer Sjögren >>>>>>>>> <stoffe(a)gmail.com&gt; >>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> That's the thing, the build doesn't complain about anything >>>>>>>>>> missing. >>>>>>>>>> But the binary seems broken because tests fail randomly. >>>>>>>>>> >>>>>>>>>> A user tried the generated binary on his machine and it worked but >>>>>>>>>> also >>>>>>>>>> said... >>>>>>>>>> >>>>>>>>>> "Hm, almost, the database file is no longer put to the requested >>>>>>>>>> directory, and its name is totally garbled. It finds it >>>>>>>>>> consistently >>>>>>>>>> though, and the contents are correct." >>>>>>>>>> >>>>>>>>>> https://github.com/deephacks/lmdbjni/issues/48 >>>>>>>>>> >>>>>>>>>> On Thu, Nov 19, 2015 at 9:57 PM, Howard Chu <hyc(a)symas.com&gt; wrote: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Kristoffer Sjögren wrote: >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Hi >>>>>>>>>>>> >>>>>>>>>>>> I'm trying to build LMDB with Java/JNI bindings with Visual C++ >>>>>>>>>>>> Project Builder 9.00.30729 (vcbuild). >>>>>>>>>>>> >>>>>>>>>>>> Unfortunately, vcbuild don't ship with inttypes.h, stdint.h, or >>>>>>>>>>>> a >>>>>>>>>>>> sane >>>>>>>>>>>> ssize_t. So I searched around and found a few candidates [1] of >>>>>>>>>>>> inttypes.h and stdint.h that seems to be working for py-lmdb. >>>>>>>>>>>> >>>>>>>>>>>> However, unistd.h seems broken for windows - and the python guys >>>>>>>>>>>> use >>>>>>>>>>>> python.h instead. >>>>>>>>>>>> >>>>>>>>>>>> Any ideas where I might find a good unistd.h? >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> I see you have one already - what is wrong with it, what >>>>>>>>>>> definitions >>>>>>>>>>> are >>>>>>>>>>> missing? >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Cheers, >>>>>>>>>>>> -Kristoffer >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> [1] >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> https://github.com/deephacks/lmdbjni/tree/master/lmdbjni-win64/headers >>>>>>>>>>>> > > > > -- > -- Howard Chu > CTO, Symas Corp. http://www.symas.com > Director, Highland Sun http://highlandsun.com/hyc/ > Chief Architect, OpenLDAP http://www.openldap.org/project/

Kristoffer Sjögren wrote: > > Just to let you know. I merged the fix manually onto LMDB_0.9.16 and it > works. Great. But as we announced just last week http://www.openldap.org/lists/openldap-technical/201511/msg00092.html LMDB 0.9.17 is in preparation for release now. Probably will be out in a few days. > > > https://github.com/deephacks/lmdbjni/commit/54a6ed3f1e0896a29e4bd10594bf9... > > > On Fri, Nov 20, 2015 at 10:34 AM, Kristoffer Sjögren <stoffe(a)gmail.com&gt; > wrote: >> >> The last build I know worked for lmdbjni on windows used LMDB_0.9.14. >> >> I was going to try a patch release on LMDB_0.9.16 but the fix doesn't >> merge cleanly. >> >> On Fri, Nov 20, 2015 at 10:26 AM, Howard Chu <hyc(a)symas.com&gt; wrote: >>> >>> Kristoffer Sjögren wrote: >>>> >>>> >>>> Yep, the commit fixes the problem. Thanks Howard! >>> >>> >>> >>> The only question now - what is "modified" in the Java UTF-8 encoding? >>> >>>> >>>> On Fri, Nov 20, 2015 at 2:14 AM, Howard Chu <hyc(a)symas.com&gt; wrote: >>>>> >>>>> >>>>> Kristoffer Sjögren wrote: >>>>>> >>>>>> >>>>>> >>>>>> Changing CharacterSet in the build file doesn't affect the garbled >>>>>> path. >>>>>> >>>>>> I tried to use explicit GetStringChars from the JNI code but with >>>>>> same >>>>>> result. >>>>>> >>>>>> Any pointers? I'm running short of ideas.. >>>>> >>>>> >>>>> >>>>> >>>>> Looks like we need to use UTF-8, as in ITS#7992. I've just merged that >>>>> patch >>>>> into mdb.master, see if that works better. You should use >>>>> GetStringUTFChars. >>>>> >>>>>> >>>>>> On Thu, Nov 19, 2015 at 11:55 PM, Kristoffer Sjögren >>>>>> <stoffe(a)gmail.com&gt; >>>>>> wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> Oh, but the build file [2] have a CharacterSet element set to >>>>>>> Unicode. >>>>>>> Let me see if I can change this. >>>>>>> >>>>>>> [2] >>>>>>> >>>>>>> >>>>>>> https://github.com/deephacks/lmdbjni/blob/master/lmdbjni/src/main/native-... >>>>>>> >>>>>>> On Thu, Nov 19, 2015 at 11:45 PM, Kristoffer Sjögren >>>>>>> <stoffe(a)gmail.com&gt; >>>>>>> wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> I can see the problem now from a local test. >>>>>>>> >>>>>>>> Hmm i'm a bit confused. The JNI code uses GetStringUTFChars calls >>>>>>>> [1] >>>>>>>> for all char * arguments going from Java to C, which is an array of >>>>>>>> bytes representing the string in modified UTF-8 encoding. Is this >>>>>>>> OK? >>>>>>>> The other option is to use GetStringChars which is unicode... >>>>>>>> >>>>>>>> [1] >>>>>>>> >>>>>>>> >>>>>>>> http://docs.oracle.com/javase/7/docs/technotes/guides/jni/spec/functions.... >>>>>>>> >>>>>>>> On Thu, Nov 19, 2015 at 10:32 PM, Howard Chu <hyc(a)symas.com&gt; wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Kristoffer Sjögren wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> The py-lmdb project have a comment [1] about reusing Python.h >>>>>>>>>> headers >>>>>>>>>> which seems to work for them. >>>>>>>>>> >>>>>>>>>> # Microsoft Visual Studio 9 ships with neither inttypes.h, >>>>>>>>>> stdint.h, >>>>>>>>>> or a >>>>>>>>>> sane >>>>>>>>>> # definition for ssize_t, so here we add lib/win32 to the search >>>>>>>>>> path, >>>>>>>>>> which >>>>>>>>>> # contains emulation header files provided by a third party. We >>>>>>>>>> force-include >>>>>>>>>> # Python.h everywhere since it has a portable definition of >>>>>>>>>> ssize_t, >>>>>>>>>> which >>>>>>>>>> # inttypes.h and stdint.h lack, and to avoid having to modify the >>>>>>>>>> LMDB >>>>>>>>>> source >>>>>>>>>> # code. Advapi32 is needed for LMDB's use of Windows security >>>>>>>>>> APIs. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Note in ITS#8067 we added an ssize_t typedef for MSVC. Also in >>>>>>>>> #8069 >>>>>>>>> we >>>>>>>>> changed to explicitly use the ANSI APIs, so if you're using >>>>>>>>> Unicode >>>>>>>>> file >>>>>>>>> paths that will certainly be garbled now. >>>>>>>>> >>>>>>>>>> >>>>>>>>>> [1] https://github.com/dw/py-lmdb/blob/master/setup.py >>>>>>>>>> >>>>>>>>>> On Thu, Nov 19, 2015 at 10:13 PM, Kristoffer Sjögren >>>>>>>>>> <stoffe(a)gmail.com&gt; >>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> The actual build command: >>>>>>>>>>> >>>>>>>>>>> $ cmd.exe /X /C "vcbuild /platform:x64 vs2008.vcproj release" >>>>>>>>>>> >>>>>>>>>>> On Thu, Nov 19, 2015 at 10:11 PM, Kristoffer Sjögren >>>>>>>>>>> <stoffe(a)gmail.com&gt; >>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> That's the thing, the build doesn't complain about anything >>>>>>>>>>>> missing. >>>>>>>>>>>> But the binary seems broken because tests fail randomly. >>>>>>>>>>>> >>>>>>>>>>>> A user tried the generated binary on his machine and it worked >>>>>>>>>>>> but >>>>>>>>>>>> also >>>>>>>>>>>> said... >>>>>>>>>>>> >>>>>>>>>>>> "Hm, almost, the database file is no longer put to the >>>>>>>>>>>> requested >>>>>>>>>>>> directory, and its name is totally garbled. It finds it >>>>>>>>>>>> consistently >>>>>>>>>>>> though, and the contents are correct." >>>>>>>>>>>> >>>>>>>>>>>> https://github.com/deephacks/lmdbjni/issues/48 >>>>>>>>>>>> >>>>>>>>>>>> On Thu, Nov 19, 2015 at 9:57 PM, Howard Chu <hyc(a)symas.com&gt; >>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Kristoffer Sjögren wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Hi >>>>>>>>>>>>>> >>>>>>>>>>>>>> I'm trying to build LMDB with Java/JNI bindings with Visual >>>>>>>>>>>>>> C++ >>>>>>>>>>>>>> Project Builder 9.00.30729 (vcbuild). >>>>>>>>>>>>>> >>>>>>>>>>>>>> Unfortunately, vcbuild don't ship with inttypes.h, stdint.h, >>>>>>>>>>>>>> or >>>>>>>>>>>>>> a >>>>>>>>>>>>>> sane >>>>>>>>>>>>>> ssize_t. So I searched around and found a few candidates [1] >>>>>>>>>>>>>> of >>>>>>>>>>>>>> inttypes.h and stdint.h that seems to be working for py-lmdb. >>>>>>>>>>>>>> >>>>>>>>>>>>>> However, unistd.h seems broken for windows - and the python >>>>>>>>>>>>>> guys >>>>>>>>>>>>>> use >>>>>>>>>>>>>> python.h instead. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Any ideas where I might find a good unistd.h? >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> I see you have one already - what is wrong with it, what >>>>>>>>>>>>> definitions >>>>>>>>>>>>> are >>>>>>>>>>>>> missing? >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Cheers, >>>>>>>>>>>>>> -Kristoffer >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> [1] >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> https://github.com/deephacks/lmdbjni/tree/master/lmdbjni-win64/headers >>>>>>>>>>>>>> >>> -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/