Russell Haley wrote:
Thanks so much for your response and thanks for the link. I've
pouring over that documentation off and on.
The developer currently doing the prototyping has said he ran into an
issue where the use of a very large key value caused the system to
slow down (he didn't give specifics and I didn't have time to ask). He
said it was a sparsely populated record set of about 100,00 rows but
he specifically used a large value for the key as we hope to support
UInt64 for keys (I don't know why, apparently we're indexing
sub-atomic particles?). I saw a setting for MDB_MAXKEYSIZE which was
0 for computed, but I don't know what that means on an ARM (v6 - 32
bit) system. Would you have any idea what would cause the performance
hit he was referring too?
At a guess, you're slowing down because you're using 64bit values on a 32bit
processor and once you exceed the range of a 32bit integer you need twice as
many memory accesses to fetch/compare keys. Anyway, if you're using
MDB_INTEGERKEY (which you should be, in this case) then you'll never get
anywhere near the max keysize.
Sorry if my questions are weird, I'm trying to learn C and
development through osmosis.
Anyway, on a personal note, I was thoroughly happy to get lmdb working
in Lua through
and can't wait to try his Mule Round-Robin Database tool
On Thu, Sep 22, 2016 at 4:25 PM, Howard Chu <hyc(a)symas.com> wrote:
> Russell Haley wrote:
>> I wasn't fully subscribed when I sent this so I'll send it again.
>> Any input or reference hints would be great. I love reading manuals. :)
>> ---------- Forwarded message ----------
>> From: Russell Haley <russ.haley(a)gmail.com>
>> Date: Tue, Sep 20, 2016 at 4:52 PM
>> Subject: Using lmdb as a pure in memory store
>> To: openldap-technical(a)openldap.org
>> Hi there,
>> We are currently evaluating in memory key value stores for ~100,000 -
>> 200,000 records in an embedded system. I suggested lmdb but it is
>> being discounted for some reasons I thought I'd validate:
>> 1) It is currently thought that a on disk file is REQUIRED for the
>> system. Does MDB_NOSYNC turn off the disk caching? Can it be run as a
>> pure in-memory database? Could the file not just be mapped to a
> A file is required. Whether the file is on-disk or not is irrelevant. It can
> of course be on a RAMdisk (or, in Linux, a tmpfs), which would then make it
> pure in-memory.
> MDB_NOSYNC turns off syncing, that's why it is named that.
>> 2) Because these values can come very fast, that the use of a lock
>> file would cause delay and too much wear on the nand based disk (SSD).
> No. That's not how the lock file is used.
>> I see a no locking option ( MDB_NOLOCK) that would stop a lock file
>> being written.
> This option should only be used if you're implementing your own lock
> manager. Hint - no matter what approach you use, any other lock manager you
> use will be slower than LMDB's.
>> Again, another option would be mapping the lock file to
>> a memdisk to handle that?
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/