On 02. des. 2016 12:26, Tim Uckun wrote:

I am writing a wrapper for liblmdb and one thing I was able to do was to store simple integers and floats as the value into the database. This seems to work pretty well but I am wondering if this is somehow against best practice.

One bad thing about this is that when you are doing a seek or a cursor operation the code does not know what kind of data is at the record. I have some routines to cast them back from the pointers but the user has to know what to cast them to. It would be very handy if the MDB_val struct had a field for metadata, even if it just a byte.

LMDB handles binary data fine. The problem is ordering:

Use a single datatype for each DB, and mdb_set_compare() mdb_set_dupsort() to order data in that DB properly. Or MDB_INTEGERKEY/MDB_INTEGERDUP which do this for unsigned int/size_t.

If that is not feasible and you are mixing different datatypes in a single database, you need to tag each data item so the compare function can figure out how to parse and order them all. Note that a slow compare function can degrade LMDB's performance.

Note that the compare functions may receive unaligned values. Though you can hack alignment 2, 4, or sizeof(size_t) = 4 or 8 by ensuring *every* key and value in a DB has size = multiple of that alignment. Keys always have alignment of at least 2. Hmm, I should add some code so data will too except when both key and data have uneven size.

Requests to add a user-defined argument to the compare functions have been rejected. If you need that, maybe your wrapper can emulate it by setting thread-specific data before calling LMDB, and then your sort function can read that data. That gets cumbersome, though.

If this is all generic, maybe define a DB with meta-info where your wrapper can look up which datatypes (sizes, compare functions, whatever) are in which database.

FYI. I have written a Crystal language wrapper for LMDB. It's here https://github.com/timuckun/crystal-liblmdb

Hope it's of some use to somebody.

On Mon, Dec 5, 2016 at 8:24 PM, Ulrich Windl < Ulrich.Windl@rz.uni-regensburg.de> wrote:

Hi!

I think you are free to implement your data dictionary any complex data structures using LMDB ;-)

Ulrich

...

...

...

Tim Uckun timuckun@gmail.com schrieb am 02.12.2016 um 12:26 in

Nachricht CAGuHJrM2wZt1jZHCqECODvFc699VgRqSMyZr1NJR0xUuxcskjw@mail.gmail.com:

...

I am writing a wrapper for liblmdb and one thing I was able to do was to store simple integers and floats as the value into the database. This seems to work pretty well but I am wondering if this is somehow against best practice.

One bad thing about this is that when you are doing a seek or a cursor operation the code does not know what kind of data is at the record. I

have

...

some routines to cast them back from the pointers but the user has to

know

...

what to cast them to. It would be very handy if the MDB_val struct had a field for metadata, even if it just a byte.

Anyway I am wondering how much trouble I am getting myself with this practice.

Thanks.