Hi, Howard.
I would like to refine my question.
Currently I develop a mdb_chk tool for deep check LMDB-files, like a fsck tool. It is needed for us, but let this stay aside for now.
On some step I plan to check the size of each key in each DBI with MDB_INTEGERKEY and/or MDB_INTEGERDUP flags.
Initially I assumed - if one of these flags is set, the the length of all keys must be sizeof(int) or sizeof(long). As will be used the corresponding comparison functions (mdb_cmp_int, mdb_cmp_long) , which can handle only keys of such sizes.
But this approach has failed. A dbi-tables created for attr by mdb_attr_dbs_open() have MDB_DUPSORT|MDB_DUPFIXED|MDB_INTEGERDUP flags and may contain records with a 5-byte keys. For instance as the 'entryCSN'.
Later you have written that MDB_INTEGERDUP is required.
Now I do not understand how it all works: - MDB_INTEGERDUP is set for attr tables. - indexer could create a 5-byte keys. - mdb-backend set custom comparison function only for MDB_DN2ID table. - under debugger I see that md_dcmp is pointed to the default. - assert for key-size in default comparators was not failed while "make test". - all test are passed without MDB_INTEGERDUP. - but you told MDB_INTEGERDUP is necessary
So, - how and when MDB_INTEGERDUP will have an impact? - in which cases keys in the table definitely should be sizeof(int) or sizeof(long)?
Thank in advance, Leonid.
Леонид Юрьев wrote:
Hi, Howard.
I would like to refine my question.
Currently I develop a mdb_chk tool for deep check LMDB-files, like a fsck tool. It is needed for us, but let this stay aside for now.
On some step I plan to check the size of each key in each DBI with MDB_INTEGERKEY and/or MDB_INTEGERDUP flags.
Initially I assumed - if one of these flags is set, the the length of all keys must be sizeof(int) or sizeof(long). As will be used the corresponding comparison functions (mdb_cmp_int, mdb_cmp_long) , which can handle only keys of such sizes.
But this approach has failed. A dbi-tables created for attr by mdb_attr_dbs_open() have MDB_DUPSORT|MDB_DUPFIXED|MDB_INTEGERDUP flags and may contain records with a 5-byte keys. For instance as the 'entryCSN'.
Later you have written that MDB_INTEGERDUP is required.
Now I do not understand how it all works:
- MDB_INTEGERDUP is set for attr tables.
- indexer could create a 5-byte keys.
- mdb-backend set custom comparison function only for MDB_DN2ID table.
- under debugger I see that md_dcmp is pointed to the default.
- assert for key-size in default comparators was not failed while "make test".
- all test are passed without MDB_INTEGERDUP.
- but you told MDB_INTEGERDUP is necessary
So,
- how and when MDB_INTEGERDUP will have an impact?
- in which cases keys in the table definitely should be sizeof(int) or
sizeof(long)?
MDB_INTGERKEY means the *keys* must be int/long. MDB_INTEGERDUP means the *values* must be int/long.
When either flag is set, all of the corresponding (keys|values) in the DB must be the same size.
The indexer only uses MDB_INTEGERDUP - it doesn't care what size the keys are, but the values must be (longs in this case).
Thanks, Howard.
It would be nice if some clarification, like these, will be in the lmdb.h.
Leonid.
2015-05-01 4:42 GMT+03:00 Howard Chu hyc@symas.com:
Леонид Юрьев wrote:
Hi, Howard.
I would like to refine my question.
Currently I develop a mdb_chk tool for deep check LMDB-files, like a fsck tool. It is needed for us, but let this stay aside for now.
On some step I plan to check the size of each key in each DBI with MDB_INTEGERKEY and/or MDB_INTEGERDUP flags.
Initially I assumed - if one of these flags is set, the the length of all keys must be sizeof(int) or sizeof(long). As will be used the corresponding comparison functions (mdb_cmp_int, mdb_cmp_long) , which can handle only keys of such sizes.
But this approach has failed. A dbi-tables created for attr by mdb_attr_dbs_open() have MDB_DUPSORT|MDB_DUPFIXED|MDB_INTEGERDUP flags and may contain records with a 5-byte keys. For instance as the 'entryCSN'.
Later you have written that MDB_INTEGERDUP is required.
Now I do not understand how it all works:
- MDB_INTEGERDUP is set for attr tables.
- indexer could create a 5-byte keys.
- mdb-backend set custom comparison function only for MDB_DN2ID table.
- under debugger I see that md_dcmp is pointed to the default.
- assert for key-size in default comparators was not failed while "make
test".
- all test are passed without MDB_INTEGERDUP.
- but you told MDB_INTEGERDUP is necessary
So,
- how and when MDB_INTEGERDUP will have an impact?
- in which cases keys in the table definitely should be sizeof(int) or
sizeof(long)?
MDB_INTGERKEY means the *keys* must be int/long. MDB_INTEGERDUP means the *values* must be int/long.
When either flag is set, all of the corresponding (keys|values) in the DB must be the same size.
The indexer only uses MDB_INTEGERDUP - it doesn't care what size the keys are, but the values must be (longs in this case).
-- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Леонид Юрьев wrote:
Thanks, Howard.
It would be nice if some clarification, like these, will be in the lmdb.h.
MDB_INTEGERDUP is already clearly documented.
Leonid.
2015-05-01 4:42 GMT+03:00 Howard Chu hyc@symas.com:
Леонид Юрьев wrote:
Hi, Howard.
I would like to refine my question.
Currently I develop a mdb_chk tool for deep check LMDB-files, like a fsck tool. It is needed for us, but let this stay aside for now.
On some step I plan to check the size of each key in each DBI with MDB_INTEGERKEY and/or MDB_INTEGERDUP flags.
Initially I assumed - if one of these flags is set, the the length of all keys must be sizeof(int) or sizeof(long). As will be used the corresponding comparison functions (mdb_cmp_int, mdb_cmp_long) , which can handle only keys of such sizes.
But this approach has failed. A dbi-tables created for attr by mdb_attr_dbs_open() have MDB_DUPSORT|MDB_DUPFIXED|MDB_INTEGERDUP flags and may contain records with a 5-byte keys. For instance as the 'entryCSN'.
Later you have written that MDB_INTEGERDUP is required.
Now I do not understand how it all works:
- MDB_INTEGERDUP is set for attr tables.
- indexer could create a 5-byte keys.
- mdb-backend set custom comparison function only for MDB_DN2ID table.
- under debugger I see that md_dcmp is pointed to the default.
- assert for key-size in default comparators was not failed while "make
test".
- all test are passed without MDB_INTEGERDUP.
- but you told MDB_INTEGERDUP is necessary
So,
- how and when MDB_INTEGERDUP will have an impact?
- in which cases keys in the table definitely should be sizeof(int) or
sizeof(long)?
MDB_INTGERKEY means the *keys* must be int/long. MDB_INTEGERDUP means the *values* must be int/long.
When either flag is set, all of the corresponding (keys|values) in the DB must be the same size.
The indexer only uses MDB_INTEGERDUP - it doesn't care what size the keys are, but the values must be (longs in this case).
-- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Oh, yes... it documented in doxygen as options for mdb_dbi_open(). Unfortunately I had looked only to comments near of MDB_INTEGERDUP definition.
Leonid.
2015-05-01 4:57 GMT+03:00 Howard Chu hyc@symas.com:
Леонид Юрьев wrote:
Thanks, Howard.
It would be nice if some clarification, like these, will be in the lmdb.h.
MDB_INTEGERDUP is already clearly documented.
Leonid.
2015-05-01 4:42 GMT+03:00 Howard Chu hyc@symas.com:
Леонид Юрьев wrote:
Hi, Howard.
I would like to refine my question.
Currently I develop a mdb_chk tool for deep check LMDB-files, like a fsck tool. It is needed for us, but let this stay aside for now.
On some step I plan to check the size of each key in each DBI with MDB_INTEGERKEY and/or MDB_INTEGERDUP flags.
Initially I assumed - if one of these flags is set, the the length of all keys must be sizeof(int) or sizeof(long). As will be used the corresponding comparison functions (mdb_cmp_int, mdb_cmp_long) , which can handle only keys of such sizes.
But this approach has failed. A dbi-tables created for attr by mdb_attr_dbs_open() have MDB_DUPSORT|MDB_DUPFIXED|MDB_INTEGERDUP flags and may contain records with a 5-byte keys. For instance as the 'entryCSN'.
Later you have written that MDB_INTEGERDUP is required.
Now I do not understand how it all works:
- MDB_INTEGERDUP is set for attr tables.
- indexer could create a 5-byte keys.
- mdb-backend set custom comparison function only for MDB_DN2ID
table.
- under debugger I see that md_dcmp is pointed to the default.
- assert for key-size in default comparators was not failed while
"make test".
- all test are passed without MDB_INTEGERDUP.
- but you told MDB_INTEGERDUP is necessary
So,
- how and when MDB_INTEGERDUP will have an impact?
- in which cases keys in the table definitely should be sizeof(int) or
sizeof(long)?
MDB_INTGERKEY means the *keys* must be int/long. MDB_INTEGERDUP means the *values* must be int/long.
When either flag is set, all of the corresponding (keys|values) in the DB must be the same size.
The indexer only uses MDB_INTEGERDUP - it doesn't care what size the keys are, but the values must be (longs in this case).
-- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
-- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
-
Howard Chu -
Леонид Юрьев