Here is a faster to do way to do what you want. Just get the last key of your range and while iterating compare the data pointer of the last key to see if you reach the end instead of the it's content. As long as you are in the same transaction the data pointer of the last key should never change. (Someone correct me if I am wrong).
Here is a quick example of what I mean.
MDB_cursor *cursor; MDB_val first_key, key, data; MDB_val last_key1, last_key2, last_data; int rc;
//Get the last key rc = mdb_cursor_get (cursor, &last_key1, &last_data, MDB_SET_RANGE); //If the current position is not equal move back one spot if (rc == MDB_SUCCESS && mdb_cmp (mdb_cursor_txn (cursor), mdb_cursor_dbi (cursor), &last_key1, &last_key2)) { rc = mdb_cursor_get (cursor, &last_key1, &last_data, MDB_PREV); } //This means that nothing is equal or greater than the last key or the last key is at the first position if (rc) { last_data.mv_data = NULL; } //Get the first key rc = mdb_cursor_get (cursor, &first_key, &data, MDB_SET_RANGE); if(rc == MDB_SUCCESS) { do { } while (mdb_cursor_get (cursor, &key, &data, MDB_NEXT) == MDB_SUCCESS && data.mv_data != last_data.mv_data); }
Hi all!
Is there a way to iterate with all key-value pairs in LMDB database,
where all keys is in specified range?
I'm trying to use mdb_cursor_get() with MDB_SET_RANGE to search first
pair.
Then I see only one way - use mdb_cursor_get() with MDB_NEXT and compare
key by memcmp().
But it seems absolutely not effective.
Is there another solution for my task?
Thanks!
James Rouzier wrote:
Here is a faster to do way to do what you want. Just get the last key of your range and while iterating compare the data pointer of the last key to see if you reach the end instead of the it's content. As long as you are in the same transaction the data pointer of the last key should never change. (Someone correct me if I am wrong).
Right, that will work in a read transaction. In a write transaction, if any writes are happening while iterating thru, it's possible for the data pointers to change.
Here is a quick example of what I mean.
MDB_cursor *cursor; MDB_val first_key, key, data; MDB_val last_key1, last_key2, last_data; int rc;
//Get the last key rc = mdb_cursor_get (cursor, &last_key1, &last_data, MDB_SET_RANGE); //If the current position is not equal move back one spot if (rc == MDB_SUCCESS && mdb_cmp (mdb_cursor_txn (cursor), mdb_cursor_dbi (cursor), &last_key1, &last_key2)) { rc = mdb_cursor_get (cursor, &last_key1, &last_data, MDB_PREV); } //This means that nothing is equal or greater than the last key or the last key is at the first position if (rc) { last_data.mv_data = NULL; } //Get the first key rc = mdb_cursor_get (cursor, &first_key, &data, MDB_SET_RANGE); if(rc == MDB_SUCCESS) { do { } while (mdb_cursor_get (cursor, &key, &data, MDB_NEXT) == MDB_SUCCESS && data.mv_data != last_data.mv_data); }
Hi all!
Is there a way to iterate with all key-value pairs in LMDB database, where
all keys is in specified range?
I'm trying to use mdb_cursor_get() with MDB_SET_RANGE to search first pair. Then I see only one way - use mdb_cursor_get() with MDB_NEXT and compare
key by memcmp().
But it seems absolutely not effective.
Is there another solution for my task?
Thanks!
Thanks for you help!
Another question: what is the best scenario for duplicates? It's normal to have 2-3 millions of duplicates for the same key, when total number of keys will be around 10000?
Thanks!
--- С уважением, Киранов Александр
2015-05-06 12:10 GMT+03:00 Howard Chu hyc@symas.com:
James Rouzier wrote:
Here is a faster to do way to do what you want. Just get the last key of your range and while iterating compare the data pointer of the last key to see if you reach the end instead of the it's content. As long as you are in the same transaction the data pointer of the last key should never change. (Someone correct me if I am wrong).
Right, that will work in a read transaction. In a write transaction, if any writes are happening while iterating thru, it's possible for the data pointers to change.
Here is a quick example of what I mean.
MDB_cursor *cursor; MDB_val first_key, key, data; MDB_val last_key1, last_key2, last_data; int rc;
//Get the last key rc = mdb_cursor_get (cursor, &last_key1, &last_data, MDB_SET_RANGE); //If the current position is not equal move back one spot if (rc == MDB_SUCCESS && mdb_cmp (mdb_cursor_txn (cursor), mdb_cursor_dbi (cursor), &last_key1, &last_key2)) { rc = mdb_cursor_get (cursor, &last_key1, &last_data, MDB_PREV); } //This means that nothing is equal or greater than the last key or the last key is at the first position if (rc) { last_data.mv_data = NULL; } //Get the first key rc = mdb_cursor_get (cursor, &first_key, &data, MDB_SET_RANGE); if(rc == MDB_SUCCESS) { do { } while (mdb_cursor_get (cursor, &key, &data, MDB_NEXT) == MDB_SUCCESS && data.mv_data != last_data.mv_data); }
Hi all!
Is there a way to iterate with all key-value pairs in LMDB database,
where all keys is in specified range?
I'm trying to use mdb_cursor_get() with MDB_SET_RANGE to search first
pair.
Then I see only one way - use mdb_cursor_get() with MDB_NEXT and
compare key by memcmp().
But it seems absolutely not effective.
Is there another solution for my task?
Thanks!
-- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Александр Киранов wrote:
Thanks for you help!
Another question: what is the best scenario for duplicates?
There are no limits there.
It's normal to have 2-3 millions of duplicates for the same key, when total number of keys will be around 10000?
Whatever you want.
openldap-technical@openldap.org