On 5/21/17 9:43 PM, Muhammed Muneer wrote:
Howard Chu wrote
"Just follow the recommendation to open all handles at the beginning of the program."
But what if I have lots of named databases like maybe 10000 or more. Wouldn't this be expensive.
I am developing a MongoDB like database (similar in query and update syntax) around LMDB. The thing is I have some enhancements on my own like the ability to generate update queries from within an ongoing update.
So in a multi threaded environment, if the name of a named dbi is generated from within a write transaction (thread1) and proceeds to mdb_dbi_open it only to find that another read transaction (thread 2) just opened the same named dbi after the write-txn of thread 1 started, the prospect of mdb_dbi_open the same named dbi for thread 1 is lost forever.
Please remember that you can have only one writing transaction at once. And first looking in a read transaction whether a database exists and then creating it in a second write transaction is definitely a bad and risky programming style, as it carries an assumption from one transaction to the next, which is typically not valid.
I have no experience with a large number of databases, but if it is a performance problem as Hallvard and the docs describe, then you still have the option to combine all your logical databases into a big single database. In this case you would maintain a database ID (e.g. four byte integer) that is prepended to the user provided key for all get and put operations. Only some care needs to be taken for range searches and cursor operations, as you might get a key/value pair that belongs to another logical database, but this is not a big deal. I use that approach for composite search keys quite a lot.
The association between database names and their IDs could be maintained in a separate database.
Regards,
Klaus