Bruno Freudensprung wrote:
Hmmm... doing it this way has several drawbacks (if I am understanding the docs correctly):
- the mutex of the write transactions prevents from writing simultaneously in my dbis (I am inserting big graphs and sometimes it takes from seconds to minutes to do so, plus those graphs are really independent and modified by independent users)
- the doc of mdb_env_set_maxdbs mentions a cost when there are more than a "moderate number" of dbis
- and, in the end, this adds another "maximum number of" to the solution
However don't get me wrong: lmdb has the best seek/get performances I have seen so far... this is why I really would like to use it (iterating over graph edges is so fast with lmdb and its zero copy API design). To me it is a the perfect fit. The last thing I really would like to do is being able to detect when I am getting close to the map size (thus my question on the mt_next_pgno field).
Even so, on a current x86-64 server you have 48 bits of virtual address space - 256 terabytes worth. Even with hundreds of DBs open you can give them a terabyte each and not have to worry about it.
Best regards,
Bruno.
-----Message d'origine----- De : Howard Chu [mailto:hyc@symas.com] Envoyé : mercredi 13 janvier 2016 23:35 À : Bruno Freudensprung bruno.freudensprung@temis.com; openldap-technical@openldap.org Objet : Re: LMBD questions (stat & copy)
Bruno Freudensprung wrote:
Hi Howard,
Thank you very much for your patient answers, and my apologies for my late reply. The "reserved space" was indeed occupied by the FREE_DBI, thanks again for pointing this out for me. Since I will have hundreds of envs (of unpredictable size) in my application,
Sounds like a misuse. For hundreds of tables you should be using individual named DBs within a single env. The point is to use a single env and not have to worry about hundreds of configuration details.
I am a bit reluctant in allocating too much space for all of them, and that's why I was trying to find a way emit a warning when room is about to lack. In this regard I think I have read something very promising in the docs: the mt_next_pgno field of the txn. It looks perfect. Do you think it would be safe to expose it in a read-only way? In any case I will follow you advice and try to find a safe compromise for the map size.