Howard Chu wrote:
Alex V. wrote:
I am new to key-value stores. I would like to use it like big persistent associative array and I want to be able to use long length keys. For example file path or URL. What technique should I use to implement such dictionary on top of LMDB? Probably some kind of hashing and collision resolution? Or just recompile with MDB_MAXKEYSIZE=2047 or something like this? Or LMDB is a wrong tool for this job?
Probably some combination of the above. Recompile with larger MAXKEYSIZE will make everything else easier.
Mozilla/Firefox/Seamonkey/whatever stores complete URLs in a history database, and uses the entire URL (including any URL parameters) as the key. This is an extremely wasteful design;
But for browsers that's the only possible way because they have to deal with everything which is out there.
first of all the URL params aren't meaningful when you just want to look back and see what sites you visited.
It very much depends.
Also, when storing keys that are inherently a hierarchical structure, you should store them hierarchically.
You can only do that if you can make strict assumptions about the URLs' structure. (Of course file pathnames are very simple subset of the problem.) It's not that simple with HTTP URLs in general given the many ways web apps, CMS whatever address data with URLs.
E.g. I saw a IAM tool fail to deal with the random looking URLs used to access Lotus Notes databases through Domino web server which actually can be somewhat hierarchical, but not necessarily the URL.
And one would also have to consider URL rewriting/redirecting in some cases.
So I hope for Alex that his data is much simpler. ;-)
Ciao, Michael.