Michael Ströder wrote:
Howard Chu wrote:
Alex V. wrote:
I am new to key-value stores. I would like to use it like big persistent associative array and I want to be able to use long length keys. For example file path or URL. What technique should I use to implement such dictionary on top of LMDB? Probably some kind of hashing and collision resolution? Or just recompile with MDB_MAXKEYSIZE=2047 or something like this? Or LMDB is a wrong tool for this job?
Probably some combination of the above. Recompile with larger MAXKEYSIZE will make everything else easier.
Mozilla/Firefox/Seamonkey/whatever stores complete URLs in a history database, and uses the entire URL (including any URL parameters) as the key. This is an extremely wasteful design;
But for browsers that's the only possible way because they have to deal with everything which is out there.
first of all the URL params aren't meaningful when you just want to look back and see what sites you visited.
It very much depends.
Also, when storing keys that are inherently a hierarchical structure, you should store them hierarchically.
You can only do that if you can make strict assumptions about the URLs' structure. (Of course file pathnames are very simple subset of the problem.) It's not that simple with HTTP URLs in general given the many ways web apps, CMS whatever address data with URLs.
Nonsense. The domain name component of an HTTP URL is purely hierarchical, and the pathname component of an HTTP URL is also. Anything after the pathname may be considered opaque, for simplicity, but the fact is URL syntax is already well defined and the components are clear-cut.