Just starting to look at implementing Time-To-Refresh, and wondering how it impacts other parameters.
proxysavequeries allows queries to be saved and reloaded across restarts. Currently it will only reload non-expired queries. The implication of setting the offline mode is that expired entries may remain in the cache. Should they in fact be reloadable too?
The proxytemplate directive has gotten a bit cumbersome, with optional negative TTL and sizelimit TTL added on. This would be the natural place to add the TTR config as well. Should there be independent TTR for negative and sizelimit too? (And it seems we should move from positional arguments to named arguments here...)
Given the existence of a TTR feature, I'm less concerned about the proxysavequeries behavior. One should simply set a long TTL and set TTR as needed to keep the cache fresh.
As a side point, I'd prefer more consistency in the config keyword names, we should have used "pcache" as the prefix for all the keywords...
pcache <database> ... pcacheattrset <index> ... pcachemaxqueries <queries> pcachevalidate <bool> pcachepersist <bool> pcacheset template=<string> attrset=<index> ttl=<secs> [negttl=<secs>] ... pcacheposition (head|tail)
Howard Chu wrote:
The proxytemplate directive has gotten a bit cumbersome, with optional negative TTL and sizelimit TTL added on. This would be the natural place to add the TTR config as well. Should there be independent TTR for negative and sizelimit too? (And it seems we should move from positional arguments to named arguments here...)
But that gets awkward fast too, just like the syncrepl config. I think in both cases, it would have been better to make them child-entries of the main config object. That would also simplify altering individual parameters of a particular config.
As a side point, I'd prefer more consistency in the config keyword names, we should have used "pcache" as the prefix for all the keywords...
pcache<database> ... pcacheattrset<index> ... pcachemaxqueries<queries> pcachevalidate<bool> pcachepersist<bool> pcacheset template=<string> attrset=<index> ttl=<secs> [negttl=<secs>] ... pcacheposition (head|tail)
I've made these changes, keeping the old keyword / attribute names as aliases for now. I didn't change pcacheTemplate to key-value format yet, but it looks like that's going to be the best way forward.
For Bind caching I want to specify some additional parameters, and they all need to be associated to a given pcacheTemplate: search base, scope, and TTR. Given that we already have 7 positional parameters for pcacheTemplate I'm not keen to make it 10 now.
Upon receiving a Bind request, pcache will first perform a search using the template's filter and attrset, with the configured base and scope. (Yes, we could just do a base-scope search on the Bind DN, but that would isolate it into its own cached query. By using the same base and scope as e.g. an NSS lookup uses, the results will go into a cached query that will also satisfy the anticipated NSS lookup.)
If the query was not already cached, or the cached query has exceeded its TTL, then this will cause a remote search to occur, and the results of that should get cached. (Unless some entry limit got exceeded, etc.) Otherwise, this should just retrieve the cached result.
If the cached result has a hashed userPassword and it is within the TTR age, then the Bind will be validated against the cached userPassword. If the hash is older than the TTR, or there is no hash, the Bind will be passed thru to the remote target and if it succeeds, it will be hashed and cached.
So, actually implementing the Bind caching is pretty straightforward. Cleaning up the config syntax, not so much...
Howard Chu wrote:
For Bind caching I want to specify some additional parameters, and they all need to be associated to a given pcacheTemplate: search base, scope, and TTR. Given that we already have 7 positional parameters for pcacheTemplate I'm not keen to make it 10 now.
I added a pcacheBind keyword. It has 5 positional parameters (sigh) with a little overlap. Because the Bind parameters must be associated to an existing pcacheTemplate, you must provide a <template_string> and <attrset_index> which correspond to the pcacheTemplate directive. However, this <template_string> is not a completely bare template like for pcacheTemplate: It is allowed to have some assertion values filled in. This is necessary to allow the Bind operation to generate a filter that matches the form that a DN-search lookup would use.
Upon receiving a Bind request, pcache will first perform a search using the template's filter and attrset, with the configured base and scope. (Yes, we could just do a base-scope search on the Bind DN, but that would isolate it into its own cached query. By using the same base and scope as e.g. an NSS lookup uses, the results will go into a cached query that will also satisfy the anticipated NSS lookup.)
For example, an NSS lookup would search for ou=people,dc=example,dc=com?<some attributes>?sub? (&(objectClass=posixAccount)(uid=joe))
On a Bind request all we have is the DN: "cn=Joe User,ou=people,dc=example,dc=com"
So the pcacheBind base, scope, and template_string are used to create a search query identical to the NSS lookup: We parse the template for a list of attributes to extract from the target entry. We then do a base scope search on the Bind DN to obtain the entry, and extract the values to substitute into the filter. We then provide this new filter to the Search response callback, before it saves the query into its cache. (So, even though we performed a base scope search on this DN, we tell the cache that it was base ou=people, scope=sub, filter=xxx)
If the query was not already cached, or the cached query has exceeded its TTL, then this will cause a remote search to occur, and the results of that should get cached. (Unless some entry limit got exceeded, etc.) Otherwise, this should just retrieve the cached result.
If the cached result has a hashed userPassword and it is within the TTR age, then the Bind will be validated against the cached userPassword. If the hash is older than the TTR, or there is no hash, the Bind will be passed thru to the remote target and if it succeeds, it will be hashed and cached.
So, actually implementing the Bind caching is pretty straightforward. Cleaning up the config syntax, not so much...
I suppose the next step is to add some tests to test020...
-
Howard Chu