Hey folks,
I've been playing with pcache for about a week (in combination with back-ldap), testing various things and trying to create a configuration that helps preserve system stability if access to the LDAP server disappears. Many things seem to work well, but I noticed that quite a good number of Unix commands, like "id", request a user's full entry (i.e., no filter string). In its current implementation, this will always fail to hit the cache, because (as the man page states) you cannot specify an empty list of attributes. This means that even if you have all the information required to return what was requested in the local cache, you will never be able to leverage that information.
Given that, I wanted to ask if it could be considered to change the design of the overlay a bit, such that if an empty attribute list is specified, pcache will return every attribute *in the cache* for that entry, instead of blindly assuming it doesn't have the information to accomplish the task. I understand the shortcomings this could introduce, most notably if one did not want to cache every attribute for a particular entry and still wanted to be able to omit the filter string and get the entry's full attribute list, but I think the benefits would make it worthwhile. I say that for a few reasons: if you're using pcache and are interested in truly optimizing performance or keeping a system that is LDAP-dependent from having major issues if the network disappears, it would make sense to cache every attribute for entries used in common system tasks (e.g., using utilities like "id"). I believe this could also significantly improve the hit/miss ratios, especially if you take a look at how many of these common system utilities make queries without specifying any kind of filter.
I would think that this could be accomplished by adding a case for the empty filter string, such that it would take a unison of each set of attributes corresponding to the cached entry for that particular LDAP filter, and returning the result. If the same attribute is present in more than one cache entry matching the specified LDAP filter, the most recent would be favored and returned, thereby preventing pcache from having choose between several query-string-dependent entries for that entry, if more than one are present.
I'm interested to hear what you guys think of this. Even if my implementation ideas aren't 100% spot-on, I would think that the wiser minds could adapt the general idea in to something more artful, provided they agree with the idea in principle. Thanks!
-Ryan
Ryan Steele wrote:
Hey folks,
You have no idea what you're talking about, and pcache doesn't work as you describe. Nor does "id" or nss for that matter.
It is illegal for an LDAP search to have an empty filter. Go back and read RFC 4511 before posting nonsense like this.
Before I respond, I should preface this message by noting that this is probably no longer a development question (or at least, it is starting to become tangential), so I'm happy to repost this to openldap-software if desired. I just didn't want to do that without verifying that it would not be considered as cross-posting.
Howard Chu wrote:
Ryan Steele wrote:
Hey folks,
You have no idea what you're talking about, and pcache doesn't work as you describe. Nor does "id" or nss for that matter.
I've got no problem being be told I'm wrong, and welcome the chance to learn in the process. If you're right (and I've got no doubt you are given your knowledge of the subject is far deeper than my own), and the second half of my response clears up the confusion with respect to the semantics of attributes versus filters in this context, how can you explain the behavior in the following slapd output? When slapd receives a query in which an attribute list is specified, everything works; when it doesn't, the call to "id" hangs. The only difference I can discern from the output is that the second query is missing an attribute list. This is, in fact, what happens when I run "id ryans". When I firewall off the LDAP server, the id call hangs when it gets to the query in which there is no "SRCH attr", which causes the request to be proxied out to my remote LDAP server (an occurrence for which the evidence is documented below). I have more complete logs if you cannot make a determination from what I provide here:
## query with SRCH attr specified conn=1 op=2 SRCH base="dc=example,dc=com" scope=2 deref=0 filter="(&(objectClass=posixAccount)(uid=ryans))" conn=1 op=2 SRCH attr=uid userPassword uidNumber gidNumber cn homeDirectory loginShell gecos description objectClass query template of incoming query = (&(objectClass=)(uid=)) Entering QC, querystr = (&(objectClass=posixAccount)(uid=ryans)) Lock QC index = 0x7f6b83aa8d10 Not answerable: Unlock QC index=0x7f6b83aa8d10 QUERY NOT ANSWERABLE QUERY CACHEABLE
## same query with no SRCH attr specified conn=1 op=3 SRCH base="dc=aweber,dc=com" scope=2 deref=0 filter="(&(objectClass=posixAccount)(uid=ryans))" query template of incoming query = (&(objectClass=)(uid=)) QUERY NOT ANSWERABLE QUERY NOT CACHEABLE ldap_create ldap_url_parse_ext(ldap://remotehost.example.com) ldap_extended_operation ldap_send_initial_request ldap_new_connection 1 1 0 ldap_int_open_connection ldap_connect_to_host: TCP remotehost.example.com:389
It is illegal for an LDAP search to have an empty filter. Go back and read RFC 4511 before posting nonsense like this.
I have read the RFC. I simply used the incorrect terminology - what I meant is, it has an empty attribute list, as denoted by the post above. I'll be more careful when articulating issues in the future to avoid this sort of miscommunication and/or confusion. And again, if you would like me to change the venue to openldap-software, I'm more than happy to repost it there, I just didn't want to cross-post before confirming.
Cheers, Ryan
-
Howard Chu -
Ryan Steele