Brett @Google wrote:
On Wed, May 6, 2009 at 5:38 PM, Pierangelo Masarati <masarati@aero.polimi.it
wrote:
I don't think slapd-meta(5) can handle cross-target paged results. It would require some non-trivial bookkeeping of the control's cookie which
<personal opinion> I wouldn't consider worth the effort for such a useless control. </personal opinion>
On the surface i would agree, but there are many situations in which it is impractical / impossible to process search results in a single gulp, without the paged results control.
One case i can think of :
A web interface where you need to present ldap search results to a user via a browser, where practicalities mean you cannot display more than say a few hundred results, either for usability, speed or browser page-size limitations (think maybe many thousands / millions of records, showing say
millions of records with a browser? are you kidding? If my browser can only inspect 500 entries at a time, I'd just put a client-side sizelimit of 500 on the search request (the server should do the same on the other side).
100 at a time).
In this situation you cannot store every complete search result for every single user, in memory or in each user's session cookie, for many practical / scalability / efficiently reasons, more particularly because the user may have what they want on the second page, so the rest of the unused search result sits in memory or session until it expires, the user having navigated away from the search results page and perhaps on to greener pastures (ui - wise).
similarly for very large databases, where the search results may exceed the physical memory of the server trying to return search results, so some part of your data set would always be inaccessible without the paged result control to load them in chunks that the ldap client can deal with.
Paged results may never be needed for small data sets, but start getting to the 100K or 1M entries and you would grind to a halt pretty fast... even 10K gets awkward 100 entries at a time :)
The lack of paged results on slapd-meta probably diminishes it's usefulness for large data sets. I would also think that users of slapd-meta would typically be at least medium sized users, who are trying to tack together several non-trivial sized data sets, so there is some probability they will stumble over this eventually.
Yes i know you added a personal opinion disclaimer, sorry about that.. but "useless" is indeed a matter of opinion, those who need it will REALLY need it. The users of large data sets must just {don't,cant,wont} be able to use back-meta :)
Users of back-meta usually need it to retrieve a single entry without the need to care about where it's located.
The only way to implement paged results in back-meta would be to intercept any request for it, chain requests to remote servers without the control, and page results locally at the proxy.
Users of paged results usually use it without requesting it, only because AD returns large data sets with that control in place, without it being requested by the client, so it wouldn't fit into this schema anyway.
In any case, adding paged results the way it's specified would not be too difficult. Probably it would be rather inefficient, though and, IMHO, not worth the effort. It could be done putting an overlay on top of back-meta that lets it run the request without the control, caches the results and deals with responding to subsequent requests (horrible, I know).
What would likely be impossible is to deal with unsolicited paged results response in any way other than bailing out.
p.