Dunno if there's already anything like this, I haven't bothered to search yet.
I was considering adding the progress meter to ldapadd. Then it was suggested that it might be useful for slapcat/ldapsearch as well. For ldapsearch we would need a means to tell the client how large the result set is expected to be. Currently it's easy for back-bdb/hdb to provide this number, because they just walk through an IDL of candidates where the IDL size is already known. Of course the real result set size may be smaller due to actual filter evaluation.
This seems to call for a control that we could attach to a search request "give me an estimate of the result set size and update me every N entries". The response control would be attached to the first entry and every N entries after that, with an updated estimate of the total number of results. A control like this would be particularly useful for administrators using GUIs to browse a large directory, to give feedback about when they will receive the complete set of entries, and give an opportunity to decide to abandon the search or continue.
Comments?
its kinda kewl idea for ui progress indicators and alike. i'm not sure about "update me every N entries" part tho. say you have a candidate list you walk thru on the server side and presumably you wanna let the client know how many you walked and evaluated against the filter. now if you have result/s to return you can communicate that by attaching a control response with that data but if you got no results to return after you walked N entries/candidates how do you push updated progress data to the client? from ui progress indicator and user perspective it would become stale til the next result is sent and that can obviously happen when you have a very large candidate list and few or no actual results. did i misunderstood and you have something different in mind?
Howard Chu wrote:
Dunno if there's already anything like this, I haven't bothered to search yet.
I was considering adding the progress meter to ldapadd. Then it was suggested that it might be useful for slapcat/ldapsearch as well. For ldapsearch we would need a means to tell the client how large the result set is expected to be. Currently it's easy for back-bdb/hdb to provide this number, because they just walk through an IDL of candidates where the IDL size is already known. Of course the real result set size may be smaller due to actual filter evaluation.
This seems to call for a control that we could attach to a search request "give me an estimate of the result set size and update me every N entries". The response control would be attached to the first entry and every N entries after that, with an updated estimate of the total number of results. A control like this would be particularly useful for administrators using GUIs to browse a large directory, to give feedback about when they will receive the complete set of entries, and give an opportunity to decide to abandon the search or continue.
Comments?
Anton Bobrov wrote:
its kinda kewl idea for ui progress indicators and alike. i'm not sure about "update me every N entries" part tho. say you have a candidate list you walk thru on the server side and presumably you wanna let the client know how many you walked and evaluated against the filter. now if you have result/s to return you can communicate that by attaching a control response with that data but if you got no results to return after you walked N entries/candidates how do you push updated progress data to the client? from ui progress indicator and user perspective it would become stale til the next result is sent and that can obviously happen when you have a very large candidate list and few or no actual results. did i misunderstood and you have something different in mind?
Good point. I hadn't really considered that because I assumed filtering the candidates will take trivial time compared to transmitting the results across a wire. Instead, how about "update every N candidates" and have the response attached to an Intermediate message if the Nth candidate doesn't actually get sent. (Here I'd worry if N is too small, and sending a lot of update messages in rapid succession.)
Howard Chu wrote:
Dunno if there's already anything like this, I haven't bothered to search yet.
I was considering adding the progress meter to ldapadd. Then it was suggested that it might be useful for slapcat/ldapsearch as well. For ldapsearch we would need a means to tell the client how large the result set is expected to be. Currently it's easy for back-bdb/hdb to provide this number, because they just walk through an IDL of candidates where the IDL size is already known. Of course the real result set size may be smaller due to actual filter evaluation.
This seems to call for a control that we could attach to a search request "give me an estimate of the result set size and update me every N entries". The response control would be attached to the first entry and every N entries after that, with an updated estimate of the total number of results. A control like this would be particularly useful for administrators using GUIs to browse a large directory, to give feedback about when they will receive the complete set of entries, and give an opportunity to decide to abandon the search or continue.
Comments?
Howard Chu wrote:
results across a wire. Instead, how about "update every N candidates" and have the response attached to an Intermediate message if the Nth candidate doesn't actually get sent. (Here I'd worry if N is too small, and sending a lot of update messages in rapid succession.)
yeah that could be a problem. i guess one other option would be firing them intermediate messages on timer of sorts with time interval of the client choosing. doesnt have to a very precise or realtime like timer.
A few comments...
Using only a response control is no good here as 99.99% of the work might go by without the server producing any response to the client. Allowing, in addition to a response control, an intermediate response message, would allow for progress updates when no responses are being produced. Also, the control should be allowed with result, to indicate for instance how far the server got before it gave up.
Nothing here should be specific to search operation. Progress is generally useful.
I think it would be good for the client, in requesting the information, give some indication of the frequency it wishes to be updated in (once per second? once per minute? once per hour). This would only a suggestion. The server must be allowed to return progress as convenient (and not overly burdensome) for it.
I'd recommend progress be measured using 0 to (2**31)-1 steps. This allows the server to hide the number of candidates from the client, and provides sufficient precision for most any application.
Lastly, I think this discussion should be moved to the IETF LDAPext list for comment from the LDAP standardization community.
-- Kurt
-
Anton Bobrov -
Howard Chu -
Kurt Zeilenga