I have a big problem whenever I have to restart a server. Obviously, the first search for objectclass=* is going to enumerate the whole directory. The problem is that even an anonymous user can cause the server to execute this search on the backend even though the ACLs and limits will keep them from getting any results. All it takes is a few poorly configured client applications to do some sort of poll and I have connections hanging for half an hour until the first objectclass=* search finishes. I run out of threads and every one of them is constantly trying to get CPU time.
What I currently do is to keep a machine from being accessible by taking it out of the load-balancer's rotation for the half hour or so that it takes for me to do a search for objectclass=*, but I figure there has got to be another way. I have both eq and pres indexes on objectclass. It's just that I have a very big directory.
I'm not trying to speed up the objectclass=* search. I'm trying to figure out how to keep it from impacting the server's responsiveness when it is being performed under circumstances where no entries will be returned, such as when sizelimits or ACLs (which are evaluated at the frontend after the backend has performed the operation, right) will block things. Any suggestions?
One thing I just thought of would be to have a single entry that would always be accessible to any searcher and then set 'limits anonymous size=1'. Would that cause the backend operation to be canceled once the first entry were returned? That might save me something.
On Fri, 2 Mar 2007, Howard Chu wrote:
(objectclass=*) is already a no-op in back-bdb/back-hdb. I made that optimization back in November 2001.
So yes, it's a nice idea, been there, done that.
I have a problem in that the first time someone performs a search for 'objectclass=*' after slapd is restarted, the server is really bogged down for a while. Once the search has completed once, this is not a problem. I assume that's due to the IDL cache. However, I currently have to keep the server unavailable after restarting slapd for upwards of half an hour while I do an 'objectclass=*' search the first time.
Changing the behavior of (objectclass=*) isn't going to make any difference. If you specify a filter that is too broad, then slapd has to trawl through every entry in the DB because every entry satisfies the condition.
Eric Irrgang wrote:
I have a big problem whenever I have to restart a server. Obviously, the first search for objectclass=* is going to enumerate the whole directory. The problem is that even an anonymous user can cause the server to execute this search on the backend even though the ACLs and limits will keep them from getting any results. All it takes is a few poorly configured client applications to do some sort of poll and I have connections hanging for half an hour until the first objectclass=* search finishes. I run out of threads and every one of them is constantly trying to get CPU time.
What I currently do is to keep a machine from being accessible by taking it out of the load-balancer's rotation for the half hour or so that it takes for me to do a search for objectclass=*, but I figure there has got to be another way. I have both eq and pres indexes on objectclass. It's just that I have a very big directory.
Get rid of the presence index. Never use a presence index for attributes that are present on the majority of your entries.
You should also probably look into sizelimit.unchecked in slapd.conf(5).
I'm not trying to speed up the objectclass=* search. I'm trying to figure out how to keep it from impacting the server's responsiveness when it is being performed under circumstances where no entries will be returned, such as when sizelimits or ACLs (which are evaluated at the frontend after the backend has performed the operation, right) will block things. Any suggestions?
One thing I just thought of would be to have a single entry that would always be accessible to any searcher and then set 'limits anonymous size=1'. Would that cause the backend operation to be canceled once the first entry were returned? That might save me something.
openldap-software@openldap.org