Laurence Field wrote:
masarati@aero.polimi.it wrote:
We are experiencing a decay in query performance over time using OpenLDAP 2.3.43 on CentOS 5.
After populating the database with about 70 000 entries, the query response time for one entry is about 0.4s. After about two hours the same query takes over 1s. The slapd.conf file and DB_CONFIG files that we use can be found at the following URLs.
http://svnweb.cern.ch/trac/gridinfo/browser/bdii/trunk/etc/bdii-slapd.conf http://svnweb.cern.ch/trac/gridinfo/browser/bdii/trunk/etc/DB_CONFIG
Does anyone have an idea about what is happening.
Not sure whether it matters or not, but your DB_CONFIG's set_cachesize looks very small, about 50MB. Are the same DB_CONFIG settings used for all databases? What database, of the 3 defined in your slapd.conf, is suffering from this problem? Are critical searches related to indexed attrs? Can you reproduce the problem with recent releases (e.g. 2.4.19)?
So here are the results of the performance test. The the machines used were Xeon 2.33GHz 2 quad core CPUs with 16 GB RAM. The database was populated with 80K entries, approximately 70MB. A query for all entries was run using a single loop with a 5s sleep. The resulting graphs of the network utilization for the different LDAP versions can be found at the URLs below.
http://lfield.web.cern.ch/lfield/2.3.43.png http://lfield.web.cern.ch/lfield/2.4.18.png
The decrease in throughput corresponds to an increase in the response time. Why is the performance of version 2.3.43 decreasing over time?
The answer to this question can probably be gleaned from reading the openldap-devel archives from the time 2.4 was being first prepared for release. A lot of profiling and refactoring went into the 2.4 code to improve performance relative to 2.3. At this point 2.3 is ancient and I've forgotten everything that we've changed internally, and it's not worth my while to dig back to remember what we fixed.