matthew sporleder wrote:
On 8/11/07, Howard Chu hyc@symas.com wrote:
Sumith Narayanan wrote:
Hi Group,
We have a coorporate openldap database in production which has more than 4 million entries. The slapd process serves three different physical dabases of sizes 4 GB , 12 GB and 24 GB respecievely. Earlier it was runnning in ldab backend and was crashing once in 3 -4 days. We upgraded to OpenLDAP 2.3.27 and BDB backend 4.4.20. Mac OS Tiger with 4GB RAM in the master and 8 GB ram in the slave. Now it crashes often may be couple of times in a day or sometimes once in 2 days. There is nothing much written in the log files but at times it shows out of memory exception. However , the search performance is still good.
- What is the maximum size of the database that OpenLDAP can support ?
- What is the maxmum number of dns it can hold ?
- Is there any solution for the above problem ?
For a machine with 4GB of RAM I think it will max out at around 1-2 million DNs. Unfortunately there is no way to control the size of the DN cache in OpenLDAP 2.3, it simply grows without bound. A config keyword for the DN cache will be in OpenLDAP 2.4. --
This is news to me. Is there a discussion somewhere of the different caches in openldap/bdb, which ones are tunable and which ones are not?
That 1-2 million estimate was wrong. This post was closer: http://www.openldap.org/lists/openldap-software/200611/msg00051.html
The limit isn't really the 4GB total RAM, but rather the available address space in a 32 bit process (usually only 2GB, sometimes as much as 3GB depending on the OS and any tweaks in the OS).
As already noted above, the DN cache is not tunable in 2.3. I frankly didn't see any wisdom in accommodating extremely large databases on 32-bit servers.
Could we get an idea with the bdb cache (DB_CONFIG) were set to the lowest values calculated from db_stat and (FAQ) "How do I determine the proper BDB/HDB database cache size?" and then working out a slapd.conf cachesize/idlcachesize?
I think Gavin Henry has written up a tuning section in the 2.4 Admin Guide. Balancing the slapd caches vs the BDB caches is already addressed in the FAQ article. Bottom of the page http://www.openldap.org/faq/index.cgi?file=1075 though of course it doesn't talk about the DN cache.