Full_Name: Quanah Gibson-Mount Version: 2.4.23 OS: Linux 2.6 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (75.111.45.108)
When using paged result searches with large databases, it quickly becomes apparent that slapd is failing to deallocate memory allocated during those searches once they complete. The example database I am using has 6,326,513 entries, and is some 8.5GB on disk (id2entry.bdb + dn2id.bdb).
What has been found is that over time when using multiple paged results searches, slapd's memory continually grows until it eventually consumes all memory on the system. Once slapd is stopped, the allocated memory is freed. This behavior can be seen in a more pronounced fashion by running multiple paged results searches and searches of the full database simultaneously.
For example, I start slapd, and preload all the caches by doing 4 full searches of the database. At this point, top reports (V/R/S):
12.2G/10G/8.4G
I then start 10 ldap searches dumping the full database (using root dn), and 10 paged result searches that only return some 2.6 million entries. After these all complete, I run this sequence again. Instead of slapd's memory remaining stable (i.e., re-using all the memory it had previously allocated), it instead continues to grow. At the end of just 2 of these runs, we end up at:
22.5G/20G/8.4G
Or a 10.3GB growth in virtual memory and 10GB growth in resident memory.
As soon as slapd is halted, all of this memory is freed (Making it impossible to track this via valgrind).
I'm currently using the heap profiler shipped with tcmalloc to gather further information and will follow up when that is complete.
--Quanah