Hi guys, I've discovered a major memory issue. We have an OpenLDAP server (2.4.11) running with almost 2 million entries. When I do a ldapsearch to retrieve the entire tree the memory consumption grows and grows and will never stop before it ate the entire RAM and swap.
My slapd.conf file looks like this:
include /var/ldap/ldap-ds/etc/schema/core.schema include /var/ldap/ldap-ds/etc/schema/netmobile.schema include /var/ldap/ldap-ds/etc/schema/netmobile.acc.schema include /var/ldap/ldap-ds/etc/schema/netmobile.zMRDB.schema
# Define global ACLs to disable default read access.
# Do not enable referrals until AFTER you have a working directory # service AND an understanding of referrals. #referral ldap://root.openldap.org
pidfile /var/ldap/ldap-ds/var/run/slapd.pid argsfile /var/ldap/ldap-ds/var/run/slapd.args
access to * by dn="cn=replica,ou=replica,ou=ldapaccounts,o=netmldap" write by users read by anonymous read
access to * by self write by users read by anonymous read
#threads 2
# all logging #loglevel -1 # only syncrepl logging loglevel 16640
####################################################################### # BDB database definitions ####################################################################### database monitor
access to dn.subtree="cn=Monitor" by dn.exact="cn=admin,ou=netm,ou=people,o=netmldap" write by dn.exact="cn=admin,ou=netm,ou=people,o=netmldap" read by * none
database bdb suffix "cn=accesslog" rootdn "cn=Admin,o=netmldap" directory /var/ldap/ldap-ds/cn=accesslog
index default eq index entryCSN,objectClass,reqEnd,reqResult,reqStart
overlay syncprov syncprov-nopresent TRUE syncprov-reloadhint TRUE
limits dn.exact="cn=Admin,o=netmldap" time.soft=unlimited time.hard=unlimited size.soft=unlimited size.hard=unlimited
database config rootdn "cn=admin,cn=config" rootpw config
database bdb suffix "o=netmldap" rootdn "cn=Admin,o=netmldap" # Cleartext passwords, especially for the rootdn, should # be avoid. See slappasswd(8) and slapd.conf(5) for details. # Use of strong authentication encouraged. rootpw {SSHA}5ycg8tSV/i0Z99FKaylr0Az5x1nBA8TC directory /var/ldap/ldap-ds/o=netmldap
#dbcachesize 1000000 cachesize 1000 idlcachesize 3000 cachefree 500 searchstack 8
#readonly on
# Indices to maintain index default pres,eq index cn index netmzMRDBPhoneNumber index netmAccNumAddr index netmAccNumPort index netmLogin2 index netmPortalName index netmCarrierID # new indices - 2007-01-15 index netmLogin pres,eq index netmClientContractUniqueName index netmPrivateMail index netmContactTecEmail index netmContactBillingEmail index netmFirmEmail index netmContactCommEmail index netmzMRDBCarrier index netmzMRDBblacklistLA index netmzMRDBPortingDate index objectClass pres,eq index entryCSN,entryUUID eq
# Save the time that the entry gets modified lastmod on
overlay syncprov syncprov-checkpoint 1000 60
overlay accesslog logdb cn=accesslog logops writes logsuccess TRUE # scan the accesslog DB every day, and purge entries older than 7 days logpurge 07+00:00 01+00:00
limits dn.exact="cn=Admin,o=netmldap" time.soft=unlimited time.hard=unlimited size.soft=unlimited size.hard=unlimited
sizelimit -1
The DB_CONFIG look like this
set_cachesize 0 8435456 1 set_lg_regionmax 262144 set_lg_bsize 2097152
# will automatically remove transaction logs # this setting isn't recommended set_flags DB_LOG_AUTOREMOVE
When I start slapd is consumes that much RAM total kB 478476
After doing the search it looks like this total kB 937316
I've done a pmap -x to see where the memory goes but this process is marked as anonymous. So I don't know how to get this memory back. I have to restart the LDAP server to get this memory released.
I would appreciate if someone could help me to solve this issue. Thanks Thorsten