--On Tuesday, June 25, 2013 11:06:50 AM -0700 Quanah Gibson-Mount quanah@zimbra.com wrote:
--On Tuesday, June 25, 2013 10:29 AM -0700 Bill MacAllister whm@stanford.edu wrote:
With the release of Debian 7 (wheezy) I was rebuilding a couple test systems and was surprised to find that the load times I am seeing for populating the mdb database with slapd have gone up dramatically. The load for a master server that was taking about 10 minutes just took 35 minutes. The slave is worse. A normal load time is 20 minutes and it is at 31 minutes now with an eta of about 2.5 hours. These systems are using OpenLDAP 2.4.35.
Here are some relevent bits from the configuration.
dn: cn=config olcToolThreads: 2
dn: olcDatabase={2}mdb,cn=config olcDbCheckpoint: 1024 5 olcDbEnvFlags: writemap olcDbEnvFlags: nometasync olcDbNoSync: FALSE olcDbMaxSize: 85899345920
The systems are Dell r610s with 16 gbyte of memory. Our database is currently 3.2G on the master server.
I have been loading wheezy/2.4.35 databases for weeks now in preparation upgrading the OS and installing the new version of OpenLDAP on our production servers. This is the first time I have seen this.
I fiddled with the hardware enough to the point I don't think it is a hardware problem. There is not really much tuning to do with mdb and I would appreciate some suggestions for what to look at next.
If you've been doing multiple tests, you likely filled RAM.
I like to:
echo 3 > /proc/sys/vm/drop_caches
before doing a MDB load to empty RAM out.
Thanks for the suggestion.
Tried that and still got a slow load. Then I thought "I know how to clear memory cache" and rebooted the system. And the slowness persists. Current load of a replica is at 46m04s with an eta of 02h10m.
I have been setting swappiness to 0 on the ldap servers for years now. I tried setting that back to the default of 60 with no discernible change.
The load starts out at a rate of about 2 M/s. In the past I remember that dropping to something like 900 k/s and staying there. Now the load starts in the same place, but after 30 seconds it alternates between stalling out right, and a rate under 100 k/s. Dips as low as under 10 k/s and sometimes as high at 700 k/s. (My undergraduate degree was in watching water boil.)
Bill