Dear Quanah,
Thank you for your generous help.
On 21/08/12 07:32 -0700, Quanah Gibson-Mount wrote:
--On Tuesday, August 21, 2012 4:03 PM +1000 Nick Urbanik nick.urbanik@optusnet.com.au wrote:
Dear Folks,
I'm upgrading a cluster of OpenLDAP servers from 2.3.43-25.el5 to 2.4.32 with BDB 4.8.30 on CentOS 5, x86_64, on HP BL460cG6 blades with two 4-core CPUs, and 12GB RAM. These are slaves, and have eleven trees on them. I have dumped and restored six of the LDAP databases in reasonable time, but the seventh is taking a long time. Here are the sizes of the slapcatted LDIF files:
It's the 2.6G LDIF file that's taking the time to slapadd: -############ 63.34% eta 02h09m elapsed 03h43m30s spd 2.7 k/s
As you can see, it has slowed to a crawl.
# cat DB_CONFIG set_flags DB_LOG_AUTOREMOVE set_cachesize 0 286162472 0
# egrep 'tool-threads|cachesize' /etc/openldap/slapd.conf tool-threads 8
cachesize and idlcachesize have no bearing on slapadd. Only toolthreads does.
Thank you.
You do not mention if you are using slapadd -q or not.
Yes, this is with the options -q -w.
Any suggestions on how to optimise this a little more towards slapadd?
If the LDIF file is 2.6GB, I fully assume the BDB database will be significantly larger than 2.6 GB. Probably closer to 6 or 8 GB, depending on your indexing. What you really need to look at from your 2.3 installation is the total size of that database (du -c -h *.bdb).
In the biggest LDAP directory on the (now finally converted) machine: 6.1G
Over all eleven LDAP directories on that same machine: 13G
Right now, you only have a 272MB BDB cache set... which is definitely going to be too small. I would change DB_CONFIG For that database to be something like:
set_cachesize 6 0 0
Okay; with 12G total RAM, and 6.9G of other directories, am I likely to overcommit memory?
I would strongly advise examining using a shared memory key for that DB as well.
Okay: I could create that with a command like ipcmk -M 16777216
But what size would be appropriate with 12G of RAM, eleven LDAP databases, and 13G of bdb files?
Finally, look at using MDB, which has zero tuning bits required, and only requires a tool-threads of 2.
So with eight cores (and 16 hyperthreaded siblings) I am better off specifying tool-threads 2?
Okay, my next one I'll convert to mdb and see how that behaves.
slapd-mdb(5) seems to indicate that mdb needs no shared memory configuration. Is that correct?