Dear Folks,
I'm upgrading a cluster of OpenLDAP servers from 2.3.43-25.el5 to 2.4.32 with BDB 4.8.30 on CentOS 5, x86_64, on HP BL460cG6 blades with two 4-core CPUs, and 12GB RAM. These are slaves, and have eleven trees on them. I have dumped and restored six of the LDAP databases in reasonable time, but the seventh is taking a long time. Here are the sizes of the slapcatted LDIF files:
320M 252M 80M 225K 246K 833K 2.6G 1.2G 24M 947M 522K
It's the 2.6G LDIF file that's taking the time to slapadd: -############ 63.34% eta 02h09m elapsed 03h43m30s spd 2.7 k/s
As you can see, it has slowed to a crawl.
# cat DB_CONFIG set_flags DB_LOG_AUTOREMOVE set_cachesize 0 286162472 0
# egrep 'tool-threads|cachesize' /etc/openldap/slapd.conf tool-threads 8 cachesize 100000 idlcachesize 100000 cachesize 100000 idlcachesize 100000 cachesize 100000 idlcachesize 100000 cachesize 100000 idlcachesize 100000 cachesize 100000 idlcachesize 100000 cachesize 100000 idlcachesize 100000 cachesize 100000 idlcachesize 100000 cachesize 100000 idlcachesize 100000 cachesize 100000 idlcachesize 100000 cachesize 100000 idlcachesize 100000 cachesize 100000 idlcachesize 100000
On another member of the cluster, not upgraded from 2.3.43 yet, we have, for this big tree: # sudo -u ldap slapd_db_stat -d id2entry.bdb Tue Aug 21 15:15:38 2012 Local time 53162 Btree magic number 9 Btree version number Little-endian Byte order Flags 2 Minimum keys per-page 16384 Underlying database page size 3 Number of levels in the tree 4611665 Number of unique keys in the tree 4611665 Number of data items in the tree 418 Number of tree internal pages 36096 Number of bytes free in tree internal pages (99% ff) 308745 Number of tree leaf pages 871M Number of bytes free in tree leaf pages (82% ff) 0 Number of tree duplicate pages 0 Number of bytes free in tree duplicate pages (0% ff) 0 Number of tree overflow pages 0 Number of bytes free in tree overflow pages (0% ff) 0 Number of empty pages 0 Number of pages on the free list # sudo -u ldap slapd_db_stat -d dn2id.bdb Tue Aug 21 15:34:30 2012 Local time 53162 Btree magic number 9 Btree version number Little-endian Byte order duplicates, sorted duplicates Flags 2 Minimum keys per-page 4096 Underlying database page size 4 Number of levels in the tree 9227811 Number of unique keys in the tree 9263210 Number of data items in the tree 2185 Number of tree internal pages 2781648 Number of bytes free in tree internal pages (68% ff) 219185 Number of tree leaf pages 287M Number of bytes free in tree leaf pages (68% ff) 37 Number of tree duplicate pages 40214 Number of bytes free in tree duplicate pages (73% ff) 0 Number of tree overflow pages 0 Number of bytes free in tree overflow pages (0% ff) 0 Number of empty pages 0 Number of pages on the free list
QUESTIONS: ==========
Any suggestions on how to optimise this a little more towards slapadd?
Would mdb possibly be faster than bdb?