wow, Quanah,
thats the best answer i ever got...
I'm gong to move to 2.4.18 and try to tune up the machine... regarding more RAM ... i can get it but not tha fast.
what can i expect with a total of 10 GB ?...
I plan to set set_cachesize 7 0 1
what is the relation between cachesize (in slapd.conf) and set_cachesize (in DB_CONFIG)?
set_cachesize =>how much memory i wiill cache cachesize => number of entries i will keep within the cache
how much one entry take ?
Lets say if i use 7 GB of cache, what cachesize should i put?
T.
On Thu, Sep 10, 2009 at 9:37 PM, Quanah Gibson-Mount quanah@zimbra.comwrote:
--On Thursday, September 10, 2009 8:56 PM +0200 Tihomir Culjaga < tculjaga@gmail.com> wrote:
So, the situation is that i have 2 ldif files i'm recreating the database
from.
/usr/local/libexec/slapadd -l /home/tculjaga/file2.ldif -f /usr/local/etc/openldap/slapd.conf /usr/local/libexec/slapadd -l /home/tculjaga/file2.ldif -f /usr/local/etc/openldap/slapd.conf
I would suggest you just make these a single file, so all the work can be done at one time.
I tried to re-index with /usr/local/libexec/slapindex -f
/usr/local/etc/openldap/slapd.conf -v restart slapd process, restart the machine ... it is always the same issue.
Nothing here indicates a problem with your indices. Running slapindex repeatedly is a waste of your time.
[root@l01lnp2 traces]# /usr/local/libexec/slapd -V
@(#) $OpenLDAP: slapd 2.4.16 (Sep 9 2009 14:39:44) $ root@l01lnp2:/home/tculjaga/openldap-2.4.16/servers/slapd
I would strongly urge you to upgrade to 2.4.18 (for reasons I will note further down)
[root@l01lnp2 traces]# /usr/local/BerkeleyDB.4.7/bin/db_stat -V
Berkeley DB 4.7.25: (May 15, 2008) - unpached!
You need to rebuild BDB 4.7.25 with the 4 patches from Oracle. There are known issues when running BDB 4.7 without them.
[root@l01lnp2 traces]# du -c -h /var/lib/ldap/*.bdb
200K /var/lib/ldap/bestMatchPrefix.bdb 3.8G /var/lib/ldap/dn2id.bdb 6.2G /var/lib/ldap/id2entry.bdb 1.8M /var/lib/ldap/objectClass.bdb 1.2M /var/lib/ldap/originatorPrefixID.bdb 48M /var/lib/ldap/uniqueID.bdb 10G total
Since your database is a total of 10 GB in size, for slapadd to work at optimum efficiency, you need at least 10GB of cache for your DB_CONFIG file. Unfortunately, you only have 10GB of RAM. Essentially, your system is under powered for your database size.
[tculjaga@l01lnp2 ~]$ cat ot.ldif | grep -c "dn: "
101588 [tculjaga@l01lnp2 ~]$ cat l01sipdir1.ldif | grep -c "dn: " 9994864 [tculjaga@l01lnp2 ~]$
So you have 10,096,452 entries total.
[root@l01lnp2 traces]# cat /var/lib/ldap/DB_CONFIG | grep -v "#"
set_cachesize 0 3221225472 1 set_lg_regionmax 262144 set_lg_bsize 2097152
You only have a 3GB DB cachesize configured here. Expect things to perform sub optimally. It would have been easier to set this by going
set_cachesize 3 0 1
Which would have the same effect, since the first number is the number of gigabytes to allocate.
Please find attached slapd.conf
Ok, so the relevant bits from here are:
cachesize 2500000 idlcachesize 7500000 cachefree 1000
Which means you have a cachesize of 2.5 million, an idlcachesize of 7.5 million, and (with OL 2.4.16) a dncachesize of 5 million.
I would highly advise you upgrade to OpenLDAP 2.4.18, and change the slapd.conf settings to:
dncachesize 0 (which means unlimited).
And setting no cache or idlcachesize, and fixing your DB_CONFIG. But you also need to buy a substantial amount of RAM for a DB of this size. :P I would advise you upgrade to at least 32GB total. Then you can more optimally tune the system.
--Quanah
--
Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc
Zimbra :: the leader in open source messaging and collaboration