Thanks Everyone !
Here is my system configuration :
OpenLDAP version : 2.3.27 Berkeley DB Version: 4.4.20 Server : Mac OSX Tiger in Power PC Memory : 8 GB Number of physical DB : 3 Size of each DB : 4 , 12 and 24 each.
DB_CONFIG file for one DB , others are also set simillarly :
========== # $OpenLDAP: pkg/ldap/servers/slapd/DB_CONFIG,v 1.1.2.3 2006/08/17 17:36:19 kurt Exp $ # Example DB_CONFIG file for use with slapd(8) BDB/HDB databases. # # See Sleepycat Berkeley DB documentation # http://www.sleepycat.com/docs/ref/env/db_config.html # for detail description of DB_CONFIG syntax and semantics. # # Hints can also be found in the OpenLDAP Software FAQ # http://www.openldap.org/faq/index.cgi?file=2 # in particular: # http://www.openldap.org/faq/index.cgi?file=1075
# Note: most DB_CONFIG settings will take effect only upon rebuilding # the DB environment.
# one 0.20 GB cache set_cachesize 0 42428800 0
# Data Directory #set_data_dir db
#set db flags #only use when using slapdADD set_flags DB_TXN_NOSYNC
set_lk_max_locks 2000
# Transaction Log settings set_lg_regionmax 1048576 set_lg_max 20485760 set_lg_bsize 2097152 set_lg_dir /Volumes/ngs/app/ldapp/openldap/var/openldap-data/db/externals/
# Automatically remove log files that are no longer needed. set_flags DB_LOG_AUTOREMOVE
# Note: special DB_CONFIG flags are no longer needed for "quick" # slapadd(8) or slapindex(8) access (see their -q option). ==========
slapd.conf file : ---------------------
==== # Do not enable referrals until AFTER you have a working directory # # See slapd.conf(5) for details on configuration options. # This file should NOT be world readable. # include /Volumes/ngs/app/ldapp/openldap/etc/openldap/schema/core.schema include /Volumes/ngs/app/ldapp/openldap/etc/openldap/schema/cosine.schema include /Volumes/ngs/app/ldapp/openldap/etc/openldap/schema/inetorgperson.schema # Custom Schema include /Volumes/ngs/app/ldapp/openldap/etc/openldap/schema/ist.schema # Define global ACLs to disable default read access.
# service AND an understanding of referrals. #referral ldap://root.openldap.org
pidfile /Volumes/ngs/app/ldapp/openldap//var/run/slapd.pid argsfile /Volumes/ngs/app/ldapp/openldap//var/run/slapd.args replogfile /Volumes/ngs/app/ldapp/openldap/var/openldap-slurp/replogfile.log
# Give the replicator account the ability to update and everyone read access.
access to attrs=userpassword by * auth access to * by dn.base="cn=replicator,o= Computer" write by * read
access to * by * write by dn.base="cn=manager,o= computer" write readonly off
loglevel 256 #conn_max_pending 300 defaultsearchbase "o= Computer" gentlehup on idletimeout 300 sizelimit 2000000 timelimit 300 password-hash {SSHA} allow bind_v2 threads 32
database bdb suffix "ou=externals,o= Computer" subordinate "o= Computer" rootdn "cn=Manager,o= Computer" dbcachesize 10000000 cachesize 100000000 directory /Volumes/ngs/app/ldapp/openldap/var/openldap-data/db/externals index objectClass eq index dsid eq index cn eq,sub index givenName eq,sub index mail eq,sub index sn eq,sub index telephonenumber eq,sub index entryUUID eq replica uri=ldap://ldapws1.corp.computer.com:3893/ binddn="cn=Replicator,o= Computer" bindmethod=simple credentials=******
============
We are planning to upgrade to 64 bit Mac Intel processors, but till then we need this to be up and running.
Any suggesstions for an optimal configuration will be appreciated.
Thanks, Sumith.
On 9/11/07, Quanah Gibson-Mount quanah@zimbra.com wrote:
--On Tuesday, September 11, 2007 8:59 AM -0400 Aaron Richton richton@nbcs.rutgers.edu wrote:
It's almost certain that this configuration is inappropriate. You're unlikely to get a decently tuned cache for a database that size in 32-bit process space.
As for your "crashes," you're likely OOM running into the DN cache issues previously discussed
http://www.openldap.org/lists/openldap-software/200708/msg00106.html
and you may find help with a 64-bit platform, OpenLDAP 2.4, or some combination thereof.
I think it is a little premature for OpenLDAP 2.4, but definitely go 64-bit, do some tuning, and actually provide useful details. You don't note the OS either, but if it is Linux, you probably want to use something like tcmalloc instead of glibc for memory management as well.
--Quanah
--
Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc
Zimbra :: the leader in open source messaging and collaboration
Sumith Narayanan wrote:
/Volumes/ngs/app/ldapp/openldap/var/openldap-data/db/externals/
This isn't a network share is it? (Volumes)
No it is not.
Thanks, Sumith.
On 9/13/07, Gavin Henry ghenry@suretecsystems.com wrote:
Sumith Narayanan wrote:
/Volumes/ngs/app/ldapp/openldap/var/openldap-data/db/externals/
This isn't a network share is it? (Volumes)
-- Kind Regards,
Gavin Henry. Managing Director.
T +44 (0) 1224 279484 M +44 (0) 7930 323266 F +44 (0) 1224 824887 E ghenry@suretecsystems.com
Open Source. Open Solutions(tm).
On Tuesday 11 September 2007 23:54:14 Sumith Narayanan wrote:
Thanks Everyone !
Here is my system configuration :
OpenLDAP version : 2.3.27 Berkeley DB Version: 4.4.20 Server : Mac OSX Tiger in Power PC Memory : 8 GB Number of physical DB : 3 Size of each DB : 4 , 12 and 24 each.
DB_CONFIG file for one DB , others are also set simillarly :
========== # $OpenLDAP: pkg/ldap/servers/slapd/DB_CONFIG,v 1.1.2.3 2006/08/17 17:36:19 kurt Exp $ # Example DB_CONFIG file for use with slapd(8) BDB/HDB databases. # # See Sleepycat Berkeley DB documentation # http://www.sleepycat.com/docs/ref/env/db_config.html # for detail description of DB_CONFIG syntax and semantics. # # Hints can also be found in the OpenLDAP Software FAQ # http://www.openldap.org/faq/index.cgi?file=2 # in particular: # http://www.openldap.org/faq/index.cgi?file=1075
# Note: most DB_CONFIG settings will take effect only upon rebuilding # the DB environment.
# one 0.20 GB cache set_cachesize 0 42428800 0
You should probably assign more BDB cache than this (and reduce your entry cache size).
# Data Directory #set_data_dir db
#set db flags #only use when using slapdADD set_flags DB_TXN_NOSYNC
Don't do this, rather run 2.3 and use the -q flag.
set_lk_max_locks 2000
# Transaction Log settings set_lg_regionmax 1048576 set_lg_max 20485760 set_lg_bsize 2097152 set_lg_dir /Volumes/ngs/app/ldapp/openldap/var/openldap-data/db/externals/
# Automatically remove log files that are no longer needed. set_flags DB_LOG_AUTOREMOVE
Which means, you should usually only have one transaction log file. How many do you have?
# Note: special DB_CONFIG flags are no longer needed for "quick"
# slapadd(8) or slapindex(8) access (see their -q option).
slapd.conf file :
==== # Do not enable referrals until AFTER you have a working directory # # See slapd.conf(5) for details on configuration options. # This file should NOT be world readable. # include /Volumes/ngs/app/ldapp/openldap/etc/openldap/schema/core.schema include /Volumes/ngs/app/ldapp/openldap/etc/openldap/schema/cosine.schema include /Volumes/ngs/app/ldapp/openldap/etc/openldap/schema/inetorgperson.schema # Custom Schema include /Volumes/ngs/app/ldapp/openldap/etc/openldap/schema/ist.schema # Define global ACLs to disable default read access.
# service AND an understanding of referrals. #referral ldap://root.openldap.org
pidfile /Volumes/ngs/app/ldapp/openldap//var/run/slapd.pid argsfile /Volumes/ngs/app/ldapp/openldap//var/run/slapd.args replogfile /Volumes/ngs/app/ldapp/openldap/var/openldap-slurp/replogfile.log
# Give the replicator account the ability to update and everyone read access.
access to attrs=userpassword by * auth access to * by dn.base="cn=replicator,o= Computer" write by * read
access to * by * write by dn.base="cn=manager,o= computer" write readonly off
loglevel 256 #conn_max_pending 300 defaultsearchbase "o= Computer" gentlehup on idletimeout 300 sizelimit 2000000 timelimit 300 password-hash {SSHA} allow bind_v2 threads 32
database bdb suffix "ou=externals,o= Computer" subordinate "o= Computer" rootdn "cn=Manager,o= Computer" dbcachesize 10000000 cachesize 100000000
There's no way you have enough memory for this (you'd probably need almost 1TB). Please read the tuning entry in the FAQ. If you don't have enough memory for your entry cache, it's better to assign more to the BDB backend cache. You don't say how many entries you have (or what the size of id2entry.bdb is), but your performance is most likely going to be very poor with this setting, as you'll start swapping quite soon ...
directory /Volumes/ngs/app/ldapp/openldap/var/openldap-data/db/externals index objectClass eq index dsid eq index cn eq,sub index givenName eq,sub index mail eq,sub index sn eq,sub index telephonenumber eq,sub index entryUUID eq replica uri=ldap://ldapws1.corp.computer.com:3893/ binddn="cn=Replicator,o= Computer" bindmethod=simple credentials=******
As I suspected, you have no checkpoint statement. So, if database recovery is required, it will be going through *all* the transaction log files.
============
We are planning to upgrade to 64 bit Mac Intel processors, but till then we need this to be up and running.
I'd rather run Linux on commodity hardware (anything recent can run 64bit) ... you have a lot of expensive ram that is more or less useless to you ...
Any suggesstions for an optimal configuration will be appreciated.
You need to provide a little bit more information (number of entries, or sizes of the id2entry.bdb files). But, drop your entry cache (cachesize in slapd.conf), increase your BDB backend cache (see the tuning entry for some guidelines), but ensure you don't assign more cache than you have the memory for ..
Regards, Buchan
openldap-software@openldap.org