Statistically, that should be relevant. I mean, I usually do.

i=0; while [ $i -lt 100 ]; do pstack <MYPID> > pstack.$i; (( i+=1 )); done;

Yes no sleep, just a burst of pstacks. That is statistically as correct as any sampling based profilers would tell, without the complexity of having to install one such tool (kernel prereq, etc…) and you can collect that in less than a minute.

Sometimes though that can considered as hard to read for people not used to it.

If you pass me with your output, I may try to help.

Best Regards

++Cyrille

From: Luca Polidoro [mailto:luca.polidoro@gmail.com]
Sent: Friday, September 06, 2013 3:08 PM
To: Maucci, Cyrille
Cc: openldap-technical@openldap.org
Subject: Re: Slapd High CPU usage on Solaris 9

Hi, I have already done these tests, but the result provides little information, none of which is useful for directing the analysis.

2013/9/6 Maucci, Cyrille <cyrille.maucci@hp.com>

When I myself face such a problem, I usually pstack the process a few times to very quickly know what the guy is doing.

And that usually gives me a good clue.

++Cyrille

From: openldap-technical-bounces@OpenLDAP.org [mailto:openldap-technical-bounces@OpenLDAP.org] On Behalf Of Luca Polidoro
Sent: Monday, August 12, 2013 3:31 PM
To: openldap-technical@openldap.org
Subject: Slapd High CPU usage on Solaris 9

Hello,

I am writing to to submit a case that has been happening in the last 2 weeks in our infrastructure. This is structured as follows:


1 provider: Solaris 9 SPARC - Sun Fire V490 - last OS patch level
CPU: 4-1500 Mhz
RAM: 32 GB

OpenLDAP version used: Berkeley DB 2.4.23 and 4.8.30 (with database bdb) all 64-bit

18 consumer: Solaris 9 SPARC - last OS patch level with different types of features (CPU, RAM)

On the following consumer products:

Consumer 1: Solaris 9 SPARC - Sun Fire 480R - last OS patch level
CPU: 4-900 Mhz
RAM: 8 GB

Consumer 2: Solaris 9 SPARC - Sun Fire 480R - last OS patch level
CPU: 4-1050 Mhz
RAM: 8 GB

Consumer 3: Solaris 9 SPARC - Sun Fire 480R - last OS patch level
CPU: 4-1050 Mhz
RAM: 8 GB

Consumer 4: Solaris 9 SPARC - Sun Fire V210 - last OS patch level
CPU: 2-1336 Mhz
RAM: 8 GB

we are noticing an increase in the cpu used by the slapd process. In fact, the process is constantly between 85% and 95%, and became completely unusable and then we are forced to restart.

LDAP with 1.000.000 objects.

This is the consumer's slapd.conf (I have omitted parts of the ACL, includes, etc..):

# See slapd.conf(5) for details on configuration options.
# This file should NOT be world readable.
#

#
# VERSION v2 -  Digital Tru64
#
allow bind_v2


Some include
...

#
#  tuning parameters - START
# ------------------------------
#
conn_max_pending 1000
conn_max_pending_auth 1000

idletimeout     500
sizelimit        unlimited
threads          8
timelimit        500
disallow bind_anon

#
#  tuning parameters - END
# ----------------------------
#

...

#######################################################################
# bdb database definitions
#######################################################################

database        bdb
suffix          "xxxxxxxxxxxx"
rootdn          "cn=root,ou=ldapusers,xxxxx"

directory       /var/openldap-2.4.23_64/var/openldap-data
#####disallow limit for syncuser
limits  dn.children="ou=syncusers,xxxx" size=unlimited         
index   objectClass,entryCSN,entryUUID eq
index   ou eq,sub,subinitial,subany,subfinal
index   uidOwner eq
index    uid eq
index    memberUid eq

#shm_key 1100
cachesize 1000000
cachefree 10000
dncachesize 1000000
idlcachesize 1000000
searchstack 16
checkpoint 1024 10

overlay ppolicy
ppolicy_default "cn=Standard,ou=Policies,xxxx"
ppolicy_use_lockout



############################SYNCREPL CONF
syncrepl   rid=011
           provider=ldap://xxxxxx
           type=refreshAndPersist
           interval=00:00:15:00
           retry="15 10 120 +"
           searchbase="xxxxx"
           filter="(objectClass=*)"
           attrs="*,+"
           scope=sub
           schemachecking=on
           bindmethod=simple
           binddn="xxxxxx"
           credentials=xxxx
############################SYNCREPL CONF


These are the bdb files:

420M    dn2id.bdb
  30M    entryCSN.bdb
  32M    entryUUID.bdb
 1,4G    id2entry.bdb
  18M    memberUid.bdb
 4,9M    objectClass.bdb
 5,3M    ou.bdb
  17M    uid.bdb
  17M    uidOwner.bdb

this is  DB CONFIG:

-----------------------------------------------------------

##########################################
###########################################
#set_cachesize 0 300000000 10
#set_lg_regionmax 262144
#set_lg_bsize 2097152
###########################################
###########################################
# replaces lockdetect directive
#set_lk_detect DB_LOCK_EXPIRE
set_lk_detect DB_LOCK_DEFAULT

# uncomment if dbnosync required
#AGGIUNTO TUTTO
#set_flags DB_TXN_WRITE_NOSYNC
####AGGIUNTO
set_flags DB_LOG_AUTOREMOVE
# multiple set_flags directives allowed

# sets max log size = 5M (BDB default=10M)
set_lg_max 25242880
set_lg_dir /var/openldap-2.4.23_64/logs

set_cachesize 2 274726912 1
# sets a database cache of 5M and
# allows fragmentation
# does NOT replace slapd.conf cachesize
# this is a database parameter

#txn_checkpoint 128 15 0
# replaces checkpoint in slap.conf
# writes checkpoint if 128K written or every 15 mins
# 0 = no writes - no update
set_lk_max_locks 2500
set_lk_max_lockers 2500
set_lk_max_objects 2500

---------------------------------------------------

We have tried to change the number of threads bringing them to 16, we lowered the parameters idletimeout and timelimit, but without result.


Appreciate your feedback.

Thanks,

Luca