Re: Index Add Failures
by Kyle Smith
Quanah, Thanks for the info, I have confirmed I'm hitting the lock maxes of
1000. And I will be upgrading to 2.4.32. I was wondering, what steps should
be done to have the changes in DB_CONFIG take effect?
stop slapd
make changes to DB_CONFIG
db_recover
start slapd
Will this also auto remove the log.* files? ( I plan on setting this:
"set_flags DB_LOG_AUTOREMOVE" in DB_CONFIG)
Thanks! - Kyle
2012/8/28 Quanah Gibson-Mount <quanah(a)zimbra.com>
> --On Tuesday, August 28, 2012 9:27 AM -0400 Kyle Smith <
> alacer.cogitatus(a)gmail.com> wrote:
>
> Good Morning All,
>>
>> I receive events that mention "attribute memberUid index add failure".
>> What does this mean? Once I notice these events, I have to restart
>> slapd, as writes will not be performed and CPU usage hits 99%
>> continuously. It seems to be associated with a high load, as our
>> semester just got underway and we are seeing an increase in logins and
>> searches (about 150-200 requests per minute). I am using the bdb
>> backend.
>>
>> Specifications
>>
>> OpenLDAP 2.4.28 and 2.4.26
>> 4-way multi-master
>>
>
> If you are using MMR, you desperately need to upgrade your OpenLDAP
> Versions... aside from the error you are seeing is pretty typical from BDB
> if you have run out of locks, lockers, or locker objects. You should check
> out db_stat to see which resource you have run out of, and fix it
> accordingly.
>
> You may with to read over <https://wiki.zimbra.com/wiki/**
> OpenLDAP_Performance_Tuning#**Berkeley_DB_DB_CONFIG_tuning<https://wiki.zimbra.com/wiki/OpenLDAP_Performance_Tuning#Berkeley_DB_DB_C...>
> >
>
> Regards,
> Quanah
>
> --
>
> Quanah Gibson-Mount
> Sr. Member of Technical Staff
> Zimbra, Inc
> A Division of VMware, Inc.
> --------------------
> Zimbra :: the leader in open source messaging and collaboration
>
11 years
Index Add Failures
by Kyle Smith
Good Morning All,
I receive events that mention "attribute memberUid index add failure".
What does this mean? Once I notice these events, I have to restart
slapd, as writes will not be performed and CPU usage hits 99%
continuously. It seems to be associated with a high load, as our
semester just got underway and we are seeing an increase in logins and
searches (about 150-200 requests per minute). I am using the bdb
backend.
Specifications
OpenLDAP 2.4.28 and 2.4.26
4-way multi-master
index gidnumber,uidnumber,objectClass,automountKey,nodisplay eq
index uid,cn,sn,givenname,employeetype,mail,eaiGroup,memberUid,destinationIndicator
eq,sub
index facsimileTelephoneNumber eq,sub
index mgrpRFC822MailMember pres
Aug 28 02:40:41 poe slapd[10882]: conn=-1 op=0: attribute "memberUid"
index add failure
Thanks!
Kyle Smith
York College of Pennsylvania
11 years
RE: Performance of MDB and BDB Please suggest?
by Chris Card
Hi all,
I am also doing some testing with mdb at the moment, and my initial testing indicates that mdb is faster for reads but slower for writes than bdb.
I am using openldap 2.4.32 on centos 6, on a 24 core box with 132 Gb RAM.
My test directory has ~ 3 million entries, and I loaded it into mdb using slapadd which took over 2 days (by comparison, the same load into bdb takes 2-3 hours).
(as an aside, I initially tried using 2.4.31, but slapadd crashed after having loaded about 90% of the data, and this was repeatable).
On disk the directory takes up ~ 13 Gb for mdb and ~ 18Gb for bdb.
Cache size for bdb is set to 63 Gb in DB_CONFIG.
Directory size for mdb is set to 63 Gb
Adding 120000 entries from an ldif file using ldapadd took ~ 10 minutes for mdb and ~ 2 minutes for bdb.
Deleting 120000 entries using ldapdelete took ~ 10 minutes for mdb and ~ 3 minutes for bdb
A search returning ~ 300000 DNs took ~ 6 seconds for mdb and for bdb it took ~ 6 minutes from a cold start of slapd and then ~ 35 seconds.
Chris
11 years
Re: slapd-meta doesn't continue with multiple uri's
by Liam Gretton
On 22/08/2012 22:14, Pierangelo Masarati wrote:
>> But what's the point of specifying multiple targets in the uri
>> option if it doesn't fall through to subsequent ones when the first
>> is not contactable?
>>
>> Have I completely missed the point of the documentation?
>
> The point is that your condition is *not* a server unreachable.
There's obviously some subtlety I'm missing here. How would you describe
it instead?
> Current failover only deals with failures/timeouts of connect(2). I
> don't think handling your case using failover is appropriate. Your
> case should be handled by removing the non-responding URI from the
> list.
I don't understand the difference. If a server is unavailable for
whatever reason (offline, firewalled, switched off, nothing listening on
the specified port), then connect() will timeout as you describe.
Which failures are the current mechanism actually expected to cope with
that don't include a server being unreachable?
--
Liam Gretton liam.gretton(a)le.ac.uk
HPC Architect http://www.le.ac.uk/its
IT Services Tel: +44 (0)116 2522254
University of Leicester, University Road
Leicestershire LE1 7RH, United Kingdom
11 years
acls
by Mundry, Marvin
Hi,
I am trying to write acl statements that implement to following scenario:
with the exception of cn=radius,ou=sa,dc=test,dc=com
every user should be able to see all objects under ou=users,dc=test,dc=com.
cn=radius,ou=sa,dc=test,dc=com should only see objects under ou=users,dc=test,dc=com with objectClass=radiusprofile
I have tried the following acl statements which unfortunately do not work:
-------------------------------
{11}to filter="(!(objectClass=radiusprofile))"
by dn.exact="cn=radius,ou=sa,dc=test,dc=com" none
by * break
{12}to dn.subtree="ou=users,dc=test,dc=com" attrs=entry,@top,cn,entryUUID
by users read
by * break
-------------------------------
statement {11} results in cn=radius,ou=sa,dc=test,dc=com not being able to see any objects.
interestingly if I set the filter in {11} to "(objectClass=radiusprofile)" (without the inversion(!))
cn=radius,ou=sa,dc=test,dc=com can see all objects not having objectClass=radiusprofile, which is exactly the opposite of what I am
trying to do.
why does the inversion (!) in the filter statement result in cn=radius,ou=sa,dc=test,dc=com
not being able to see any objects?
Marvin
11 years, 1 month
BDB: Converting mmapped files to shared memory: Need slap{cat,add}?
by Nick Urbanik
Dear Folks,
I have a large production 2.4.32 LDAP server slaving many trees. I
set it up with memory-mapped files, and want to change it to use
shared memory instead, since shared memory BDB seems to work better
after watching the other servers that have been upgraded 2.3 => 2.4.32
with shared memory.
To convert this, do I need to slapcat the directories and restore them
after the configuration change with slapadd, or is it sufficient to
use db_recover in each LDAP database after changing the configuration
while slapd is stopped?
--
Nick Urbanik http://nicku.org 808-71011 nick.urbanik(a)optusnet.com.au
GPG: 7FFA CDC7 5A77 0558 DC7A 790A 16DF EC5B BB9D 2C24 ID: BB9D2C24
I disclaim, therefore I am.
11 years, 1 month
Performance of MDB and BDB Please suggest?
by aryan rawat
Hi,
I am working on Open-Ldap for one of my application and I was testing to
find out which DB is FAST MDB or BDB for ldapsearch.
Tested both the DB’s on same platform for .6 Million records.
Configuration Done for
BDB (openldap2.4.24)
MDB (openldap2.4.32)
Slapd.conf
database hdb
threads 32
Slapd.conf
database mdb
threads 32
maxsize 2147483648
DB_CONFIG
set_cachesize 2 0 1
To test the same developed a java tool to send a bulk search request on
Open Ldap for both the DB’s.
Pumping the load for 400,500,1000,2000,2500 etc.. search request to
openldap.
*The outcome was BDB was almost 8 time faster than MDB. *
To my surprise it was written that for search MDB is faster than BDB.
Please suggest if I have done some wrong configuration in MDB..
BR's,
Haroon
11 years, 1 month
compiled the Openldap 2.4.32 with these setting are they the Best..
by aryan rawat
Hi ,
I have compiled the Openldap 2.4.32 with these options for MDB
./configure --prefix=/root/haroon/openldap/new1 --enable-debug=yes
--enable-bdb=no --enable-hdb=no --enable-mdb=yes --with-cyrus-sasl
--with-threads --with-tls=openssl CC="gcc" LDFLAGS="-L/usr/lib64/sasl2"
CPPFLAGS="-I/usr/include/sasl"
Are they the best setting or I need to set some thing different.
BR's,
Haroon
11 years, 1 month
