openldap-technical

openldap-technical@openldap.org

3 participants
9937 discussions

Re: Performance of MDB and BDB Please suggest?
by Yajuvendra Singh 31 Aug '12

31 Aug '12

Hi Quanah, Thanks for replying, I have few more observation regrading my load runs, real time of load run with MDB is 98% even at 10 TPS. What version of OpenLDAP are you using? ---2.4.32 What type of disk? --root@tspatca2103> fdisk -l /dev/mapper/vg00-root Disk /dev/mapper/vg00-root: 10.7 GB, 10737418240 bytes 255 heads, 63 sectors/track, 1305 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000 Disk /dev/mapper/vg00-root doesn't contain a valid partition table root@tspatca2103> What type of file system? --ext4 (/dev/mapper/vg00-root on / type ext4 (rw,nouser_xattr)) What is your *exact* slapadd command? --/opt/openldap/yaju/sbin/slapadd -q -w -f /opt/openldap/yaju/etc/openldap/slapd.conf -l /root/yaju/db.ldif I am using jmeter to simulate the load. -- Thanks and Regards Yajuvendra On Tue, Aug 28, 2012 at 8:19 PM, Quanah Gibson-Mount <quanah(a)zimbra.com>wrote: > --On Tuesday, August 28, 2012 5:29 PM +0530 Yajuvendra Singh < > yajuvendra.singh(a)gmail.com> wrote: > > Dear Experts, >> >> >> Today we tried with to run the load with the below schema. We added about >> .6M entries in the DB. >> Still our performance is severely poor. (10 TPS) >> >> >> Can anybody review our slapd.conf file and point us where we are wrong, >> is there any other config we have missed out. >> > > You don't provide any useful or relevant information, so it is impossible > to help you. > > What version of OpenLDAP are you using? > What type of disk? > What type of file system? > What is your *exact* slapadd command? > etc. > > There is virtually no tuning involved with MDB, although I strongly > recommend you read Howard's notes about the writeback bits for EXT4 etc he > made in a recent post to -technical about MDB. > > > --Quanah > > > -- > > Quanah Gibson-Mount > Sr. Member of Technical Staff > Zimbra, Inc > A Division of VMware, Inc. > -------------------- > Zimbra :: the leader in open source messaging and collaboration >

4 8

help with setting up replication
by Jeff Dickens 30 Aug '12

30 Aug '12

I've been following this page from the Ubuntu Server Guide with generally good results: https://help.ubuntu.com/12.04/serverguide/openldap-server.html Now I'm down to the replication section, at https://help.ubuntu.com/12.04/serverguide/openldap-server.html#openldap-ser… . So far it isn't working. First things first: I create the following ldif file to configure the sync provider: # Add indexes to the frontend db. dn: olcDatabase={1}hdb,cn=config changetype: modify add: olcDbIndex olcDbIndex: entryCSN eq - add: olcDbIndex olcDbIndex: entryUUID eq #Load the syncprov and accesslog modules. dn: cn=module{0},cn=config changetype: modify add: olcModuleLoad olcModuleLoad: syncprov - add: olcModuleLoad olcModuleLoad: accesslog # Accesslog database definitions dn: olcDatabase={2}hdb,cn=config objectClass: olcDatabaseConfig objectClass: olcHdbConfig olcDatabase: {2}hdb olcDbDirectory: /var/lib/ldap/accesslog olcSuffix: cn=accesslog olcRootDN: cn=admin,dc=intranet,dc=seamanpaper,dc=com olcDbIndex: default eq olcDbIndex: entryCSN,objectClass,reqEnd,reqResult,reqStart # Accesslog db syncprov. dn: olcOverlay=syncprov,olcDatabase={2}hdb,cn=config changetype: add objectClass: olcOverlayConfig objectClass: olcSyncProvConfig olcOverlay: syncprov olcSpNoPresent: TRUE olcSpReloadHint: TRUE # syncrepl Provider for primary db dn: olcOverlay=syncprov,olcDatabase={1}hdb,cn=config changetype: add objectClass: olcOverlayConfig objectClass: olcSyncProvConfig olcOverlay: syncprov olcSpNoPresent: TRUE # accesslog overlay definitions for primary db dn: olcOverlay=accesslog,olcDatabase={1}hdb,cn=config objectClass: olcOverlayConfig objectClass: olcAccessLogConfig olcOverlay: accesslog olcAccessLogDB: cn=accesslog olcAccessLogOps: writes olcAccessLogSuccess: TRUE # scan the accesslog DB every day, and purge entries older than 7 days olcAccessLogPurge: 07+00:00 01+00:00 The guide says you can test the provider with this command: root@grackle:~# ldapsearch -z1 -LLLQY EXTERNAL -H ldapi:/// -s base contextCSN dn: root@grackle:~# but as you see that doesn't return anything. However, this command does find it: root@grackle:~# slapcat | grep -C 10 contextCSN objectClass: organization o: intranet.seamanpaper.com dc: intranet structuralObjectClass: organization entryUUID: 99e43416-73a1-1031-9d82-4f560555aca0 creatorsName: cn=admin,dc=intranet,dc=seamanpaper,dc=com createTimestamp: 20120805233244Z entryCSN: 20120805233244.262007Z#000000#000#000000 modifiersName: cn=admin,dc=intranet,dc=seamanpaper,dc=com modifyTimestamp: 20120805233244Z contextCSN: 20120829024252.920832Z#000000#000#000000 dn: cn=admin,dc=intranet,dc=seamanpaper,dc=com objectClass: simpleSecurityObject objectClass: organizationalRole cn: admin description: LDAP administrator userPassword:: e1NTSEF9Nm9zUVlmUStzd1RCOVJCQXUyL3NhQURpYTZ1R0NuRC8= structuralObjectClass: organizationalRole entryUUID: 99e4f9fa-73a1-1031-9d83-4f560555aca0 creatorsName: cn=admin,dc=intranet,dc=seamanpaper,dc=com root@grackle:~# Before I go on to figure out why the sync isn't working, why isn't the ldapsearch command above returning anything? Thanks in advance for your help. -- * Jeff Dickens* IT Manager 978-632-1513

3 3

LinuxCon Presentation 2012-08-29
by Howard Chu 30 Aug '12

30 Aug '12

Just FYI, I'll be giving a talk on MDB, OpenLDAP's new memory-mapped database library, at LinuxCon in San Diego next month. http://events.linuxfoundation.org/events/linuxcon/schedule Some of the benchmark data I posted here will be part of that talk: http://highlandsun.com/hyc/mdb/microbench/ If you haven't looked into MDB yet, here's your chance to find out all about it! -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

3 3

How to config LDAP client to get data from multiple sub domains in LDAP server?
by Qian Zhang 30 Aug '12

30 Aug '12

Hi All, I encountered one issue when configuring the LDAP client. My question is how to configure the LDAP client to make it get the users and groups information from two sub domains. The base domain of the LDAP server is "dc=main,dc=com", and under the base domain, there are more than one sub domains. Such as, sub domain 1 ("dc=sub1,dc=main,dc=com"), sub domain 2 ("dc=sub2,dc=main,dc=com"), sub domain 3 ("dc=sub3,dc=main,dc=com"), .... Now I just wanna list the users and groups information from sub domain sub1 and sub domain sub2 (by command "getent passwd"). The information from sub domain sub3 must be excluded. My test env is : Rehl6.2_x86_64, openldap-2.4.23 I have tried the following way: Step1, configure the LDAP server. I used the access control item "olcAccess" in file /etc/openldap/slapd.d/cn\=config/olcDatabase\=\{2\}bdb.ldif olcAccess: to dn.subtree="dc=sub1,dc=main,dc=com" by dn="uid=user1,dc=sub1,dc=main,dc=com" read olcAccess: to dn.subtree="dc=sub2,dc=main,dc=com" by dn="uid=user1,dc=sub1,dc=main,dc=com" read See above, I added the access controls, they mean that only the user "uid=user1,dc=sub1,dc=main,dc=com" can access the information of the two sub domains (sub1 and sub2). After added the configurations, I restarted slapd service in server side. Step2, configure the LDAP client, in /etc/nslcd and /etc/pam_ldap.conf, I modified the items blow: base dc=main,dc=com binddn uid=user1,dc=sub1,dc=main,dc=com bindpw *** Then restart the service nslcd, but I can't get any users information by command "getent passwd" in LDAP client side. ( I confirmed my others configuration are correct, if I change the base to "base dc=sub1,dc=main,dc=com", I can get all the information from sub1). I think the problem is from LDAP server side, the user "uid=user1,dc=sub1,dc=main,dc=com" in sub domain "sub1" is only allowed to access the sub doamins (sub1 and sub2) by access control. But I can't change the "base" from "dc=main,dc=com" to "dc=sub1,dc=main,dc=com" or "dc=sub2,dc=main,dc=com". That's because only one sub domain can be the "base", so I only can get the information from one of the sub domains, but this is not my purpose. Then I try to set the search scope to "scope sub", but still can't work. Any comments? I will be really appreciated for your feedback.

1 0

Re: Index Add Failures
by Kyle Smith 29 Aug '12

29 Aug '12

Quanah, Thanks for the info, I have confirmed I'm hitting the lock maxes of 1000. And I will be upgrading to 2.4.32. I was wondering, what steps should be done to have the changes in DB_CONFIG take effect? stop slapd make changes to DB_CONFIG db_recover start slapd Will this also auto remove the log.* files? ( I plan on setting this: "set_flags DB_LOG_AUTOREMOVE" in DB_CONFIG) Thanks! - Kyle 2012/8/28 Quanah Gibson-Mount <quanah(a)zimbra.com> > --On Tuesday, August 28, 2012 9:27 AM -0400 Kyle Smith < > alacer.cogitatus(a)gmail.com> wrote: > > Good Morning All, >> >> I receive events that mention "attribute memberUid index add failure". >> What does this mean? Once I notice these events, I have to restart >> slapd, as writes will not be performed and CPU usage hits 99% >> continuously. It seems to be associated with a high load, as our >> semester just got underway and we are seeing an increase in logins and >> searches (about 150-200 requests per minute). I am using the bdb >> backend. >> >> Specifications >> >> OpenLDAP 2.4.28 and 2.4.26 >> 4-way multi-master >> > > If you are using MMR, you desperately need to upgrade your OpenLDAP > Versions... aside from the error you are seeing is pretty typical from BDB > if you have run out of locks, lockers, or locker objects. You should check > out db_stat to see which resource you have run out of, and fix it > accordingly. > > You may with to read over <https://wiki.zimbra.com/wiki/** > OpenLDAP_Performance_Tuning#**Berkeley_DB_DB_CONFIG_tuning<https://wiki.zimbra.com/wiki/OpenLDAP_Performance_Tuning#Berkeley_DB_DB_CON…> > > > > Regards, > Quanah > > -- > > Quanah Gibson-Mount > Sr. Member of Technical Staff > Zimbra, Inc > A Division of VMware, Inc. > -------------------- > Zimbra :: the leader in open source messaging and collaboration >

2 1

Index Add Failures
by Kyle Smith 28 Aug '12

28 Aug '12

Good Morning All, I receive events that mention "attribute memberUid index add failure". What does this mean? Once I notice these events, I have to restart slapd, as writes will not be performed and CPU usage hits 99% continuously. It seems to be associated with a high load, as our semester just got underway and we are seeing an increase in logins and searches (about 150-200 requests per minute). I am using the bdb backend. Specifications OpenLDAP 2.4.28 and 2.4.26 4-way multi-master index gidnumber,uidnumber,objectClass,automountKey,nodisplay eq index uid,cn,sn,givenname,employeetype,mail,eaiGroup,memberUid,destinationIndicator eq,sub index facsimileTelephoneNumber eq,sub index mgrpRFC822MailMember pres Aug 28 02:40:41 poe slapd[10882]: conn=-1 op=0: attribute "memberUid" index add failure Thanks! Kyle Smith York College of Pennsylvania

2 1

RE: Performance of MDB and BDB Please suggest?
by Chris Card 28 Aug '12

28 Aug '12

Hi all, I am also doing some testing with mdb at the moment, and my initial testing indicates that mdb is faster for reads but slower for writes than bdb. I am using openldap 2.4.32 on centos 6, on a 24 core box with 132 Gb RAM. My test directory has ~ 3 million entries, and I loaded it into mdb using slapadd which took over 2 days (by comparison, the same load into bdb takes 2-3 hours). (as an aside, I initially tried using 2.4.31, but slapadd crashed after having loaded about 90% of the data, and this was repeatable). On disk the directory takes up ~ 13 Gb for mdb and ~ 18Gb for bdb. Cache size for bdb is set to 63 Gb in DB_CONFIG. Directory size for mdb is set to 63 Gb Adding 120000 entries from an ldif file using ldapadd took ~ 10 minutes for mdb and ~ 2 minutes for bdb. Deleting 120000 entries using ldapdelete took ~ 10 minutes for mdb and ~ 3 minutes for bdb A search returning ~ 300000 DNs took ~ 6 seconds for mdb and for bdb it took ~ 6 minutes from a cold start of slapd and then ~ 35 seconds. Chris

5 9

Re: slapd-meta doesn't continue with multiple uri's
by Liam Gretton 28 Aug '12

28 Aug '12

On 22/08/2012 22:14, Pierangelo Masarati wrote: >> But what's the point of specifying multiple targets in the uri >> option if it doesn't fall through to subsequent ones when the first >> is not contactable? >> >> Have I completely missed the point of the documentation? > > The point is that your condition is *not* a server unreachable. There's obviously some subtlety I'm missing here. How would you describe it instead? > Current failover only deals with failures/timeouts of connect(2). I > don't think handling your case using failover is appropriate. Your > case should be handled by removing the non-responding URI from the > list. I don't understand the difference. If a server is unavailable for whatever reason (offline, firewalled, switched off, nothing listening on the specified port), then connect() will timeout as you describe. Which failures are the current mechanism actually expected to cope with that don't include a server being unreachable? -- Liam Gretton liam.gretton(a)le.ac.uk HPC Architect http://www.le.ac.uk/its IT Services Tel: +44 (0)116 2522254 University of Leicester, University Road Leicestershire LE1 7RH, United Kingdom

5 9

acls
by Mundry, Marvin 27 Aug '12

27 Aug '12

Hi, I am trying to write acl statements that implement to following scenario: with the exception of cn=radius,ou=sa,dc=test,dc=com every user should be able to see all objects under ou=users,dc=test,dc=com. cn=radius,ou=sa,dc=test,dc=com should only see objects under ou=users,dc=test,dc=com with objectClass=radiusprofile I have tried the following acl statements which unfortunately do not work: ------------------------------- {11}to filter="(!(objectClass=radiusprofile))" by dn.exact="cn=radius,ou=sa,dc=test,dc=com" none by * break {12}to dn.subtree="ou=users,dc=test,dc=com" attrs=entry,@top,cn,entryUUID by users read by * break ------------------------------- statement {11} results in cn=radius,ou=sa,dc=test,dc=com not being able to see any objects. interestingly if I set the filter in {11} to "(objectClass=radiusprofile)" (without the inversion(!)) cn=radius,ou=sa,dc=test,dc=com can see all objects not having objectClass=radiusprofile, which is exactly the opposite of what I am trying to do. why does the inversion (!) in the filter statement result in cn=radius,ou=sa,dc=test,dc=com not being able to see any objects? Marvin

2 4

BDB: Converting mmapped files to shared memory: Need slap{cat,add}?
by Nick Urbanik 27 Aug '12

27 Aug '12

Dear Folks, I have a large production 2.4.32 LDAP server slaving many trees. I set it up with memory-mapped files, and want to change it to use shared memory instead, since shared memory BDB seems to work better after watching the other servers that have been upgraded 2.3 => 2.4.32 with shared memory. To convert this, do I need to slapcat the directories and restore them after the configuration change with slapadd, or is it sufficient to use db_recover in each LDAP database after changing the configuration while slapd is stopped? -- Nick Urbanik http://nicku.org 808-71011 nick.urbanik(a)optusnet.com.au GPG: 7FFA CDC7 5A77 0558 DC7A 790A 16DF EC5B BB9D 2C24 ID: BB9D2C24 I disclaim, therefore I am.

3 3

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

openldap-technical