openldap-technical April 2013

openldap-technical@openldap.org

69 participants
74 discussions

How to improve performance with MDB backend?
by Saša-Stjepan Bakša 08 Apr '13

08 Apr '13

Hi, Maybe this is not the best way to ask but I would like to get some performance expectations or maybe suggestions how to improve performance. I do have relatively long experience with OpenLDAP as a precompiled package and with much less users so performance was not an issue for those installations. Now I need to put few million users (now one million for test), custom tailored schema and search performance is crucial but also modify performance is big issue. Before MDB I have used HDB as backend. What to do to improve write part of performance or performance in general (when adding data – from time to time – OpenLDAP stalls in a way)? Just around 5 add/mod operations during that time then it continue with much higher speed. I have used many different sources to find out other peoples experience and I didn’t choose to write to list lightly but I really need some help/hints. As a hardware I am using two ProLiant DL360p Gen8 servers with: 48GB RAM, HP Smart Array P420i Controller, 2 x EG0600FBDSR 558 GB hard disk in RAID 1. 2 Ethernet ports are bonded to achieve redundancy and performance and connected to Cisco 3750 switch OS is Ubuntu 12.04.2 LTS server with no unnecessary daemons installed. Disk is partitioned this way. Filesystem Size Used Avail Use% Mounted on /dev/sda1 46G 1.4G 43G 4% / udev 24G 4.0K 24G 1% /dev tmpfs 9.5G 240K 9.5G 1% /run none 5.0M 0 5.0M 0% /run/lock none 24G 0 24G 0% /run/shm /dev/sda3 232G 32G 189G 15% /var /dev/sda4 230G 2.4G 216G 2% /opt OpenLDAP is latest one – 2.4.35 compiled on Ubuntu server with addition of Berkley db 5.3.21 (I am not using it but …) and no special switches are used during configuration except the on which puts installation to /opt/openldap directory. I am using also libhoard.so as a memory manager (latest one downloaded from HOARD web site). OpenLDAP is configured as MultiMaster N Way and I am using MDB as backend database. Config is in conf.d stile. root@spr1:~# more /opt/openldap/etc/openldap/slapd.d/cn\=config.ldif # AUTO-GENERATED FILE - DO NOT EDIT!! Use ldapmodify. # CRC32 47664f80 dn: cn=config objectClass: olcGlobal cn: config olcArgsFile: /var/run/slapd/slapd.args olcLogLevel: none olcPidFile: /var/run/slapd/slapd.pid structuralObjectClass: olcGlobal creatorsName: cn=config entryUUID: a8df00ce-80fb-1031-8652-fff9cf1d6a3e createTimestamp: 20120822232009Z olcServerID: 1 ldap://spr1.lab.os olcServerID: 2 ldap://spr2.lab.os olcIdleTimeout: 5 olcThreads: 6 entryCSN: 20130404141239.680528Z#000000#001#000000 modifiersName: cn=admin,cn=config modifyTimestamp: 20130404141239Z contextCSN: 20130404141404.368953Z#000000#001#000000 contextCSN: 20130329141804.133907Z#000000#002#000000 And database: root@spr1:~# more /opt/openldap/etc/openldap/slapd.d/cn\=config/olcDatabase={1}mdb.ldif # AUTO-GENERATED FILE - DO NOT EDIT!! Use ldapmodify. # CRC32 405c8aaf dn: olcDatabase={1}mdb objectClass: olcDatabaseConfig objectClass: olcMdbConfig olcDatabase: {1}mdb olcDbDirectory: /var/lib/ldap olcSuffix: dc=spr olcAccess: {0}to attrs=userPassword,shadowLastChange by self write by anonymou s auth by dn="cn=admin,dc=spr" write by * none olcAccess: {1}to attrs=shadowLastChange by self write by * read olcAccess: {2}to dn.base="" by * read olcAccess: {3}to * by self write by dn="cn=admin,dc=spr" write by * read olcLastMod: TRUE olcRootDN: cn=admin,dc=spr olcRootPW:: e1NTSEF9eDRVRTBiV2Z4YnFSNnZDVDdKRWEwSWRhWFRhMDN2M3I= olcDbNoSync: TRUE olcDbMaxSize: 107374182400 structuralObjectClass: olcMdbConfig entryUUID: 804a8ede-2cc3-1032-9b7a-c7b2eb845e03 creatorsName: gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth createTimestamp: 20130329135129Z olcDbIndex: objectClass eq olcDbIndex: uid eq olcDbIndex: MSISDN eq olcDbIndex: IMSI eq olcDbIndex: pfUsername eq olcDbIndex: entryCSN eq olcDbIndex: entryUUID eq olcDbIndex: contextCSN eq olcMirrorMode: TRUE olcSyncrepl: {0}rid=003 provider=ldap://spr1.lab.os binddn="cn=admin,dc=spr" b indmethod=simple credentials=siemens searchbase="dc=spr" type=refreshOnly int erval=00:00:00:10 retry="5 5 300 5" timeout=1 olcSyncrepl: {1}rid=004 provider=ldap://spr2.lab.os binddn="cn=admin,dc=spr" b indmethod=simple credentials=siemens searchbase="dc=spr" type=refreshOnly int erval=00:00:00:10 retry="5 5 300 5" timeout=1 olcDbCheckpoint: 8192 15 entryCSN: 20130404141404.368953Z#000000#001#000000 modifiersName: cn=admin,cn=config modifyTimestamp: 20130404141404Z This is output from iotop during ldapmodify test (1 mil user, two clients modifying each user in sequence , 1 DN / 1 attribute per user): Total DISK READ: 0.00 B/s | Total DISK WRITE: 15.30 M/s TID PRIO USER DISK READ DISK WRITE SWAPIN IO> COMMAND 4403 be/4 root 0.00 B/s 0.00 B/s 0.00 % 98.88 % [flush-8:0] 4838 be/4 openldap 0.00 B/s 3.41 M/s 0.00 % 21.68 % slapd -h ldap:/// -F /opt/openldap/etc/~ldap/slapd.d -g openldap -u openldap -4 4837 be/4 openldap 0.00 B/s 3.19 M/s 0.00 % 19.13 % slapd -h ldap:/// -F /opt/openldap/etc/~ldap/slapd.d -g openldap -u openldap -4 4836 be/4 openldap 0.00 B/s 2.83 M/s 0.00 % 17.48 % slapd -h ldap:/// -F /opt/openldap/etc/~ldap/slapd.d -g openldap -u openldap -4 10889 be/4 openldap 0.00 B/s 2.94 M/s 0.00 % 16.30 % slapd -h ldap:/// -F /opt/openldap/etc/~ldap/slapd.d -g openldap -u openldap -4 10962 be/4 openldap 0.00 B/s 2.92 M/s 0.00 % 15.25 % slapd -h ldap:/// -F /opt/openldap/etc/~ldap/slapd.d -g openldap -u openldap -4 1816 be/3 root 0.00 B/s 0.00 B/s 0.00 % 3.23 % [jbd2/sda3-8] And top: top - 11:56:27 up 4 days, 1:43, 1 user, load average: 2.71, 2.88, 3.19 Tasks: 224 total, 1 running, 223 sleeping, 0 stopped, 0 zombie Cpu(s): 2.6%us, 0.1%sy, 0.0%ni, 91.3%id, 6.0%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 49424228k total, 35362892k used, 14061336k free, 127844k buffers Swap: 46874876k total, 0k used, 46874876k free, 32631568k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 4833 openldap 20 0 101g 30g 29g S 91 64.1 196:30.20 slapd 4403 root 20 0 0 0 0 D 2 0.0 2:41.89 flush-8:0 1 root 20 0 24472 2444 1360 S 0 0.0 0:04.36 init 2 root 20 0 0 0 0 S 0 0.0 0:00.03 kthreadd 3 root 20 0 0 0 0 S 0 0.0 0:00.37 ksoftirqd/0

3 2

Loosing existing entries when implementing syncrepl N-Way Multi-Master Replication (openldap-2.4.32)
by Joe Phan 07 Apr '13

07 Apr '13

Hi, I have some general questions related to syncrepl N-Way Multi-Master Replication (openldap-2.4.32 on Solaris 10). I already configured a LDAP server and added user entries to DIT. From a client machine, I am able to retrieve user entries via ldapsearch, and login using LDAP user entries is also successful. When I try to configure the LDAP server to be syncrepl N-Way Multi-Master Replication - Server1, existing entries in DIT are gone. From the client machine, ldapsearch command returns empty. - For existing entries in DIT, can we keep them when implementing syncrepl N-Way Multi-Master Replication or we must create them again from the beginning? Regards, Joe

2 1

Re: Anonymous Bind ACL Problems
by Shawn Morford 07 Apr '13

07 Apr '13

On Fri, Apr 5, 2013 at 1:11 PM, Quanah Gibson-Mount <quanah(a)zimbra.com>wrote: > --On Friday, April 05, 2013 12:46 PM -0700 Dark Morford < > darkmorford+ldap(a)gmail.com> wrote: > > I'm setting up my first LDAP server; just using it as an auth provider >> for Apache until I'm more comfortable with things. I was able to get it >> up and running with a few user entries, but I can't get anonymous >> searching to work the way I want. >> >> It's configured (cn=config) style, and the ACLs are: >> {0}to attrs=uid by anonymous read by users read >> {1}to attrs=userPassword by anonymous auth by self write >> {2}to * by users read >> > > access to entry by * read needs to be in there too before {2}. > > --Quanah > > -- > > Quanah Gibson-Mount > Sr. Member of Technical Staff > Zimbra, Inc > A Division of VMware, Inc. > -------------------- > Zimbra :: the leader in open source messaging and collaboration > Adding the access entry exactly like you have it gave me an error; I managed to figure out that it needed to be 'to attrs=entry by * read'. And now it seems to be working, so thanks for that. I'm not sure I understand why it's necessary, though. The client service (Apache) just needs to find out if a particular uid exists. Why does it need access to the whole entry?

1 0

None
by Bob Fitch 06 Apr '13

06 Apr '13

http://www.octevillenatation.com/nnqv/xdxapmiuaxur.lsu 4/6/2013 8:23:01 PM 4/6/2013 8:23:01 PM fitchbob

1 0

Anonymous Bind ACL Problems
by Dark Morford 05 Apr '13

05 Apr '13

I'm setting up my first LDAP server; just using it as an auth provider for Apache until I'm more comfortable with things. I was able to get it up and running with a few user entries, but I can't get anonymous searching to work the way I want. It's configured (cn=config) style, and the ACLs are: {0}to attrs=uid by anonymous read by users read {1}to attrs=userPassword by anonymous auth by self write {2}to * by users read Searching for a user as the rootDN works fine: shawn@aquamarine:~$ ldapsearch -x -D 'cn=Manager,dc=darkmorford,dc=net' -W -b 'dc=darkmorford,dc=net' '(uid=smorford)' uid Enter LDAP Password: # extended LDIF # # LDAPv3 # base <dc=darkmorford,dc=net> with scope subtree # filter: (uid=smorford) # requesting: uid # # smorford, Users, darkmorford.net dn: uid=smorford,ou=Users,dc=darkmorford,dc=net uid: smorford # search result search: 2 result: 0 Success # numResponses: 2 # numEntries: 1 But doing the same search anonymously can't find the user: shawn@aquamarine:~$ ldapsearch -x -b 'dc=darkmorford,dc=net' '(uid=smorford)' uid # extended LDIF # # LDAPv3 # base <dc=darkmorford,dc=net> with scope subtree # filter: (uid=smorford) # requesting: uid # # search result search: 2 result: 32 No such object # numResponses: 1 I have to assume that something in the ACL is blocking the anonymous search. How do I fix this?

2 1

Replication using cn=config
by Andre Rodier 05 Apr '13

05 Apr '13

Hello Everyone, I am trying to implement mirorring replication on OpenLDAP on debian wheezy, and I am struggling since a few days. All the documentation I have found so far on the web site is related to OpenLDAP with a configuration file (slapd.conf) Can anyone gives me a clue on how I can adapt the examples given on the web site to the database configuration backend (cn=config). Thanks for your help. André Rodier.

3 3

How to use LDAP to get user information from MySql database?
by jupiter 05 Apr '13

05 Apr '13

Hi, I have a user management which using MySql database. How can I add and configure LDAP to get user information (username, password, email, public key ....) which installed in MySql database? I know there is a shell in LDAP configuration which can be used to get information from external database such as MySql, but I am not sure if it is the good way or only way to run LDAP and MySql together. Appreciate any advice. Thank you. Kind regards. j

3 3

Re: openldap-technical Digest, Vol 64, Issue 28
by Francois Gnu 03 Apr '13

03 Apr '13

Hello Bill, I'm very interesting to know your method to create a debian package. Thanks a lot! Librement, ------ Francois Trachez (kiko) Team Fedora|Lyon (France) http://stg.fedoraproject.org/fr/ http://stg.fedoraproject.org/es/ 2013/3/28 <openldap-technical-request(a)openldap.org>: > > --On Tuesday, March 26, 2013 06:14:33 PM -0700 Quanah Gibson-Mount <quanah(a)zimbra.com> wrote: > >> If you know how to build OpenLDAP manually, and would like to participate in testing the next set of code for the 2.4.35 release, please do so. >> >> Generally, get the code for RE24: >> >> <http://www.openldap.org/devel/gitweb.cgi?p=openldap.git;a=snapshot;h=refs/h…> >> >> Configure & build. >> >> Execute the test suite (via make test) after it is built. > > Created a debian package for wheezy and installed it on our development > servers. In addition to passing all of the build tests, my initial > tests have all been successful. > > To build the package I did have to update ol_patch in version.var. > > Bill > > -- > > Bill MacAllister > Infrastructure Delivery Group, Stanford University > > > > ------------------------------ > > _______________________________________________ > openldap-technical mailing list > openldap-technical(a)openldap.org > http://www.openldap.org/lists/mm/listinfo/openldap-technical > > > End of openldap-technical Digest, Vol 64, Issue 28 > **************************************************

1 0

::Bob Fitch::
by Bob Fitch 03 Apr '13

03 Apr '13

http://www.drkaria.com/xecoyox/tose.keyh?pni 4/3/2013 2:28:49 PM 4/3/2013 2:28:49 PM fitchbob

1 0

Chaining without TLS
by jeevan kc 02 Apr '13

02 Apr '13

I've configured a chaining overlay on two slaves with a master to enable update privileges on the slave servers. But every-time I try to modify entries on the Slave I get error code 8.(strong authentication required) Can chaining work without TLS ? If so how? Also I created the overlay only on the slaves ( I looked at the references and it's obvious it shouldn't be on the master). Please help.

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

openldap-technical April 2013