openldap-technical

openldap-technical@openldap.org

9 participants
9876 discussions

Performance issues lately.
by Jorgen Lundman 14 Nov '10

14 Nov '10

Hello list, Solaris 10u8 db-4.8.30.NC openldap-2.4.23 So in the last week, simple queries have been really slow. For example: # time /usr/local/bin/ldapsearch -h 0 -D cn=admin,dc=company,dc=com -x -w pass -b ou=mail,dc=company,dc=com mail=testlookup(a)domain.com 1m21s Of course "mail" in indexed. The server is dedicated to LDAP (test server) so there are no other connections. Not even syncrepl connection. I have checked the usual things, memory usage, BDB cache, and disk IO. All seem to be within reasonable limits. Then I went deeper. I started trussing it, and it seems to send the query reply nearly right away, then sit around 200 seconds before we send a few more bytes and exit. I tried out db_stat on various db, and found it is also really slow on stat on "id2entry.bdb". For example: # time /usr/local/BerkeleyDB.4.8/bin/db_stat -d id2entry.bdb 2m32s So, trussing that call (including all calls in libdb-4.2): 19700/1@1: 0.0001 -> libdb-4.8:__db_stat_print_pp(0x419920, 0x0, 0x412c30, 0x4146a0, 0xfffffd7fffdff880, 0xfffffd7fffdff824) 19700/1: 0.0009 time() = 128972 5450 19700/1: 0.0001 brk(0x0041EC20) = 0 19700/1: 0.0001 brk(0x00422C20) = 0 19700/1: 0.0001 open("/usr/share/lib/zoneinfo/Japan", O_RDONLY) = 4 19700/1: 0.0000 fstat(4, 0xFFFFFD7FFFDFF880) = 0 19700/1: 0.0001 read(4, 0xFFFFFD7FFEEA0880, 125) = 125 19700/1: T Z i f\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\003\0\0\003\0\0\0 \0 19700/1: \0\0\0\t\0\0\003\0\0\0\fC3CE85 pD7 >1E90D7EC1680D8F91690D9CBF8 80 19700/1: DB071D10DBABDA80DCE6FF10DD8BBC80020102010201020102\0\0 ~90\0\0 \0 19700/1: \08CA00104\0\0 ~90\0\b C J T\0 J D T\0 J S T\0\0\0\0\0\0\0 19700/1: 0.0000 close(4) = 0 19700/1: 0.0002 ioctl(1, TCGETA, 0xFFFFFD7FFFDFE8B0) = 0 19700/1: 0.0000 fstat(1, 0xFFFFFD7FFFDFE840) = 0 19700/1: 0.0001 write(1, 0xFFFFFD7FFF006764, 36) = 36 19700/1: S u n N o v 1 4 1 8 : 0 4 : 1 0 2 0 1 0\t L o c a l t 19700/1: i m e\n 19700/1: 589.8794 write(1, 0xFFFFFD7FFF006764, 25) = 25 19700/1: 5 3 1 6 2\t B t r e e m a g i c n u m b e r\n 19700/1: 0.0046 write(1, 0xFFFFFD7FFF006764, 23) = 23 Not entirely sure why a write (everything leading up) to stdout takes 589 seconds. I have tried rebuilding the whole database with slapadd but made no difference. I put all DB files in /tmp on x4540 (has 32G RAM) and yet, still 2mins to do db_stat. General db_stat -m output: 4GB Total cache size 8 Number of caches 8 Maximum number of caches 512MB Pool individual cache size 0 Maximum memory-mapped file size 0 Maximum open file descriptors 0 Maximum sequential buffer writes 0 Sleep after writing maximum sequential buffers 0 Requested pages mapped into the process' address space 407M Requested pages found in the cache (99%) 38 Requested pages not found in the cache 425405 Pages created in the cache 38 Pages read into the cache 425427 Pages written from the cache to the backing file 0 Clean pages forced from the cache 0 Dirty pages forced from the cache 0 Dirty pages written by trickle-sync thread 425427 Current total page count 425427 Current clean page count 0 Current dirty page count 524296 Number of hash buckets used for page location 4096 Assumed page size used 407M Total number of times hash chains searched for a page (407904893) 32 The longest hash chain searched for a page 466M Total number of hash chain entries checked for page (466861223) 0 The number of hash bucket locks that required waiting (0%) 0 The maximum number of times any hash bucket lock was waited for (0%) 0 The number of region locks that required waiting (0%) 0 The number of buffers frozen 0 The number of buffers thawed 0 The number of frozen buffers freed 425538 The number of page allocations 0 The number of hash buckets examined during allocations 0 The maximum number of hash buckets examined for an allocation 0 The number of pages examined during allocations 0 The max number of pages examined for an allocation 0 Threads waited on page I/O 0 The number of times a sync is interrupted Pool File: id2entry.bdb 16384 Page size 0 Requested pages mapped into the process' address space 13M Requested pages found in the cache (99%) 2 Requested pages not found in the cache 163449 Pages created in the cache 2 Pages read into the cache 163451 Pages written from the cache to the backing file Pool File: mail.bdb 4096 Page size 0 Requested pages mapped into the process' address space 2797594 Requested pages found in the cache (99%) 2 Requested pages not found in the cache 3335 Pages created in the cache 2 Pages read into the cache 3337 Pages written from the cache to the backing file -rw-r--r-- 1 root root 168 Nov 14 12:54 DB_CONFIG -rw------- 1 root root 3.8M Nov 14 14:03 DNSData.bdb -rw------- 1 root root 5.4M Nov 14 14:03 DNSHostName.bdb -rw------- 1 root root 3.3M Nov 14 14:03 DNSIPAddr.bdb -rw------- 1 root root 2.7M Nov 14 14:03 DNSType.bdb -rw------- 1 root root 24K Nov 14 12:54 __db.001 -rw------- 1 root root 180M Nov 14 18:28 __db.002 -rw------- 1 root root 512M Nov 14 13:45 __db.003 -rw------- 1 root root 512M Nov 14 14:35 __db.004 -rw------- 1 root root 512M Nov 14 14:40 __db.005 -rw------- 1 root root 512M Nov 14 13:45 __db.006 -rw------- 1 root root 512M Nov 14 13:45 __db.007 -rw------- 1 root root 512M Nov 14 13:45 __db.008 -rw------- 1 root root 512M Nov 14 14:35 __db.009 -rw------- 1 root root 512M Nov 14 14:27 __db.010 -rw------- 1 root root 160K Nov 14 12:54 __db.011 -rw------- 1 root root 1.2M Nov 14 18:25 __db.012 -rw------- 1 root root 32K Nov 14 12:54 __db.013 -rw------- 1 root root 2.1M Nov 14 14:03 accountStatus.bdb -rw-r--r-- 1 root root 2.0K Nov 14 16:12 alock -rw------- 1 root root 536K Nov 14 14:03 deliveryMode.bdb -rw------- 1 root root 621M Nov 14 14:07 dn2id.bdb -rw------- 1 root root 172M Nov 14 14:03 entryCSN.bdb -rw------- 1 root root 125M Nov 14 14:03 entryUUID.bdb -rw------- 1 root root 17M Nov 14 14:03 gecos.bdb -rw------- 1 root root 1.8M Nov 14 14:03 gidNumber.bdb -rw------- 1 root root 2.5G Nov 14 14:40 id2entry.bdb -rw------- 1 root root 50M Nov 14 16:12 log.0000000266 -rw------- 1 root root 13M Nov 14 14:03 mail.bdb -rw------- 1 root root 3.2M Nov 14 14:03 mailAlternateAddress.bdb -rw------- 1 root root 8.7M Nov 14 14:03 o.bdb -rw------- 1 root root 4.9M Nov 14 14:03 objectClass.bdb -rw------- 1 root root 1.3M Nov 14 14:03 radiusGroupName.bdb -rw------- 1 root root 20M Nov 14 14:03 uid.bdb -rw------- 1 root root 17M Nov 14 14:03 uidNumber.bdb checkpoint 128 15 cachesize 5000 idlcachesize 15000 overlay syncprov syncprov-checkpoint 100 10 syncprov-sessionlog 1000 # DB_CONFIG FILE. Warning, these entries are only written if the DB_CONFIG # file DOES NOT EXIST. So if you change any of these values, you will need # to remove the file, then restart slapd. dbconfig set_lk_detect DB_LOCK_DEFAULT dbconfig set_lg_max 52428800 dbconfig set_cachesize 4 0 8 dbconfig set_flags db_log_autoremove dbconfig set_lk_max_objects 1500 dbconfig set_lk_max_locks 1500 dbconfig set_lk_max_lockers 1500 # DB_CONFIG FILE. Any suggestions? Must have hit some limit somewhere I guess. Lund

4 9

Chaining not working
by Jaap Winius 14 Nov '10

14 Nov '10

Hi folks, While testing the current Debian squeeze version of OpenLDAP, v2.4.23-6, in a provider/consumer syncprov/syncrepl (refreshAndPersist) configuration, using a patch(1) written by Pierangelo, I have not been able to get chaining to work. The consumer, ldaps2, was configured with a referral(2) to the provider, ldaps1, as well as a chaining configuration(3). A couple of authzTo rules(4) were added to its entry in the DIT, which immediately replicated to the consumer, and the provider was configured with an olcAuthzPolicy directive for "to"(5). So far, so good. However, when using ldapmodify on the consumer to test that an entry in the DIT could actually be modified (the description attr of the consumer's entry) from there as a result, I got this response: ------------------------------------------------------------ modifying entry "cn=ldaps2,dc=example,dc=com" ldap_modify: Referral (10) referrals: ldap://ldaps.example.com/cn=ldaps2,dc=example,dc=com ------------------------------------------------------------ I know ldapmodify doesn't understand referrals; this is where chaining should have worked instead. So, I removed the referral from the consumer's configuration to see what would then happen with the same command: ------------------------------------------------------------ modifying entry "cn=ldaps2,dc=example,dc=com" ldap_modify: Server is unwilling to perform (53) additional info: shadow context; no update referral ------------------------------------------------------------ (shadow context?). In both cases, this shows up in the syslog as a result: ------------------------------------------------------------ Nov 12 00:48:54 ldaps2 slapd[23862]: conn=1002 fd=19 ACCEPT from IP=127.0.1.1:43982 (IP=0.0.0.0:389) Nov 12 00:48:54 ldaps2 slapd[23862]: conn=1002 op=0 BIND dn="cn=admin,dc=example,dc=com" method=128 Nov 12 00:48:54 ldaps2 slapd[23862]: conn=1002 op=0 BIND dn="cn=admin,dc=example,dc=com" mech=SIMPLE ssf=0 Nov 12 00:48:54 ldaps2 slapd[23862]: conn=1002 op=0 RESULT tag=97 err=0 text= Nov 12 00:48:54 ldaps2 slapd[23862]: conn=1002 op=1 MOD dn="cn=ldaps2,dc=example,dc=com" Nov 12 00:48:54 ldaps2 slapd[23862]: conn=1002 op=1 MOD attr=description Nov 12 00:48:54 ldaps2 slapd[23862]: conn=1002 op=1 RESULT tag=103 err=53 text=shadow context; no update referral Nov 12 00:48:54 ldaps2 slapd[23862]: conn=1002 op=2 UNBIND Nov 12 00:48:54 ldaps2 slapd[23862]: conn=1002 fd=19 closed ------------------------------------------------------------ Have I made a mistake somewhere, or could this be another bug? Thanks, Jaap 1) ftp://ftp.openldap.org/incoming/pierangelo-masarati-2010-04-29-chain.1.patch 2) LDIF applied to ldaps2 (the consumer) to create the referral to ldaps1 (the provider) via an alias (ldaps): --------------------------------- dn: olcDatabase={1}hdb,cn=config changetype: modify add: olcUpdateref olcUpdateref: ldap://ldaps.example.com --------------------------------- 3) LDIF applied to ldaps2 to create the chaining configuration: --------------------------------- dn: cn=module{0},cn=config changetype: modify add: olcModuleLoad olcModuleLoad: {1}back_ldap dn: olcOverlay={0}chain,olcDatabase={1}hdb,cn=config objectClass: olcOverlayConfig objectClass: olcChainConfig olcOverlay: {0}chain olcChainReturnError: TRUE dn: olcDatabase={0}ldap,olcOverlay={0}chain,olcDatabase={1}hdb,cn=config objectClass: olcLDAPConfig objectClass: olcChainDatabase olcDatabase: {0}ldap olcDbURI: ldap://ldaps.example.com olcDbRebindAsUser: TRUE olcDbIDAssertBind: bindmethod=simple binddn="cn=ldaps2,dc=example,dc=com" credentials=bilineatus mode=self --------------------------------- 4) LDIF to create a couple of authzTo rules for the consumer: --------------------------------- dn: cn=ldaps2,dc=example,dc=com changetype: modify add: authzTo authzTo: {0}dn.regex:^uid=[^,]+,ou=people,dc=example,dc=com$ authzTo: {1}dn.exact:cn=admin,dc=example,dc=com --------------------------------- 5) LDIF to add an olcAuthzPolicy directive to the provider, ldaps1: --------------------------------- dn: cn=config changetype: modify add: olcAuthzPolicy olcAuthzPolicy: to ---------------------------------

2 2

Re: Trying to understand an error message
by Khaled Blah 13 Nov '10

13 Nov '10

Thanks for your reply, Hallvard. Do I understand it correctly that I can re-use LDAP structures on which a bind has already been done? Do you know of any other stateful data I would have to look at? Regards, Khaled 2010/11/12 Hallvard B Furuseth <h.b.furuseth(a)usit.uio.no>: > Khaled Blah writes: >> My application creates LDAP structures (using ldap_init) and re-uses >> these structures whenever possible so that the least number of LDAP >> binds have to done. Is there anything I need to do after an LDAP >> operation has finished or maybe do I have to unbind at some point to >> make sure the LDAP structure remains in a valid state? I would be >> happy to receive any insight into this. > > With LDAP *ld, call ldap_unbind(ld) when you are finished with ld, and > not before. ldap_unbind() is actually the destructor for an LDAP*, in > addition to sending an Unbind message. > > -- > Hallvard >

2 2

Out of memory Errors when starting up Slurpd: Openldap-server-2.3.43
by Traiano Welcome 13 Nov '10

13 Nov '10

Hi List We're using openldap-server 2.3.43 on FreeBSD 6.2-RELEASE-p9 #0. When trying to start up slurpd (/usr/local/etc/rc.d/slurpd start), slurpd runs for a while then dies at various points during it's replication with error "Out of memory". The following are various attempts to start slurpd, with the last messages it dies with before "Out of Memory". My question is: Is this a known bug? What could be the cause of these out of memory errors in Slurpd? Test one: --- Initializing session to ldap://server.ops.aanet.co.za:389 Error: ldap_initialize(0, ldap://server.ops.aanet.co.za:389) failed: Out of memory --- Test two: --- malloc of 3 bytes failed Error: ldap_modify_s failed modifying DN "uid=sherbert.buth,ou=mycast.co.za,ou=AuthSystem,dc=za,dc=oo,dc=com": Out of memory --- Test three: ---- Error: ldap operation failed, data written to "/var/db/openldap-slurp/replica/server.ops.aanet.co.za:389.rej" ---- Test four: --- Error: ldap_modify_s failed modifying DN "uid=bruce,ou=fatnet.co.za,ou=AuthSystem,dc=za,dc=uu,dc=net": Out of memory --- Test 5: --- Error: ldap operation failed, data written to "/var/db/openldap-slurp/replica/server.ops.aanet.co.za:389.rej" --- Test 6: --- ldif_parse_line: type malloc failed Error: malformed replog line "replace: sn" Warning: freeing re (dn: uid=dericks,ou=anb.co.za,ou=AuthSystem,dc=za,dc=uu,dc=net) with nonzero refcnt error: malformed replog entry (begins with "replica: server.ops.aanet.co.za:389") Error: ldap_modify_s failed modifying DN "uid=tyrone,ou=business.net,ou=AuthSystem,dc=za,dc=,dc=net": Out of memory Error: ldap operation failed, data written to "/var/db/openldap-slurp/replica/server.ops.aanet.co.za:389.rej" ldif_parse_line: value malloc failed Error: malformed replog line "olcPasswordHash: 92XXXXXXXXXXXXXXXXXXXX" Warning: freeing re (dn: uid=adi,ou=anb.co.za,ou=AuthSystem,dc=za,dc=oo,dc=net) with nonzero refcnt error: malformed replog entry (begins with "replica: server.ops.aanet.co.za:389") malloc of 12 bytes failed malloc of 8 bytes failed --- Thanks in Advance, Traiano Welcome

2 1

Assigning a Member to Multiple Groups
by Anton Chu 12 Nov '10

12 Nov '10

Unfortunately there's can only be one GID number per user entry into the DIT, how can I assign multiple groups to one user? So when that user logs into a server, he or she will have multiple group IDs.

2 1

How to convert Solaris m5 passwords to LDAP?
by Christian Schmidt 12 Nov '10

12 Nov '10

Hi all, we want to switch a server machine from Solaris (credentials stored in "traditional" passwd and shadow file) to Debian with OpenLDAP for authentication. Creating LDIF files from /etc/passwd and /etc/shadow using PADL's migrationtools is working fine. The only problem is, that many user passwords on the Solaris machine have been encrypted using Sun's md5 scheme which results in hashes beginning with the characters "$md5$". These hashes can be "imported" into our LDAP directory, but they cannot be used for authentication: Each attempt results in "access denied" on the client side and LDAP bind errors on the server side. Even when adding the user information to /etc/passwd and /etc/shadow on the Linux machine, there's no success. With CRYPT password hashes, everything works fine. Do you know any means to "convert" these Solaris-md5-hashed password strings into something we can use with OpenLDAP? I appreciate your helpful answers. Thanks in advance! Gruss/Regards, Christian Schmidt -- You have an ability to sense and know higher truth.

4 7

Syncrepl from OSX slapd to Centos5 slapd
by Markku Tavasti 12 Nov '10

12 Nov '10

Hi! We have OSX as primary ldap server (slapd 2.3.27). Schema is OSX tuned schema, with Apple provided extensions. I'm trying to set up backup-ldap to Centos5 server (slapd 2.3.43). I have copied schemas from OSX /etc/openldap/schema, and edited & commented out attributetype 'authAuthority' definition from apple.schema. OSX is running with configuration from cn=config, so how I can check if schema it uses is 100% same as in schema files? Here is my syncrepl config on client side: syncrepl rid=23 provider=ldap://xxxx.domain.com type=refreshOnly interval=00:00:10:00 retry="60 10 300 +" searchbase="dc=xxxx,dc=domain,dc=com" scope=sub schemachecking=off bindmethod=simple binddn="uid=replicator,cn=users,dc=xxxx,dc=domain,dc=com" credentials=secret Client-slapd starts ok, and replication gets only one entry: # xxxx, computers, xxxx.domain.com dn: cn=xxxx,cn=computers,dc=xxxx,dc=domain,dc=com To log on client side I get only these: Nov 11 05:31:46 xxxx slapd[13962]: do_syncrep2: rid 023 LDAP_RES_SEARCH_RESULT Nov 11 05:41:46 xxxx slapd[13962]: daemon: epoll: listen=7 active_threads=0 tvp =zero Nov 11 05:41:46 xxxx slapd[13962]: daemon: epoll: listen=8 active_threads=0 tvp =zero Any ideas what is problem? -- M. Tavasti / tavasti(a)tavasti.fi / +358-40-5078254

1 1

Trying to understand an error message
by Khaled Blah 12 Nov '10

12 Nov '10

Hello all, I have written an application which uses OpenLDAP and I keep receiving this error message: "sasl.c:75: ldap_sasl_bind: Assertion `( (ld)->ld_options.ldo_valid == 0x2 )' failed." The code at this position says the following: "assert( LDAP_VALID( ld ) );" My application creates LDAP structures (using ldap_init) and re-uses these structures whenever possible so that the least number of LDAP binds have to done. Is there anything I need to do after an LDAP operation has finished or maybe do I have to unbind at some point to make sure the LDAP structure remains in a valid state? I would be happy to receive any insight into this. Regards, Khaled

1 0

German Umlauts are converted and also not accepted
by Götz Reinicke - IT-Koordinator 11 Nov '10

11 Nov '10

Hallo, I'v searched for an answer but did not found one yet. I do have a openldap-2.3.43 server (centos 5.5.) in test and tried to add a ldif-file with my data using the apache directory studio (on mac os x). As my name contains a german umlaut I do get an error adding 'Götz Reinicke' for the gecko. Also for 'displayName', 'cn' and 'givenName' the 'ö' is automatically replaced by 'oe'. If I change the entry in the ldap by hand in the apache DS, the 'ö' is accepted accept for the geckos. An 'ü' in 'o' is manually accepted to. Any hints, explanations, suggestions? Thanks and best regards, Götz -- Götz Reinicke IT-Koordinator Tel. +49 7141 969 420 Fax +49 7141 969 55 420 E-Mail goetz.reinicke(a)filmakademie.de Filmakademie Baden-Württemberg GmbH Akademiehof 10 71638 Ludwigsburg www.filmakademie.de Eintragung Amtsgericht Stuttgart HRB 205016 Vorsitzende des Aufsichtsrats: Prof. Dr. Claudia Hübner Geschäftsführer: Prof. Thomas Schadt

3 3

olcDbURI error
by Jaap Winius 11 Nov '10

11 Nov '10

Hi folks, After applying some changes to a consumer server used for testing purposes, my attempts to run slapcat result in the following error: #################################### slapd-chain: first underlying database "olcDatabase={0}ldap,olcOverlay={0}chain,olcDatabase={1}hdb,cn=config" cannot contain attribute "olcDbURI". config error processing olcDatabase={0}ldap,olcOverlay={0}chain,olcDatabase={1}hdb,cn=config: slapcat: bad configuration file! #################################### (Debian squeeze, slapd v2.4.23-6) After having configure syncrepl on the same host, I used slapcat to compare its newly replicated database to the one on the provider. At that point it still worked and the databases proved to be identical. I then proceeded to configure chaining and a referral. The above slapcat error appeared after I made the following changes (which applied successfully): ########################################################################### # 1. dn: olcDatabase={1}hdb,cn=config changetype: modify add: olcUpdateref olcUpdateref: ldap://ldaps.example.com # 2. dn: cn=module{0},cn=config changetype: modify add: olcModuleLoad olcModuleLoad: {1}back_ldap # 3. dn: olcOverlay={0}chain,olcDatabase={1}hdb,cn=config objectClass: olcOverlayConfig objectClass: olcChainConfig olcOverlay: {0}chain olcChainReturnError: TRUE # 4. dn: olcDatabase={0}ldap,olcOverlay={0}chain,olcDatabase={1}hdb,cn=config objectClass: olcLDAPConfig objectClass: olcChainDatabase olcDatabase: {0}ldap olcDbURI: ldap://ldaps.example.com olcDbRebindAsUser: TRUE olcDbIDAssertBind: bindmethod=simple binddn="cn=ldaps2,dc=example,dc=com" credentials=bilineatus mode=self ########################################################################### Afterwards, slapd could no longer be restarted either, with the same error showing up in the syslog. I am aware that this is a known bug: http://www.openldap.org/its/index.cgi/Software%20Bugs?id=6540;selectid=6540 According to the report, a patch was made available on April the 29th, but I'm not sure if it actually fixed anything. At any rate, my Debian slapd version is dated from the 23rd of September and still has this problem. However, Followup 17 (31 July) in the bug report refers to a "simple fix". Unfortunately, I haven't yet been able to find what that's supposed to be. This seems to me to be a serious bug, but for the time being I'd be happy with simple workaround. Can anyone help? Thanks, Jaap

2 3

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

openldap-technical