openldap-software December 2009

openldap-software@openldap.org

28 participants
29 discussions

searching for attributes without index in 2.4.19 with bdb 4.5
by Christoph Herrmann 09 Feb '10

09 Feb '10

Hello, after upgrading from openldap-2.3.39 with bdb-4.2.52 to openldap 2.4.19 with bdb-4.5 searching for attributes without index is about three times slower. (same machine, same data, all data fit in DB cache) Are there any known problems or magic tuning options we have missed? regards Christoph &:-) -- Vorstand/Board of Management: Dr. Bernd Finkbeiner, Dr. Roland Niemeier, Dr. Arno Steitz, Dr. Ingrid Zech Vorsitzender des Aufsichtsrats/ Chairman of the Supervisory Board: Michel Lepert Sitz/Registered Office: Tuebingen Registergericht/Registration Court: Stuttgart Registernummer/Commercial Register No.: HRB 382196

1 1

dynlist overlay and ldapsearch
by ben thielsen 03 Feb '10

03 Feb '10

hi- i'm using the dynlist overlay and am not getting back the search results i expected. i'm using 2.4.11 courtesy of debian. here is my overlay config: >ldapsearch -xWLLLD 'cn=admin,cn=config' -b 'cn=config' "(objectclass=olcdynamiclist)" dn: olcOverlay={5}dynlist,olcDatabase={2}bdb,cn=config objectClass: olcOverlayConfig objectClass: olcDynamicList olcOverlay: {5}dynlist olcDLattrSet: {0}groupOfNames memberURL member olcDLattrSet: {1}mailGroup labeledURI here is the entry in question: >ldapsearch -xWLLLD 'cn=admin,dc=groundnoise,dc=net' -s base -b 'cn=abuse,ou=distribution_groups,ou=all_domains,ou=domains,ou=mail,dc=groundnoise,dc=net' dn: cn=abuse,ou=distribution_groups,ou=all_domains,ou=domains,ou=mail,dc=groun dnoise,dc=net objectClass: mailGroup objectClass: top objectClass: extensibleObject cn: abuse member: cn=postmaster,ou=distribution_groups,ou=all_domains,ou=domains,ou=mail ,dc=groundnoise,dc=net labeledURI: ldap:///ou=domains,ou=mail,dc=groundnoise,dc=net?host?sub?(objectC lass=mailDomain) host: phone.dipswitch.net host: luna.mpls.mn.us host: groundnoise.net host: thielsen.org host: sjva1991.org host: dipswitch.net host: bitrate.net searched for another way: >ldapsearch -xWLLLD 'cn=admin,dc=groundnoise,dc=net' '(&(objectclass=mailgroup)(cn=abuse))' host dn: cn=abuse,ou=distribution_groups,ou=all_domains,ou=domains,ou=mail,dc=groun dnoise,dc=net host: phone.dipswitch.net host: luna.mpls.mn.us host: groundnoise.net host: thielsen.org host: sjva1991.org host: dipswitch.net host: bitrate.net however, the results from this search are missing that entry: >ldapsearch -xWLLLD 'cn=admin,dc=groundnoise,dc=net' '(host=dipswitch.net)' dn dn: host=dipswitch.net,ou=domains,ou=mail,dc=groundnoise,dc=net or another search: ldapsearch -xvWD 'cn=admin,dc=groundnoise,dc=net' '(&(objectclass=mailgroup)(host=*))' host ldap_initialize( <DEFAULT> ) Enter LDAP Password: filter: (&(objectclass=mailgroup)(host=*)) requesting: host # extended LDIF # # LDAPv3 # base <dc=groundnoise, dc=net> (default) with scope subtree # filter: (&(objectclass=mailgroup)(host=*)) # requesting: host # # search result search: 2 result: 0 Success # numResponses: 1 if i remove the labeledURI attribute and populate with static entries, things appear to work as expected: here's the entry: >ldapsearch -xWLLLD 'cn=admin,dc=groundnoise,dc=net' '(&(objectclass=mailgroup)(cn=abuse))' dn: cn=abuse,ou=distribution_groups,ou=all_domains,ou=domains,ou=mail,dc=groun dnoise,dc=net objectClass: mailGroup objectClass: top objectClass: extensibleObject cn: abuse member: cn=postmaster,ou=distribution_groups,ou=all_domains,ou=domains,ou=mail ,dc=groundnoise,dc=net host: foo host: bar host: com host: net host: org and a search: >ldapsearch -xWLLLD 'cn=admin,dc=groundnoise,dc=net' '(host=foo)' dn dn: cn=abuse,ou=distribution_groups,ou=all_domains,ou=domains,ou=mail,dc=groun dnoise,dc=net what am i doing wrong? thanks -ben

3 3

trouble with openldap >2.4.16
by Brett ＠Google 25 Jan '10

25 Jan '10

Hello, I am having trouble with Segfaults on Solaris Sparc, it seems like the sporadic error that some people have coming up during testing, but not others. Oddly, this only happens for me on an older not-patched box myunhappyserver, and NOT a more recently patched box myhappyserver (same slapd/bdb binaries and libraries) The "new" box (happily runs all openldap versions i have tried, up to and including the stable version based on 2.4.19) is : SunOS myhappyserver 5.10 Generic_141414-07 sun4v sparc SUNW,Sun-Fire-T200 The "old" box (segfaults even on simple start sometimes - but very occasionally it will run for awhile before it segfaults) is : SunOS myunhappyserver 5.10 Generic_127111-11 sun4v sparc SUNW,Sun-Fire-T200 >From adb on "myunhappyserver" i get (in response to "adb slapd core_qgpro01_slapd_404_404_1260924086_4710") : ::status debugging core file of slapd (64-bit) from myunhappyserver file: slapd initial argv: /usr/local/openldap/libexec/slapd -f /usr/local/openldap/etc/openldap/slapd_who threading model: multi-threaded using native lwps status: process terminated by SIGSEGV (Segmentation Fault) ::regs %g0 = 0x0000000000000000 %l0 = 0x000000010050ef48 %g1 = 0x000000010551e0d8 %l1 = 0x0000000105522440 %g2 = 0x0000000105551220 %l2 = 0x000000010050f020 %g3 = 0x0000000000000000 %l3 = 0x00000001001225e0 %g4 = 0x0000000000000000 %l4 = 0x0000000100122610 %g5 = 0x0000000000000008 %l5 = 0x0000000000000000 %g6 = 0x0000000000000000 %l6 = 0x00000001001bbdf8 avl_dup_error %g7 = 0xffffffff7e802200 %l7 = 0x0000000105522440 %o0 = 0x0000000000000000 %i0 = 0x000000010551d110 %o1 = 0xffffffffffffffff %i1 = 0x0000000000000000 %o2 = 0x0000000105551220 %i2 = 0x000000010050ef40 %o3 = 0xffffffff00000001 %i3 = 0x0000000105522440 %o4 = 0x0000000000000000 %i4 = 0x000000010050ef60 %o5 = 0x00000000ffffffff %i5 = 0x000000010050eef0 %o6 = 0xffffffff6c27e4f1 %i6 = 0xffffffff6c27e6d1 %o7 = 0x0000000100122bd8 hdb_cache_find_parent+0x124 %i7 = 0x00000001001233a8 hdb_cache_find_id+0x13c %ccr = 0x44 xcc=nZvc icc=nZvc %y = 0x0000000000000000 %pc = 0x0000000100122c8c hdb_cache_find_parent+0x1d8 %npc = 0x0000000100122c90 hdb_cache_find_parent+0x1dc %sp = 0xffffffff6c27e4f1 %fp = 0xffffffff6c27e6d1 %asi = 0x82 %fprs = 0x07 If i can do any more useful things with adb or the like, please provide some example and i'll run it. The ldd for slapd on the "new" server returns : -bash-3.00$ ldd /usr/local/openldap/libexec/slapd libdb-4.8.so => /usr/local/openldap/lib/libdb-4.8.so librt.so.1 => /lib/64/librt.so.1 libperl.so => /usr/local/lib/perl5/5.10.1/sun4-solaris-thread-multi-64/CORE/libperl.so libm.so.2 => /lib/64/libm.so.2 libpthread.so.1 => /lib/64/libpthread.so.1 libicuuc.so.3 => /usr/lib/64/libicuuc.so.3 libicudata.so.3 => /usr/lib/64/libicudata.so.3 libsasl2.so.2 => /usr/local/openldap/lib/libsasl2.so.2 libdl.so.1 => /lib/64/libdl.so.1 libssl.so.0.9.7 => /usr/sfw/lib/sparcv9/libssl.so.0.9.7 libcrypto.so.0.9.7 => /usr/sfw/lib/sparcv9/libcrypto.so.0.9.7 libresolv.so.2 => /lib/64/libresolv.so.2 libgen.so.1 => /lib/64/libgen.so.1 libnsl.so.1 => /lib/64/libnsl.so.1 libsocket.so.1 => /lib/64/libsocket.so.1 libc.so.1 => /lib/64/libc.so.1 libaio.so.1 => /lib/64/libaio.so.1 libmd.so.1 => /lib/64/libmd.so.1 libCrun.so.1 => /usr/lib/64/libCrun.so.1 libmp.so.2 => /lib/64/libmp.so.2 libscf.so.1 => /lib/64/libscf.so.1 libdoor.so.1 => /lib/64/libdoor.so.1 libuutil.so.1 => /lib/64/libuutil.so.1 libssl_extra.so.0.9.7 => /usr/sfw/lib/sparcv9/libssl_extra.so.0.9.7 libcrypto_extra.so.0.9.7 => /usr/sfw/lib/sparcv9/libcrypto_extra.so.0.9.7 /platform/SUNW,Sun-Fire-T200/lib/sparcv9/libc_psr.so.1 /platform/SUNW,Sun-Fire-T200/lib/sparcv9/libmd_psr.so.1 The ldd for slapd on the "old" server adds (which would appear to be harmless) : libgss.so.1 => /usr/lib/64/libgss.so.1 libcmd.so.1 => /lib/64/libcmd.so.1 All compiled libraries are 64 bit, linked against the 64 bit solaris libraries. I am using the same libraries for berkeley 4.8, on all servers. Cheers Brett

3 3

Re: solaris compile options
by Brett ＠Google 24 Jan '10

24 Jan '10

i am using CFLAGS="-fast -xtarget=ultraT1 -xarch=sparcvis2 -xcode=pic32 -g -xs -O" one set of solaris docs i read implied that -xarch=sparcvis2 was equivalent to -xarch=v9 (which used to trigger 64 bit), but looking at the sun studio 12 compiler options, the more specific versions of -xarch (ie. other than -xarch=v9 or v9a or v9b) may no longer imply that the 64 bit memory model should be used. so maybe i need to add a -m64 to the above ? (compiling on a Sun T2000, with a homegenous build / execute environment, so favouring speed over cpu compatibility is ok) On Thu, Mar 12, 2009 at 1:31 AM, Aaron Richton <richton(a)nbcs.rutgers.edu>wrote: > On Wed, 11 Mar 2009, Brett @Google wrote: > > /data/openldap/backups/ldap_090302.ldif: Value too large for defined data >> type >> > > man lfcompile, and/or switch to 64-bit binaries? >

2 3

Understanding Synchronization Problem using syncrepl
by Andrea Cirulli 04 Jan '10

04 Jan '10

Hi All, I work on an important infrastructure of 20 LDAP Servers, One Producer and 19 consumers. We are migrating on multimaster-mode achieving 22 server, 2 Producers ( in mirror mode ) and 20 consumers. I'm here to asking all which is the better way for understanding if all the LDAP servers are synchronized. We want to create some tools for checking periodically whether there is some consumer not yet synchronized or not. Could you suggest me something? We were thinking on analyzing entryCSN, however if we are going to analyze all of them could lead in an overload problem since we are dealing with more then 3 Millions of objects, and we want to check the sync at least every X minutes and not Hours or day..... Many Thanks in advance -- Andrea Cirulli

3 5

DNCache in hdb stays over limits
by Ondřej Kuzník 31 Dec '09

31 Dec '09

Hello, I am using OpenLDAP with a large database using HDB backend that cannot fit in the RAM. The bulk of the database is located under a single suffix like this: ou=data, dc=example, dc=com In this example, I used a smaller and simpler database (~20000 entries) and set DN cache to 1000 entries: $ ldapsearch -LLL -b "olcDatabase={2}hdb,cn=config" olcDbCacheFree \ olcDbDNcacheSize olcDbCacheSize dn: olcDatabase={2}hdb,cn=config olcDbCacheSize: 1000 olcDbCacheFree: 1 olcDbDNcacheSize: 1000 example entry: dn: uid=6102328959,ou=data,dc=example,dc=com objectClass: top objectClass: account uid: 6102328959 When querying the database for some of the entries, sometimes the DN cache gets to a state when it contains all of them (and much more than the configured maximum) even though no search is being processed. Here are some observations I made thus far: 1. When performing a search with a base of "ou=data, dc=example, dc=com" the following happens (no indexes are present on the database): $ ldapsearch -b "ou=data, dc=example, dc=com" "*" i. First everything under "ou=data, dc=example, dc=com" is loaded into the DN cache (exhausting available memory and being swapped out if needed). ii. Then the server starts returning the matching entries, gradually freeing the DN cache. If a limit on the number of returned entries is set with the search (e.g. -z 10), DNs of the entries not returned are not freed until they are visited by a search in the future. $ ldapsearch -LLL -b "cn=Database 2,cn=Databases,cn=Monitor" \ olmBDBEntryCache olmBDBDNCache olmBDBEntryCache: 1000 olmBDBDNCache: 20324 2. When the same search is performed with a base higher ("dc=example, dc=com") I get the expected behaviour: $ ldapsearch -b "dc=example, dc=com" "objectClass=account" The search starts returning the entries instantly and DN cache size never exceeds 1002 entries (the entry limit "-z" changes nothing). Is this behaviour expected? Because the slapd-hdb man page is a little blurry on this. Can something be done on the configuration side to prevent such behaviour or the only way out is to use BDB backend for the database? I have tested this with both 2.4.20 and the latest cvs snapshot. Regards, Ondřej Kuzník

1 0

Re: how can add componnet matching module to openldap?
by Aaron Richton 21 Dec '09

21 Dec '09

On Mon, 21 Dec 2009, med s wrote: > Thank you the libtool error is solved. Now i am having trouble making > the comp match module. it has errors when compiling the componentlib.c > file.Is there anything else the includes in the makefile that i should > add? Please keep replies on the list. I'm not sure if additional flags are necessary for componentlib.c. Perhaps the exact text of the compiler error would be helpful.

1 0

delta sync repl out of sync
by ST Wong (ITSC) 18 Dec '09

18 Dec '09

Hi all, I'm using openldap 2.4.19 with 1 provider and consumer. Everything works fine with syncrepl setup. However, when changed to delta syncrepl, most of the updates can't be updated on consumer. However,when I do ldapsearch from consumer to provider using same binddn and search filter for accesslog content as defined in the syncrepl statement, access log entries can be retrived. Here comes my configuration files. Would anyone please help? Sorry for newbie question. Thanks a lot. ST Wong Consumer: ------------------------- cut here ------------------------------- syncrepl rid=005 provider=ldap://provider1.my.com bindmethod=simple binddn="cn=replicator,dc=my,dc=com" credentials="mysecret" retry="60 +" searchbase="dc=my,dc=com" schemachecking=off type=refreshAndPersist interval=00:00:00:05 starttls=yes tls_reqcert=never tls_cacert=/etc/pki/tls/certs/cacert.pem logbase="cn=accesslog" <---- logfilter="(&(objectClass=auditWriteObject)(reqResult=0))" <---- replication fails after adding these lines. syncdata=accesslog <---- ------------------------- cut here ------------------------------- Provider: ------------------------- cut here ------------------------------- database bdb suffix cn=accesslog directory /usr/local/var/openldap-accesslog rootdn cn=accesslog index default eq index entryUUID,entryCSN,objectClass,reqEnd,reqResult,reqStart limits dn.exact="cn=replicator,dc=my,dc=com" time.soft=unlimited time.hard=unlimited size.soft=unlimited size.hard=unlimited overlay syncprov syncprov-reloadhint TRUE syncprov-nopresent TRUE .... database bdb suffix "dc=my,dc=com" rootdn "cn=Manager,dc=my,dc=com" rootpw mysecret directory /usr/local/var/openldap-data index entryCSN,entryUUID eq index contextCSN eq index objectClass eq overlay syncprov syncprov-checkpoint 100 10 overlay accesslog logdb cn=accesslog logops writes logsuccess TRUE logpurge 01+00:00 01+00:00 ------------------------- cut here -------------------------------

2 6

how can add componnet matching module to openldap?
by med s 18 Dec '09

18 Dec '09

Hello am using openldap 2.4 I want my server to support componnet matching when i run it like this env CPPFLAGS="-I/usr/local/BerkeleyDB.4.7/include" LDFLAGS="-L/usr/local/BerkeleyDB.4.7/lib" ./configure -enable-modules=yes it gives error could not locate libtool ltdl.h so how should i configure slapd to support component matchingg?

2 1

Re: trouble with openldap >2.4.16
by Brett ＠Google 18 Dec '09

18 Dec '09

On Fri, Dec 18, 2009 at 3:11 AM, Quanah Gibson-Mount <quanah(a)zimbra.com>wrote: > From adb on "myunhappyserver" i get (in response to "adb slapd >> core_qgpro01_slapd_404_404_1260924086_4710") : >> > > Sounds like a question for Sun to me. If their patch levels fix the bug, > then it must be a problem they know about? I expect that patching (or staying at 2.4.16) might fix my problems, but was thinking more along the lines that if this was an instance of an already existing but subtle openldap bug that is hard to track down, then this might be an opportunity to track it down whilst it is visible, regardless of the patching issue trigger. There will be some delay until i can patch the server in question, due to external dependancies, but there would be a small window for investigation. I was thinking of ITS#6200 but is looks like : This trace looks like the problem fixed in ITS#6335. The fix is in CVS HEAD and RE24 (2.4.20 pre-release) If this is the case, i'll just wait for 2.4.20 to be released. Cheers Brett

2 1

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-software December 2009