searching for attributes without index in 2.4.19 with bdb 4.5
by Christoph Herrmann
Hello,
after upgrading from openldap-2.3.39 with bdb-4.2.52 to openldap 2.4.19 with bdb-4.5
searching for attributes without index is about three times slower. (same machine, same
data, all data fit in DB cache)
Are there any known problems or magic tuning options we have missed?
regards
Christoph &:-)
--
Vorstand/Board of Management:
Dr. Bernd Finkbeiner, Dr. Roland Niemeier,
Dr. Arno Steitz, Dr. Ingrid Zech
Vorsitzender des Aufsichtsrats/
Chairman of the Supervisory Board:
Michel Lepert
Sitz/Registered Office: Tuebingen
Registergericht/Registration Court: Stuttgart
Registernummer/Commercial Register No.: HRB 382196
13 years, 1 month
dynlist overlay and ldapsearch
by ben thielsen
hi-
i'm using the dynlist overlay and am not getting back the search results i expected. i'm using 2.4.11 courtesy of debian.
here is my overlay config:
>ldapsearch -xWLLLD 'cn=admin,cn=config' -b 'cn=config' "(objectclass=olcdynamiclist)"
dn: olcOverlay={5}dynlist,olcDatabase={2}bdb,cn=config
objectClass: olcOverlayConfig
objectClass: olcDynamicList
olcOverlay: {5}dynlist
olcDLattrSet: {0}groupOfNames memberURL member
olcDLattrSet: {1}mailGroup labeledURI
here is the entry in question:
>ldapsearch -xWLLLD 'cn=admin,dc=groundnoise,dc=net' -s base -b 'cn=abuse,ou=distribution_groups,ou=all_domains,ou=domains,ou=mail,dc=groundnoise,dc=net'
dn: cn=abuse,ou=distribution_groups,ou=all_domains,ou=domains,ou=mail,dc=groun
dnoise,dc=net
objectClass: mailGroup
objectClass: top
objectClass: extensibleObject
cn: abuse
member: cn=postmaster,ou=distribution_groups,ou=all_domains,ou=domains,ou=mail
,dc=groundnoise,dc=net
labeledURI: ldap:///ou=domains,ou=mail,dc=groundnoise,dc=net?host?sub?(objectC
lass=mailDomain)
host: phone.dipswitch.net
host: luna.mpls.mn.us
host: groundnoise.net
host: thielsen.org
host: sjva1991.org
host: dipswitch.net
host: bitrate.net
searched for another way:
>ldapsearch -xWLLLD 'cn=admin,dc=groundnoise,dc=net' '(&(objectclass=mailgroup)(cn=abuse))' host
dn: cn=abuse,ou=distribution_groups,ou=all_domains,ou=domains,ou=mail,dc=groun
dnoise,dc=net
host: phone.dipswitch.net
host: luna.mpls.mn.us
host: groundnoise.net
host: thielsen.org
host: sjva1991.org
host: dipswitch.net
host: bitrate.net
however, the results from this search are missing that entry:
>ldapsearch -xWLLLD 'cn=admin,dc=groundnoise,dc=net' '(host=dipswitch.net)' dn
dn: host=dipswitch.net,ou=domains,ou=mail,dc=groundnoise,dc=net
or another search:
ldapsearch -xvWD 'cn=admin,dc=groundnoise,dc=net' '(&(objectclass=mailgroup)(host=*))' host
ldap_initialize( <DEFAULT> )
Enter LDAP Password:
filter: (&(objectclass=mailgroup)(host=*))
requesting: host
# extended LDIF
#
# LDAPv3
# base <dc=groundnoise, dc=net> (default) with scope subtree
# filter: (&(objectclass=mailgroup)(host=*))
# requesting: host
#
# search result
search: 2
result: 0 Success
# numResponses: 1
if i remove the labeledURI attribute and populate with static entries, things appear to work as expected:
here's the entry:
>ldapsearch -xWLLLD 'cn=admin,dc=groundnoise,dc=net' '(&(objectclass=mailgroup)(cn=abuse))'
dn: cn=abuse,ou=distribution_groups,ou=all_domains,ou=domains,ou=mail,dc=groun
dnoise,dc=net
objectClass: mailGroup
objectClass: top
objectClass: extensibleObject
cn: abuse
member: cn=postmaster,ou=distribution_groups,ou=all_domains,ou=domains,ou=mail
,dc=groundnoise,dc=net
host: foo
host: bar
host: com
host: net
host: org
and a search:
>ldapsearch -xWLLLD 'cn=admin,dc=groundnoise,dc=net' '(host=foo)' dn
dn: cn=abuse,ou=distribution_groups,ou=all_domains,ou=domains,ou=mail,dc=groun
dnoise,dc=net
what am i doing wrong?
thanks
-ben
13 years, 1 month
trouble with openldap >2.4.16
by Brett ＠Google
Hello,
I am having trouble with Segfaults on Solaris Sparc, it seems like the
sporadic error that some people have coming up during testing, but not
others.
Oddly, this only happens for me on an older not-patched box myunhappyserver,
and NOT a more recently patched box myhappyserver (same slapd/bdb binaries
and libraries)
The "new" box (happily runs all openldap versions i have tried, up to and
including the stable version based on 2.4.19) is :
SunOS myhappyserver 5.10 Generic_141414-07 sun4v sparc SUNW,Sun-Fire-T200
The "old" box (segfaults even on simple start sometimes - but very
occasionally it will run for awhile before it segfaults) is :
SunOS myunhappyserver 5.10 Generic_127111-11 sun4v sparc SUNW,Sun-Fire-T200
>From adb on "myunhappyserver" i get (in response to "adb slapd
core_qgpro01_slapd_404_404_1260924086_4710") :
::status
debugging core file of slapd (64-bit) from myunhappyserver
file: slapd
initial argv: /usr/local/openldap/libexec/slapd -f
/usr/local/openldap/etc/openldap/slapd_who
threading model: multi-threaded using native lwps
status: process terminated by SIGSEGV (Segmentation Fault)
::regs
%g0 = 0x0000000000000000 %l0 = 0x000000010050ef48
%g1 = 0x000000010551e0d8 %l1 = 0x0000000105522440
%g2 = 0x0000000105551220 %l2 = 0x000000010050f020
%g3 = 0x0000000000000000 %l3 = 0x00000001001225e0
%g4 = 0x0000000000000000 %l4 = 0x0000000100122610
%g5 = 0x0000000000000008 %l5 = 0x0000000000000000
%g6 = 0x0000000000000000 %l6 = 0x00000001001bbdf8
avl_dup_error
%g7 = 0xffffffff7e802200 %l7 = 0x0000000105522440
%o0 = 0x0000000000000000 %i0 = 0x000000010551d110
%o1 = 0xffffffffffffffff %i1 = 0x0000000000000000
%o2 = 0x0000000105551220 %i2 = 0x000000010050ef40
%o3 = 0xffffffff00000001 %i3 = 0x0000000105522440
%o4 = 0x0000000000000000 %i4 = 0x000000010050ef60
%o5 = 0x00000000ffffffff %i5 = 0x000000010050eef0
%o6 = 0xffffffff6c27e4f1 %i6 = 0xffffffff6c27e6d1
%o7 = 0x0000000100122bd8 hdb_cache_find_parent+0x124 %i7 =
0x00000001001233a8 hdb_cache_find_id+0x13c
%ccr = 0x44 xcc=nZvc icc=nZvc
%y = 0x0000000000000000
%pc = 0x0000000100122c8c hdb_cache_find_parent+0x1d8
%npc = 0x0000000100122c90 hdb_cache_find_parent+0x1dc
%sp = 0xffffffff6c27e4f1
%fp = 0xffffffff6c27e6d1
%asi = 0x82
%fprs = 0x07
If i can do any more useful things with adb or the like, please provide some
example and i'll run it.
The ldd for slapd on the "new" server returns :
-bash-3.00$ ldd /usr/local/openldap/libexec/slapd
libdb-4.8.so => /usr/local/openldap/lib/libdb-4.8.so
librt.so.1 => /lib/64/librt.so.1
libperl.so =>
/usr/local/lib/perl5/5.10.1/sun4-solaris-thread-multi-64/CORE/libperl.so
libm.so.2 => /lib/64/libm.so.2
libpthread.so.1 => /lib/64/libpthread.so.1
libicuuc.so.3 => /usr/lib/64/libicuuc.so.3
libicudata.so.3 => /usr/lib/64/libicudata.so.3
libsasl2.so.2 => /usr/local/openldap/lib/libsasl2.so.2
libdl.so.1 => /lib/64/libdl.so.1
libssl.so.0.9.7 => /usr/sfw/lib/sparcv9/libssl.so.0.9.7
libcrypto.so.0.9.7 => /usr/sfw/lib/sparcv9/libcrypto.so.0.9.7
libresolv.so.2 => /lib/64/libresolv.so.2
libgen.so.1 => /lib/64/libgen.so.1
libnsl.so.1 => /lib/64/libnsl.so.1
libsocket.so.1 => /lib/64/libsocket.so.1
libc.so.1 => /lib/64/libc.so.1
libaio.so.1 => /lib/64/libaio.so.1
libmd.so.1 => /lib/64/libmd.so.1
libCrun.so.1 => /usr/lib/64/libCrun.so.1
libmp.so.2 => /lib/64/libmp.so.2
libscf.so.1 => /lib/64/libscf.so.1
libdoor.so.1 => /lib/64/libdoor.so.1
libuutil.so.1 => /lib/64/libuutil.so.1
libssl_extra.so.0.9.7 =>
/usr/sfw/lib/sparcv9/libssl_extra.so.0.9.7
libcrypto_extra.so.0.9.7 =>
/usr/sfw/lib/sparcv9/libcrypto_extra.so.0.9.7
/platform/SUNW,Sun-Fire-T200/lib/sparcv9/libc_psr.so.1
/platform/SUNW,Sun-Fire-T200/lib/sparcv9/libmd_psr.so.1
The ldd for slapd on the "old" server adds (which would appear to be
harmless) :
libgss.so.1 => /usr/lib/64/libgss.so.1
libcmd.so.1 => /lib/64/libcmd.so.1
All compiled libraries are 64 bit, linked against the 64 bit solaris
libraries.
I am using the same libraries for berkeley 4.8, on all servers.
Cheers
Brett
13 years, 1 month
Re: solaris compile options
by Brett ＠Google
i am using CFLAGS="-fast -xtarget=ultraT1 -xarch=sparcvis2 -xcode=pic32 -g
-xs -O"
one set of solaris docs i read implied that -xarch=sparcvis2 was equivalent
to -xarch=v9 (which used to trigger 64 bit), but looking at the sun studio
12 compiler options, the more specific versions of -xarch (ie. other than
-xarch=v9 or v9a or v9b) may no longer imply that the 64 bit memory model
should be used. so maybe i need to add a -m64 to the above ?
(compiling on a Sun T2000, with a homegenous build / execute environment, so
favouring speed over cpu compatibility is ok)
On Thu, Mar 12, 2009 at 1:31 AM, Aaron Richton <richton(a)nbcs.rutgers.edu>wrote:
> On Wed, 11 Mar 2009, Brett @Google wrote:
>
> /data/openldap/backups/ldap_090302.ldif: Value too large for defined data
>> type
>>
>
> man lfcompile, and/or switch to 64-bit binaries?
>
13 years, 2 months
Understanding Synchronization Problem using syncrepl
by Andrea Cirulli
Hi All,
I work on an important infrastructure of 20 LDAP Servers, One Producer and
19 consumers.
We are migrating on multimaster-mode achieving 22 server, 2 Producers ( in
mirror mode ) and 20 consumers.
I'm here to asking all which is the better way for understanding if all the
LDAP servers are synchronized.
We want to create some tools for checking periodically whether there is some
consumer not yet synchronized or not.
Could you suggest me something?
We were thinking on analyzing entryCSN, however if we are going to analyze
all of them could lead in an overload problem since we are dealing with more
then 3 Millions of objects, and we want to check the sync at least every X
minutes and not Hours or day.....
Many Thanks in advance
--
Andrea Cirulli
13 years, 2 months
DNCache in hdb stays over limits
by Ondřej Kuzník
Hello,
I am using OpenLDAP with a large database using HDB backend that cannot
fit in the RAM. The bulk of the database is located under a single
suffix like this:
ou=data, dc=example, dc=com
In this example, I used a smaller and simpler database (~20000 entries)
and set DN cache to 1000 entries:
$ ldapsearch -LLL -b "olcDatabase={2}hdb,cn=config" olcDbCacheFree \
olcDbDNcacheSize olcDbCacheSize
dn: olcDatabase={2}hdb,cn=config
olcDbCacheSize: 1000
olcDbCacheFree: 1
olcDbDNcacheSize: 1000
example entry:
dn: uid=6102328959,ou=data,dc=example,dc=com
objectClass: top
objectClass: account
uid: 6102328959
When querying the database for some of the entries, sometimes the
DN cache gets to a state when it contains all of them (and much more
than the configured maximum) even though no search is being processed.
Here are some observations I made thus far:
1. When performing a search with a base of "ou=data, dc=example, dc=com"
the following happens (no indexes are present on the database):
$ ldapsearch -b "ou=data, dc=example, dc=com" "*"
i. First everything under "ou=data, dc=example, dc=com" is loaded into
the DN cache (exhausting available memory and being swapped out if
needed).
ii. Then the server starts returning the matching entries, gradually
freeing the DN cache.
If a limit on the number of returned entries is set with the search
(e.g. -z 10), DNs of the entries not returned are not freed until
they are visited by a search in the future.
$ ldapsearch -LLL -b "cn=Database 2,cn=Databases,cn=Monitor" \
olmBDBEntryCache olmBDBDNCache
olmBDBEntryCache: 1000
olmBDBDNCache: 20324
2. When the same search is performed with a base higher ("dc=example,
dc=com") I get the expected behaviour:
$ ldapsearch -b "dc=example, dc=com" "objectClass=account"
The search starts returning the entries instantly and DN cache size
never exceeds 1002 entries (the entry limit "-z" changes nothing).
Is this behaviour expected? Because the slapd-hdb man page is a little
blurry on this. Can something be done on the configuration side to
prevent such behaviour or the only way out is to use BDB backend for the
database? I have tested this with both 2.4.20 and the latest cvs
snapshot.
Regards,
Ondřej Kuzník
13 years, 2 months
Re: how can add componnet matching module to openldap?
by Aaron Richton
On Mon, 21 Dec 2009, med s wrote:
> Thank you the libtool error is solved. Now i am having trouble making
> the comp match module. it has errors when compiling the componentlib.c
> file.Is there anything else the includes in the makefile that i should
> add?
Please keep replies on the list.
I'm not sure if additional flags are necessary for componentlib.c. Perhaps
the exact text of the compiler error would be helpful.
13 years, 3 months
delta sync repl out of sync
by ST Wong (ITSC)
Hi all,
I'm using openldap 2.4.19 with 1 provider and consumer. Everything
works fine with syncrepl setup. However, when changed to delta
syncrepl, most of the updates can't be updated on consumer.
However,when I do ldapsearch from consumer to provider using same binddn
and search filter for accesslog content as defined in the syncrepl
statement, access log entries can be retrived.
Here comes my configuration files. Would anyone please help? Sorry for
newbie question. Thanks a lot.
ST Wong
Consumer:
------------------------- cut here -------------------------------
syncrepl rid=005
provider=ldap://provider1.my.com
bindmethod=simple
binddn="cn=replicator,dc=my,dc=com"
credentials="mysecret"
retry="60 +"
searchbase="dc=my,dc=com"
schemachecking=off
type=refreshAndPersist
interval=00:00:00:05
starttls=yes
tls_reqcert=never
tls_cacert=/etc/pki/tls/certs/cacert.pem
logbase="cn=accesslog"
<----
logfilter="(&(objectClass=auditWriteObject)(reqResult=0))"
<---- replication fails after adding these lines.
syncdata=accesslog
<----
------------------------- cut here -------------------------------
Provider:
------------------------- cut here -------------------------------
database bdb
suffix cn=accesslog
directory /usr/local/var/openldap-accesslog
rootdn cn=accesslog
index default eq
index entryUUID,entryCSN,objectClass,reqEnd,reqResult,reqStart
limits dn.exact="cn=replicator,dc=my,dc=com" time.soft=unlimited
time.hard=unlimited size.soft=unlimited size.hard=unlimited
overlay syncprov
syncprov-reloadhint TRUE
syncprov-nopresent TRUE
....
database bdb
suffix "dc=my,dc=com"
rootdn "cn=Manager,dc=my,dc=com"
rootpw mysecret
directory /usr/local/var/openldap-data
index entryCSN,entryUUID eq
index contextCSN eq
index objectClass eq
overlay syncprov
syncprov-checkpoint 100 10
overlay accesslog
logdb cn=accesslog
logops writes
logsuccess TRUE
logpurge 01+00:00 01+00:00
------------------------- cut here -------------------------------
13 years, 3 months
how can add componnet matching module to openldap?
by med s
Hello am using openldap 2.4
I want my server to support componnet matching
when i run it like this
env CPPFLAGS="-I/usr/local/BerkeleyDB.4.7/include"
LDFLAGS="-L/usr/local/BerkeleyDB.4.7/lib" ./configure -enable-modules=yes
it gives error could not locate libtool ltdl.h
so how should i configure slapd to support component matchingg?
13 years, 3 months
Re: trouble with openldap >2.4.16
by Brett ＠Google
On Fri, Dec 18, 2009 at 3:11 AM, Quanah Gibson-Mount <quanah(a)zimbra.com>wrote:
> From adb on "myunhappyserver" i get (in response to "adb slapd
>> core_qgpro01_slapd_404_404_1260924086_4710") :
>>
>
> Sounds like a question for Sun to me. If their patch levels fix the bug,
> then it must be a problem they know about?
I expect that patching (or staying at 2.4.16) might fix my problems, but was
thinking more along the lines that if this was an instance of an already
existing but subtle openldap bug that is hard to track down, then this might
be an opportunity to track it down whilst it is visible, regardless of the
patching issue trigger. There will be some delay until i can patch the
server in question, due to external dependancies, but there would be a small
window for investigation.
I was thinking of ITS#6200 but is looks like :
This trace looks like the problem fixed in ITS#6335. The fix is in CVS HEAD
and RE24 (2.4.20 pre-release)
If this is the case, i'll just wait for 2.4.20 to be released.
Cheers
Brett
13 years, 3 months
