openldap-technical

openldap-technical@openldap.org

9897 discussions

Re: Index for objectclass does not work...
by Steeg Carson 03 Jan '11

03 Jan '11

2011/1/4 Quanah Gibson-Mount <quanah(a)zimbra.com>: > --On Tuesday, January 04, 2011 1:08 AM +0100 Steeg Carson > <steeg.carson(a)googlemail.com> wrote: > >>> This does not look like a valid ldapsearch: >>> >>> ldapsearch -x -h localhost -D"uid=admin,ou=root" -b"cn=ou=root>" >>> "(ObjectClass=subEngine)" dn | grep "^dn:" | wc -l) >>> >>> The -b setting in particular looks broken, which is problem why you are >>> getting that message. >> >> Sorry, the base DN in the search should be -b"ou=root", my mistake. I >> tried to shorten it in the message... > > Please provide the exact ldapsearch in its entirety then (minus any > password). > I simulate this on my database just right now: =>set logelevel 33 echo >/var/log/messages # time ldapsearch -x -h localhost -wpassword -D"uid=admin,ou=root" -b"ou=root" "(objectClass=subEngine)" dn | grep "^dn:" | wc -l 104384 real 1m45.518s user 0m1.208s sys 0m1.332s grep "hdb_search:" /var/log/messages | wc -l 366201 => result form messages: ################################################### backend_startup_one: starting "ou=root" hdb_db_open: database "ou=root": dbenv_open(/var/lib/ldap_hdb). hdb_monitor_db_open: monitoring disabled; configure monitor database to enable slapd starting slap_listener_activate(7): >>> slap_listener(ldap:///) connection_get(12): got connid=1000 connection_read(12): checking for input on id=1000 op tag 0x60, time 1294100496 conn=1000 op=0 do_bind >>> dnPrettyNormal: <> <<< dnPrettyNormal: <>, <> do_bind: version=3 dn="" method=128 send_ldap_result: conn=1000 op=0 p=3 send_ldap_response: msgid=1 tag=97 err=48 do_bind: v3 anonymous bind connection_get(12): got connid=1000 connection_read(12): checking for input on id=1000 ber_get_next on fd 12 failed errno=0 (Success) connection_close: conn=1000 sd=12 slap_listener_activate(7): >>> slap_listener(ldap:///) connection_get(12): got connid=1001 connection_read(12): checking for input on id=1001 op tag 0x60, time 1294100553 conn=1001 op=0 do_bind >>> dnPrettyNormal: <uid=admin,ou=root> <<< dnPrettyNormal: <uid=admin,ou=root>, <cn=root,ou=admins,ou=root> do_bind: version=3 dn="uid=admin,ou=root" method=128 do_bind: v3 bind: "uid=admin,ou=root" to "uid=admin,ou=root" send_ldap_result: conn=1001 op=0 p=3 send_ldap_response: msgid=1 tag=97 err=0 connection_get(12): got connid=1001 connection_read(12): checking for input on id=1001 op tag 0x63, time 1294100553 conn=1001 op=1 do_search >>> dnPrettyNormal: <ou=root> <<< dnPrettyNormal: <ou=root>, <ou=root> begin get_filter EQUALITY end get_filter 0 => hdb_search bdb_dn2entry("ou=root") => hdb_dn2id("ou=root") <= hdb_dn2id: got id=0x1 entry_decode: "" <= entry_decode() search_candidates: base="ou=root" (0x00000001) scope=2 => hdb_dn2idl("ou=root") => bdb_filter_candidates #011AND => bdb_list_candidates 0xa0 => bdb_filter_candidates #011OR => bdb_list_candidates 0xa1 => bdb_filter_candidates #011EQUALITY => bdb_equality_candidates (objectClass) => key_read <= bdb_index_read: failed (-30989) <= bdb_equality_candidates: id=0, first=0, last=0 <= bdb_filter_candidates: id=0 first=0 last=0 => bdb_filter_candidates #011EQUALITY => bdb_equality_candidates (objectClass) => key_read <= bdb_index_read 470601 candidates <= bdb_equality_candidates: id=-1, first=228, last=470828 <= bdb_filter_candidates: id=-1 first=228 last=470828 <= bdb_list_candidates: id=-1 first=228 last=470828 <= bdb_filter_candidates: id=-1 first=228 last=470828 <= bdb_list_candidates: id=-1 first=228 last=470828 <= bdb_filter_candidates: id=-1 first=228 last=470828 bdb_search_candidates: id=-1 first=228 last=470828 entry_decode: "" <= entry_decode() => test_filter EQUALITY <= test_filter 6 entry_decode: "" <= entry_decode() => test_filter EQUALITY <= test_filter 5 hdb_search: 229 does not match filter entry_decode: "" <= entry_decode() => test_filter EQUALITY <= test_filter 5 hdb_search: 230 does not match filter entry_decode: "" <= entry_decode() => test_filter EQUALITY <= test_filter 5 hdb_search: 231 does not match filter entry_decode: "" <= entry_decode() => test_filter EQUALITY , , , and so on Thanks Steeg

2 1

Re: Index for objectclass does not work...
by Steeg Carson 03 Jan '11

03 Jan '11

2011/1/4 Quanah Gibson-Mount <quanah(a)zimbra.com>: > --On Tuesday, January 04, 2011 12:50 AM +0100 Steeg Carson > <steeg.carson(a)googlemail.com> wrote: > >> 2011/1/3 Quanah Gibson-Mount <quanah(a)zimbra.com>: >>>>> >>>>> The DB is always read off disk into the BDB backing cache the very >>>>> first time slapd is run (assuming back-bdb or back-hdb are your >>>>> backends). I don't see any bug/issue here. >>>>> >>>> >>>> But does the slapd process during this time _write_ (heavy the whole >>>> time) to the disk? >>> >>> Did you configure you BDB backing cache to be in memory, or on disk? >>> The default is for it to be on the disk. If you want it to be in >>> memory, set an shm_key value in the database configuration. >>> >>> --Quanah >>> >> >> Yes, thanks, thats it :). But why are the shared Memory on disk? Does >> it not make the slapd slower? > > I've only noticed it making things slower on databases that are 8GB or later > in size. > >> My second problem is, that I don't really understand, how indexes >> work, or is the a problem with slapd, my configuration or my Database? >> As I said in the message before, >> the searched objectClass=subEngine exists 104384 times in the entire >> directory: (=> ldapsearch -x -h localhost -D"uid=admin,ou=root" >> -b"cn=ou=root>" "(ObjectClass=subEngine)" dn | grep "^dn:" | wc -l) > > This does not look like a valid ldapsearch: > > ldapsearch -x -h localhost -D"uid=admin,ou=root" -b"cn=ou=root>" > "(ObjectClass=subEngine)" dn | grep "^dn:" | wc -l) > > The -b setting in particular looks broken, which is problem why you are > getting that message. Sorry, the base DN in the search should be -b"ou=root", my mistake. I tried to shorten it in the message... Thanks Steeg

2 1

Re: Index for objectclass does not work...
by Steeg Carson 03 Jan '11

03 Jan '11

2011/1/3 Quanah Gibson-Mount <quanah(a)zimbra.com>: >>> The DB is always read off disk into the BDB backing cache the very first >>> time slapd is run (assuming back-bdb or back-hdb are your backends). I >>> don't see any bug/issue here. >>> >> >> But does the slapd process during this time _write_ (heavy the whole >> time) to the disk? > > Did you configure you BDB backing cache to be in memory, or on disk? The > default is for it to be on the disk. If you want it to be in memory, set an > shm_key value in the database configuration. > > --Quanah > Yes, thanks, thats it :). But why are the shared Memory on disk? Does it not make the slapd slower? My second problem is, that I don't really understand, how indexes work, or is the a problem with slapd, my configuration or my Database? As I said in the message before, the searched objectClass=subEngine exists 104384 times in the entire directory: (=> ldapsearch -x -h localhost -D"uid=admin,ou=root" -b"cn=ou=root>" "(ObjectClass=subEngine)" dn | grep "^dn:" | wc -l) If I configure with index for Objectclass, I get for the search following messages: ####################################################################### => bdb_equality_candidates (objectClass) => key_read <= bdb_index_read 470601 candidates <= bdb_equality_candidates: id=-1, first=228, last=470828 <= bdb_filter_candidates: id=-1 first=228 last=470828 <= bdb_list_candidates: id=-1 first=228 last=470828 <= bdb_filter_candidates: id=-1 first=228 last=470828 <= bdb_list_candidates: id=-1 first=40595 last=470828 <= bdb_filter_candidates: id=-1 first=40595 last=470828 bdb_search_candidates: id=-1 first=40595 last=470828 ####################################################################### If I search then in the logfile, I see 430233 messages like: "hdb_search: <candidate> <message: does not match filter | scope not okay>" Why does the it the search (hdbsearch) 430233 times, instead only 104384 times? I've also rebuild the index, should the index not only return 104384 candidates? Thanks for helping! Kindly regards, Steeg

2 1

Re: Index for objectclass does not work...
by Steeg Carson 03 Jan '11

03 Jan '11

2011/1/3 Quanah Gibson-Mount <quanah(a)zimbra.com>: > --On Monday, January 03, 2011 9:22 PM +0100 Steeg Carson > <steeg.carson(a)googlemail.com> wrote: > >> I also used strace to catch the problem, but i did not find where the >> slapd write to disk. It seems, it comes via set up the futex write >> locks? I'm not a kernel programmer and the code from openldap is hard >> to read. But strace I use for several years to find errors, it was >> also the first choice for me, to find the error. It is not a normal >> file that will be written... > > The DB is always read off disk into the BDB backing cache the very first > time slapd is run (assuming back-bdb or back-hdb are your backends). I > don't see any bug/issue here. > But does the slapd process during this time _write_ (heavy the whole time) to the disk? I try to check this with oprofile too, but it seems not work proper with vmware ... Thanks Steeg

2 1

Index for objectclass does not work...
by Steeg Carson 03 Jan '11

03 Jan '11

Hello again, I can't solve my problem on my own... please help. As I can see at the moment, the Index for objectclass is not used. The searchtime with and without index for objectclass is the same! The search looks like: ldapsearch -x -h localhost -wpassword -D"uid=admin,ou=root" -b"cn=ownPath,ou=root" "(ObjectClass=subEngine)" The first serach, it takes about 40seconds. So after the cache is filled, the next searches takes only about 2 seconds (during both logging is turned off!)... It doesn't matter whether the index is set up in the config or not. To check, that the index is really set up (or not), I set loglevel 64 ind the slapd.conf: ####################################################################### @(#) $OpenLDAP: slapd 2.4.23 $#012#011opensuse-buildservice(a)opensuse.org line 30 (rootpw ***) line 31 (directory#011/var/lib/ldap_hdb) line 32 (logfile#011#011/var/log/openldap.log) line 33 (cachesize 1000000) line 34 (dncachesize 1000000) line 34 (idlcachesize 3000000) line 36 (dbnosync) line 38 (index objectClass eq) index objectClass 0x0004 line 39 (index cn eq,sub) index cn 0x0714 hdb_monitor_db_open: monitoring disabled; configure monitor database to enable slapd starting ####################################################################### The searched objectClass=subEngine exists 104384 times in the entire directory: (=> ldapsearch -x -h localhost -D"uid=admin,ou=root" -b"cn=ou=root>" "(ObjectClass=subEngine)" dn | grep "^dn:" | wc -l) If I configure with index for Objectclass, I get for the search following messages: ####################################################################### => bdb_equality_candidates (objectClass) => key_read <= bdb_index_read 470601 candidates <= bdb_equality_candidates: id=-1, first=228, last=470828 <= bdb_filter_candidates: id=-1 first=228 last=470828 <= bdb_list_candidates: id=-1 first=228 last=470828 <= bdb_filter_candidates: id=-1 first=228 last=470828 <= bdb_list_candidates: id=-1 first=40595 last=470828 <= bdb_filter_candidates: id=-1 first=40595 last=470828 bdb_search_candidates: id=-1 first=40595 last=470828 ####################################################################### If I search then in the logfile, I see 430233 messages like: "hdb_search: <candidate> <message: does not match filter | scope not okay>" If I configure without index for objectclass, I get for the search following messages: ####################################################################### => bdb_equality_candidates (objectClass) <= bdb_equality_candidates: (objectClass) not indexed <= bdb_filter_candidates: id=-1 first=1 last=470828 <= bdb_list_candidates: id=-1 first=1 last=470828 <= bdb_filter_candidates: id=-1 first=1 last=470828 <= bdb_list_candidates: id=-1 first=40595 last=470828 <= bdb_filter_candidates: id=-1 first=40595 last=470828 bdb_search_candidates: id=-1 first=40595 last=470828 ####################################################################### During initial set up of the Database (slapadd, copied from production, off-line with slapcat), the index for objectclass was build. I also tried to rebuild the index with slapindex. I configured a 64bit system, with 16GByte RAM and 2 CPUs under VMware ESX with own SAS-Storage (RAID10) for only this Machine. The entire directory holds 470812 entires. (=> ldapsearch -x -h localhost -wpassword -D"uid=admin,ou=root" -b"ou=root" "(objectClass=*)" dn | grep "^dn:" | wc -l) I configured slapd.conf as following: ####################################################################### include /etc/openldap/schema/core.schema include /etc/openldap/schema/cosine.schema include /etc/openldap/schema/inetorgperson.schema include /etc/openldap/schema/rfc2307bis.schema include /etc/openldap/schema/own.schema pidfile /var/run/slapd/slapd.pid argsfile /var/run/slapd/slapd.args modulepath /usr/lib/ldap moduleload back_hdb sizelimit -1 timelimit 300 disallow bind_anon gentlehup on tool-threads 2 # hdb database definitions database hdb suffix "ou=root" rootdn "uid=admin,ou=root" checkpoint 4096 15 # loglevel only for test, not during time measuring loglevel 33 # or for config file processing # loglevel 64 rootpw password directory /var/lib/ldap_hdb logfile /var/log/openldap.log cachesize 1000000 dncachesize 1000000 idlcachesize 3000000 dbnosync index objectClass,entryUUID,entryCSN eq index cn eq,sub ####################################################################### In DB_CONF I set 2GB BDB page cache (set_cachesize 2 0 1) Thanks for help, kindly regards Steeg PS: During the first search, the disk is heavy used (iotop) by writing from slapd.conf ... why it - should only read ...? -

3 3

Re: Windows ldp.exe kills openldap 2.4.23
by masarati＠aero.polimi.it 03 Jan '11

03 Jan '11

> On Sun, Jan 2, 2011 at 3:45 AM, <masarati(a)aero.polimi.it> wrote: >> >> >>>> Is this a know bug (see ITS#6570) or something new? Is there a >>>> workaround >>>> it? >>> >>> AFAIK it is not known. I am able to reproduce it (Linux, HEAD code), >>> so I >>> suggest you file an ITS. With respect to a workaround, it is not clear >>> what's the purpose of the operation you're trying to accomplish. In >>> general, a modrdn on the empty DN should be meaningless (although it >>> should not crash the server). >> >> A fix is in HEAD (slapd/modrdn.c); please test. p. Please keep replies on the thread. > Thanks, I'll give it a try. And sorry for the confusion, by "work > around" a meant securing server from a such easy DOS attack, > i.e.preventing a crush. > > Do you know when 2.4.24 is scheduled to come out? Do you still want me > to create an ITS? Yes, please, so we can track the issue. Thanks, p.

2 2

Re: Kerberos/GSSAPI issues
by Brian Candler 02 Jan '11

02 Jan '11

On Thu, Dec 30, 2010 at 11:49:23AM -0800, Russ Allbery wrote: > > Have you got the one-line patch? > > Included below. Thank you Russ. Discussed further on kerberos(a)mit.edu list, and ticket raised as http://bugzilla.cyrusimap.org/show_bug.cgi?id=3380 Regards, Brian.

1 0

SASL proxy auth problem
by Jaap Winius 02 Jan '11

02 Jan '11

Hi folks, Thanks to Pierangelo's last reply, I now know what I suspected: that my consumer servers are configured to authenticate to their providers using SASL/GSSAPI, but that sometimes they don't do this, especially with proxy authorization. I've documented the entire install process: * OpenLDAP provider with MIT Kerberos V on Debian squeeze http://www.rjsystems.nl/en/2100-d6-openldap-provider-kerberos.php * OpenLDAP consumer with MIT Kerberos V on Debian squeeze http://www.rjsystems.nl/en/2100-d6-openldap-consumer-kerberos.php The last time I followed these instructions to the letter, proxy authorization worked. Now I've booted up the same machines again a few days later and it no longer works: the consumer still uses SASL to bind with the provider for replication, but it uses a SIMPLE bind for proxy authorization. Of course that results in an error. Yet, the configuration seems unchanged. Has anyone else experienced this problem? Thanks, Jaap PS -- If anyone is interested, I can supply plenty of details. See also my post of 12/24/2010 03:25:51 AM CET with subject "No ProxyAuthz with SASL-GSSAPI?"

2 6

Re: Windows ldp.exe kills openldap 2.4.23
by masarati＠aero.polimi.it 02 Jan '11

02 Jan '11

>> Is this a know bug (see ITS#6570) or something new? Is there a >> workaround >> it? > > AFAIK it is not known. I am able to reproduce it (Linux, HEAD code), so I > suggest you file an ITS. With respect to a workaround, it is not clear > what's the purpose of the operation you're trying to accomplish. In > general, a modrdn on the empty DN should be meaningless (although it > should not crash the server). A fix is in HEAD (slapd/modrdn.c); please test. p.

1 0

Windows ldp.exe kills openldap 2.4.23
by Serge Dubrouski 01 Jan '11

01 Jan '11

Hello - I just found that ldp.exe tool that comes with Windows is able to successfully kill OpenLDAP 2.4.23 when one tries tp modify DN and submits an empty old DN. Log files on openldap show this: Dec 29 15:19:32 ottawa slapd[11818]: conn=1000 op=2 do_modrdn Dec 29 15:19:32 ottawa slapd[11818]: do_modrdn: dn () newrdn (C=US) newsuperior () Dec 29 15:19:32 ottawa slapd[11818]: >>> dnPrettyNormal: <> Dec 29 15:19:32 ottawa slapd[11818]: <<< dnPrettyNormal: <>, <> Dec 29 15:19:32 ottawa slapd[11818]: >>> dnPrettyNormal: <C=US> Dec 29 15:19:32 ottawa slapd[11818]: <<< dnPrettyNormal: <c=US>, <c=us> Dec 29 15:19:32 ottawa slapd[11818]: conn=1000 op=2 MODRDN dn="" Is this a know bug (see ITS#6570) or something new? Is there a workaround it? -- Serge Dubrouski.

2 1

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

openldap-technical