- openldap-bugs - openldap.org

Re: (ITS#7745) olcAuthzRegexp lowers cases substitution variables
by pierangelo.masarati＠polimi.it 13 Nov '13

13 Nov '13

On 11/13/2013 07:11 PM, whm(a)stanford.edu wrote: > Full_Name: Bill MacAllister > Version: 2.4.35 > OS: Debian 7 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (67.180.239.194) > > > We have noticed a problem with authentication for some Kerberos > principals. The olcAuthzRegexp processing seems to be lower casing > the input when forming substition variables. Since Kerberos > principals are case sensitive this causes authentications to fail. > > We have the following olcAuthzRegexp entries in our directory. > > % ldapsearch -b cn=config -LLL -Q "olcAuthzRegexp=*" olcAuthzRegexp > dn: cn=config > olcAuthzRegexp: {0}"uid=(.*)(a)win.stanford.edu,cn=stanford.edu,cn=gssapi,cn=aut > h" "ldap:///cn=service-win,cn=Applications,dc=stanford,dc=edu??sub?krb5Princi > palName=$1(a)WIN.STANFORD.EDU" > olcAuthzRegexp: {1}"uid=(.*)/cgi,cn=stanford.edu,cn=gssapi,cn=auth" "ldap:///c > n=cgi,cn=applications,dc=stanford,dc=edu??sub?krb5PrincipalName=$1/cgi@stanfo > rd.edu" > olcAuthzRegexp: {2}"uid=host/(.*),cn=stanford.edu,cn=gssapi,cn=auth" "ldap:/// > cn=Host,cn=Applications,dc=stanford,dc=edu??sub?krb5PrincipalName=host/$1@sta > nford.edu" > olcAuthzRegexp: {3}"uid=service/(.*),cn=stanford.edu,cn=gssapi,cn=auth" "ldap: > ///cn=Service,cn=Applications,dc=stanford,dc=edu??sub?krb5PrincipalName=servi > ce/$1(a)stanford.edu" > olcAuthzRegexp: {4}"uid=smtp/(.*),cn=stanford.edu,cn=gssapi,cn=auth" "ldap:/// > cn=smtp,cn=Applications,dc=stanford,dc=edu??sub?krb5PrincipalName=smtp/$1@sta > nford.edu" > olcAuthzRegexp: {5}"uid=webauth/(.*),cn=stanford.edu,cn=gssapi,cn=auth" "ldap: > ///cn=Webauth,cn=Applications,dc=stanford,dc=edu??sub?krb5PrincipalName=webau > th/$1(a)stanford.edu" > olcAuthzRegexp: {6}"uid=(.*),cn=stanford.edu,cn=gssapi,cn=auth" "ldap:///uid=$ > 1,cn=Accounts,dc=stanford,dc=edu??sub?suSeasStatus=active" > > We have the following entries in the directory holding Kerberos > principals. > > % ldapsearch -Q -LLL -b cn=service-win,cn=Applications,dc=stanford,dc=edu > "krb5PrincipalName=*" krb5PrincipalName > dn: cn=adharvservice,cn=service-win,cn=applications,dc=stanford,dc=edu > krb5PrincipalName: adharvservice(a)WIN.STANFORD.EDU > > dn: cn=EmailGroupMgr.svc,cn=service-win,cn=applications,dc=stanford,dc=edu > krb5PrincipalName: EmailGroupMgr.svc(a)WIN.STANFORD.EDU > > Searches using adharvservice(a)WIN.STANFORD.EDU work as expected, but searches > using EmailGroupMgr.svc(a)WIN.STANFORD.EDU fail. Here is what we see in > the log for a EmailGroupMgr.svc(a)WIN.STANFORD.EDU search: > > Nov 12 14:36:04 directory1 slapd[8804]: conn=2649928 fd=572 ACCEPT from > IP=171.64.18.139:3902 (IP=0.0.0.0:389) > Nov 12 14:36:04 directory1 slapd[8804]: conn=2649928 op=0 SRCH base="" scope=0 > deref=0 filter="(objectClass=*)" > Nov 12 14:36:04 directory1 slapd[8804]: conn=2649928 op=0 SRCH > attr=subschemaSubentry dsServiceName namingContexts defaultNamingContext > schemaNamingContext configurationNamingContext rootDomainNamingContext > supportedControl supportedLDAPVersion supportedLDAPPolicies > supportedSASLMechanisms dnsHostName ldapServiceName serverName > supportedCapabilities > Nov 12 14:36:04 directory1 slapd[8804]: conn=2649928 op=0 SEARCH RESULT tag=101 > err=0 nentries=1 text= > Nov 12 14:36:04 directory1 slapd[8804]: conn=2649928 op=1 BIND dn="" method=163 > Nov 12 14:36:04 directory1 slapd[8804]: conn=2649928 op=1 RESULT tag=97 err=14 > text=SASL(0): successful result: mech EXTERNAL is too weak > Nov 12 14:36:04 directory1 slapd[8804]: conn=2649928 op=2 BIND dn="" method=163 > Nov 12 14:36:04 directory1 slapd[8804]: conn=2649928 op=2 RESULT tag=97 err=14 > text=SASL(0): successful result: mech EXTERNAL is too weak > Nov 12 14:36:04 directory1 slapd[8804]: conn=2649928 op=3 BIND dn="" method=163 > Nov 12 14:36:04 directory1 slapd[8804]: conn=2649928 op=3 BIND > authcid="EmailGroupMgr.svc(a)WIN.STANFORD.EDU" > authzid="EmailGroupMgr.svc(a)WIN.STANFORD.EDU" > Nov 12 14:36:04 directory1 slapd[8804]: conn=2649928 op=3 BIND > dn="uid=emailgroupmgr.svc(a)win.stanford.edu,cn=stanford.edu,cn=gssapi,cn=auth" > mech=GSSAPI sasl_ssf=56 ssf=56 > Nov 12 14:36:04 directory1 slapd[8804]: conn=2649928 op=3 RESULT tag=97 err=0 > text= > Nov 12 14:36:04 directory1 slapd[8804]: conn=2649928 op=4 SRCH > base="cn=accounts,dc=stanford,dc=edu" scope=2 deref=0 filter="(uid=whm)" > Nov 12 14:36:04 directory1 slapd[8804]: conn=2649928 op=4 SRCH attr=uid > suSeasLocal > Nov 12 14:36:04 directory1 slapd[8804]: conn=2649928 op=4 SEARCH RESULT tag=101 > err=0 nentries=0 text= > Nov 12 14:37:43 directory1 slapd[8804]: conn=2649928 op=5 UNBIND > Nov 12 14:37:43 directory1 slapd[8804]: conn=2649928 fd=572 closed > > Note that the authcid is EmailGroupMgr.svc(a)WIN.STANFORD.EDU, but it > gets mapped into: > > uid=emailgroupmgr.svc(a)win.stanford.edu,cn=stanford.edu,cn=stanford.edu,cn=gssapi,cn=auth This is because the construction of such DN makes use of "uid", whose matching rule is case insensitive. As a workaround (assuming this is acceptable for you), you could probably modify your olcAuthzRegexp rules to force case-insensitive lookup of krb5PrincipalName, something like olcAuthzRegexp: {0}"uid=(.*)(a)win.stanford.edu,cn=stanford.edu,cn=gssapi,cn=auth" "ldap:///cn=service-win,cn=Applications,dc=stanford,dc=edu??sub?(krb5PrincipalName:caseIgnoreIA5Match:=$1@WIN.STANFORD.EDU)" or something like that. p. > > I guessed that the substitution in the olcAuthzRegex is the problem > and that the principal is being converted to lower case. > > Our attribute definition for krb5PrincipalName is: > > olcAttributeTypes: {0}( 1.3.6.1.4.1.5322.10.1.1 > NAME 'krb5PrincipalName' > DESC 'The unparsed Kerberos principal name' > EQUALITY caseExactIA5Match > SYNTAX 1.3.6.1.4.1.1466.115.121.1.26 > SINGLE-VALUE ) > > But, if I change the krb5PrincipalName in the directory to be all > lower case as follows: > > % ldapsearch -Q -LLL -b cn=service-win,cn=Applications,dc=stanford,dc=edu > krb5PrincipalName=* krb5PrincipalName > dn: cn=adharvservice,cn=service-win,cn=applications,dc=stanford,dc=edu > krb5PrincipalName: adharvservice(a)WIN.STANFORD.EDU > > dn: cn=EmailGroupMgr.svc,cn=service-win,cn=applications,dc=stanford,dc=edu > krb5PrincipalName: emailgroupmgr.svc(a)WIN.STANFORD.EDU > > Then the authentication succeeds: > > Nov 13 08:13:38 directory2 slapd[31590]: conn=3197185 fd=46 ACCEPT from > IP=171.64.18.139:5897 (IP=0.0.0.0:389) > Nov 13 08:13:38 directory2 slapd[31590]: conn=3197185 op=0 SRCH base="" scope=0 > deref=0 filter="(objectClass=*)" > Nov 13 08:13:38 directory2 slapd[31590]: conn=3197185 op=0 SRCH > attr=subschemaSubentry dsServiceName namingContexts defaultNamingContext > schemaNamingContext configurationNamingContext rootDomainNamingContext > supportedControl supportedLDAPVersion supportedLDAPPolicies > supportedSASLMechanisms dnsHostName ldapServiceName serverName > supportedCapabilities > Nov 13 08:13:38 directory2 slapd[31590]: conn=3197185 op=0 SEARCH RESULT tag=101 > err=0 nentries=1 text= > Nov 13 08:13:38 directory2 slapd[31590]: conn=3197185 op=1 BIND dn="" > method=163 > Nov 13 08:13:38 directory2 slapd[31590]: conn=3197185 op=1 RESULT tag=97 err=14 > text=SASL(0): successful result: mech EXTERNAL is too weak > Nov 13 08:13:38 directory2 slapd[31590]: conn=3197185 op=2 BIND dn="" > method=163 > Nov 13 08:13:38 directory2 slapd[31590]: conn=3197185 op=2 RESULT tag=97 err=14 > text=SASL(0): successful result: mech EXTERNAL is too weak > Nov 13 08:13:38 directory2 slapd[31590]: conn=3197185 op=3 BIND dn="" > method=163 > Nov 13 08:13:38 directory2 slapd[31590]: conn=3197185 op=3 BIND > authcid="EmailGroupMgr.svc(a)WIN.STANFORD.EDU" > authzid="EmailGroupMgr.svc(a)WIN.STANFORD.EDU" > Nov 13 08:13:38 directory2 slapd[31590]: conn=3197185 op=3 BIND > dn="cn=emailgroupmgr.svc,cn=service-win,cn=applications,dc=stanford,dc=edu" > mech=GSSAPI sasl_ssf=56 ssf=56 > Nov 13 08:13:38 directory2 slapd[31590]: conn=3197185 op=3 RESULT tag=97 err=0 > text= > Nov 13 08:13:38 directory2 slapd[31590]: conn=3197185 op=4 SRCH > base="cn=accounts,dc=stanford,dc=edu" scope=2 deref=0 filter="(uid=whm)" > Nov 13 08:13:38 directory2 slapd[31590]: conn=3197185 op=4 SRCH attr=uid > suSeasLocal > Nov 13 08:13:38 directory2 slapd[31590]: conn=3197185 op=4 SEARCH RESULT tag=101 > err=0 nentries=1 text= > Nov 13 08:15:16 directory2 slapd[31590]: conn=3197185 op=5 UNBIND > Nov 13 08:15:16 directory2 slapd[31590]: conn=3197185 fd=46 closed > > > > -- Pierangelo Masarati Associate Professor Dipartimento di Scienze e Tecnologie Aerospaziali Politecnico di Milano

1 0

(ITS#7745) olcAuthzRegexp lowers cases substitution variables
by whm＠stanford.edu 13 Nov '13

13 Nov '13

Full_Name: Bill MacAllister Version: 2.4.35 OS: Debian 7 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (67.180.239.194) We have noticed a problem with authentication for some Kerberos principals. The olcAuthzRegexp processing seems to be lower casing the input when forming substition variables. Since Kerberos principals are case sensitive this causes authentications to fail. We have the following olcAuthzRegexp entries in our directory. % ldapsearch -b cn=config -LLL -Q "olcAuthzRegexp=*" olcAuthzRegexp dn: cn=config olcAuthzRegexp: {0}"uid=(.*)(a)win.stanford.edu,cn=stanford.edu,cn=gssapi,cn=aut h" "ldap:///cn=service-win,cn=Applications,dc=stanford,dc=edu??sub?krb5Princi palName=$1(a)WIN.STANFORD.EDU" olcAuthzRegexp: {1}"uid=(.*)/cgi,cn=stanford.edu,cn=gssapi,cn=auth" "ldap:///c n=cgi,cn=applications,dc=stanford,dc=edu??sub?krb5PrincipalName=$1/cgi@stanfo rd.edu" olcAuthzRegexp: {2}"uid=host/(.*),cn=stanford.edu,cn=gssapi,cn=auth" "ldap:/// cn=Host,cn=Applications,dc=stanford,dc=edu??sub?krb5PrincipalName=host/$1@sta nford.edu" olcAuthzRegexp: {3}"uid=service/(.*),cn=stanford.edu,cn=gssapi,cn=auth" "ldap: ///cn=Service,cn=Applications,dc=stanford,dc=edu??sub?krb5PrincipalName=servi ce/$1(a)stanford.edu" olcAuthzRegexp: {4}"uid=smtp/(.*),cn=stanford.edu,cn=gssapi,cn=auth" "ldap:/// cn=smtp,cn=Applications,dc=stanford,dc=edu??sub?krb5PrincipalName=smtp/$1@sta nford.edu" olcAuthzRegexp: {5}"uid=webauth/(.*),cn=stanford.edu,cn=gssapi,cn=auth" "ldap: ///cn=Webauth,cn=Applications,dc=stanford,dc=edu??sub?krb5PrincipalName=webau th/$1(a)stanford.edu" olcAuthzRegexp: {6}"uid=(.*),cn=stanford.edu,cn=gssapi,cn=auth" "ldap:///uid=$ 1,cn=Accounts,dc=stanford,dc=edu??sub?suSeasStatus=active" We have the following entries in the directory holding Kerberos principals. % ldapsearch -Q -LLL -b cn=service-win,cn=Applications,dc=stanford,dc=edu "krb5PrincipalName=*" krb5PrincipalName dn: cn=adharvservice,cn=service-win,cn=applications,dc=stanford,dc=edu krb5PrincipalName: adharvservice(a)WIN.STANFORD.EDU dn: cn=EmailGroupMgr.svc,cn=service-win,cn=applications,dc=stanford,dc=edu krb5PrincipalName: EmailGroupMgr.svc(a)WIN.STANFORD.EDU Searches using adharvservice(a)WIN.STANFORD.EDU work as expected, but searches using EmailGroupMgr.svc(a)WIN.STANFORD.EDU fail. Here is what we see in the log for a EmailGroupMgr.svc(a)WIN.STANFORD.EDU search: Nov 12 14:36:04 directory1 slapd[8804]: conn=2649928 fd=572 ACCEPT from IP=171.64.18.139:3902 (IP=0.0.0.0:389) Nov 12 14:36:04 directory1 slapd[8804]: conn=2649928 op=0 SRCH base="" scope=0 deref=0 filter="(objectClass=*)" Nov 12 14:36:04 directory1 slapd[8804]: conn=2649928 op=0 SRCH attr=subschemaSubentry dsServiceName namingContexts defaultNamingContext schemaNamingContext configurationNamingContext rootDomainNamingContext supportedControl supportedLDAPVersion supportedLDAPPolicies supportedSASLMechanisms dnsHostName ldapServiceName serverName supportedCapabilities Nov 12 14:36:04 directory1 slapd[8804]: conn=2649928 op=0 SEARCH RESULT tag=101 err=0 nentries=1 text= Nov 12 14:36:04 directory1 slapd[8804]: conn=2649928 op=1 BIND dn="" method=163 Nov 12 14:36:04 directory1 slapd[8804]: conn=2649928 op=1 RESULT tag=97 err=14 text=SASL(0): successful result: mech EXTERNAL is too weak Nov 12 14:36:04 directory1 slapd[8804]: conn=2649928 op=2 BIND dn="" method=163 Nov 12 14:36:04 directory1 slapd[8804]: conn=2649928 op=2 RESULT tag=97 err=14 text=SASL(0): successful result: mech EXTERNAL is too weak Nov 12 14:36:04 directory1 slapd[8804]: conn=2649928 op=3 BIND dn="" method=163 Nov 12 14:36:04 directory1 slapd[8804]: conn=2649928 op=3 BIND authcid="EmailGroupMgr.svc(a)WIN.STANFORD.EDU" authzid="EmailGroupMgr.svc(a)WIN.STANFORD.EDU" Nov 12 14:36:04 directory1 slapd[8804]: conn=2649928 op=3 BIND dn="uid=emailgroupmgr.svc(a)win.stanford.edu,cn=stanford.edu,cn=gssapi,cn=auth" mech=GSSAPI sasl_ssf=56 ssf=56 Nov 12 14:36:04 directory1 slapd[8804]: conn=2649928 op=3 RESULT tag=97 err=0 text= Nov 12 14:36:04 directory1 slapd[8804]: conn=2649928 op=4 SRCH base="cn=accounts,dc=stanford,dc=edu" scope=2 deref=0 filter="(uid=whm)" Nov 12 14:36:04 directory1 slapd[8804]: conn=2649928 op=4 SRCH attr=uid suSeasLocal Nov 12 14:36:04 directory1 slapd[8804]: conn=2649928 op=4 SEARCH RESULT tag=101 err=0 nentries=0 text= Nov 12 14:37:43 directory1 slapd[8804]: conn=2649928 op=5 UNBIND Nov 12 14:37:43 directory1 slapd[8804]: conn=2649928 fd=572 closed Note that the authcid is EmailGroupMgr.svc(a)WIN.STANFORD.EDU, but it gets mapped into: uid=emailgroupmgr.svc(a)win.stanford.edu,cn=stanford.edu,cn=stanford.edu,cn=gssapi,cn=auth I guessed that the substitution in the olcAuthzRegex is the problem and that the principal is being converted to lower case. Our attribute definition for krb5PrincipalName is: olcAttributeTypes: {0}( 1.3.6.1.4.1.5322.10.1.1 NAME 'krb5PrincipalName' DESC 'The unparsed Kerberos principal name' EQUALITY caseExactIA5Match SYNTAX 1.3.6.1.4.1.1466.115.121.1.26 SINGLE-VALUE ) But, if I change the krb5PrincipalName in the directory to be all lower case as follows: % ldapsearch -Q -LLL -b cn=service-win,cn=Applications,dc=stanford,dc=edu krb5PrincipalName=* krb5PrincipalName dn: cn=adharvservice,cn=service-win,cn=applications,dc=stanford,dc=edu krb5PrincipalName: adharvservice(a)WIN.STANFORD.EDU dn: cn=EmailGroupMgr.svc,cn=service-win,cn=applications,dc=stanford,dc=edu krb5PrincipalName: emailgroupmgr.svc(a)WIN.STANFORD.EDU Then the authentication succeeds: Nov 13 08:13:38 directory2 slapd[31590]: conn=3197185 fd=46 ACCEPT from IP=171.64.18.139:5897 (IP=0.0.0.0:389) Nov 13 08:13:38 directory2 slapd[31590]: conn=3197185 op=0 SRCH base="" scope=0 deref=0 filter="(objectClass=*)" Nov 13 08:13:38 directory2 slapd[31590]: conn=3197185 op=0 SRCH attr=subschemaSubentry dsServiceName namingContexts defaultNamingContext schemaNamingContext configurationNamingContext rootDomainNamingContext supportedControl supportedLDAPVersion supportedLDAPPolicies supportedSASLMechanisms dnsHostName ldapServiceName serverName supportedCapabilities Nov 13 08:13:38 directory2 slapd[31590]: conn=3197185 op=0 SEARCH RESULT tag=101 err=0 nentries=1 text= Nov 13 08:13:38 directory2 slapd[31590]: conn=3197185 op=1 BIND dn="" method=163 Nov 13 08:13:38 directory2 slapd[31590]: conn=3197185 op=1 RESULT tag=97 err=14 text=SASL(0): successful result: mech EXTERNAL is too weak Nov 13 08:13:38 directory2 slapd[31590]: conn=3197185 op=2 BIND dn="" method=163 Nov 13 08:13:38 directory2 slapd[31590]: conn=3197185 op=2 RESULT tag=97 err=14 text=SASL(0): successful result: mech EXTERNAL is too weak Nov 13 08:13:38 directory2 slapd[31590]: conn=3197185 op=3 BIND dn="" method=163 Nov 13 08:13:38 directory2 slapd[31590]: conn=3197185 op=3 BIND authcid="EmailGroupMgr.svc(a)WIN.STANFORD.EDU" authzid="EmailGroupMgr.svc(a)WIN.STANFORD.EDU" Nov 13 08:13:38 directory2 slapd[31590]: conn=3197185 op=3 BIND dn="cn=emailgroupmgr.svc,cn=service-win,cn=applications,dc=stanford,dc=edu" mech=GSSAPI sasl_ssf=56 ssf=56 Nov 13 08:13:38 directory2 slapd[31590]: conn=3197185 op=3 RESULT tag=97 err=0 text= Nov 13 08:13:38 directory2 slapd[31590]: conn=3197185 op=4 SRCH base="cn=accounts,dc=stanford,dc=edu" scope=2 deref=0 filter="(uid=whm)" Nov 13 08:13:38 directory2 slapd[31590]: conn=3197185 op=4 SRCH attr=uid suSeasLocal Nov 13 08:13:38 directory2 slapd[31590]: conn=3197185 op=4 SEARCH RESULT tag=101 err=0 nentries=1 text= Nov 13 08:15:16 directory2 slapd[31590]: conn=3197185 op=5 UNBIND Nov 13 08:15:16 directory2 slapd[31590]: conn=3197185 fd=46 closed

1 0

(ITS#7744) [Patch] TLS_REQCERT section in ldap.conf is confusing
by jsynacek＠redhat.com 13 Nov '13

13 Nov '13

Full_Name: Jan Synacek Version: master OS: Linux - Fedora 19 URL: http://jsynacek.fedorapeople.org/openldap/jsynacek-20131113-0001-Fix-client… Submission from: (NULL) (209.132.186.34) Quoting ldap.conf(5): TLS_REQCERT <level> ... try The server certificate is requested. If no certificate is provided, the session proceeds normally. If a bad certificate is provided, the session is immediately terminated. There is currently no way how to "provide no server certificate" and successfully connect via a client (e.g. ldapsearch). For additional discussion, see http://www.openldap.org/lists/openldap-technical/201311/msg00099.html.

1 0

Re: (ITS#7723) slapd crashes on multi core machines if a search request is *immediately* followed by an unbind
by jsynacek＠redhat.com 13 Nov '13

13 Nov '13

On 11/04/2013 03:11 PM, hyc(a)symas.com wrote: > jsynacek(a)redhat.com wrote: >> On 10/15/2013 01:10 PM, michael.vishchers(a)7p-group.com wrote: >>> It is not the client loop that is multithreading but the ldap server. >>> >>> And it is not a misuse of the API but a problem that may be raised by day t= >>> o day network problems. >>> >>> I've boiled down the problem to a few simple configurations that work (or b= >>> etter, fail ;-) with both 2.4.23 and 2.4.36. A tgz file containing a setup = >>> with start script and testclient is attached. It should be sufficient to re= >>> produce the fault. >>> >>> The problem occurs only if we use session variable substitution in the rwm = >>> overlay, and only if a search is *immediately* (e.g. caused by network loss= >>> and client timeout) followed by an unbind. >>> >> >> I modified the reproducer a bit (the start script) and find out a few things. >> You can find the reproducer I'm using at [1]. >> >> Valgrind's helgrind shows some lock problems in the rwm overlay and also in >> back-ldap and connection.c. After correcting those the issue seems to be gone. >> >> You can find helgrind logs at [2] (before the fix) and [3] (after). >> >> Also, ElectricFence reveals some problems [4], which I didn't fix yet. >> >> A fix attempt can be found at [5]. I'm not sure if that is a correct fix, or it >> just masked the real issue. But I didn't to manage to reproduce the problem >> after applying it. > > I already explained the problem. The other issues you identified are not > relevant, and your patch is not correct. Reread Followup #4 of this ITS. > Another take on the fix: http://jsynacek.fedorapeople.org/openldap/its7723/0001-ITS-7723-fix-referen… -- Jan Synacek Software Engineer, Red Hat

1 0

Re: (ITS#7743) bdb_idl_intersection() seems to expand the search candidates unnecessarily
by nishino-yoshinori＠mxc.nes.nec.co.jp 12 Nov '13

12 Nov '13

Dear Howard, Thank you for your prompt reply. >This is certainly no bug in idl_intersection(), it is an >optimizatio that makes subsequent intersection operations faster. >As such, your suggested fix is wrong. I understood the line 1094-1099 part of idl.c was implemented to optimize subsequent intersection oprations. On the other hands, in this case, bdb_idl_intersection(C,D) (as I described before) returns the range-type candidate set despite its having only two elements. Then, by subsequent bdb_idl_union() with A (list-type: the number of elements=296, minID=296,maxID=858689), the candidate set becomes very huge. #bdb_idl_union() seems to change the candidate set into "range-type" #if one of the two sets is a "range-type". The impact of removing the optimization of bdb_idl_intersection() is only that subsequent bdb_idl_intersection() oprations become slower in certain cases? If the aforementioned optimization becomes effective when the number of the element is more than certain threshold number(ex.1024), I would like to change bdb_idl_intersection() like that: ex) 1091 if ( BDB_IDL_IS_RANGE( b ) 1092 && BDB_IDL_RANGE_FIRST( b ) <= BDB_IDL_RANGE_FIRST( a ) 1093 && BDB_IDL_RANGE_LAST( b ) >= BDB_IDL_RANGE_LAST( a ) ) { 1094 if (idmax - idmin + 1 == a[0] && a[0] > 1024) 1095 { 1096 a[0] = NOID; 1097 a[1] = idmin; 1098 a[2] = idmax; 1099 } 1100 goto done; 1101 } The issue occurs in the environment of our user and makes some trouble because of the slow response of the ldapsearch. So, I would like to find some fix of the issue. >If you want to suggest a fix, you should look at idl_union() instead. Thank you for your advice. I'll also try to check bdb_idl_union(). If you have other good idea to fix the issue, could you let me know? Best Regards, -- ********************************************* Yoshinori Nishino NEC Soft, Ltd. E-MAIL: nishino-yoshinori(a)mxc.nes.nec.co.jp *********************************************

1 0

Re: (ITS#7743) bdb_idl_intersection() seems to expand the search candidates unnecessarily
by nishino-yoshinori＠mxc.nes.nec.co.jp 12 Nov '13

12 Nov '13

Dear Quanah, Thank you for your prompt reply. > back-bdb is deprecated, I suggest you use back-mdb instead. Would you tell me why you recommend using back-mdb? It's because the collision of the hash key in the bdb index occurs more easily than in the mdb index? Best Regards, -- ********************************************* Yoshinori Nishino NEC Soft, Ltd. E-MAIL: nishino-yoshinori(a)mxc.nes.nec.co.jp *********************************************

1 0

Re: (ITS#7743) bdb_idl_intersection() seems to expand the search candidates unnecessarily
by hyc＠symas.com 12 Nov '13

12 Nov '13

nishino-yoshinori(a)mxc.nes.nec.co.jp wrote: > Full_Name: Yoshinori Nishino > Version: 2.4.37 > OS: RedHatEL 5.5 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (210.143.35.12) > > > Dear sir, > > OS:Red Hat Enterprise Linux 5.5(x86) (2.6.18-194.el5) > Version: 2.4.37 + BDB 4.4.20 > > In our environment, the issue occurs that the ldapsearch command using the > following filter always took about 15`30 seconds. > The response of the ldapsearch command becomes faster by modifying idl.c like > that: > > 1087 /* If a range completely covers the list, the result is > > 1088 * just the list. If idmin to idmax is contiguous, just > > 1089 * turn it into a range. > > 1090 */ > 1091 if ( BDB_IDL_IS_RANGE( b ) > > 1092 && BDB_IDL_RANGE_FIRST( b ) <= BDB_IDL_RANGE_FIRST( a ) > > 1093 && BDB_IDL_RANGE_LAST( b ) >= BDB_IDL_RANGE_LAST( a ) ) > { > 1094 /* if (idmax - idmin + 1 == a[0]) > > 1095 { > > 1096 a[0] = NOID; > > 1097 a[1] = idmin; > > 1098 a[2] = idmax; > > 1099 } */ > > 1100 goto done; > > 1101 } > > By the aforementioned modification, the search candidate set becomes (list-type: > the number of elements=298, minID=296,maxID=858689). > > I think the line 1094-1099 part of idl.c seems to be a bug. > The part seems to be implemented in order to just make "ids[]" smaller. > If the reason is so, would you please consider fixing idl.c? This is certainly no bug in idl_intersection(), it is an optimization that makes subsequent intersection operations faster. As such, your suggested fix is wrong. If you want to suggest a fix, you should look at idl_union() instead. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7743) bdb_idl_intersection() seems to expand the search candidates unnecessarily
by quanah＠zimbra.com 12 Nov '13

12 Nov '13

--On Tuesday, November 12, 2013 3:25 AM +0000 quanah(a)zimbra.com wrote: > --On Tuesday, November 12, 2013 3:09 AM +0000 > nishino-yoshinori(a)mxc.nes.nec.co.jp wrote: > >> Full_Name: Yoshinori Nishino >> Version: 2.4.37 >> OS: RedHatEL 5.5 >> URL: ftp://ftp.openldap.org/incoming/ >> Submission from: (NULL) (210.143.35.12) > > back-bdb is deprecated, I suggest you use back-mdb instead. Hm, I see the same bit exists for back-mdb too at line 720ish. Oh well. :P --Quanah -- Quanah Gibson-Mount Architect - Server Zimbra, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

Re: (ITS#7743) bdb_idl_intersection() seems to expand the search candidates unnecessarily
by quanah＠zimbra.com 12 Nov '13

12 Nov '13

--On Tuesday, November 12, 2013 3:09 AM +0000 nishino-yoshinori(a)mxc.nes.nec.co.jp wrote: > Full_Name: Yoshinori Nishino > Version: 2.4.37 > OS: RedHatEL 5.5 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (210.143.35.12) back-bdb is deprecated, I suggest you use back-mdb instead. --Quanah -- Quanah Gibson-Mount Architect - Server Zimbra, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

(ITS#7743) bdb_idl_intersection() seems to expand the search candidates unnecessarily
by nishino-yoshinori＠mxc.nes.nec.co.jp 12 Nov '13

12 Nov '13

Full_Name: Yoshinori Nishino Version: 2.4.37 OS: RedHatEL 5.5 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (210.143.35.12) Dear sir, OS:Red Hat Enterprise Linux 5.5(x86) (2.6.18-194.el5) Version: 2.4.37 + BDB 4.4.20 In our environment, the issue occurs that the ldapsearch command using the following filter always took about 15`30 seconds. '(|(mailLocalAddress=XXXXXXX@xxxx.yy.zz)(mailRoutingAddress=XXXXXXX@xxxx.yy.zz)(&(xxxXXXXAlias=XXXXXXX)(xxxXXXYYYYYYList=bb)))' # masked the private information for the security reason. As far as I checked the debug.log("loglevel -1" in slapd.conf) and servers/slapd/back-bdb/idl.c, bdb_idl_intersection() seems to expand the search candidates unnecessarily, which results in this slow response. #Please see the log below(masked the private information for the security reason) The detail that I expect is as follows: == 1. By seaeching the index datebases, slapd got the candidate sets for each of the attributes. A.The candidate set of mailLocalAddress: (list-type: the number of elements=296, minID=296,maxID=858689) B.The candidate set of mailRoutingAddress:(list-type: the number of elements=0, minID=0,maxID=0) C.The candidate set of xxxXXXXAlias:(list-type: the number of elements=2, minID=676406, maxID=676407) D.The candidate set of xxxXXXYYYYYYList:(range-type: minID=628610, maxID=868129) #become range-type because the number of elements is more than 65536 2. First, slapd ran bdb_idl_intersection(C,D) according to the search filter, which resulted in the following candidate set. E.(range-type: minID=676406, maxID=676407) 3. Next, slapd ran bdb_idl_union(E,A), which resulted in the following. F.(range-type: minID=296, maxID=858689) 4. Finally, slapd ran bdb_idl_union(F,B), which resulted in the following search candidate. G.(range-type: minID=296, maxID=858689) Therefore, slapd searched many candidates unnecessarily. == The issues seems to occur because the following part of idl.c. 1087 /* If a range completely covers the list, the result is 1088 * just the list. If idmin to idmax is contiguous, just 1089 * turn it into a range. 1090 */ 1091 if ( BDB_IDL_IS_RANGE( b ) 1092 && BDB_IDL_RANGE_FIRST( b ) <= BDB_IDL_RANGE_FIRST( a ) 1093 && BDB_IDL_RANGE_LAST( b ) >= BDB_IDL_RANGE_LAST( a ) ) { 1094 if (idmax - idmin + 1 == a[0]) 1095 { 1096 a[0] = NOID; 1097 a[1] = idmin; 1098 a[2] = idmax; 1099 } 1100 goto done; 1101 } In the aforementioned "2", the set C matches the "if" condition of line 1094. Therefore the result of bdb_idl_intersection() becomes the range-type candidate set. If the set C didn't matche the condition, slapd would run "goto"(line 1094), according to which the result of bdb_idl_intersection(C,D) would be just "C"(list-type: the number of elements=2, minID=676406, maxID=676407). However, because the bdb_idl_intersection() make the result set changed into "range-type" unnecessarily, slapd is finally forced to search about 800,000 entries in vain. The response of the ldapsearch command becomes faster by modifying idl.c like that: 1087 /* If a range completely covers the list, the result is 1088 * just the list. If idmin to idmax is contiguous, just 1089 * turn it into a range. 1090 */ 1091 if ( BDB_IDL_IS_RANGE( b ) 1092 && BDB_IDL_RANGE_FIRST( b ) <= BDB_IDL_RANGE_FIRST( a ) 1093 && BDB_IDL_RANGE_LAST( b ) >= BDB_IDL_RANGE_LAST( a ) ) { 1094 /* if (idmax - idmin + 1 == a[0]) 1095 { 1096 a[0] = NOID; 1097 a[1] = idmin; 1098 a[2] = idmax; 1099 } */ 1100 goto done; 1101 } By the aforementioned modification, the search candidate set becomes (list-type: the number of elements=298, minID=296,maxID=858689). I think the line 1094-1099 part of idl.c seems to be a bug. The part seems to be implemented in order to just make "ids[]" smaller. If the reason is so, would you please consider fixing idl.c? *debug log(masked for security reason) Nov 12 10:42:58 server2 slapd[714]: daemon: activity on 1 descriptor Nov 12 10:42:58 server2 slapd[714]: daemon: activity on: Nov 12 10:42:58 server2 slapd[714]: Nov 12 10:42:58 server2 slapd[714]: slap_listener_activate(8): Nov 12 10:42:58 server2 slapd[714]: daemon: epoll: listen=7 active_threads=0 tvp=zero Nov 12 10:42:58 server2 slapd[714]: daemon: epoll: listen=8 busy Nov 12 10:42:58 server2 slapd[714]: >>> slap_listener(ldap:///) Nov 12 10:42:58 server2 slapd[714]: daemon: listen=8, new connection on 10 Nov 12 10:42:58 server2 slapd[714]: daemon: added 10r (active) listener=(nil) Nov 12 10:42:58 server2 slapd[714]: conn=1000 fd=10 ACCEPT from IP=127.0.0.1:51095 (IP=0.0.0.0:389) Nov 12 10:42:58 server2 slapd[714]: daemon: activity on 2 descriptors Nov 12 10:42:58 server2 slapd[714]: daemon: activity on: Nov 12 10:42:58 server2 slapd[714]: 10r Nov 12 10:42:58 server2 slapd[714]: Nov 12 10:42:58 server2 slapd[714]: daemon: read active on 10 Nov 12 10:42:58 server2 slapd[714]: daemon: epoll: listen=7 active_threads=0 tvp=zero Nov 12 10:42:58 server2 slapd[714]: daemon: epoll: listen=8 active_threads=0 tvp=zero Nov 12 10:42:58 server2 slapd[714]: connection_get(10) Nov 12 10:42:58 server2 slapd[714]: connection_get(10): got connid=1000 Nov 12 10:42:58 server2 slapd[714]: connection_read(10): checking for input on id=1000 Nov 12 10:42:58 server2 slapd[714]: op tag 0x60, time 1384220578 Nov 12 10:42:58 server2 slapd[714]: daemon: activity on 1 descriptor Nov 12 10:42:58 server2 slapd[714]: daemon: activity on: Nov 12 10:42:58 server2 slapd[714]: Nov 12 10:42:58 server2 slapd[714]: daemon: epoll: listen=7 active_threads=0 tvp=zero Nov 12 10:42:58 server2 slapd[714]: daemon: epoll: listen=8 active_threads=0 tvp=zero Nov 12 10:42:58 server2 slapd[714]: conn=1000 op=0 do_bind Nov 12 10:42:58 server2 slapd[714]: >>> dnPrettyNormal: <cn=XXXXXXXX> Nov 12 10:42:58 server2 slapd[714]: <<< dnPrettyNormal: <cn=XXXXXXXX>, <cn=XXXXXXXX> Nov 12 10:42:58 server2 slapd[714]: conn=1000 op=0 BIND dn="cn=XXXXXXXX" method=128 Nov 12 10:42:58 server2 slapd[714]: do_bind: version=3 dn="cn=XXXXXXXX" method=128 Nov 12 10:42:58 server2 slapd[714]: ==> bdb_bind: dn: cn=XXXXXXXX Nov 12 10:42:58 server2 slapd[714]: conn=1000 op=0 BIND dn="cn=XXXXXXXX" mech=SIMPLE ssf=0 Nov 12 10:42:58 server2 slapd[714]: do_bind: v3 bind: "cn=XXXXXXXX" to "cn=XXXXXXXX" Nov 12 10:42:58 server2 slapd[714]: send_ldap_result: conn=1000 op=0 p=3 Nov 12 10:42:58 server2 slapd[714]: send_ldap_result: err=0 matched="" text="" Nov 12 10:42:58 server2 slapd[714]: send_ldap_response: msgid=1 tag=97 err=0 Nov 12 10:42:58 server2 slapd[714]: daemon: activity on 1 descriptor Nov 12 10:42:58 server2 slapd[714]: daemon: activity on: Nov 12 10:42:58 server2 slapd[714]: 10r Nov 12 10:42:58 server2 slapd[714]: Nov 12 10:42:58 server2 slapd[714]: daemon: read active on 10 Nov 12 10:42:58 server2 slapd[714]: daemon: epoll: listen=7 active_threads=0 tvp=zero Nov 12 10:42:58 server2 slapd[714]: daemon: epoll: listen=8 active_threads=0 tvp=zero Nov 12 10:42:58 server2 slapd[714]: conn=1000 op=0 RESULT tag=97 err=0 text= Nov 12 10:42:58 server2 slapd[714]: connection_get(10) Nov 12 10:42:58 server2 slapd[714]: connection_get(10): got connid=1000 Nov 12 10:42:58 server2 slapd[714]: connection_read(10): checking for input on id=1000 Nov 12 10:42:58 server2 slapd[714]: op tag 0x63, time 1384220578 Nov 12 10:42:58 server2 slapd[714]: daemon: activity on 1 descriptor Nov 12 10:42:58 server2 slapd[714]: daemon: activity on: Nov 12 10:42:58 server2 slapd[714]: Nov 12 10:42:58 server2 slapd[714]: daemon: epoll: listen=7 active_threads=0 tvp=zero Nov 12 10:42:58 server2 slapd[714]: daemon: epoll: listen=8 active_threads=0 tvp=zero Nov 12 10:42:58 server2 slapd[714]: conn=1000 op=1 do_search Nov 12 10:42:58 server2 slapd[714]: >>> dnPrettyNormal: <ou=People,dc=yyyy,dc=zz> Nov 12 10:42:58 server2 slapd[714]: <<< dnPrettyNormal: <ou=People,dc=yyyy,dc=zz>, <ou=people,dc=yyyy,dc=zz> Nov 12 10:42:58 server2 slapd[714]: SRCH "ou=People,dc=yyyy,dc=zz" 2 0 Nov 12 10:42:58 server2 slapd[714]: 0 0 0 Nov 12 10:42:58 server2 slapd[714]: begin get_filter Nov 12 10:42:58 server2 slapd[714]: OR Nov 12 10:42:58 server2 slapd[714]: begin get_filter_list Nov 12 10:42:58 server2 slapd[714]: begin get_filter Nov 12 10:42:58 server2 slapd[714]: EQUALITY Nov 12 10:42:58 server2 slapd[714]: end get_filter 0 Nov 12 10:42:58 server2 slapd[714]: begin get_filter Nov 12 10:42:58 server2 slapd[714]: EQUALITY Nov 12 10:42:58 server2 slapd[714]: end get_filter 0 Nov 12 10:42:58 server2 slapd[714]: begin get_filter Nov 12 10:42:58 server2 slapd[714]: AND Nov 12 10:42:58 server2 slapd[714]: begin get_filter_list Nov 12 10:42:58 server2 slapd[714]: begin get_filter Nov 12 10:42:58 server2 slapd[714]: EQUALITY Nov 12 10:42:58 server2 slapd[714]: end get_filter 0 Nov 12 10:42:58 server2 slapd[714]: begin get_filter Nov 12 10:42:58 server2 slapd[714]: EQUALITY Nov 12 10:42:58 server2 slapd[714]: end get_filter 0 Nov 12 10:42:58 server2 slapd[714]: end get_filter_list Nov 12 10:42:58 server2 slapd[714]: end get_filter 0 Nov 12 10:42:58 server2 slapd[714]: end get_filter_list Nov 12 10:42:58 server2 slapd[714]: end get_filter 0 Nov 12 10:42:58 server2 slapd[714]: filter: (|(mailLocalAddress=XXXXXXX@xxxx.yy.zz)(mailRoutingAddress=XXXXXXX@xxxx.yy.zz)(&(xxxXXXXAlias=XXXXXXX)(xxxXXXYYYYYYList=bb))) Nov 12 10:42:58 server2 slapd[714]: attrs: Nov 12 10:42:58 server2 slapd[714]: Nov 12 10:42:58 server2 slapd[714]: conn=1000 op=1 SRCH base="ou=People,dc=yyyy,dc=zz" scope=2 deref=0 filter="(|(mailLocalAddress=XXXXXXX@xxxx.yy.zz)(mailRoutingAddress=XXXXXXX@xxxx.yy.zz)(&(xxxXXXXAlias=XXXXXXX)(xxxXXXYYYYYYList=bb)))" Nov 12 10:42:58 server2 slapd[714]: => bdb_search Nov 12 10:42:58 server2 slapd[714]: bdb_dn2entry("ou=people,dc=yyyy,dc=zz") Nov 12 10:42:58 server2 slapd[714]: => bdb_dn2id("dc=zz") Nov 12 10:42:58 server2 slapd[714]: <= bdb_dn2id: got id=0x1 Nov 12 10:42:58 server2 slapd[714]: => bdb_dn2id("dc=yyyy,dc=zz") Nov 12 10:42:58 server2 slapd[714]: <= bdb_dn2id: got id=0x2 Nov 12 10:42:58 server2 slapd[714]: => bdb_dn2id("ou=people,dc=yyyy,dc=zz") Nov 12 10:42:58 server2 slapd[714]: <= bdb_dn2id: got id=0x19 Nov 12 10:42:58 server2 slapd[714]: entry_decode: "ou=People,dc=yyyy,dc=zz" Nov 12 10:42:58 server2 slapd[714]: <= entry_decode(ou=People,dc=yyyy,dc=zz) Nov 12 10:42:58 server2 slapd[714]: => access_allowed: search access to "ou=People,dc=yyyy,dc=zz" "entry" requested Nov 12 10:42:58 server2 slapd[714]: <= root access granted Nov 12 10:42:58 server2 slapd[714]: => access_allowed: search access granted by manage(=mwrscxd) Nov 12 10:42:58 server2 slapd[714]: search_candidates: base="ou=people,dc=yyyy,dc=zz" (0x00000019) scope=2 Nov 12 10:42:58 server2 slapd[714]: => bdb_dn2idl("ou=people,dc=yyyy,dc=zz") Nov 12 10:42:58 server2 slapd[714]: bdb_idl_fetch_key: @ou=people,dc=yyyy,dc=zz Nov 12 10:42:58 server2 slapd[714]: <= bdb_dn2idl: id=-1 first=25 last=868130 Nov 12 10:42:58 server2 slapd[714]: => bdb_filter_candidates Nov 12 10:42:58 server2 slapd[714]: AND Nov 12 10:42:58 server2 slapd[714]: => bdb_list_candidates 0xa0 Nov 12 10:42:58 server2 slapd[714]: => bdb_filter_candidates Nov 12 10:42:58 server2 slapd[714]: OR Nov 12 10:42:58 server2 slapd[714]: => bdb_list_candidates 0xa1 Nov 12 10:42:58 server2 slapd[714]: => bdb_filter_candidates Nov 12 10:42:58 server2 slapd[714]: EQUALITY Nov 12 10:42:58 server2 slapd[714]: => bdb_equality_candidates (objectClass) Nov 12 10:42:58 server2 slapd[714]: => key_read Nov 12 10:42:58 server2 slapd[714]: bdb_idl_fetch_key: [b49d1940] Nov 12 10:42:58 server2 slapd[714]: <= bdb_index_read: failed (-30989) Nov 12 10:42:58 server2 slapd[714]: <= bdb_equality_candidates: id=0, first=0, last=0 Nov 12 10:42:58 server2 slapd[714]: <= bdb_filter_candidates: id=0 first=0 last=0 Nov 12 10:42:58 server2 slapd[714]: => bdb_filter_candidates Nov 12 10:42:58 server2 slapd[714]: OR Nov 12 10:42:58 server2 slapd[714]: => bdb_list_candidates 0xa1 Nov 12 10:42:58 server2 slapd[714]: => bdb_filter_candidates Nov 12 10:42:58 server2 slapd[714]: EQUALITY Nov 12 10:42:58 server2 slapd[714]: => bdb_equality_candidates (mailLocalAddress) Nov 12 10:42:58 server2 slapd[714]: => key_read Nov 12 10:42:58 server2 slapd[714]: bdb_idl_fetch_key: [6d316d9b] Nov 12 10:42:58 server2 slapd[714]: <= bdb_index_read 296 candidates Nov 12 10:42:58 server2 slapd[714]: <= bdb_equality_candidates: id=296, first=296, last=858689 Nov 12 10:42:58 server2 slapd[714]: <= bdb_filter_candidates: id=296 first=296 last=858689 Nov 12 10:42:58 server2 slapd[714]: => bdb_filter_candidates Nov 12 10:42:58 server2 slapd[714]: EQUALITY Nov 12 10:42:58 server2 slapd[714]: => bdb_equality_candidates (mailRoutingAddress) Nov 12 10:42:58 server2 slapd[714]: => key_read Nov 12 10:42:58 server2 slapd[714]: bdb_idl_fetch_key: [0afa05dc] Nov 12 10:42:58 server2 slapd[714]: <= bdb_index_read: failed (-30989) Nov 12 10:42:58 server2 slapd[714]: <= bdb_equality_candidates: id=0, first=0, last=0 Nov 12 10:42:58 server2 slapd[714]: <= bdb_filter_candidates: id=0 first=0 last=0 Nov 12 10:42:58 server2 slapd[714]: => bdb_filter_candidates Nov 12 10:42:58 server2 slapd[714]: AND Nov 12 10:42:58 server2 slapd[714]: => bdb_list_candidates 0xa0 Nov 12 10:42:58 server2 slapd[714]: => bdb_filter_candidates Nov 12 10:42:58 server2 slapd[714]: EQUALITY Nov 12 10:42:58 server2 slapd[714]: => bdb_equality_candidates (xxxXXXXAlias) Nov 12 10:42:58 server2 slapd[714]: => key_read Nov 12 10:42:58 server2 slapd[714]: bdb_idl_fetch_key: [a0867a62] Nov 12 10:42:58 server2 slapd[714]: <= bdb_index_read 2 candidates Nov 12 10:42:58 server2 slapd[714]: <= bdb_equality_candidates: id=2, first=676406, last=676407 Nov 12 10:42:58 server2 slapd[714]: <= bdb_filter_candidates: id=2 first=676406 last=676407 Nov 12 10:42:58 server2 slapd[714]: => bdb_filter_candidates Nov 12 10:42:58 server2 slapd[714]: EQUALITY Nov 12 10:42:58 server2 slapd[714]: => bdb_equality_candidates (xxxXXXYYYYYYList) Nov 12 10:42:58 server2 slapd[714]: => key_read Nov 12 10:42:58 server2 slapd[714]: bdb_idl_fetch_key: [d752aaac] Nov 12 10:42:58 server2 slapd[714]: <= bdb_index_read 239520 candidates Nov 12 10:42:58 server2 slapd[714]: <= bdb_equality_candidates: id=-1, first=628610, last=868129 Nov 12 10:42:58 server2 slapd[714]: <= bdb_filter_candidates: id=-1 first=628610 last=868129 Nov 12 10:42:58 server2 slapd[714]: <= bdb_list_candidates: id=-1 first=676406 last=676407 Nov 12 10:42:58 server2 slapd[714]: <= bdb_filter_candidates: id=-1 first=676406 last=676407 Nov 12 10:42:58 server2 slapd[714]: <= bdb_list_candidates: id=-1 first=296 last=858689 Nov 12 10:42:58 server2 slapd[714]: <= bdb_filter_candidates: id=-1 first=296 last=858689 Nov 12 10:42:58 server2 slapd[714]: <= bdb_list_candidates: id=-1 first=296 last=858689 Nov 12 10:42:58 server2 slapd[714]: <= bdb_filter_candidates: id=-1 first=296 last=858689 Nov 12 10:42:58 server2 slapd[714]: <= bdb_list_candidates: id=-1 first=296 last=858689 Nov 12 10:42:58 server2 slapd[714]: <= bdb_filter_candidates: id=-1 first=296 last=858689 Nov 12 10:42:58 server2 slapd[714]: bdb_search_candidates: id=-1 first=296 last=858689 Nov 12 10:42:58 server2 slapd[714]: entry_decode: "mailRoutingAddress=000000130(a)xxxx-xxxxx.yyyy.zz,ou=People,dc=yyyy,dc=zz" Nov 12 10:42:58 server2 slapd[714]: <= entry_decode(mailRoutingAddress=100000130(a)xxxx-xxxxx.yyyy.zz,ou=People,dc=yyyy,dc=zz) Nov 12 10:42:58 server2 slapd[714]: => bdb_dn2id("mailRoutingAddress=000000130(a)xxxx-xxxxx.yyyy.zz,ou=people,dc=yyyy,dc=zz") Nov 12 10:42:58 server2 slapd[714]: <= bdb_dn2id: got id=0x128 Nov 12 10:42:58 server2 slapd[714]: => test_filter Nov 12 10:42:58 server2 slapd[714]: OR Nov 12 10:42:58 server2 slapd[714]: => test_filter_or Nov 12 10:42:58 server2 slapd[714]: => test_filter Nov 12 10:42:58 server2 slapd[714]: EQUALITY Nov 12 10:42:58 server2 slapd[714]: => access_allowed: search access to "mailRoutingAddress=100000130(a)xxxx-xxxxx.yyyy.zz,ou=People,dc=yyyy,dc=zz" "mailLocalAddress" requested Nov 12 10:42:58 server2 slapd[714]: <= root access granted Nov 12 10:42:58 server2 slapd[714]: => access_allowed: search access granted by manage(=mwrscxd) Nov 12 10:42:58 server2 slapd[714]: <= test_filter 5 Nov 12 10:42:58 server2 slapd[714]: => test_filter Nov 12 10:42:58 server2 slapd[714]: EQUALITY Nov 12 10:42:58 server2 slapd[714]: => access_allowed: search access to "mailRoutingAddress=100000130(a)xxxx-xxxxx.yyyy.zz,ou=People,dc=yyyy,dc=zz" "mailRoutingAddress" requested Nov 12 10:42:58 server2 slapd[714]: <= root access granted Nov 12 10:42:58 server2 slapd[714]: => access_allowed: search access granted by manage(=mwrscxd) Nov 12 10:42:58 server2 slapd[714]: <= test_filter 5 Nov 12 10:42:58 server2 slapd[714]: => test_filter Nov 12 10:42:58 server2 slapd[714]: AND Nov 12 10:42:58 server2 slapd[714]: => test_filter_and Nov 12 10:42:58 server2 slapd[714]: => test_filter Nov 12 10:42:58 server2 slapd[714]: EQUALITY Nov 12 10:42:58 server2 slapd[714]: => access_allowed: search access to "mailRoutingAddress=100000130(a)xxxx-xxxxx.yyyy.zz,ou=People,dc=yyyy,dc=zz" "xxxXXXXAlias" requested Nov 12 10:42:58 server2 slapd[714]: <= root access granted Nov 12 10:42:58 server2 slapd[714]: => access_allowed: search access granted by manage(=mwrscxd) Nov 12 10:42:58 server2 slapd[714]: <= test_filter 5 Nov 12 10:42:58 server2 slapd[714]: <= test_filter_and 5 Nov 12 10:42:58 server2 slapd[714]: <= test_filter 5 Nov 12 10:42:58 server2 slapd[714]: <= test_filter_or 5 Nov 12 10:42:58 server2 slapd[714]: <= test_filter 5 Nov 12 10:42:58 server2 slapd[714]: bdb_search: 296 does not match filter Nov 12 10:42:58 server2 slapd[714]: entry_decode: "gecos=000000130(a)xxxx-xxxxx.yyyy.zz,ou=Ftp,dc=yyyy,dc=zz" Nov 12 10:42:58 server2 slapd[714]: <= entry_decode(gecos=000000130(a)xxxx-xxxxx.yyyy.zz,ou=Ftp,dc=yyyy,dc=zz) Nov 12 10:42:58 server2 slapd[714]: => bdb_dn2id("ou=ftp,dc=yyyy,dc=zz") Nov 12 10:42:58 server2 slapd[714]: <= bdb_dn2id: got id=0x24 Nov 12 10:42:58 server2 slapd[714]: => bdb_dn2id("gecos=000000130(a)xxxx-xxxxx.yyyy.zz,ou=ftp,dc=yyyy,dc=zz") Nov 12 10:42:58 server2 slapd[714]: <= bdb_dn2id: got id=0x129 Nov 12 10:42:58 server2 slapd[714]: bdb_search: 297 scope not okay Nov 12 10:42:58 server2 slapd[714]: entry_decode: "mailRoutingAddress=000000131(a)xxxx-xxxxx.yyyy.zz,ou=People,dc=yyyy,dc=zz" Nov 12 10:42:58 server2 slapd[714]: <= entry_decode(mailRoutingAddress=100000131(a)xxxx-xxxxx.yyyy.zz,ou=People,dc=yyyy,dc=zz) Nov 12 10:42:58 server2 slapd[714]: => bdb_dn2id("mailRoutingAddress=000000131(a)xxxx-xxxxx.yyyy.zz,ou=people,dc=yyyy,dc=zz") Nov 12 10:42:58 server2 slapd[714]: <= bdb_dn2id: got id=0x12a Nov 12 10:42:58 server2 slapd[714]: => test_filter Nov 12 10:42:58 server2 slapd[714]: OR Nov 12 10:42:58 server2 slapd[714]: => test_filter_or Nov 12 10:42:58 server2 slapd[714]: => test_filter Nov 12 10:42:58 server2 slapd[714]: EQUALITY Nov 12 10:42:58 server2 slapd[714]: => access_allowed: search access to "mailRoutingAddress=100000131(a)xxxx-xxxxx.yyyy.zz,ou=People,dc=yyyy,dc=zz" "mailLocalAddress" requested Nov 12 10:42:58 server2 slapd[714]: <= root access granted Nov 12 10:42:58 server2 slapd[714]: => access_allowed: search access granted by manage(=mwrscxd) Nov 12 10:42:58 server2 slapd[714]: <= test_filter 5 Nov 12 10:42:58 server2 slapd[714]: => test_filter Nov 12 10:42:58 server2 slapd[714]: EQUALITY Nov 12 10:42:58 server2 slapd[714]: => access_allowed: search access to "mailRoutingAddress=100000131(a)xxxx-xxxxx.yyyy.zz,ou=People,dc=yyyy,dc=zz" "mailRoutingAddress" requested Nov 12 10:42:58 server2 slapd[714]: <= root access granted Nov 12 10:42:58 server2 slapd[714]: => access_allowed: search access granted by manage(=mwrscxd) Nov 12 10:42:58 server2 slapd[714]: <= test_filter 5 Nov 12 10:42:58 server2 slapd[714]: => test_filter Nov 12 10:42:58 server2 slapd[714]: AND Nov 12 10:42:58 server2 slapd[714]: => test_filter_and Nov 12 10:42:58 server2 slapd[714]: => test_filter Nov 12 10:42:58 server2 slapd[714]: EQUALITY Nov 12 10:42:58 server2 slapd[714]: => access_allowed: search access to "mailRoutingAddress=100000131(a)xxxx-xxxxx.yyyy.zz,ou=People,dc=yyyy,dc=zz" "xxxXXXXAlias" requested Nov 12 10:42:58 server2 slapd[714]: <= root access granted Nov 12 10:42:58 server2 slapd[714]: => access_allowed: search access granted by manage(=mwrscxd) Nov 12 10:42:58 server2 slapd[714]: <= test_filter 5 Nov 12 10:42:58 server2 slapd[714]: <= test_filter_and 5 Nov 12 10:42:58 server2 slapd[714]: <= test_filter 5 Nov 12 10:42:58 server2 slapd[714]: <= test_filter_or 5 Nov 12 10:42:58 server2 slapd[714]: <= test_filter 5 Nov 12 10:42:58 server2 slapd[714]: bdb_search: 298 does not match filter Nov 12 10:42:58 server2 slapd[714]: entry_decode: "gecos=000000131(a)xxxx-xxxxx.yyyy.zz,ou=Ftp,dc=yyyy,dc=zz" Nov 12 10:42:58 server2 slapd[714]: <= entry_decode(gecos=000000131(a)xxxx-xxxxx.yyyy.zz,ou=Ftp,dc=yyyy,dc=zz) Nov 12 10:42:58 server2 slapd[714]: => bdb_dn2id("gecos=000000131(a)xxxx-xxxxx.yyyy.zz,ou=ftp,dc=yyyy,dc=zz") Nov 12 10:42:58 server2 slapd[714]: <= bdb_dn2id: got id=0x12b Nov 12 10:42:58 server2 slapd[714]: bdb_search: 299 scope not okay (omitted) Best Regards,

1 0

(ITS#7742) New: PBKDF2 module for OpenLDAP
by hamano＠osstech.co.jp 08 Nov '13

08 Nov '13

Full_Name: HAMANO Tsukasa Version: git master OS: Linux URL: ftp://ftp.openldap.org/incoming/0001-New-PBKDF2-module.patch Submission from: (NULL) (240b:10:2640:bf0:c1b8:b6c9:76f:8964) PBKDF2(RFC 2898) suport for OpenLDAP See README for details. -- The attached patch file is derived from OpenLDAP Software. All of the modifications to OpenLDAP Software represented in the following patch(es) were developed by HAMANO Tsukasa <hamano(a)osstech.co.jp>. I have not assigned rights and/or interest in this work to any party. Copyright 2013 HAMANO Tsukasa <hamano(a)osstech.co.jp> Redistribution and use in source and binary forms, with or without modification, are permitted only as authorized by the OpenLDAP Public License.

1 0

Re: (ITS#7741) Incorrect indexing of entryDN
by hyc＠symas.com 06 Nov '13

06 Nov '13

raphael.ouazana(a)linagora.com wrote: > Full_Name: Raphael Ouazana > Version: 2.4.26 > OS: Linux > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (88.173.78.196) > > > Hi, > > I have a pretty big database (hdb, 300 000 entries) with low memory (1/4 GB). > If I try a search with scope sub, base dc=suffix, and filter > (entryDN=uid=nonexisting,dc=people,dc=suffix), the search is very slow (I > stopped after some minutes). > But if I try with scope base, base uid=nonexisting,dc=people,dc=suffix, filter > (objectClass=*), the result is immediate, telling there is no such entry. > > I have read that since 2.4.13 entryDN is indexed by dn2id, but it doesn't seem > to be the case. Only positive results from the index were being used. > Regards, > Raphaël Ouazana. > > PS : my test database run OpenLDAP 2.4.26 only, but I have not seen any change > in the area of entryDN indexing. Will try with 2.4.37 if you think something has > changed. Nothing has changed in this section of code in 5 years. Fixed now in master. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7741) Incorrect indexing of entryDN
by quanah＠zimbra.com 06 Nov '13

06 Nov '13

--On Wednesday, November 06, 2013 4:27 PM +0000 raphael.ouazana(a)linagora.com wrote: > Full_Name: Raphael Ouazana > Version: 2.4.26 > OS: Linux > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (88.173.78.196) > > > Hi, > > I have a pretty big database (hdb, 300 000 entries) with low memory (1/4 > GB). If I try a search with scope sub, base dc=suffix, and filter > (entryDN=uid=nonexisting,dc=people,dc=suffix), the search is very slow (I > stopped after some minutes). > But if I try with scope base, base uid=nonexisting,dc=people,dc=suffix, > filter (objectClass=*), the result is immediate, telling there is no such > entry. What makes you feel there is a bug present? Nothing in your report indicates the presence of a bug. --Quanah -- Quanah Gibson-Mount Architect - Server Zimbra, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

(ITS#7741) Incorrect indexing of entryDN
by raphael.ouazana＠linagora.com 06 Nov '13

06 Nov '13

Full_Name: Raphael Ouazana Version: 2.4.26 OS: Linux URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (88.173.78.196) Hi, I have a pretty big database (hdb, 300 000 entries) with low memory (1/4 GB). If I try a search with scope sub, base dc=suffix, and filter (entryDN=uid=nonexisting,dc=people,dc=suffix), the search is very slow (I stopped after some minutes). But if I try with scope base, base uid=nonexisting,dc=people,dc=suffix, filter (objectClass=*), the result is immediate, telling there is no such entry. I have read that since 2.4.13 entryDN is indexed by dn2id, but it doesn't seem to be the case. Regards, Raphaël Ouazana. PS : my test database run OpenLDAP 2.4.26 only, but I have not seen any change in the area of entryDN indexing. Will try with 2.4.37 if you think something has changed.

1 0

Re: (ITS#7739) Include of include badly handled while converting from slapd.conf to slapd.d
by hyc＠symas.com 05 Nov '13

05 Nov '13

Raphaël Ouazana-Sustowski wrote: > Hi, > > Le 04/11/2013 17:06, Howard Chu a écrit : >> raphael.ouazana(a)linagora.com wrote: >>> Full_Name: Raphael Ouazana >>> Version: 2.4.37 >>> OS: Linux >>> URL: ftp://ftp.openldap.org/incoming/ >>> Submission from: (NULL) (88.173.78.196) >>> >>> >>> Hi, >>> >>> If I have for example a main slapd.conf with: >>> include <path>schema.conf >>> >>> And a simple schema.conf: >>> include <path>core.schema >>> include <path>cosine.schema >>> include <path>inetorgperson.schema >>> >>> When I try to convert slapd.conf to slapd.d with slaptest, the >>> resulting >>> configuration is incorrect: the schema file are not included. >>> >> >> Try again, using a different name than "schema.conf" > > It doesn't work neither. > > Steps to reproduce: > > 1. basic slapd.conf, but: > a. replace include <schemapath>core.schema by include > <configpath>morethings.conf > b. add index sn eq at the end > > 2. create <configpath>morethings.conf including only > <schemapath>core.schema Fixed now in master. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7740) back-meta lock threads when remote node is unreachable
by hyc＠symas.com 05 Nov '13

05 Nov '13

jorge.perez.burgos(a)ericsson.com wrote: > Full_Name: Jorge Perez Burgos > Version: 2.4.35 > OS: SLES 11 > URL: ftp://ftp.openldap.org/incoming/0001-Don-t-lock-other-threads-while-trying-… > Submission from: (NULL) (195.235.15.243) > > > When there is meta proxy traffic to another node and this node becomes > unreachable the thread in meta retrying to connect to the remote node locks all > the other threads making the slapd unresponsive until the timeout expires. > Attach a possible fix > ftp://ftp.openldap.org/incoming/0001-Don-t-lock-other-threads-while-trying-… > > and a backtraceo of some threads when the problem happens: Thanks, the patch looks OK, committed to git master. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#7740) back-meta lock threads when remote node is unreachable
by jorge.perez.burgos＠ericsson.com 05 Nov '13

05 Nov '13

Full_Name: Jorge Perez Burgos Version: 2.4.35 OS: SLES 11 URL: ftp://ftp.openldap.org/incoming/0001-Don-t-lock-other-threads-while-trying-… Submission from: (NULL) (195.235.15.243) When there is meta proxy traffic to another node and this node becomes unreachable the thread in meta retrying to connect to the remote node locks all the other threads making the slapd unresponsive until the timeout expires. Attach a possible fix ftp://ftp.openldap.org/incoming/0001-Don-t-lock-other-threads-while-trying-… and a backtraceo of some threads when the problem happens: Thread 259 (Thread 0x7ff9a771e700 (LWP 27085)): #0 0x00007ff9fc2a6294 in __lll_lock_wait () from /lib64/libpthread.so.0 #1 0x00007ff9fc2a1619 in _L_lock_1008 () from /lib64/libpthread.so.0 #2 0x00007ff9fc2a142e in pthread_mutex_lock () from /lib64/libpthread.so.0 #3 0x00000000004b6007 in meta_back_search (op=0x31a8800, rs=0x7ff9a771d930) at search.c:2012 #4 0x000000000043ab31 in fe_op_search (op=0x31a8800, rs=0x7ff9a771d930) at search.c:402 #5 0x00000000004a0ef7 in overlay_op_walk (op=0x31a8800, rs=0x7ff9a771d930, which=op_search, oi=0x87cc000, on=0x0) at backover.c:671 #6 0x00000000004a1970 in over_op_func (op=0x31a8800, rs=0x7ff9a771bdc8, which=op_bind) at backover.c:723 #7 0x000000000043b37c in do_search (op=0x31a8800, rs=0x7ff9a771d930) at search.c:247 #8 0x0000000000437eef in connection_operation (ctx=0x7ff9a771db30, arg_v=<optimized out>) at connection.c:1313 #9 0x0000000000438b1b in connection_read_thread (ctx=0x7ff9a771db30, argv=<optimized out>) at connection.c:1449 #10 0x00000000004ee750 in ldap_int_thread_pool_wrapper (xpool=<optimized out>) at tpool.c:688 #11 0x00007ff9fc29f7b6 in start_thread () from /lib64/libpthread.so.0 #12 0x00007ff9fbffac5d in clone () from /lib64/libc.so.6 #13 0x0000000000000000 in ?? () 226 Transfer complete (unique file name:0001-Don-t-lock-other-threads-while-trying-to-connect-to.patch). Thread 258 (Thread 0x7ff9a6f1d700 (LWP 27086)): #0 0x00007ff9fbff1786 in poll () from /lib64/libc.so.6 #1 0x0000000000504b19 in ldap_int_poll (ld=0x5f03e030, s=462, tvp=0x7ff9a6f1afd0, wr=<optimized out>) at os-ip.c:302 #2 0x0000000000505248 in ldap_pvt_connect (async=<optimized out>, addrlen=<optimized out>, sin=<optimized out>, s=<optimized out>, ld=<optimized out>) at os-ip.c:465 #3 ldap_connect_to_host (ld=0x5f03e030, sb=0x592c0ed0, proto=1, srv=0x6e26edc0, async=0) at os-ip.c:658 #4 0x00000000004ef4d7 in ldap_int_open_connection (ld=0x5f03e030, conn=0x574bfc20, srv=0x6e26edc0, async=0) at open.c:367 #5 0x000000000050243a in ldap_new_connection (ld=0x5f03e030, srvlist=0x6cf79598, use_ldsb=1, connect=<optimized out>, bind=0x0, m_req=0, m_res=0) at request.c:481 #6 0x00000000004ef41f in ldap_open_defconn (ld=0x7ff9a6f1aef0) at open.c:41 #7 0x0000000000503158 in ldap_send_initial_request (ld=0x5f03e030, msgtype=<optimized out>, dn=<optimized out>, ber=0x6db20780, msgid=1) at request.c:130 #8 0x00000000004f8ebc in ldap_sasl_bind (ld=0x5f03e030, dn=0x89e9660 <error reading variable>, mechanism=0x0, cred=0x7ff9a6f1b220, sctrls=0x0, cctrls=<optimized out>, msgidp=0x7ff9a6f1b248) at sasl.c:148 #9 0x00000000004cb270 in meta_back_proxy_authz_bind (mc=0x12165ba0, candidate=2, op=0x342ac00, rs=0x7ff9a6f1ca60, sendok=LDAP_BACK_SENDERR, dolock=0) at bind.c:1582 #10 0x00000000004cb49f in meta_back_single_dobind (op=0x342ac00, rs=0x7ff9a6f1ca60, mcp=0x7ff9a6f1b560, candidate=2, sendok=LDAP_BACK_SENDERR, nretries=<optimized out>, dolock=0) at bind.c:605 #11 0x00000000004d5df3 in meta_back_retry (op=0x342ac00, rs=0x7ff9a6f1ca60, mcp=0x7ff9a6f1b560, candidate=2, sendok=LDAP_BACK_SENDERR) at conn.c:766 #12 0x00000000004b42c5 in meta_back_modify (op=0x342ac00, rs=0x7ff9a6f1ca60) at modify.c:194 #13 0x00000000004506d8 in fe_op_modify (op=0x342ac00, rs=0x7ff9a6f1ca60) at modify.c:303 #14 0x00000000004a0ef7 in overlay_op_walk (op=0x342ac00, rs=0x7ff9a6f1ca60, which=op_modify, oi=0x87cc000, on=0x0) at backover.c:671 #15 0x00000000004a1970 in over_op_func (op=0x342ac00, rs=0x7ff9a6f1a918, which=4000) at backover.c:723 #16 0x0000000000451007 in do_modify (op=0x342ac00, rs=0x7ff9a6f1ca60) at modify.c:177 #17 0x0000000000437eef in connection_operation (ctx=0x7ff9a6f1cb30, arg_v=<optimized out>) at connection.c:1313 #18 0x00000000004ee750 in ldap_int_thread_pool_wrapper (xpool=<optimized out>) at tpool.c:688 #19 0x00007ff9fc29f7b6 in start_thread () from /lib64/libpthread.so.0 #20 0x00007ff9fbffac5d in clone () from /lib64/libc.so.6 #21 0x0000000000000000 in ?? () Thread 257 (Thread 0x7ff9a671c700 (LWP 27088)): #0 0x00007ff9fc2a6294 in __lll_lock_wait () from /lib64/libpthread.so.0 #1 0x00007ff9fc2a1619 in _L_lock_1008 () from /lib64/libpthread.so.0 #2 0x00007ff9fc2a142e in pthread_mutex_lock () from /lib64/libpthread.so.0 #3 0x00000000004d4540 in meta_back_getconn (op=0x46fe000, rs=0x7ff9a671b930, candidate=0x0, sendok=LDAP_BACK_SENDERR) at conn.c:1129 #4 0x00000000004b5b61 in meta_back_search (op=0x46fe000, rs=0x7ff9a671b930) at search.c:803 #5 0x000000000043ab31 in fe_op_search (op=0x46fe000, rs=0x7ff9a671b930) at search.c:402 #6 0x00000000004a0ef7 in overlay_op_walk (op=0x46fe000, rs=0x7ff9a671b930, which=op_search, oi=0x87cc000, on=0x0) at backover.c:671 #7 0x00000000004a1970 in over_op_func (op=0x46fe000, rs=0x0, which=op_bind) at backover.c:723 #8 0x000000000043b37c in do_search (op=0x46fe000, rs=0x7ff9a671b930) at search.c:247 #9 0x0000000000437eef in connection_operation (ctx=0x7ff9a671bb30, arg_v=<optimized out>) at connection.c:1313 #10 0x0000000000438b1b in connection_read_thread (ctx=0x7ff9a671bb30, argv=<optimized out>) at connection.c:1449 #11 0x00000000004ee750 in ldap_int_thread_pool_wrapper (xpool=<optimized out>) at tpool.c:688 #12 0x00007ff9fc29f7b6 in start_thread () from /lib64/libpthread.so.0 #13 0x00007ff9fbffac5d in clone () from /lib64/libc.so.6 #14 0x0000000000000000 in ?? () Thread 256 (Thread 0x7ff9a5f1b700 (LWP 27089)): #0 0x00007ff9fc2a6294 in __lll_lock_wait () from /lib64/libpthread.so.0 #1 0x00007ff9fc2a1619 in _L_lock_1008 () from /lib64/libpthread.so.0 #2 0x00007ff9fc2a142e in pthread_mutex_lock () from /lib64/libpthread.so.0 #3 0x00000000004d4540 in meta_back_getconn (op=0x46f5000, rs=0x7ff9a5f1a930, candidate=0x0, sendok=LDAP_BACK_SENDERR) at conn.c:1129 #4 0x00000000004b5b61 in meta_back_search (op=0x46f5000, rs=0x7ff9a5f1a930) at search.c:803 #5 0x000000000043ab31 in fe_op_search (op=0x46f5000, rs=0x7ff9a5f1a930) at search.c:402 #6 0x00000000004a0ef7 in overlay_op_walk (op=0x46f5000, rs=0x7ff9a5f1a930, which=op_search, oi=0x87cc000, on=0x0) at backover.c:671 #7 0x00000000004a1970 in over_op_func (op=0x46f5000, rs=0x0, which=op_bind) at backover.c:723 #8 0x000000000043b37c in do_search (op=0x46f5000, rs=0x7ff9a5f1a930) at search.c:247 #9 0x0000000000437eef in connection_operation (ctx=0x7ff9a5f1ab30, arg_v=<optimized out>) at connection.c:1313 #10 0x0000000000438b1b in connection_read_thread (ctx=0x7ff9a5f1ab30, argv=<optimized out>) at connection.c:1449 #11 0x00000000004ee750 in ldap_int_thread_pool_wrapper (xpool=<optimized out>) at tpool.c:688 #12 0x00007ff9fc29f7b6 in start_thread () from /lib64/libpthread.so.0 #13 0x00007ff9fbffac5d in clone () from /lib64/libc.so.6 #14 0x0000000000000000 in ?? ()

1 0

Re: (ITS#7735) Memory leaks on Multi-Master replications
by dmitrii.fonariuk＠gmail.com 05 Nov '13

05 Nov '13

--047d7b33cb784de6d004ea68c04b Content-Type: text/plain; charset=ISO-8859-1 Hi Quanah, I have installed two patches, compiled binary and run modify requests test. I see that the memory is no longer leaking. great!!! patches helped. excellent work! Thanks! --Dmitrii --047d7b33cb784de6d004ea68c04b Content-Type: text/html; charset=ISO-8859-1 <div dir="ltr"><div class="gmail_extra"> <p class="MsoNormal">Hi Quanah,</p><p class="MsoNormal"><br></p><p class="MsoNormal">I have installed two patches, compiled binary and run modify requests test. I see that the memory is no longer leaking. great!!! patches helped. excellent work!</p><p class="MsoNormal"> </p><p class="MsoNormal"><br></p><p class="MsoNormal">Thanks!</p><p class="MsoNormal"><br></p><p class="MsoNormal">--Dmitrii</p> </div></div> --047d7b33cb784de6d004ea68c04b--

1 0

Re: (ITS#7739) Include of include badly handled while converting from slapd.conf to slapd.d
by hyc＠symas.com 04 Nov '13

04 Nov '13

raphael.ouazana(a)linagora.com wrote: > Full_Name: Raphael Ouazana > Version: 2.4.37 > OS: Linux > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (88.173.78.196) > > > Hi, > > If I have for example a main slapd.conf with: > include <path>schema.conf > > And a simple schema.conf: > include <path>core.schema > include <path>cosine.schema > include <path>inetorgperson.schema > > When I try to convert slapd.conf to slapd.d with slaptest, the resulting > configuration is incorrect: the schema file are not included. > Try again, using a different name than "schema.conf" -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#7739) Include of include badly handled while converting from slapd.conf to slapd.d
by raphael.ouazana＠linagora.com 04 Nov '13

04 Nov '13

Full_Name: Raphael Ouazana Version: 2.4.37 OS: Linux URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (88.173.78.196) Hi, If I have for example a main slapd.conf with: include <path>schema.conf And a simple schema.conf: include <path>core.schema include <path>cosine.schema include <path>inetorgperson.schema When I try to convert slapd.conf to slapd.d with slaptest, the resulting configuration is incorrect: the schema file are not included. Regards, Raphaël Ouazana.

1 0

Re: (ITS#7723) slapd crashes on multi core machines if a search request is *immediately* followed by an unbind
by hyc＠symas.com 04 Nov '13

04 Nov '13

jsynacek(a)redhat.com wrote: > On 10/15/2013 01:10 PM, michael.vishchers(a)7p-group.com wrote: >> It is not the client loop that is multithreading but the ldap server. >> >> And it is not a misuse of the API but a problem that may be raised by day t= >> o day network problems. >> >> I've boiled down the problem to a few simple configurations that work (or b= >> etter, fail ;-) with both 2.4.23 and 2.4.36. A tgz file containing a setup = >> with start script and testclient is attached. It should be sufficient to re= >> produce the fault. >> >> The problem occurs only if we use session variable substitution in the rwm = >> overlay, and only if a search is *immediately* (e.g. caused by network loss= >> and client timeout) followed by an unbind. >> > > I modified the reproducer a bit (the start script) and find out a few things. > You can find the reproducer I'm using at [1]. > > Valgrind's helgrind shows some lock problems in the rwm overlay and also in > back-ldap and connection.c. After correcting those the issue seems to be gone. > > You can find helgrind logs at [2] (before the fix) and [3] (after). > > Also, ElectricFence reveals some problems [4], which I didn't fix yet. > > A fix attempt can be found at [5]. I'm not sure if that is a correct fix, or it > just masked the real issue. But I didn't to manage to reproduce the problem > after applying it. I already explained the problem. The other issues you identified are not relevant, and your patch is not correct. Reread Followup #4 of this ITS. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7723) slapd crashes on multi core machines if a search request is *immediately* followed by an unbind
by jsynacek＠redhat.com 04 Nov '13

04 Nov '13

On 10/15/2013 01:10 PM, michael.vishchers(a)7p-group.com wrote: > It is not the client loop that is multithreading but the ldap server. > > And it is not a misuse of the API but a problem that may be raised by day t= > o day network problems. > > I've boiled down the problem to a few simple configurations that work (or b= > etter, fail ;-) with both 2.4.23 and 2.4.36. A tgz file containing a setup = > with start script and testclient is attached. It should be sufficient to re= > produce the fault. > > The problem occurs only if we use session variable substitution in the rwm = > overlay, and only if a search is *immediately* (e.g. caused by network loss= > and client timeout) followed by an unbind. > I modified the reproducer a bit (the start script) and find out a few things. You can find the reproducer I'm using at [1]. Valgrind's helgrind shows some lock problems in the rwm overlay and also in back-ldap and connection.c. After correcting those the issue seems to be gone. You can find helgrind logs at [2] (before the fix) and [3] (after). Also, ElectricFence reveals some problems [4], which I didn't fix yet. A fix attempt can be found at [5]. I'm not sure if that is a correct fix, or it just masked the real issue. But I didn't to manage to reproduce the problem after applying it. [1] http://jsynacek.fedorapeople.org/openldap/its7723/reproducer/ [2] http://jsynacek.fedorapeople.org/openldap/its7723/results/slapd1-helgrind-b… [3] http://jsynacek.fedorapeople.org/openldap/its7723/results/slapd1-helgrind-f… [4] http://jsynacek.fedorapeople.org/openldap/its7723/results/slapd1-broken-efe… [5] http://jsynacek.fedorapeople.org/openldap/its7723/0001-fix-possible-race-co… Cheers, -- Jan Synacek Software Engineer, Red Hat

1 0

Re: (ITS#7735) Memory leaks on Multi-Master replications
by quanah＠zimbra.com 03 Nov '13

03 Nov '13

--On Friday, November 01, 2013 10:39 PM +0400 Dmitrii Fonariuk <dmitrii.fonariuk(a)gmail.com> wrote: > > > hi Quanah, > > it's a good news! unfortunately I can't test the changes now. Tuesday I > rebuild the source with patch, do the tests and report to you. > > thank you for your efforts. You will also need to grab <http://www.openldap.org/devel/gitweb.cgi?p=openldap.git;a=patch;h=ed0922296…> --Quanah -- Quanah Gibson-Mount Architect - Server Zimbra, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

Re: (ITS#7700) DOC: OL AdminGuide not updated
by marco.pizzoli＠gmail.com 02 Nov '13

02 Nov '13

On Sat, Nov 2, 2013 at 10:56 PM, Gavin Henry <openldap-its(a)openldap.org> wrote: >> I noticed other errors on the Admin Guide. > > Hi Marco, > > Many thanks for taking the time to read our documentation and report missing > items and errors. It's been a pleasure :-) [cut] >> 6.2.3.3 -> I think it could be useful to specify that in case that >> server is consumer, all modifications coming from syncrepl still >> continue to be received and written > > Do you have any more wording you'd like to contribute as per: > > http://www.openldap.org/devel/contributing.html I'm afraid I cannot get what you are asking me to do. Are you asking me to just add the disclaimer or to reword my point in order to be feasible for direct inclusion in the AdminGuide? > >> 6.2.3.6 -> "Note: When the backend to pass a query to is selected, >> slapd looks at the suffix line(s) in each database definition in the >> order they appear in the file. >> Thus, if one database suffix is a prefix of another, it must >> appear BEFORE it in the config file." > > And with the ldif backend for dynamic configuration, cn=config Of course :-) And what if I found errors also on this page? :-) http://www.openldap.org/devel/contributing.html#patches "Each software change should be conCise, self-contained, and made against up-to-date source code." "One patch per functional change, and" -> and what? Regards Marco

1 0

Re: (ITS#7700) DOC: OL AdminGuide not updated
by marco.pizzoli＠gmail.com 02 Nov '13

02 Nov '13

I noticed other errors on the Admin Guide. I put all of my findings in the following list: 6.2.2.1 -> mdb is not present as a choice 6.2.3.3 -> I think it could be useful to specify that in case that server is consumer, all modifications coming from syncrepl still continue to be received and written 6.2.3.6 -> "Note: When the backend to pass a query to is selected, slapd looks at the suffix line(s) in each database definition in the order they appear in the file. Thus, if one database suffix is a prefix of another, it must appear BEFORE it in the config file." 6.2.3.7 - as "syncrepl" options don't appear the following: "keepalive", "network-timeout", "timeout", "tls_protocol_min" and "suffixmassage" -> "The syncrepl replication mechanism is supported by the bdb, hdb AND MDB backends" 9.4.1 -> "Table ZZZ.ZZZ: Entity Specifiers"

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs