openldap-bugs April 2017

openldap-bugs@openldap.org

25 participants
116 discussions

Re: (ITS#7013) OpenLDAP freezes and doesn't respond
by quanah＠symas.com 05 Apr '17

05 Apr '17

Hello, Due to inactivity and the noted improper configuration, I am closing this ITS. I would note there have been hundreds of fixes to OpenLDAP since this report was filed, including a new BDB-free backend. If you ever drop OpenDJ and move back to OpenLDAP and can reproduce the issue using back-mdb (based on LMDB), please file a new ITS. Thanks! --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

Re: (ITS#6545) delta-syncrepl rejects modification master accepted
by ondra＠mistotebe.net 05 Apr '17

05 Apr '17

On Wed, Apr 05, 2017 at 04:14:12PM +0200, Michael Ströder wrote: > ondra(a)mistotebe.net wrote: >> On Wed, Apr 05, 2017 at 07:32:46AM -0400, Frank Swasey wrote: >>> Thanks for the patch to provide a test script that just shows the same >>> thing. >>> >>> I see two possible solutions: >>> >>> 1) replacing the same attribute twice in the same modify LDIF is illegal >>> (as it was in older releases) >> >> AFAIK, LDAP doesn't forbid it so I don't see that going away. > > Yes, there's no text in RFC 4511 which forbids this: > https://tools.ietf.org/html/rfc4511#section-4.6 > > However personally I consider LDAP clients sending modify requests like this to be > broken/mis-behaving. (And I'd like to know which LDAP clients were causing this ITS.) I'm not saying it's common or good practice ;) > => There could be a slapd per-backend configuation directive to disallow it with a strong > hint in the docs recommending to disallow it when using delta-syncrepl. > > Suggestion: > disallow mod_attr_repeated In my view, that's more pain than it's worth. >>> 2) the accesslog/syncrepl needs to record the final result, not every bit >>> of the change. >> >> Two problems: >> - the log is meant to record the request for review/statistics purposes >> as well and should record the intent, not just the result >> - the modification might fail, in that case you still need an accurate >> record of the request > > IIRC slapo-accesslog was primarily implemented for delta-syncrepl. > > Personally I'm in the (ab)use-slapo-accesslog-as-auditlog camp. ;-) > But still I'd consider an optimized changes list written to accesslog to be perfectly > fine for security auditing because it would represent what caused the modification of the > database content. I don't think it matters what it was written for, the fact that it's officially useful for that as you and others have realised means they might object to us making drastic changes in 2.4, a series the project have tried to freeze even maintenance for. > Also note that not every failed modification is written to accesslog-DB anyway because > most malformed write operations already fail in slapd's front-end (schema check etc.) and > never reach the DB backend (and thus slapo-accesslog). So debugging errornous clients is > something for which you have to use Wireshark etc. in most cases anyway. I think most of the ones that don't reach accesslog get a protocol error or come from anonymous connections, schema checking and ACL should happen later. > I'm not familiar with the inner workings of slapd but these things should be carefully > considered when optimizing the changes list of a modify request: > 1. constraint checking (slapo-unique and slapo-constraint) > 2. impact on indexing > 3. access control, especially with val=foo part in the ACL > > 1. and 2. are hopefully already done on the "resulting entry after the entire list of > modifications is performed" (RFC 4511). Not sure I follow. On the consumer, syncrepl bypasses all ACLs and the overlays you mention should already be capable of passing the operation intact. On the provider side it depends on the overlay order and the administrator has the option to decide that if they wish. Consistency and data model are generally maintained quite alright. -- Ondřej Kuzník Senior Software Engineer Symas Corporation http://www.symas.com Packaged, certified, and supported LDAP solutions powered by OpenLDAP

1 0

Re: (ITS#6545) delta-syncrepl rejects modification master accepted
by michael＠stroeder.com 05 Apr '17

05 Apr '17

ondra(a)mistotebe.net wrote: > On Wed, Apr 05, 2017 at 07:32:46AM -0400, Frank Swasey wrote: >> Thanks for the patch to provide a test script that just shows the same >> thing. >> >> I see two possible solutions: >> >> 1) replacing the same attribute twice in the same modify LDIF is illegal >> (as it was in older releases) > > AFAIK, LDAP doesn't forbid it so I don't see that going away. Yes, there's no text in RFC 4511 which forbids this: https://tools.ietf.org/html/rfc4511#section-4.6 However personally I consider LDAP clients sending modify requests like this to be broken/mis-behaving. (And I'd like to know which LDAP clients were causing this ITS.) => There could be a slapd per-backend configuation directive to disallow it with a strong hint in the docs recommending to disallow it when using delta-syncrepl. Suggestion: disallow mod_attr_repeated >> 2) the accesslog/syncrepl needs to record the final result, not every bit >> of the change. > > Two problems: > - the log is meant to record the request for review/statistics purposes > as well and should record the intent, not just the result > - the modification might fail, in that case you still need an accurate > record of the request IIRC slapo-accesslog was primarily implemented for delta-syncrepl. Personally I'm in the (ab)use-slapo-accesslog-as-auditlog camp. ;-) But still I'd consider an optimized changes list written to accesslog to be perfectly fine for security auditing because it would represent what caused the modification of the database content. Also note that not every failed modification is written to accesslog-DB anyway because most malformed write operations already fail in slapd's front-end (schema check etc.) and never reach the DB backend (and thus slapo-accesslog). So debugging errornous clients is something for which you have to use Wireshark etc. in most cases anyway. I'm not familiar with the inner workings of slapd but these things should be carefully considered when optimizing the changes list of a modify request: 1. constraint checking (slapo-unique and slapo-constraint) 2. impact on indexing 3. access control, especially with val=foo part in the ACL 1. and 2. are hopefully already done on the "resulting entry after the entire list of modifications is performed" (RFC 4511). Ciao, Michael.

1 0

Re: (ITS#6545) delta-syncrepl rejects modification master accepted
by ondra＠mistotebe.net 05 Apr '17

05 Apr '17

On Wed, Apr 05, 2017 at 07:32:46AM -0400, Frank Swasey wrote: > Thanks for the patch to provide a test script that just shows the same > thing. > > I see two possible solutions: > > 1) replacing the same attribute twice in the same modify LDIF is illegal > (as it was in older releases) AFAIK, LDAP doesn't forbid it so I don't see that going away. > 2) the accesslog/syncrepl needs to record the final result, not every bit > of the change. Two problems: - the log is meant to record the request for review/statistics purposes as well and should record the intent, not just the result - the modification might fail, in that case you still need an accurate record of the request > I believe the easiest solution is #1. However, #2 would be proper if this > kind of abuse is legal. > > Consider the following deck: > > [...] > > Is that legal? Well, it works and results in an entry in the accesslog that > contains: > > [...] > > and has no issues in the replication. > > Therefore, why does this deck: > > dn: cn=Oh Noes,ou=People,dc=example,dc=com > changetype: modify > replace: sn > sn: Thomas > - > replace: sn > sn: George > - > > which creates an accesslog entry that contains: > > reqMod: sn:= Thomas > reqMod: sn:= George > > Interestingly, this morning when I was performing this, the replica ate that > accesslog entry and put both of those sn's into the LDAP entry on the > replica (instead of the normal barking in the logs that it failed). Perhaps > because I had done the former test with the delete/add/delete/add sequence? Your original example did two replacements with the same value, the test case runs two replacements with different values as that will cause the two servers to de-sync silently. > At this point, I'm thinking the accesslog should probably generate the > minimal changes necessary to get to the right state. That would mean the > former test should produce: > > reqMod: sn:- George > reqMod: sn:+ George > > and the latter: > > reqMod:sn:= George > > Either that, or the syncrepl processing of what accesslog is sending needs > some serious investigation. I'll see what I can find for time to look into > that serious investigation. It does need a fair bit of attention, the problem is it's something that would require a format change and there are already existing consumers that would be affected. I'm going to try with a patch to record something like this: regMod: sn:= George regMod: : regMod: sn:= George Together with the relevant syncrepl updates. While consumers would still be affected, this should be quite rare and the current handling of that case would have been incorrect anyway. The change would then only make the silent failure a noisy one. -- Ondřej Kuzník Senior Software Engineer Symas Corporation http://www.symas.com Packaged, certified, and supported LDAP solutions powered by OpenLDAP

1 0

Re: (ITS#6545) delta-syncrepl rejects modification master accepted
by Frank.Swasey＠uvm.edu 05 Apr '17

05 Apr '17

This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --0-1015574222-1491391973=:71829 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8BIT Thanks for the patch to provide a test script that just shows the same thing. I see two possible solutions: 1) replacing the same attribute twice in the same modify LDIF is illegal (as it was in older releases) 2) the accesslog/syncrepl needs to record the final result, not every bit of the change. I believe the easiest solution is #1. However, #2 would be proper if this kind of abuse is legal. Consider the following deck: dn: cn=Oh Noes,ou=People,dc=example,dc=com changetype: modify delete: sn sn: George - add: sn sn: Thomas - delete: sn sn: Thomas - add: sn sn: George - Is that legal? Well, it works and results in an entry in the accesslog that contains: reqMod: sn:- George reqMod: sn:+ Thomas reqMod: sn:- Thomas reqMod: sn:+ George and has no issues in the replication. Therefore, why does this deck: dn: cn=Oh Noes,ou=People,dc=example,dc=com changetype: modify replace: sn sn: Thomas - replace: sn sn: George - which creates an accesslog entry that contains: reqMod: sn:= Thomas reqMod: sn:= George Interestingly, this morning when I was performing this, the replica ate that accesslog entry and put both of those sn's into the LDAP entry on the replica (instead of the normal barking in the logs that it failed). Perhaps because I had done the former test with the delete/add/delete/add sequence? At this point, I'm thinking the accesslog should probably generate the minimal changes necessary to get to the right state. That would mean the former test should produce: reqMod: sn:- George reqMod: sn:+ George and the latter: reqMod:sn:= George Either that, or the syncrepl processing of what accesslog is sending needs some serious investigation. I'll see what I can find for time to look into that serious investigation. - Frank On Tue, 4 Apr 2017 at 1:54pm, Ondřej Kuzník wrote: > The attached patch updates the testsuite to reproduce the issue: > ftp://ftp.openldap.org/incoming/Ondrej-Kuznik-20170404-ITS-6545-testcase.pa… > > The issue is that there is information lost when creating the log entry: > - the ordering of values is not guaranteed by LDAP, even though this is > rarely an issue with OpenLDAP with main backends (unless some overlay > interferes) > - there is no way to record where one modification ends and another > begins > > The obvious way to fix this is to record that a modification has ended. > This is technically a change to the accesslog format but delta syncrepl > will recover by falling back to plain syncrepl (which it already does in > the reported case). > > We could also record the order modifications as well (implementing the > X-ORDERED trait on the attribute), but that would break most consumers > of this format. > > -- Frank Swasey | http://www.uvm.edu/~fcs Sr Systems Administrator | Always remember: You are UNIQUE, University of Vermont | just like everyone else. "I am not young enough to know everything." - Oscar Wilde (1854-1900) --0-1015574222-1491391973=:71829--

1 0

Re: (ITS#6545) delta-syncrepl rejects modification master accepted
by ondra＠mistotebe.net 04 Apr '17

04 Apr '17

The attached patch updates the testsuite to reproduce the issue: ftp://ftp.openldap.org/incoming/Ondrej-Kuznik-20170404-ITS-6545-testcase.pa… The issue is that there is information lost when creating the log entry: - the ordering of values is not guaranteed by LDAP, even though this is rarely an issue with OpenLDAP with main backends (unless some overlay interferes) - there is no way to record where one modification ends and another begins The obvious way to fix this is to record that a modification has ended. This is technically a change to the accesslog format but delta syncrepl will recover by falling back to plain syncrepl (which it already does in the reported case). We could also record the order modifications as well (implementing the X-ORDERED trait on the attribute), but that would break most consumers of this format. -- Ondřej Kuzník Senior Software Engineer Symas Corporation http://www.symas.com Packaged, certified, and supported LDAP solutions powered by OpenLDAP

1 0

Re: (ITS#8630) Objectclass name being overwritten when loading schema
by quanah＠symas.com 04 Apr '17

04 Apr '17

--On Tuesday, April 04, 2017 7:30 AM +0000 thomas.fazekas(a)gmail.com wrote: > Full_Name: Thomas Fazekas > Version: 2.4.44 > OS: raspbian > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (213.181.38.213) Hi Thomas, The back-sql backend is clearly noted as being experimental in the man page. Currently there is no active maintainer/developer for the backend, so no promises if/when this will be fixed. I would note that there have been a number of people who have working installations of back-sql, but I don't know the specifics of their setup. Hope that helps. --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

(ITS#8630) Objectclass name being overwritten when loading schema
by thomas.fazekas＠gmail.com 03 Apr '17

03 Apr '17

Full_Name: Thomas Fazekas Version: 2.4.44 OS: raspbian URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (213.181.38.213) My system info : Linux berry 4.4.34+ #930 Wed Nov 23 15:12:30 GMT 2016 armv6l GNU/Linux (raspberry pi 3 with latest raspbian) openldap 2.4.44 built from sources : "./configure --prefix=/opt/openldap-2.4.44 --enable-sql" Mysql deployed from repository : 5.5.54-0+deb8u1 (Raspbian) followed the mysql specific README from the rdbms sources folder (created the db, schema & test data) Now, when I try to start-up the server I get : 58e33726 backend_startup_one: starting "dc=berry,dc=office,dc=company,dc=com" 58e33726 ==>backsql_db_open(): testing RDBMS connection 58e33726 backsql_db_open(): concat func not specified (use "concat_pattern" directive in slapd.conf) 58e33726 backsql_db_open(): children search SQL condition not specified (use "children_cond" directive in slapd.conf); preparing default 58e33726 backsql_db_open(): setting "ldap_entries.dn LIKE CONCAT('%,',?)" as default "children_cond" 58e33726 backsql_db_open(): DN match search SQL condition not specified (use "dn_match_cond" directive in slapd.conf); preparing default 58e33726 backsql_db_open(): setting "ldap_entries.dn=?" as default "dn_match_cond" 58e33726 backsql_db_open(): objectclass mapping SQL statement not specified (use "oc_query" directive in slapd.conf) 58e33726 backsql_db_open(): setting "SELECT id,name,keytbl,keycol,create_proc,delete_proc,expect_return FROM ldap_oc_mappings" by default 58e33726 backsql_db_open(): attribute mapping SQL statement not specified (use "at_query" directive in slapd.conf) 58e33726 backsql_db_open(): setting "SELECT name,sel_expr,from_tbls,join_where,add_proc,delete_proc,param_order,expect_return,sel_expr_u FROM ldap_attr_mappings WHERE oc_map_id=?" by default 58e33726 backsql_db_open(): entry deletion SQL statement not specified (use "delentry_stmt" directive in slapd.conf) 58e33726 backsql_db_open(): setting "DELETE FROM ldap_entries WHERE id=?" by default 58e33726 backsql_db_open(): entry deletion SQL statement not specified (use "renentry_stmt" directive in slapd.conf) 58e33726 backsql_db_open(): setting "UPDATE ldap_entries SET dn=?,parent=?,keyval=? WHERE id=?" by default 58e33726 backsql_db_open(): objclasses deletion SQL statement not specified (use "delobjclasses_stmt" directive in slapd.conf) 58e33726 backsql_db_open(): setting "DELETE FROM ldap_entry_objclasses WHERE entry_id=?" by default 58e33726 ==>backsql_get_db_conn() 58e33726 ==>backsql_open_db_handle() 58e33726 <==backsql_open_db_handle() 58e33726 <==backsql_get_db_conn() 58e33726 ==>backsql_load_schema_map() 58e33726 backsql_load_schema_map(): oc_query "SELECT id,name,keytbl,keycol,create_proc,delete_proc,expect_return FROM ldap_oc_mappings" 58e33726 objectClass: id="1" name="inetOrgPerson" keytbl="persons" keycol="id" create_proc="" create_keyval="" delete_proc="" expect_return="0"create_hint="" 58e33726 backsql_load_schema_map(): objectClass "inetOrgPerson": keytbl="persons" keycol="id" 58e33726 expect_return: add=0, del=0; attributes: 58e33726 objectClass: id="2" name="documenterson" keytbl="documents" keycol="id" create_proc="" create_keyval="" delete_proc="" expect_return="0"create_hint="" 58e33726 backsql_load_schema_map(): objectClass "documenterson" is not defined in schema 58e33726 backsql_db_open(): schema mapping failed, exiting 58e33726 backend_startup_one (type=sql, suffix="dc=berry,dc=office,dc=company,dc=com"): bi_db_open failed! (1) 58e33726 slapd shutdown: initiated 58e33726 ==>backsql_db_close() 58e33726 <==backsql_db_close() 58e33726 slapd destroy: freeing system resources. My related DB content : mysql> select * from ldap_oc_mappings; +----+---------------+------------+--------+-------------+-------------+---------------+ | id | name | keytbl | keycol | create_proc | delete_proc | expect_return | +----+---------------+------------+--------+-------------+-------------+---------------+ | 1 | inetOrgPerson | persons | id | NULL | NULL | 0 | | 2 | document | documents | id | NULL | NULL | 0 | | 3 | organization | institutes | id | NULL | NULL | 0 | | 4 | referral | referrals | id | NULL | NULL | 0 | +----+---------------+------------+--------+-------------+-------------+---------------+ 4 rows in set (0.00 sec) Now, as far as I can tell what is happening is that when loading the schema the first one (inetOrgPerson) is being overwritten by the second (document) without a terminator, ending up with an "documenterson" which obviously doesn't exist.

1 0

Re: (ITS#8425)
by quanah＠symas.com 03 Apr '17

03 Apr '17

Hello, The issue you opened is not about a software bug, but how to configure OpenLDAP. The ITS system is for issue reports only. Please use the OpenLDAP Technical list for help on how to configure OpenLDAP: <http://www.openldap.org/lists/mm/listinfo/openldap-technical> This ITS will be closed. Regards, Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

Re: (ITS#8444) Out-of-sync issue with memberOf overlay, Delta-syncrepl MMR and >2 nodes
by quanah＠symas.com 03 Apr '17

03 Apr '17

--On Friday, December 23, 2016 2:17 AM +0000 henson(a)acm.org wrote: > I sent this to the mailing list and was advised to add it to ITS 8444, > so here it is: Looks like this has actually been reported before: <http://www.openldap.org/its/index.cgi/?findid=7400> --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs April 2017