- openldap-bugs - openldap.org

Re: (ITS#6545) delta-syncrepl rejects modification master accepted
by ondra＠mistotebe.net 06 Apr '17

06 Apr '17

On Thu, Apr 06, 2017 at 05:14:15PM +0200, Michael Ströder wrote: > ondra(a)mistotebe.net wrote: >> On Wed, Apr 05, 2017 at 04:14:12PM +0200, Michael StrĂśder wrote: >>> => There could be a slapd per-backend configuation directive to disallow it with a >>> strong hint in the docs recommending to disallow it when using delta-syncrepl. >>> >>> Suggestion: >>> disallow mod_attr_repeated >> >> In my view, that's more pain than it's worth. > > Hmm, I think slapd should be able to disallow a crazy modify request like this: > > dn: cn=foobar,dc=example,dc=com > changetype: modify > replace: description > description: foobar1 > - > replace: description > description: foobar2 > - > .. > replace: description > description: foobar1000 > - Well, the clients are allowed to request a lot of strange things, some of which border on a DoS: e.g. right now slapd can't disallow a modify request like: dn: cn=foobar,dc=example,dc=com changetype: modify replace: description description: foobar1 description: foobar2 ... description: foobar1000 So there. If we can agree on a way to handle that, we might see whether it could be repurposed. I should have a patch for the accesslog issue soon. -- Ondřej Kuzník Senior Software Engineer Symas Corporation http://www.symas.com Packaged, certified, and supported LDAP solutions powered by OpenLDAP

1 0

Re: (ITS#6545) delta-syncrepl rejects modification master accepted
by michael＠stroeder.com 06 Apr '17

06 Apr '17

ondra(a)mistotebe.net wrote: > On Wed, Apr 05, 2017 at 04:14:12PM +0200, Michael Ströder wrote: >> ondra(a)mistotebe.net wrote: >>> On Wed, Apr 05, 2017 at 07:32:46AM -0400, Frank Swasey wrote: >>>> Thanks for the patch to provide a test script that just shows the same >>>> thing. >>>> >>>> I see two possible solutions: >>>> >>>> 1) replacing the same attribute twice in the same modify LDIF is illegal >>>> (as it was in older releases) >>> >>> AFAIK, LDAP doesn't forbid it so I don't see that going away. >> >> Yes, there's no text in RFC 4511 which forbids this: >> https://tools.ietf.org/html/rfc4511#section-4.6 >> >> However personally I consider LDAP clients sending modify requests like this to be >> broken/mis-behaving. (And I'd like to know which LDAP clients were causing this ITS.) > > I'm not saying it's common or good practice ;) > >> => There could be a slapd per-backend configuation directive to disallow it with a >> strong hint in the docs recommending to disallow it when using delta-syncrepl. >> >> Suggestion: >> disallow mod_attr_repeated > > In my view, that's more pain than it's worth. Hmm, I think slapd should be able to disallow a crazy modify request like this: dn: cn=foobar,dc=example,dc=com changetype: modify replace: description description: foobar1 - replace: description description: foobar2 - .. replace: description description: foobar1000 - Ciao, Michael.

1 0

Re: (ITS#6545) delta-syncrepl rejects modification master accepted
by Frank.Swasey＠uvm.edu 06 Apr '17

06 Apr '17

On Wed, 5 Apr 2017 at 8:18am, ondra(a)mistotebe.net wrote: > AFAIK, LDAP doesn't forbid it so I don't see that going away. Well, then that means that the current behavior is a bug in either the syncrepl implementation or the accesslog recording. Since the MASTER gets updated correctly and the REPLICAs (and other MMR members) all fall down when attempting to process the accesslog entry. > It does need a fair bit of attention, the problem is it's something that > would require a format change and there are already existing consumers > that would be affected. > > I'm going to try with a patch to record something like this: > regMod: sn:= George > regMod: : > regMod: sn:= George > > Together with the relevant syncrepl updates. > > While consumers would still be affected, this should be quite rare and > the current handling of that case would have been incorrect anyway. The > change would then only make the silent failure a noisy one. Noisy in that the replica/mmr falls over dead or just logs more stuff? To be honest, my goal is consistency. I want the replicas (and MMR mates) to all wind up with the same values the master has. -- Frank Swasey | http://www.uvm.edu/~fcs Sr Systems Administrator | Always remember: You are UNIQUE, University of Vermont | just like everyone else. "I am not young enough to know everything." - Oscar Wilde (1854-1900)

1 0

Re: (ITS#6545) delta-syncrepl rejects modification master accepted
by Frank.Swasey＠uvm.edu 06 Apr '17

06 Apr '17

On Wed, 5 Apr 2017 at 10:14am, michael(a)stroeder.com wrote: > However personally I consider LDAP clients sending modify requests like this to be > broken/mis-behaving. (And I'd like to know which LDAP clients were causing this ITS.) My original report was from 2010 - I presume it was something that the perl code (using Net::LDAP module) that I had written at the time tripped onto [or it was some awful edgecase test that I had devised]. To the best of my knowledge, this was not something that I had a client doing to the LDAP servers from the wild. -- Frank Swasey | http://www.uvm.edu/~fcs Sr Systems Administrator | Always remember: You are UNIQUE, University of Vermont | just like everyone else. "I am not young enough to know everything." - Oscar Wilde (1854-1900)

1 0

Re: (ITS#8629) slapauth segfault with GSSAPI + olcAuthzRegexp using internal LDAP search + ppolicy overlay
by clement.oudot＠savoirfairelinux.com 06 Apr '17

06 Apr '17

Hello, I would like to know if some of you have an idea on how to fix this bug? I am able to test a patch if you can provide one. Regards, Cl=E9ment.

1 0

Re: (ITS#8444) Out-of-sync issue with memberOf overlay, Delta-syncrepl MMR and >2 nodes
by henson＠acm.org 06 Apr '17

06 Apr '17

On Mon, Apr 03, 2017 at 11:15:17AM -0700, Quanah Gibson-Mount wrote: > Looks like this has actually been reported before: > <http://www.openldap.org/its/index.cgi/?findid=7400> Yikes, from 2012, and never resolved 8-/. I'm curious though why Howard says the work has to be distributed to all the replicas locally as it wouldn't scale otherwise? For adding or removing a user from a group, replicating the memberOf attribute on the user in addition to the member attribute on the group only doubles the replication load which doesn't seem excessive. I suppose if you delete a group with a large membership that would result in updating a large number of users, but that's no different than if a delete a single user from a large number of groups, which doesn't seem to cause any issues. Unless I'm misunderstanding something? In any case, I suppose this ITS can be closed now too, with the same resolution that the memberof overlay is no longer supported in a replicated configuration and to switch to the dynlist implementation...

1 0

Re: (ITS#8609) segfault in mods.c - modify_add_values
by henson＠acm.org 06 Apr '17

06 Apr '17

On Mon, Apr 03, 2017 at 02:44:48PM +0100, Howard Chu wrote: > You should probably switch to a non-optimized build. The backtrace you > posted initially had most parameters optimized out, which also makes > things harder to trace. Ok, I installed binaries built with -O0 on the master server that's been failing; I'll post a new backtrace the next time it crashes, thanks.

1 0

Re: (ITS#8533) Support OpenSSL-1.1.0c
by quanah＠symas.com 06 Apr '17

06 Apr '17

--On Thursday, April 06, 2017 2:02 AM +0100 Howard Chu <hyc(a)symas.com> wrote: >> +#if OPENSSL_VERSION_NUMBER < 0x10100000 > > Are you sure about exactly when these functions were removed? The title > of this ITS is for 1.1.0c, and the earlier patches were for 1.1.0a. For > version 1.1.0c we should be comparing to 0x1010003f, not 0x10100000. Yes, I checked against OpenSSL 1.1.0 (released before 1.1.0a). The OpenSSL shipped headers are clear on this as well, for example: #if OPENSSL_API_COMPAT < 0x10100000L # define SSL_library_init() OPENSSL_init_ssl(0, NULL) #endif #if OPENSSL_API_COMPAT < 0x10100000L # define SSL_load_error_strings() \ OPENSSL_init_ssl(OPENSSL_INIT_LOAD_SSL_STRINGS \ | OPENSSL_INIT_LOAD_CRYPTO_STRINGS, NULL) #endif etc -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

Re: (ITS#8533) Support OpenSSL-1.1.0c
by hyc＠symas.com 06 Apr '17

06 Apr '17

quanah(a)symas.com wrote: > --On Wednesday, April 05, 2017 11:12 PM +0000 quanah(a)symas.com wrote: > >>> The following 5 function calls are problematic: >>> >>> ./.libs/libldap.so: undefined reference to `OpenSSL_add_all_digests' >>> ./.libs/libldap.so: undefined reference to `SSL_load_error_strings' >>> ./.libs/libldap.so: undefined reference to `ERR_free_strings' >>> ./.libs/libldap.so: undefined reference to `EVP_cleanup' >>> ./.libs/libldap.so: undefined reference to `SSL_library_init' > > The above can be fixed with the following patch: > > build@u12build:~/git/symas-packages/thirdparty/openldap/build/UBUNTU12_64/symas-openldap/libraries/libldap$ > diff -u tls_o.c.orig tls_o.c > --- tls_o.c.orig 2017-04-05 14:40:02.849559862 -0700 > +++ tls_o.c 2017-04-05 15:13:09.371718493 -0700 > @@ -154,9 +154,16 @@ > (void) tlso_seed_PRNG( lo->ldo_tls_randfile ); > #endif > > +#if OPENSSL_VERSION_NUMBER < 0x10100000 Are you sure about exactly when these functions were removed? The title of this ITS is for 1.1.0c, and the earlier patches were for 1.1.0a. For version 1.1.0c we should be comparing to 0x1010003f, not 0x10100000. > SSL_load_error_strings(); > SSL_library_init(); > OpenSSL_add_all_digests(); > +#else > + OPENSSL_init_ssl(OPENSSL_INIT_LOAD_SSL_STRINGS \ > + | OPENSSL_INIT_LOAD_CRYPTO_STRINGS, NULL); > + OPENSSL_init_ssl(0, NULL); > + OPENSSL_init_crypto(OPENSSL_INIT_ADD_ALL_DIGESTS, NULL); > +#endif > > /* FIXME: mod_ssl does this */ > X509V3_add_standard_extensions(); > @@ -172,6 +179,7 @@ > { > struct ldapoptions *lo = LDAP_INT_GLOBAL_OPT(); > > +#if OPENSSL_VERSION_NUMBER < 0x10100000 > EVP_cleanup(); > #if OPENSSL_VERSION_NUMBER < 0x10000000 > ERR_remove_state(0); > @@ -179,6 +187,9 @@ > ERR_remove_thread_state(NULL); > #endif > ERR_free_strings(); > +#else > + ERR_remove_thread_state(NULL); > +#endif > > if ( lo->ldo_tls_randfile ) { > LDAP_FREE( lo->ldo_tls_randfile ); > > > --Quanah > > > -- > > Quanah Gibson-Mount > Product Architect > Symas Corporation > Packaged, certified, and supported LDAP solutions powered by OpenLDAP: > <http://www.symas.com> > > > > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8533) Support OpenSSL-1.1.0c
by hyc＠symas.com 06 Apr '17

06 Apr '17

quanah(a)symas.com wrote: > --On Wednesday, April 05, 2017 10:38 PM +0000 quanah(a)symas.com wrote: > >> --On Wednesday, April 05, 2017 7:58 PM +0000 quanah(a)symas.com wrote: >> >>> If openssl 1.1.0 is built with the option "no-deprecated" the build will >>> fail, as portions of the code still use the pre 1.1 API. This needs >>> fixing before release. >> >> The following 5 function calls are problematic: >> >> ./.libs/libldap.so: undefined reference to `OpenSSL_add_all_digests' >> ./.libs/libldap.so: undefined reference to `SSL_load_error_strings' >> ./.libs/libldap.so: undefined reference to `ERR_free_strings' >> ./.libs/libldap.so: undefined reference to `EVP_cleanup' >> ./.libs/libldap.so: undefined reference to `SSL_library_init' >> >> >> Looking at the best way in which to fix. > > These also need fixing, as they don't exist in 1.1 when the old API is > disabled. > > CRYPTO_num_locks() > CRYPTO_LOCK > CRYPTO_set_locking_callback() > CRYPTO_set_id_callback() These only exist because OpenSSL's support for threading was mostly nonexistent in the past. They should just be #ifdef'd away for 1.1. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8533) Support OpenSSL-1.1.0c
by quanah＠symas.com 06 Apr '17

06 Apr '17

--On Wednesday, April 05, 2017 11:12 PM +0000 quanah(a)symas.com wrote: >> The following 5 function calls are problematic: >> >> ./.libs/libldap.so: undefined reference to `OpenSSL_add_all_digests' >> ./.libs/libldap.so: undefined reference to `SSL_load_error_strings' >> ./.libs/libldap.so: undefined reference to `ERR_free_strings' >> ./.libs/libldap.so: undefined reference to `EVP_cleanup' >> ./.libs/libldap.so: undefined reference to `SSL_library_init' The above can be fixed with the following patch: build@u12build:~/git/symas-packages/thirdparty/openldap/build/UBUNTU12_64/symas-openldap/libraries/libldap$ diff -u tls_o.c.orig tls_o.c --- tls_o.c.orig 2017-04-05 14:40:02.849559862 -0700 +++ tls_o.c 2017-04-05 15:13:09.371718493 -0700 @@ -154,9 +154,16 @@ (void) tlso_seed_PRNG( lo->ldo_tls_randfile ); #endif +#if OPENSSL_VERSION_NUMBER < 0x10100000 SSL_load_error_strings(); SSL_library_init(); OpenSSL_add_all_digests(); +#else + OPENSSL_init_ssl(OPENSSL_INIT_LOAD_SSL_STRINGS \ + | OPENSSL_INIT_LOAD_CRYPTO_STRINGS, NULL); + OPENSSL_init_ssl(0, NULL); + OPENSSL_init_crypto(OPENSSL_INIT_ADD_ALL_DIGESTS, NULL); +#endif /* FIXME: mod_ssl does this */ X509V3_add_standard_extensions(); @@ -172,6 +179,7 @@ { struct ldapoptions *lo = LDAP_INT_GLOBAL_OPT(); +#if OPENSSL_VERSION_NUMBER < 0x10100000 EVP_cleanup(); #if OPENSSL_VERSION_NUMBER < 0x10000000 ERR_remove_state(0); @@ -179,6 +187,9 @@ ERR_remove_thread_state(NULL); #endif ERR_free_strings(); +#else + ERR_remove_thread_state(NULL); +#endif if ( lo->ldo_tls_randfile ) { LDAP_FREE( lo->ldo_tls_randfile ); --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

Re: (ITS#8533) Support OpenSSL-1.1.0c
by quanah＠symas.com 06 Apr '17

06 Apr '17

--On Wednesday, April 05, 2017 10:38 PM +0000 quanah(a)symas.com wrote: > --On Wednesday, April 05, 2017 7:58 PM +0000 quanah(a)symas.com wrote: > >> If openssl 1.1.0 is built with the option "no-deprecated" the build will >> fail, as portions of the code still use the pre 1.1 API. This needs >> fixing before release. > > The following 5 function calls are problematic: > > ./.libs/libldap.so: undefined reference to `OpenSSL_add_all_digests' > ./.libs/libldap.so: undefined reference to `SSL_load_error_strings' > ./.libs/libldap.so: undefined reference to `ERR_free_strings' > ./.libs/libldap.so: undefined reference to `EVP_cleanup' > ./.libs/libldap.so: undefined reference to `SSL_library_init' > > > Looking at the best way in which to fix. These also need fixing, as they don't exist in 1.1 when the old API is disabled. CRYPTO_num_locks() CRYPTO_LOCK CRYPTO_set_locking_callback() CRYPTO_set_id_callback() --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

Re: (ITS#8533) Support OpenSSL-1.1.0c
by quanah＠symas.com 05 Apr '17

05 Apr '17

--On Wednesday, April 05, 2017 7:58 PM +0000 quanah(a)symas.com wrote: > If openssl 1.1.0 is built with the option "no-deprecated" the build will > fail, as portions of the code still use the pre 1.1 API. This needs > fixing before release. The following 5 function calls are problematic: ./.libs/libldap.so: undefined reference to `OpenSSL_add_all_digests' ./.libs/libldap.so: undefined reference to `SSL_load_error_strings' ./.libs/libldap.so: undefined reference to `ERR_free_strings' ./.libs/libldap.so: undefined reference to `EVP_cleanup' ./.libs/libldap.so: undefined reference to `SSL_library_init' Looking at the best way in which to fix. --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

Re: (ITS#8533) Support OpenSSL-1.1.0c
by quanah＠symas.com 05 Apr '17

05 Apr '17

If openssl 1.1.0 is built with the option "no-deprecated" the build will fail, as portions of the code still use the pre 1.1 API. This needs fixing before release. --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

Re: (ITS#7013) OpenLDAP freezes and doesn't respond
by quanah＠symas.com 05 Apr '17

05 Apr '17

Hello, Due to inactivity and the noted improper configuration, I am closing this ITS. I would note there have been hundreds of fixes to OpenLDAP since this report was filed, including a new BDB-free backend. If you ever drop OpenDJ and move back to OpenLDAP and can reproduce the issue using back-mdb (based on LMDB), please file a new ITS. Thanks! --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

Re: (ITS#6545) delta-syncrepl rejects modification master accepted
by ondra＠mistotebe.net 05 Apr '17

05 Apr '17

On Wed, Apr 05, 2017 at 04:14:12PM +0200, Michael Ströder wrote: > ondra(a)mistotebe.net wrote: >> On Wed, Apr 05, 2017 at 07:32:46AM -0400, Frank Swasey wrote: >>> Thanks for the patch to provide a test script that just shows the same >>> thing. >>> >>> I see two possible solutions: >>> >>> 1) replacing the same attribute twice in the same modify LDIF is illegal >>> (as it was in older releases) >> >> AFAIK, LDAP doesn't forbid it so I don't see that going away. > > Yes, there's no text in RFC 4511 which forbids this: > https://tools.ietf.org/html/rfc4511#section-4.6 > > However personally I consider LDAP clients sending modify requests like this to be > broken/mis-behaving. (And I'd like to know which LDAP clients were causing this ITS.) I'm not saying it's common or good practice ;) > => There could be a slapd per-backend configuation directive to disallow it with a strong > hint in the docs recommending to disallow it when using delta-syncrepl. > > Suggestion: > disallow mod_attr_repeated In my view, that's more pain than it's worth. >>> 2) the accesslog/syncrepl needs to record the final result, not every bit >>> of the change. >> >> Two problems: >> - the log is meant to record the request for review/statistics purposes >> as well and should record the intent, not just the result >> - the modification might fail, in that case you still need an accurate >> record of the request > > IIRC slapo-accesslog was primarily implemented for delta-syncrepl. > > Personally I'm in the (ab)use-slapo-accesslog-as-auditlog camp. ;-) > But still I'd consider an optimized changes list written to accesslog to be perfectly > fine for security auditing because it would represent what caused the modification of the > database content. I don't think it matters what it was written for, the fact that it's officially useful for that as you and others have realised means they might object to us making drastic changes in 2.4, a series the project have tried to freeze even maintenance for. > Also note that not every failed modification is written to accesslog-DB anyway because > most malformed write operations already fail in slapd's front-end (schema check etc.) and > never reach the DB backend (and thus slapo-accesslog). So debugging errornous clients is > something for which you have to use Wireshark etc. in most cases anyway. I think most of the ones that don't reach accesslog get a protocol error or come from anonymous connections, schema checking and ACL should happen later. > I'm not familiar with the inner workings of slapd but these things should be carefully > considered when optimizing the changes list of a modify request: > 1. constraint checking (slapo-unique and slapo-constraint) > 2. impact on indexing > 3. access control, especially with val=foo part in the ACL > > 1. and 2. are hopefully already done on the "resulting entry after the entire list of > modifications is performed" (RFC 4511). Not sure I follow. On the consumer, syncrepl bypasses all ACLs and the overlays you mention should already be capable of passing the operation intact. On the provider side it depends on the overlay order and the administrator has the option to decide that if they wish. Consistency and data model are generally maintained quite alright. -- Ondřej Kuzník Senior Software Engineer Symas Corporation http://www.symas.com Packaged, certified, and supported LDAP solutions powered by OpenLDAP

1 0

Re: (ITS#6545) delta-syncrepl rejects modification master accepted
by michael＠stroeder.com 05 Apr '17

05 Apr '17

ondra(a)mistotebe.net wrote: > On Wed, Apr 05, 2017 at 07:32:46AM -0400, Frank Swasey wrote: >> Thanks for the patch to provide a test script that just shows the same >> thing. >> >> I see two possible solutions: >> >> 1) replacing the same attribute twice in the same modify LDIF is illegal >> (as it was in older releases) > > AFAIK, LDAP doesn't forbid it so I don't see that going away. Yes, there's no text in RFC 4511 which forbids this: https://tools.ietf.org/html/rfc4511#section-4.6 However personally I consider LDAP clients sending modify requests like this to be broken/mis-behaving. (And I'd like to know which LDAP clients were causing this ITS.) => There could be a slapd per-backend configuation directive to disallow it with a strong hint in the docs recommending to disallow it when using delta-syncrepl. Suggestion: disallow mod_attr_repeated >> 2) the accesslog/syncrepl needs to record the final result, not every bit >> of the change. > > Two problems: > - the log is meant to record the request for review/statistics purposes > as well and should record the intent, not just the result > - the modification might fail, in that case you still need an accurate > record of the request IIRC slapo-accesslog was primarily implemented for delta-syncrepl. Personally I'm in the (ab)use-slapo-accesslog-as-auditlog camp. ;-) But still I'd consider an optimized changes list written to accesslog to be perfectly fine for security auditing because it would represent what caused the modification of the database content. Also note that not every failed modification is written to accesslog-DB anyway because most malformed write operations already fail in slapd's front-end (schema check etc.) and never reach the DB backend (and thus slapo-accesslog). So debugging errornous clients is something for which you have to use Wireshark etc. in most cases anyway. I'm not familiar with the inner workings of slapd but these things should be carefully considered when optimizing the changes list of a modify request: 1. constraint checking (slapo-unique and slapo-constraint) 2. impact on indexing 3. access control, especially with val=foo part in the ACL 1. and 2. are hopefully already done on the "resulting entry after the entire list of modifications is performed" (RFC 4511). Ciao, Michael.

1 0

Re: (ITS#6545) delta-syncrepl rejects modification master accepted
by ondra＠mistotebe.net 05 Apr '17

05 Apr '17

On Wed, Apr 05, 2017 at 07:32:46AM -0400, Frank Swasey wrote: > Thanks for the patch to provide a test script that just shows the same > thing. > > I see two possible solutions: > > 1) replacing the same attribute twice in the same modify LDIF is illegal > (as it was in older releases) AFAIK, LDAP doesn't forbid it so I don't see that going away. > 2) the accesslog/syncrepl needs to record the final result, not every bit > of the change. Two problems: - the log is meant to record the request for review/statistics purposes as well and should record the intent, not just the result - the modification might fail, in that case you still need an accurate record of the request > I believe the easiest solution is #1. However, #2 would be proper if this > kind of abuse is legal. > > Consider the following deck: > > [...] > > Is that legal? Well, it works and results in an entry in the accesslog that > contains: > > [...] > > and has no issues in the replication. > > Therefore, why does this deck: > > dn: cn=Oh Noes,ou=People,dc=example,dc=com > changetype: modify > replace: sn > sn: Thomas > - > replace: sn > sn: George > - > > which creates an accesslog entry that contains: > > reqMod: sn:= Thomas > reqMod: sn:= George > > Interestingly, this morning when I was performing this, the replica ate that > accesslog entry and put both of those sn's into the LDAP entry on the > replica (instead of the normal barking in the logs that it failed). Perhaps > because I had done the former test with the delete/add/delete/add sequence? Your original example did two replacements with the same value, the test case runs two replacements with different values as that will cause the two servers to de-sync silently. > At this point, I'm thinking the accesslog should probably generate the > minimal changes necessary to get to the right state. That would mean the > former test should produce: > > reqMod: sn:- George > reqMod: sn:+ George > > and the latter: > > reqMod:sn:= George > > Either that, or the syncrepl processing of what accesslog is sending needs > some serious investigation. I'll see what I can find for time to look into > that serious investigation. It does need a fair bit of attention, the problem is it's something that would require a format change and there are already existing consumers that would be affected. I'm going to try with a patch to record something like this: regMod: sn:= George regMod: : regMod: sn:= George Together with the relevant syncrepl updates. While consumers would still be affected, this should be quite rare and the current handling of that case would have been incorrect anyway. The change would then only make the silent failure a noisy one. -- Ondřej Kuzník Senior Software Engineer Symas Corporation http://www.symas.com Packaged, certified, and supported LDAP solutions powered by OpenLDAP

1 0

Re: (ITS#6545) delta-syncrepl rejects modification master accepted
by Frank.Swasey＠uvm.edu 05 Apr '17

05 Apr '17

This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --0-1015574222-1491391973=:71829 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8BIT Thanks for the patch to provide a test script that just shows the same thing. I see two possible solutions: 1) replacing the same attribute twice in the same modify LDIF is illegal (as it was in older releases) 2) the accesslog/syncrepl needs to record the final result, not every bit of the change. I believe the easiest solution is #1. However, #2 would be proper if this kind of abuse is legal. Consider the following deck: dn: cn=Oh Noes,ou=People,dc=example,dc=com changetype: modify delete: sn sn: George - add: sn sn: Thomas - delete: sn sn: Thomas - add: sn sn: George - Is that legal? Well, it works and results in an entry in the accesslog that contains: reqMod: sn:- George reqMod: sn:+ Thomas reqMod: sn:- Thomas reqMod: sn:+ George and has no issues in the replication. Therefore, why does this deck: dn: cn=Oh Noes,ou=People,dc=example,dc=com changetype: modify replace: sn sn: Thomas - replace: sn sn: George - which creates an accesslog entry that contains: reqMod: sn:= Thomas reqMod: sn:= George Interestingly, this morning when I was performing this, the replica ate that accesslog entry and put both of those sn's into the LDAP entry on the replica (instead of the normal barking in the logs that it failed). Perhaps because I had done the former test with the delete/add/delete/add sequence? At this point, I'm thinking the accesslog should probably generate the minimal changes necessary to get to the right state. That would mean the former test should produce: reqMod: sn:- George reqMod: sn:+ George and the latter: reqMod:sn:= George Either that, or the syncrepl processing of what accesslog is sending needs some serious investigation. I'll see what I can find for time to look into that serious investigation. - Frank On Tue, 4 Apr 2017 at 1:54pm, Ondřej Kuzník wrote: > The attached patch updates the testsuite to reproduce the issue: > ftp://ftp.openldap.org/incoming/Ondrej-Kuznik-20170404-ITS-6545-testcase.pa… > > The issue is that there is information lost when creating the log entry: > - the ordering of values is not guaranteed by LDAP, even though this is > rarely an issue with OpenLDAP with main backends (unless some overlay > interferes) > - there is no way to record where one modification ends and another > begins > > The obvious way to fix this is to record that a modification has ended. > This is technically a change to the accesslog format but delta syncrepl > will recover by falling back to plain syncrepl (which it already does in > the reported case). > > We could also record the order modifications as well (implementing the > X-ORDERED trait on the attribute), but that would break most consumers > of this format. > > -- Frank Swasey | http://www.uvm.edu/~fcs Sr Systems Administrator | Always remember: You are UNIQUE, University of Vermont | just like everyone else. "I am not young enough to know everything." - Oscar Wilde (1854-1900) --0-1015574222-1491391973=:71829--

1 0

Re: (ITS#6545) delta-syncrepl rejects modification master accepted
by ondra＠mistotebe.net 04 Apr '17

04 Apr '17

The attached patch updates the testsuite to reproduce the issue: ftp://ftp.openldap.org/incoming/Ondrej-Kuznik-20170404-ITS-6545-testcase.pa… The issue is that there is information lost when creating the log entry: - the ordering of values is not guaranteed by LDAP, even though this is rarely an issue with OpenLDAP with main backends (unless some overlay interferes) - there is no way to record where one modification ends and another begins The obvious way to fix this is to record that a modification has ended. This is technically a change to the accesslog format but delta syncrepl will recover by falling back to plain syncrepl (which it already does in the reported case). We could also record the order modifications as well (implementing the X-ORDERED trait on the attribute), but that would break most consumers of this format. -- Ondřej Kuzník Senior Software Engineer Symas Corporation http://www.symas.com Packaged, certified, and supported LDAP solutions powered by OpenLDAP

1 0

Re: (ITS#8630) Objectclass name being overwritten when loading schema
by quanah＠symas.com 04 Apr '17

04 Apr '17

--On Tuesday, April 04, 2017 7:30 AM +0000 thomas.fazekas(a)gmail.com wrote: > Full_Name: Thomas Fazekas > Version: 2.4.44 > OS: raspbian > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (213.181.38.213) Hi Thomas, The back-sql backend is clearly noted as being experimental in the man page. Currently there is no active maintainer/developer for the backend, so no promises if/when this will be fixed. I would note that there have been a number of people who have working installations of back-sql, but I don't know the specifics of their setup. Hope that helps. --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

(ITS#8630) Objectclass name being overwritten when loading schema
by thomas.fazekas＠gmail.com 04 Apr '17

04 Apr '17

Full_Name: Thomas Fazekas Version: 2.4.44 OS: raspbian URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (213.181.38.213) My system info : Linux berry 4.4.34+ #930 Wed Nov 23 15:12:30 GMT 2016 armv6l GNU/Linux (raspberry pi 3 with latest raspbian) openldap 2.4.44 built from sources : "./configure --prefix=/opt/openldap-2.4.44 --enable-sql" Mysql deployed from repository : 5.5.54-0+deb8u1 (Raspbian) followed the mysql specific README from the rdbms sources folder (created the db, schema & test data) Now, when I try to start-up the server I get : 58e33726 backend_startup_one: starting "dc=berry,dc=office,dc=company,dc=com" 58e33726 ==>backsql_db_open(): testing RDBMS connection 58e33726 backsql_db_open(): concat func not specified (use "concat_pattern" directive in slapd.conf) 58e33726 backsql_db_open(): children search SQL condition not specified (use "children_cond" directive in slapd.conf); preparing default 58e33726 backsql_db_open(): setting "ldap_entries.dn LIKE CONCAT('%,',?)" as default "children_cond" 58e33726 backsql_db_open(): DN match search SQL condition not specified (use "dn_match_cond" directive in slapd.conf); preparing default 58e33726 backsql_db_open(): setting "ldap_entries.dn=?" as default "dn_match_cond" 58e33726 backsql_db_open(): objectclass mapping SQL statement not specified (use "oc_query" directive in slapd.conf) 58e33726 backsql_db_open(): setting "SELECT id,name,keytbl,keycol,create_proc,delete_proc,expect_return FROM ldap_oc_mappings" by default 58e33726 backsql_db_open(): attribute mapping SQL statement not specified (use "at_query" directive in slapd.conf) 58e33726 backsql_db_open(): setting "SELECT name,sel_expr,from_tbls,join_where,add_proc,delete_proc,param_order,expect_return,sel_expr_u FROM ldap_attr_mappings WHERE oc_map_id=?" by default 58e33726 backsql_db_open(): entry deletion SQL statement not specified (use "delentry_stmt" directive in slapd.conf) 58e33726 backsql_db_open(): setting "DELETE FROM ldap_entries WHERE id=?" by default 58e33726 backsql_db_open(): entry deletion SQL statement not specified (use "renentry_stmt" directive in slapd.conf) 58e33726 backsql_db_open(): setting "UPDATE ldap_entries SET dn=?,parent=?,keyval=? WHERE id=?" by default 58e33726 backsql_db_open(): objclasses deletion SQL statement not specified (use "delobjclasses_stmt" directive in slapd.conf) 58e33726 backsql_db_open(): setting "DELETE FROM ldap_entry_objclasses WHERE entry_id=?" by default 58e33726 ==>backsql_get_db_conn() 58e33726 ==>backsql_open_db_handle() 58e33726 <==backsql_open_db_handle() 58e33726 <==backsql_get_db_conn() 58e33726 ==>backsql_load_schema_map() 58e33726 backsql_load_schema_map(): oc_query "SELECT id,name,keytbl,keycol,create_proc,delete_proc,expect_return FROM ldap_oc_mappings" 58e33726 objectClass: id="1" name="inetOrgPerson" keytbl="persons" keycol="id" create_proc="" create_keyval="" delete_proc="" expect_return="0"create_hint="" 58e33726 backsql_load_schema_map(): objectClass "inetOrgPerson": keytbl="persons" keycol="id" 58e33726 expect_return: add=0, del=0; attributes: 58e33726 objectClass: id="2" name="documenterson" keytbl="documents" keycol="id" create_proc="" create_keyval="" delete_proc="" expect_return="0"create_hint="" 58e33726 backsql_load_schema_map(): objectClass "documenterson" is not defined in schema 58e33726 backsql_db_open(): schema mapping failed, exiting 58e33726 backend_startup_one (type=sql, suffix="dc=berry,dc=office,dc=company,dc=com"): bi_db_open failed! (1) 58e33726 slapd shutdown: initiated 58e33726 ==>backsql_db_close() 58e33726 <==backsql_db_close() 58e33726 slapd destroy: freeing system resources. My related DB content : mysql> select * from ldap_oc_mappings; +----+---------------+------------+--------+-------------+-------------+---------------+ | id | name | keytbl | keycol | create_proc | delete_proc | expect_return | +----+---------------+------------+--------+-------------+-------------+---------------+ | 1 | inetOrgPerson | persons | id | NULL | NULL | 0 | | 2 | document | documents | id | NULL | NULL | 0 | | 3 | organization | institutes | id | NULL | NULL | 0 | | 4 | referral | referrals | id | NULL | NULL | 0 | +----+---------------+------------+--------+-------------+-------------+---------------+ 4 rows in set (0.00 sec) Now, as far as I can tell what is happening is that when loading the schema the first one (inetOrgPerson) is being overwritten by the second (document) without a terminator, ending up with an "documenterson" which obviously doesn't exist.

1 0

Re: (ITS#8425)
by quanah＠symas.com 03 Apr '17

03 Apr '17

Hello, The issue you opened is not about a software bug, but how to configure OpenLDAP. The ITS system is for issue reports only. Please use the OpenLDAP Technical list for help on how to configure OpenLDAP: <http://www.openldap.org/lists/mm/listinfo/openldap-technical> This ITS will be closed. Regards, Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

Re: (ITS#8444) Out-of-sync issue with memberOf overlay, Delta-syncrepl MMR and >2 nodes
by quanah＠symas.com 03 Apr '17

03 Apr '17

--On Friday, December 23, 2016 2:17 AM +0000 henson(a)acm.org wrote: > I sent this to the mailing list and was advised to add it to ITS 8444, > so here it is: Looks like this has actually been reported before: <http://www.openldap.org/its/index.cgi/?findid=7400> --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

Re: (ITS#7183) Open LDAP - Failed user id creation
by quanah＠symas.com 03 Apr '17

03 Apr '17

Hello, This report does not contain any useful or actionable items. The first step I would advise is to use a current release of OpenLDAP, and if you continue to have issues, please read the FAQ on how to report an issue: <http://www.openldap.org/faq/data/cache/59.html> This ITS will be closed Regards, Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs