- openldap-bugs - openldap.org

Re: (ITS#8980) Async mode and TLS Non-Blocking Issue
by quanah＠symas.com 19 Feb '19

19 Feb '19

--On Tuesday, February 19, 2019 2:12 AM -0500 Vern Smith <vsmith(a)interlinknetworks.com> wrote: > ITS#8957 and ITS#8968 are my submits and I was using those changes. > ITS#8963 is for OpenLDAP servers and I only an using the libraries so it > does not apply to my updates. Please keep replies to the ITS system. When filing an ITS, please note specifically which commits past a release you've applied, so the ITS is a clear representation of the issue you're reporting (and thus avoiding questions such as those I posited). Thanks! --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

Re: (ITS#8980) Async mode and TLS Non-Blocking Issue
by quanah＠symas.com 19 Feb '19

19 Feb '19

--On Tuesday, February 19, 2019 5:57 AM +0000 vsmith(a)interlinknetworks.com wrote: > Full_Name: Vernon Smith > Version: 2.4.47 > OS: Linux, Solaris > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (2601:40d:4300:679a:d18c:3060:e826:d35c) > > > I am using libldap library built with -DLDAP_USE_NON_BLOCKING_TLS and > configured for Async connection mode. I test making connections using the > library to servers that are hung to verify that my application will not > hang in those cases. I have found 3 issues. The first is that the > ldap_pvt_connect() clears non-blocking socket setup after the connection > is made even when Async mode was configured. So here is my patch for that. Did you pick up the fix for ITS#8957 from RE24? Did you pick up the fix for ITS#8968 from RE24? Did you pick up the fix for ITS#8963 from RE24? Those are all post 2.4.47 fixes. --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

(ITS#8981) Async mode and TLS Non-Blocking Issue
by vsmith＠interlinknetworks.com 19 Feb '19

19 Feb '19

Full_Name: Vernon Smith Version: 2.4.47 OS: Linux, Solaris URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (2601:40d:4300:679a:d18c:3060:e826:d35c) I am using libldap library built with -DLDAP_USE_NON_BLOCKING_TLS and configured for Async connection mode. I test making connections using the library to servers that are hung to verify that my application will not hang in those cases. I have found 3 issues. The first is that the ldap_pvt_connect() clears non-blocking socket setup after the connection is made even when Async mode was configured. So here is my patch for that. diff --git a/libraries/libldap/os-ip.c b/libraries/libldap/os-ip.c index a823cc6..d7927e5 100644 --- a/libraries/libldap/os-ip.c +++ b/libraries/libldap/os-ip.c @@ -443,7 +443,7 @@ ldap_pvt_connect(LDAP *ld, ber_socket_t s, if ( connect(s, sin, addrlen) != AC_SOCKET_ERROR ) { osip_debug(ld, "connect success\n", 0, 0, 0); - if ( opt_tv && ldap_pvt_ndelay_off(ld, s) == -1 ) + if ( !async && opt_tv && ldap_pvt_ndelay_off(ld, s) == -1 ) return ( -1 ); return ( 0 ); } The second issue is that the tlso_session_connect() routine does not correctly handle the return code from SSL_connect(), it just returns it to the caller. For LDAP_USE_NON_BLOCKING_TLS, the return code must be checked and an appropriate return used. Here is my patch. diff --git a/libraries/libldap/tls_o.c b/libraries/libldap/tls_o.c index e95a448..7a31b5e 100644 --- a/libraries/libldap/tls_o.c +++ b/libraries/libldap/tls_o.c @@ -531,7 +531,23 @@ tlso_session_connect( LDAP *ld, tls_session *sess ) tlso_session *s = (tlso_session *)sess; /* Caller expects 0 = success, OpenSSL returns 1 = success */ - return SSL_connect( s ) - 1; + int rc = SSL_connect( s ) - 1; + +#ifdef LDAP_USE_NON_BLOCKING_TLS + int sslerr = SSL_get_error(s, rc+1); + int sockerr = sock_errno(); + + if ( rc < 0 ) { + if ( sslerr == SSL_ERROR_WANT_READ || sslerr == SSL_ERROR_WANT_WRITE ) { + rc = 0; + } else if (( sslerr == SSL_ERROR_SYSCALL ) && + ( sockerr == EAGAIN || sockerr == ENOTCONN )) { + rc = 0; + } + } +#endif /* LDAP_USE_NON_BLOCKING_TLS */ + + return rc; } static int The third issue is that ldap_int_tls_start() compiled with -DLDAP_USE_NON_BLOCKING_TLS Plays with the socket non-blocking setting even if Async mode is configured. Here is mt patch to only play with the socket setting if not in Async mode. diff --git a/libraries/libldap/tls2.c b/libraries/libldap/tls2.c index d9b2d27..69b749c 100644 --- a/libraries/libldap/tls2.c +++ b/libraries/libldap/tls2.c @@ -1075,8 +1075,11 @@ ldap_int_tls_start ( LDAP *ld, LDAPConn *conn, LDAPURLDesc *srv ) /* * Use non-blocking io during SSL Handshake when a timeout is configured */ + int async = LDAP_BOOL_GET( &ld->ld_options, LDAP_BOOL_CONNECT_ASYNC ); if ( ld->ld_options.ldo_tm_net.tv_sec >= 0 ) { - ber_sockbuf_ctrl( sb, LBER_SB_OPT_SET_NONBLOCK, (void*)1 ); + if ( ! async ) { + ber_sockbuf_ctrl( sb, LBER_SB_OPT_SET_NONBLOCK, (void*)1 ); + } ber_sockbuf_ctrl( sb, LBER_SB_OPT_GET_FD, &sd ); tv = ld->ld_options.ldo_tm_net; tv0 = tv; @@ -1110,8 +1113,10 @@ ldap_int_tls_start ( LDAP *ld, LDAPConn *conn, LDAPURLDesc *srv ) ld->ld_errno = LDAP_TIMEOUT; break; } else { - /* ldap_int_poll called ldap_pvt_ndelay_off */ - ber_sockbuf_ctrl( sb, LBER_SB_OPT_SET_NONBLOCK, (void*)1 ); + /* ldap_int_poll called ldap_pvt_ndelay_off if not in async mode */ + if ( ! async ) { + ber_sockbuf_ctrl( sb, LBER_SB_OPT_SET_NONBLOCK, (void*)1 ); + } ret = ldap_int_tls_connect( ld, conn, host ); if ( ret > 0 ) { /* need to call tls_connect once more */ struct timeval curr_time_tv, delta_tv; @@ -1159,7 +1164,9 @@ ldap_int_tls_start ( LDAP *ld, LDAPConn *conn, LDAPURLDesc *srv ) } } if ( ld->ld_options.ldo_tm_net.tv_sec >= 0 ) { - ber_sockbuf_ctrl( sb, LBER_SB_OPT_SET_NONBLOCK, NULL ); + if ( ! async ) { + ber_sockbuf_ctrl( sb, LBER_SB_OPT_SET_NONBLOCK, NULL ); + } } #endif /* LDAP_USE_NON_BLOCKING_TLS */ Thanks, Vern

1 0

(ITS#8980) Async mode and TLS Non-Blocking Issue
by vsmith＠interlinknetworks.com 19 Feb '19

19 Feb '19

Full_Name: Vernon Smith Version: 2.4.47 OS: Linux, Solaris URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (2601:40d:4300:679a:d18c:3060:e826:d35c) I am using libldap library built with -DLDAP_USE_NON_BLOCKING_TLS and configured for Async connection mode. I test making connections using the library to servers that are hung to verify that my application will not hang in those cases. I have found 3 issues. The first is that the ldap_pvt_connect() clears non-blocking socket setup after the connection is made even when Async mode was configured. So here is my patch for that. diff --git a/libraries/libldap/os-ip.c b/libraries/libldap/os-ip.c index a823cc6..d7927e5 100644 --- a/libraries/libldap/os-ip.c +++ b/libraries/libldap/os-ip.c @@ -443,7 +443,7 @@ ldap_pvt_connect(LDAP *ld, ber_socket_t s, if ( connect(s, sin, addrlen) != AC_SOCKET_ERROR ) { osip_debug(ld, "connect success\n", 0, 0, 0); - if ( opt_tv && ldap_pvt_ndelay_off(ld, s) == -1 ) + if ( !async && opt_tv && ldap_pvt_ndelay_off(ld, s) == -1 ) return ( -1 ); return ( 0 ); } The second issue is that the tlso_session_connect() routine does not correctly handle the return code from SSL_connect(), it just returns it to the caller. For LDAP_USE_NON_BLOCKING_TLS, the return code must be checked and an appropriate return used. Here is my patch. diff --git a/libraries/libldap/tls_o.c b/libraries/libldap/tls_o.c index e95a448..7a31b5e 100644 --- a/libraries/libldap/tls_o.c +++ b/libraries/libldap/tls_o.c @@ -531,7 +531,23 @@ tlso_session_connect( LDAP *ld, tls_session *sess ) tlso_session *s = (tlso_session *)sess; /* Caller expects 0 = success, OpenSSL returns 1 = success */ - return SSL_connect( s ) - 1; + int rc = SSL_connect( s ) - 1; + +#ifdef LDAP_USE_NON_BLOCKING_TLS + int sslerr = SSL_get_error(s, rc+1); + int sockerr = sock_errno(); + + if ( rc < 0 ) { + if ( sslerr == SSL_ERROR_WANT_READ || sslerr == SSL_ERROR_WANT_WRITE ) { + rc = 0; + } else if (( sslerr == SSL_ERROR_SYSCALL ) && + ( sockerr == EAGAIN || sockerr == ENOTCONN )) { + rc = 0; + } + } +#endif /* LDAP_USE_NON_BLOCKING_TLS */ + + return rc; } static int The third issue is that ldap_int_tls_start() compiled with -DLDAP_USE_NON_BLOCKING_TLS Plays with the socket non-blocking setting even if Async mode is configured. Here is mt patch to only play with the socket setting if not in Async mode. diff --git a/libraries/libldap/tls2.c b/libraries/libldap/tls2.c index d9b2d27..69b749c 100644 --- a/libraries/libldap/tls2.c +++ b/libraries/libldap/tls2.c @@ -1075,8 +1075,11 @@ ldap_int_tls_start ( LDAP *ld, LDAPConn *conn, LDAPURLDesc *srv ) /* * Use non-blocking io during SSL Handshake when a timeout is configured */ + int async = LDAP_BOOL_GET( &ld->ld_options, LDAP_BOOL_CONNECT_ASYNC ); if ( ld->ld_options.ldo_tm_net.tv_sec >= 0 ) { - ber_sockbuf_ctrl( sb, LBER_SB_OPT_SET_NONBLOCK, (void*)1 ); + if ( ! async ) { + ber_sockbuf_ctrl( sb, LBER_SB_OPT_SET_NONBLOCK, (void*)1 ); + } ber_sockbuf_ctrl( sb, LBER_SB_OPT_GET_FD, &sd ); tv = ld->ld_options.ldo_tm_net; tv0 = tv; @@ -1110,8 +1113,10 @@ ldap_int_tls_start ( LDAP *ld, LDAPConn *conn, LDAPURLDesc *srv ) ld->ld_errno = LDAP_TIMEOUT; break; } else { - /* ldap_int_poll called ldap_pvt_ndelay_off */ - ber_sockbuf_ctrl( sb, LBER_SB_OPT_SET_NONBLOCK, (void*)1 ); + /* ldap_int_poll called ldap_pvt_ndelay_off if not in async mode */ + if ( ! async ) { + ber_sockbuf_ctrl( sb, LBER_SB_OPT_SET_NONBLOCK, (void*)1 ); + } ret = ldap_int_tls_connect( ld, conn, host ); if ( ret > 0 ) { /* need to call tls_connect once more */ struct timeval curr_time_tv, delta_tv; @@ -1159,7 +1164,9 @@ ldap_int_tls_start ( LDAP *ld, LDAPConn *conn, LDAPURLDesc *srv ) } } if ( ld->ld_options.ldo_tm_net.tv_sec >= 0 ) { - ber_sockbuf_ctrl( sb, LBER_SB_OPT_SET_NONBLOCK, NULL ); + if ( ! async ) { + ber_sockbuf_ctrl( sb, LBER_SB_OPT_SET_NONBLOCK, NULL ); + } } #endif /* LDAP_USE_NON_BLOCKING_TLS */ Thanks, Vern

1 0

Re: (ITS#8979) The -l <timelimit> or -o nettimeout=<timelimit> don't limit DNS lookup time
by hyc＠symas.com 18 Feb '19

18 Feb '19

=C3=86var Arnfj=C3=B6r=C3=B0 Bjarmason wrote: > On Mon, Feb 18, 2019 at 5:18 PM Howard Chu <hyc(a)symas.com> wrote: >> >> avarab(a)gmail.com wrote: >>> Full_Name: .var Arnfj.r. Bjarmason >>> Version: 2.4.44-21 >>> OS: CentOS 7.6 >>> URL: ftp://ftp.openldap.org/incoming/ >>> Submission from: (NULL) (5.57.21.154) >>> >>> >>> On a setup where you have a blackholed DNS server: >>> >>> $ grep ^name /etc/resolv.conf >>> nameserver 1.2.3.4 >>> >>> Running e.g.: >>> >>> $ time ldapsearch -l 2 -o nettimeout=3D1 [...] >>> >>> Will (on my system) eventually return: >>> >>> ldap_sasl_bind(SIMPLE): Can't contact LDAP server (-1) >>> >>> real 0m24.039s >>> user 0m0.004s >>> sys 0m0.007s >>> >>> It'll take around 48 seconds if I have two DNS servers. Running strac= e(1) on it >>> reveals that it's sitting in a socket/connect/poll loop trying to loo= kup the >>> hostname of the LDAP server I'm trying to talk to. >>> >>> Instead one of these options should limit time spent on DNS lookups, = or there >>> should be another option, so that you can run ldapsearch with a combi= nation of >>> these options and be sure that it'll run in at most the <timeout> you= give it. >> >> The current code in libldap uses gethostbyname() and this API doesn't = offer >> any parameters for setting a timeout. You can configure a timeout in y= our >> system's /etc/resolv.conf or using RES_OPTIONS environment variable. R= ead your >> system's resolver(5) manpage. >> >> If you know of any widely available resolver API that allows specifyin= g a timeout >> on individual queries, you're welcome to submit a patch supporting thi= s feature. >> The standard -lresolv doesn't provide such an interface. >=20 > That resolv.conf had "options timeout:2". That it didn't kick in is > probably a bug somewhere else, but it would still be handy if > ldapsearch had this itself, since the timeout you use for general > resolutions might not be what you want for ldapsearch. If all you need is a timeout in the ldapsearch command, wrap it in a scri= pt that sets the RES_OPTIONS environment variable. > But implementing that is a mess. This SO post has a summary: > https://stackoverflow.com/questions/24403435/socket-hostname-lookup-tim= eout-how-to-implement-it >=20 > So e.g. there could be a probe for getaddrinfo_a() on GNU systems. >=20 >> Closing this ITS. >=20 > Fair enough, but is that a "we don't want this ever" or "if someone > submitted a patch for (possibly OS-specific) DNS timeout we'd be > interested". Generally not fond of features that aren't available cross-platform, part= icularly if that means it can only be tested on a particular OS. I guess for GNU e= xtensions that are easily detected and widely supported it may be OK. We already use getaddrinfo() if it's available, so adding getaddrinfo_a()= may not be too messy. --=20 -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8979) The -l <timelimit> or -o nettimeout=<timelimit> don't limit DNS lookup time
by quanah＠symas.com 18 Feb '19

18 Feb '19

--On Monday, February 18, 2019 4:32 PM +0000 avarab(a)gmail.com wrote: >> Closing this ITS. > > Fair enough, but is that a "we don't want this ever" or "if someone > submitted a patch for (possibly OS-specific) DNS timeout we'd be > interested". This was already answered in Howard's reply: > If you know of any widely available resolver API that allows specifying a timeout > on individual queries, you're welcome to submit a patch supporting this feature. > The standard -lresolv doesn't provide such an interface. I.e., the ITS can always be reopened if a working resolver API can be found and a patch is supplied. --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

Re: (ITS#8979) The -l <timelimit> or -o nettimeout=<timelimit> don't limit DNS lookup time
by avarab＠gmail.com 18 Feb '19

18 Feb '19

On Mon, Feb 18, 2019 at 5:18 PM Howard Chu <hyc(a)symas.com> wrote: > > avarab(a)gmail.com wrote: > > Full_Name: .var Arnfj.r. Bjarmason > > Version: 2.4.44-21 > > OS: CentOS 7.6 > > URL: ftp://ftp.openldap.org/incoming/ > > Submission from: (NULL) (5.57.21.154) > > > > > > On a setup where you have a blackholed DNS server: > > > > $ grep ^name /etc/resolv.conf > > nameserver 1.2.3.4 > > > > Running e.g.: > > > > $ time ldapsearch -l 2 -o nettimeout=1 [...] > > > > Will (on my system) eventually return: > > > > ldap_sasl_bind(SIMPLE): Can't contact LDAP server (-1) > > > > real 0m24.039s > > user 0m0.004s > > sys 0m0.007s > > > > It'll take around 48 seconds if I have two DNS servers. Running strace(1) on it > > reveals that it's sitting in a socket/connect/poll loop trying to lookup the > > hostname of the LDAP server I'm trying to talk to. > > > > Instead one of these options should limit time spent on DNS lookups, or there > > should be another option, so that you can run ldapsearch with a combination of > > these options and be sure that it'll run in at most the <timeout> you give it. > > The current code in libldap uses gethostbyname() and this API doesn't offer > any parameters for setting a timeout. You can configure a timeout in your > system's /etc/resolv.conf or using RES_OPTIONS environment variable. Read your > system's resolver(5) manpage. > > If you know of any widely available resolver API that allows specifying a timeout > on individual queries, you're welcome to submit a patch supporting this feature. > The standard -lresolv doesn't provide such an interface. That resolv.conf had "options timeout:2". That it didn't kick in is probably a bug somewhere else, but it would still be handy if ldapsearch had this itself, since the timeout you use for general resolutions might not be what you want for ldapsearch. But implementing that is a mess. This SO post has a summary: https://stackoverflow.com/questions/24403435/socket-hostname-lookup-timeout… So e.g. there could be a probe for getaddrinfo_a() on GNU systems. > Closing this ITS. Fair enough, but is that a "we don't want this ever" or "if someone submitted a patch for (possibly OS-specific) DNS timeout we'd be interested". > > As a workaround I'm using ldapsearch with /usr/bin/timeout, but since it kills > > it if it exceeds the timeout I don't get a meaningful error. > > > -- > -- Howard Chu > CTO, Symas Corp. http://www.symas.com > Director, Highland Sun http://highlandsun.com/hyc/ > Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8979) The -l <timelimit> or -o nettimeout=<timelimit> don't limit DNS lookup time
by hyc＠symas.com 18 Feb '19

18 Feb '19

avarab(a)gmail.com wrote: > Full_Name: .var Arnfj.r. Bjarmason > Version: 2.4.44-21 > OS: CentOS 7.6 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (5.57.21.154) > > > On a setup where you have a blackholed DNS server: > > $ grep ^name /etc/resolv.conf > nameserver 1.2.3.4 > > Running e.g.: > > $ time ldapsearch -l 2 -o nettimeout=1 [...] > > Will (on my system) eventually return: > > ldap_sasl_bind(SIMPLE): Can't contact LDAP server (-1) > > real 0m24.039s > user 0m0.004s > sys 0m0.007s > > It'll take around 48 seconds if I have two DNS servers. Running strace(1) on it > reveals that it's sitting in a socket/connect/poll loop trying to lookup the > hostname of the LDAP server I'm trying to talk to. > > Instead one of these options should limit time spent on DNS lookups, or there > should be another option, so that you can run ldapsearch with a combination of > these options and be sure that it'll run in at most the <timeout> you give it. The current code in libldap uses gethostbyname() and this API doesn't offer any parameters for setting a timeout. You can configure a timeout in your system's /etc/resolv.conf or using RES_OPTIONS environment variable. Read your system's resolver(5) manpage. If you know of any widely available resolver API that allows specifying a timeout on individual queries, you're welcome to submit a patch supporting this feature. The standard -lresolv doesn't provide such an interface. Closing this ITS. > As a workaround I'm using ldapsearch with /usr/bin/timeout, but since it kills > it if it exceeds the timeout I don't get a meaningful error. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#8979) The -l <timelimit> or -o nettimeout=<timelimit> don't limit DNS lookup time
by avarab＠gmail.com 18 Feb '19

18 Feb '19

Full_Name: .var Arnfj.r. Bjarmason Version: 2.4.44-21 OS: CentOS 7.6 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (5.57.21.154) On a setup where you have a blackholed DNS server: $ grep ^name /etc/resolv.conf nameserver 1.2.3.4 Running e.g.: $ time ldapsearch -l 2 -o nettimeout=1 [...] Will (on my system) eventually return: ldap_sasl_bind(SIMPLE): Can't contact LDAP server (-1) real 0m24.039s user 0m0.004s sys 0m0.007s It'll take around 48 seconds if I have two DNS servers. Running strace(1) on it reveals that it's sitting in a socket/connect/poll loop trying to lookup the hostname of the LDAP server I'm trying to talk to. Instead one of these options should limit time spent on DNS lookups, or there should be another option, so that you can run ldapsearch with a combination of these options and be sure that it'll run in at most the <timeout> you give it. As a workaround I'm using ldapsearch with /usr/bin/timeout, but since it kills it if it exceeds the timeout I don't get a meaningful error.

1 0

Re: (ITS#8978) Fix mdb_env_open2() failing when getting handle for NTDLL.dll
by hyc＠symas.com 17 Feb '19

17 Feb '19

khng300(a)gmail.com wrote: > Full_Name: Ka Ho Ng > Version: mdb.master > OS: Windows 10 > URL: ftp://ftp.openldap.org/incoming/Ka-Ho-Ng-190218.patch > Submission from: (NULL) (2001:470:fa95:1300::1) Thanks for the patch, committed to mdb.master > > Fix mdb_env_open2() failing when getting handle for NTDLL.dll > > Always call GetModuleHandleW() with Unicode string, as mdb_fopen() is > calling CreateFileW() already. Otherwise GetModuleHandle() will fail with > non-Unicode string. > > I, Ka Ho Ng, hereby place the following modifications to OpenLDAP Software (and > only these modifications) into the public domain. Hence, these modifications may > be freely used and/or redistributed for any purpose with or without attribution > and/or other notice. > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#8978) Fix mdb_env_open2() failing when getting handle for NTDLL.dll
by khng300＠gmail.com 17 Feb '19

17 Feb '19

Full_Name: Ka Ho Ng Version: mdb.master OS: Windows 10 URL: ftp://ftp.openldap.org/incoming/Ka-Ho-Ng-190218.patch Submission from: (NULL) (2001:470:fa95:1300::1) Fix mdb_env_open2() failing when getting handle for NTDLL.dll Always call GetModuleHandleW() with Unicode string, as mdb_fopen() is calling CreateFileW() already. Otherwise GetModuleHandle() will fail with non-Unicode string. I, Ka Ho Ng, hereby place the following modifications to OpenLDAP Software (and only these modifications) into the public domain. Hence, these modifications may be freely used and/or redistributed for any purpose with or without attribution and/or other notice.

1 0

Re: (ITS#8976) LMDB > use posix_madvise(3) instead of madvise(3) for SmartOS
by matthieu.guegan＠smile-suisse.com 15 Feb '19

15 Feb '19

On Fri, Feb 15, 2019 at 5:16 PM Howard Chu <hyc(a)symas.com> wrote: > > matthieu.guegan(a)smile-suisse.com wrote: > > On Fri, Feb 15, 2019 at 4:33 PM Howard Chu <hyc(a)symas.com> wrote: > >> > >> Matthieu GUEGAN wrote: > >>> On Fri, Feb 15, 2019 at 3:56 PM Howard Chu <hyc(a)symas.com> wrote: > >>>> > >>>> mguegan(a)virtua.ch wrote: > >>>>> Full_Name: Matthieu Guegan > >>>>> Version: current > >>>>> OS: SmartOS > >>>>> URL: > >>>>> Submission from: (NULL) (109.190.148.77) > >>>>> > >>>>> > >>>>> Hi, > >>>>> > >>>>> In order to compile knot[1] on SmartOS, I have done a series of little patches. > >>>>> One of them is linked to the LMDB project, which redirects me here. > >>>>> > >>>>> The idea is to tell SmartOS (and I think the SunOS family) to make use of > >>>>> posix_madvise(3), instead of madvise(3) as implementation on this OS[2] is > >>>>> different than, for example, the Linux[3] one. > >>>>> > >>>>> So, I kindly ask you to take a look at the merge request of the knot project[4], > >>>>> in particular the `src/contrib/lmdb/mdb.c` patch (that's just a ifdef > >>>>> modification) : would it be possible to apply this patch directly in this > >>>>> repository ? > >>>> > >>>> I see no functional difference between your references [2] and [3]. > >>>> What is the benefit of this patch? > >>> > >>> Well, the Solaris/SunOS's madvise(3) is defined with an old `caddr_t` > >>> type whereas Linux (and *BSD OSes too) use a more modern approach with > >>> `void *`. > >>> On the `knot` project, which embed the lmdb part of openldap, > >>> compilation failed when using madvise(3) on SmartOS, so I did a > >>> replacement with posix_madvise(3) which uses the `void *` argument > >>> too. > >>> I don't know what's the best way to handle this case, that's why I ask here. > >> > >> A caddr_t is just a typedef of "char *". It should always be legal to > >> pass a void * to such a parameter. What is your compile error message? > >> > > > > ``` > > contrib/lmdb/mdb.c: In function 'mdb_env_map': > > contrib/lmdb/mdb.c:3998:3: error: implicit declaration of function > > 'madvise'; did you mean 'raise'? > > [-Werror=implicit-function-declaration] > > madvise(env->me_map, env->me_mapsize, MADV_RANDOM); > > ^~~~~~~ > > raise > > This has absolutely nothing to do with whether the parameter is a caddr_t or a void *. > Why would you even mention something totally irrelevant to the actual error? Sorry if I mislead you with that, that was not my intention. I was just trying to answer you when asking the difference using posix_madvise() vs madvise() on SmartOS. And that was the only thing I found. Indeed, it seems completely irrelevant to the error I get. > Strange that you have MADV_RANDOM defined but no declaration of madvise(). Indeed, searching in the header file `sys/mman.h` [5], it seems to be possible to have a condition that enable madvise() and not MADV_RANDOM or vice-versa. > Sounds like your system's header files are playing stupid games with definitions. > If you can find out what flags must be used to make the header files properly > declare the madvise function, a patch for that may be acceptable. I should say that I'm using the pkgsrc framework in order to compile the whole thing and found multiple packages on which was applied the same type of patch (excluding __sun when trying to use madvise(), thus using the posix_madvise variant). Flags can be overridden here, so it must be something related with the pkgsrc's Makefile variables. Anyway, maybe this should be done this way, instead of trying to change things upstream. Thanks for your help. [5] https://github.com/illumos/illumos-gate/blob/4e0c5eff9af325c80994e9527b7cb8…

1 0

Re: (ITS#8976) LMDB > use posix_madvise(3) instead of madvise(3) for SmartOS
by hyc＠symas.com 15 Feb '19

15 Feb '19

matthieu.guegan(a)smile-suisse.com wrote: > On Fri, Feb 15, 2019 at 4:33 PM Howard Chu <hyc(a)symas.com> wrote: >> >> Matthieu GUEGAN wrote: >>> On Fri, Feb 15, 2019 at 3:56 PM Howard Chu <hyc(a)symas.com> wrote: >>>> >>>> mguegan(a)virtua.ch wrote: >>>>> Full_Name: Matthieu Guegan >>>>> Version: current >>>>> OS: SmartOS >>>>> URL: >>>>> Submission from: (NULL) (109.190.148.77) >>>>> >>>>> >>>>> Hi, >>>>> >>>>> In order to compile knot[1] on SmartOS, I have done a series of little patches. >>>>> One of them is linked to the LMDB project, which redirects me here. >>>>> >>>>> The idea is to tell SmartOS (and I think the SunOS family) to make use of >>>>> posix_madvise(3), instead of madvise(3) as implementation on this OS[2] is >>>>> different than, for example, the Linux[3] one. >>>>> >>>>> So, I kindly ask you to take a look at the merge request of the knot project[4], >>>>> in particular the `src/contrib/lmdb/mdb.c` patch (that's just a ifdef >>>>> modification) : would it be possible to apply this patch directly in this >>>>> repository ? >>>> >>>> I see no functional difference between your references [2] and [3]. >>>> What is the benefit of this patch? >>> >>> Well, the Solaris/SunOS's madvise(3) is defined with an old `caddr_t` >>> type whereas Linux (and *BSD OSes too) use a more modern approach with >>> `void *`. >>> On the `knot` project, which embed the lmdb part of openldap, >>> compilation failed when using madvise(3) on SmartOS, so I did a >>> replacement with posix_madvise(3) which uses the `void *` argument >>> too. >>> I don't know what's the best way to handle this case, that's why I ask here. >> >> A caddr_t is just a typedef of "char *". It should always be legal to >> pass a void * to such a parameter. What is your compile error message? >> > > ``` > contrib/lmdb/mdb.c: In function 'mdb_env_map': > contrib/lmdb/mdb.c:3998:3: error: implicit declaration of function > 'madvise'; did you mean 'raise'? > [-Werror=implicit-function-declaration] > madvise(env->me_map, env->me_mapsize, MADV_RANDOM); > ^~~~~~~ > raise This has absolutely nothing to do with whether the parameter is a caddr_t or a void *. Why would you even mention something totally irrelevant to the actual error? Strange that you have MADV_RANDOM defined but no declaration of madvise(). Sounds like your system's header files are playing stupid games with definitions. If you can find out what flags must be used to make the header files properly declare the madvise function, a patch for that may be acceptable. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8976) LMDB > use posix_madvise(3) instead of madvise(3) for SmartOS
by matthieu.guegan＠smile-suisse.com 15 Feb '19

15 Feb '19

On Fri, Feb 15, 2019 at 4:33 PM Howard Chu <hyc(a)symas.com> wrote: > > Matthieu GUEGAN wrote: > > On Fri, Feb 15, 2019 at 3:56 PM Howard Chu <hyc(a)symas.com> wrote: > >> > >> mguegan(a)virtua.ch wrote: > >>> Full_Name: Matthieu Guegan > >>> Version: current > >>> OS: SmartOS > >>> URL: > >>> Submission from: (NULL) (109.190.148.77) > >>> > >>> > >>> Hi, > >>> > >>> In order to compile knot[1] on SmartOS, I have done a series of little patches. > >>> One of them is linked to the LMDB project, which redirects me here. > >>> > >>> The idea is to tell SmartOS (and I think the SunOS family) to make use of > >>> posix_madvise(3), instead of madvise(3) as implementation on this OS[2] is > >>> different than, for example, the Linux[3] one. > >>> > >>> So, I kindly ask you to take a look at the merge request of the knot project[4], > >>> in particular the `src/contrib/lmdb/mdb.c` patch (that's just a ifdef > >>> modification) : would it be possible to apply this patch directly in this > >>> repository ? > >> > >> I see no functional difference between your references [2] and [3]. > >> What is the benefit of this patch? > > > > Well, the Solaris/SunOS's madvise(3) is defined with an old `caddr_t` > > type whereas Linux (and *BSD OSes too) use a more modern approach with > > `void *`. > > On the `knot` project, which embed the lmdb part of openldap, > > compilation failed when using madvise(3) on SmartOS, so I did a > > replacement with posix_madvise(3) which uses the `void *` argument > > too. > > I don't know what's the best way to handle this case, that's why I ask here. > > A caddr_t is just a typedef of "char *". It should always be legal to > pass a void * to such a parameter. What is your compile error message? > ``` contrib/lmdb/mdb.c: In function 'mdb_env_map': contrib/lmdb/mdb.c:3998:3: error: implicit declaration of function 'madvise'; did you mean 'raise'? [-Werror=implicit-function-declaration] madvise(env->me_map, env->me_mapsize, MADV_RANDOM); ^~~~~~~ raise ```

1 0

Re: (ITS#8976) LMDB > use posix_madvise(3) instead of madvise(3) for SmartOS
by hyc＠symas.com 15 Feb '19

15 Feb '19

Matthieu GUEGAN wrote: > On Fri, Feb 15, 2019 at 3:56 PM Howard Chu <hyc(a)symas.com> wrote: >> >> mguegan(a)virtua.ch wrote: >>> Full_Name: Matthieu Guegan >>> Version: current >>> OS: SmartOS >>> URL: >>> Submission from: (NULL) (109.190.148.77) >>> >>> >>> Hi, >>> >>> In order to compile knot[1] on SmartOS, I have done a series of little patches. >>> One of them is linked to the LMDB project, which redirects me here. >>> >>> The idea is to tell SmartOS (and I think the SunOS family) to make use of >>> posix_madvise(3), instead of madvise(3) as implementation on this OS[2] is >>> different than, for example, the Linux[3] one. >>> >>> So, I kindly ask you to take a look at the merge request of the knot project[4], >>> in particular the `src/contrib/lmdb/mdb.c` patch (that's just a ifdef >>> modification) : would it be possible to apply this patch directly in this >>> repository ? >> >> I see no functional difference between your references [2] and [3]. >> What is the benefit of this patch? > > Well, the Solaris/SunOS's madvise(3) is defined with an old `caddr_t` > type whereas Linux (and *BSD OSes too) use a more modern approach with > `void *`. > On the `knot` project, which embed the lmdb part of openldap, > compilation failed when using madvise(3) on SmartOS, so I did a > replacement with posix_madvise(3) which uses the `void *` argument > too. > I don't know what's the best way to handle this case, that's why I ask here. A caddr_t is just a typedef of "char *". It should always be legal to pass a void * to such a parameter. What is your compile error message? -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8976) LMDB > use posix_madvise(3) instead of madvise(3) for SmartOS
by matthieu.guegan＠smile-suisse.com 15 Feb '19

15 Feb '19

On Fri, Feb 15, 2019 at 3:56 PM Howard Chu <hyc(a)symas.com> wrote: > > mguegan(a)virtua.ch wrote: > > Full_Name: Matthieu Guegan > > Version: current > > OS: SmartOS > > URL: > > Submission from: (NULL) (109.190.148.77) > > > > > > Hi, > > > > In order to compile knot[1] on SmartOS, I have done a series of little patches. > > One of them is linked to the LMDB project, which redirects me here. > > > > The idea is to tell SmartOS (and I think the SunOS family) to make use of > > posix_madvise(3), instead of madvise(3) as implementation on this OS[2] is > > different than, for example, the Linux[3] one. > > > > So, I kindly ask you to take a look at the merge request of the knot project[4], > > in particular the `src/contrib/lmdb/mdb.c` patch (that's just a ifdef > > modification) : would it be possible to apply this patch directly in this > > repository ? > > I see no functional difference between your references [2] and [3]. > What is the benefit of this patch? Well, the Solaris/SunOS's madvise(3) is defined with an old `caddr_t` type whereas Linux (and *BSD OSes too) use a more modern approach with `void *`. On the `knot` project, which embed the lmdb part of openldap, compilation failed when using madvise(3) on SmartOS, so I did a replacement with posix_madvise(3) which uses the `void *` argument too. I don't know what's the best way to handle this case, that's why I ask here. > > > > Thanks > > > > > > [1] https://www.knot-dns.cz/ > > [2] https://docs.oracle.com/cd/E36784_01/html/E36874/madvise-3c.html > > [3] http://man7.org/linux/man-pages/man2/madvise.2.html > > [4] https://gitlab.labs.nic.cz/knot/knot-dns/merge_requests/979/diffs?commit_id… > > > > > > > -- > -- Howard Chu > CTO, Symas Corp. http://www.symas.com > Director, Highland Sun http://highlandsun.com/hyc/ > Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8976) LMDB > use posix_madvise(3) instead of madvise(3) for SmartOS
by hyc＠symas.com 15 Feb '19

15 Feb '19

mguegan(a)virtua.ch wrote: > Full_Name: Matthieu Guegan > Version: current > OS: SmartOS > URL: > Submission from: (NULL) (109.190.148.77) > > > Hi, > > In order to compile knot[1] on SmartOS, I have done a series of little patches. > One of them is linked to the LMDB project, which redirects me here. > > The idea is to tell SmartOS (and I think the SunOS family) to make use of > posix_madvise(3), instead of madvise(3) as implementation on this OS[2] is > different than, for example, the Linux[3] one. > > So, I kindly ask you to take a look at the merge request of the knot project[4], > in particular the `src/contrib/lmdb/mdb.c` patch (that's just a ifdef > modification) : would it be possible to apply this patch directly in this > repository ? I see no functional difference between your references [2] and [3]. What is the benefit of this patch? > > Thanks > > > [1] https://www.knot-dns.cz/ > [2] https://docs.oracle.com/cd/E36784_01/html/E36874/madvise-3c.html > [3] http://man7.org/linux/man-pages/man2/madvise.2.html > [4] https://gitlab.labs.nic.cz/knot/knot-dns/merge_requests/979/diffs?commit_id… > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#8977) Make IDL sizes configurable in back-mdb
by hyc＠openldap.org 15 Feb '19

15 Feb '19

Full_Name: Howard Chu Version: 2.5 OS: URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (80.233.42.124) Submitted by: hyc IDLs used for index slots are hardcoded to a max of 65536 elements in the DB and twice that in memory. Sites with larger DBs tend to need larger IDLs; if they're running with sufficient memory it ought to be possible to configure larger IDLs.

1 0

(ITS#8976) LMDB > use posix_madvise(3) instead of madvise(3) for SmartOS
by mguegan＠virtua.ch 15 Feb '19

15 Feb '19

Full_Name: Matthieu Guegan Version: current OS: SmartOS URL: Submission from: (NULL) (109.190.148.77) Hi, In order to compile knot[1] on SmartOS, I have done a series of little patches. One of them is linked to the LMDB project, which redirects me here. The idea is to tell SmartOS (and I think the SunOS family) to make use of posix_madvise(3), instead of madvise(3) as implementation on this OS[2] is different than, for example, the Linux[3] one. So, I kindly ask you to take a look at the merge request of the knot project[4], in particular the `src/contrib/lmdb/mdb.c` patch (that's just a ifdef modification) : would it be possible to apply this patch directly in this repository ? Thanks [1] https://www.knot-dns.cz/ [2] https://docs.oracle.com/cd/E36784_01/html/E36874/madvise-3c.html [3] http://man7.org/linux/man-pages/man2/madvise.2.html [4] https://gitlab.labs.nic.cz/knot/knot-dns/merge_requests/979/diffs?commit_id…

1 0

Re: (ITS#8841) Fix for an assertion error in back-meta
by nivanova＠symas.com 14 Feb '19

14 Feb '19

A new patch with added comments has been added at ftp://ftp.openldap.org/incoming/nadezhda-ivanova-190214.patch More details about this issue: The customer reported that when they simulated a "network unreachable" error, SLAPD exited with the following assertion error: #0 0x00007fa1d19e4885 in raise () from /lib64/libc.so.6 #1 0x00007fa1d19e5e61 in abort () from /lib64/libc.so.6 #2 0x00007fa1d19dd740 in __assert_fail () from /lib64/libc.so.6 #3 0x0000000000449c17 in slap_send_ldap_result (op=0x1870000, rs=0x7fa17c742930) at result.c:830 #4 0x00000000004b6a53 in meta_back_search (op=0x1870000, rs=0x7fa17c742930) at search.c:1183 #5 0x000000000043aef1 in fe_op_search (op=0x1870000, rs=0x7fa17c742930) at search.c:402 #6 0x00000000004a1a62 in overlay_op_walk (op=0x1870000, rs=0x7fa17c742930, which=op_search, oi=0x88583c0, on=0x0) at backover.c:677 #7 0x00000000004a24dc in over_op_func (op=0x1870000, rs=0x8, which=op_add) at backover.c:730 #8 0x000000000043b73c in do_search (op=0x1870000, rs=0x7fa17c742930) at search.c:247 #9 0x00000000004382af in connection_operation (ctx=0x7fa17c742b30, arg_v=<optimized out>) at connection.c:1316 #10 0x0000000000438edb in connection_read_thread (ctx=0x7fa17c742b30, argv=<optimized out>) at connection.c:1452 #11 0x00000000004f06a0 in ldap_int_thread_pool_wrapper (xpool=<optimized out>) at tpool.c:696 #12 0x00007fa1d1d35806 in start_thread () from /lib64/libpthread.so.0 #13 0x00007fa1d1a9065d in clone () from /lib64/libc.so.6 #14 0x0000000000000000 in ?? () This happened because meta used to directly return the ldap_sasl_bind return value to the client, and in this case it is not an LDAP error. We fixed the issue by mapping it to an LDAP error. Nadezhda Ivanova Software Engineer Symas Corporation http://www.symas.com

1 0

Re: (ITS#8847) New LDAP URL syntax to support binding to specific IP address at client side
by hyc＠symas.com 13 Feb '19

13 Feb '19

Sharma, Ramakant 2. (Nokia - IN/Bangalore) wrote: > Hi Howard, > > Please provide your valuable inputs for the below patch. It is a little strange that ldap_set_option() takes a string but ldap_get_option() returns a binary structure. get/set should be totally symmetric. It would be simplest to just save a copy of the string given to set and return the string in get. In libldap/os-ip.c, you should not be using ldap_get_option() to retrieve the values. You're operating inside libldap, you can access the structure directly. In particular, it's a waste to use ldap_get_option here and create a dynamically allocated copy of the address struct. Look at how the existing code works. Notice that it just uses ld->ld_options directly. This is rule #1 in software development - when extending existing code, your new code should do things the same way existing code does. Corollaries to this rule are: don't patch code without reading it first, and don't patch code without studying its complete context. > > Also let us know how to proceed further. > > BR, > Ramakant Sharma > > -----Original Message----- > From: Sharma, Ramakant 2. (Nokia - IN/Bangalore) > Sent: Monday, January 28, 2019 10:10 AM > To: Howard Chu <hyc(a)symas.com>; openldap-its(a)OpenLDAP.org > Cc: Singam, Sudhir (Nokia - IN/Bangalore) <sudhir.singam(a)nokia.com> > Subject: RE: (ITS#8847) New LDAP URL syntax to support binding to specific IP address at client side > > Hi Howard, > > I have uploaded the new patch which addresses the previous comments. > > Kindly check and provide your valuable comments . > > ftp://ftp.openldap.org/incoming/ramakant-sharma-190121.patch > > BR, > Ramakant Sharma > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8975) Calling mdb_env_set_mapsize on a shared DB fails sometimes
by hyc＠symas.com 10 Feb '19

10 Feb '19

kriszyp(a)gmail.com wrote: > Full_Name: Kristopher William Zyp > Version: LMDB 0.9.23 > OS: Windows Server 2012 R2, Windows 10 > URL: https://github.com/kriszyp/node-lmdb/commit/6df903907f5516320e9a8afce45bd32… > Submission from: (NULL) (71.199.6.148) > Thanks for the report and patch, added to mdb.RE/0.9 > Calling mdb_env_set_mapsize to increase the map size, when a DB is also in use > by other processes, when MDB_WRITEMAP is enabled (and the db file has been > opened with PAGE_READWRITE access), on Windows, will occasionally (seems like > about 1/100 attempts fails) produce an error "The requested operation cannot be > performed on a file with a user-mapped section open", or segfaults. The error > occurs in the SetFilePointer (or SetEndOfFile) call in mdb_env_map that is > performed to increase the allocated file size to the map size, prior to > CreateFileMapping. > > As it turns out this is pretty easy to solve, because manually expanding the > file size is not necessary when calling CreateFileMapping with PAGE_READWRITE > access, as Windows will automatically expand the file size for us, when opened > with the page write access enabled. Of course, this means all processes must be > consistent in use of MDB_WRITEMAP, but the documentation already makes this > explicit and clear. > > I believe this can be fixed by simply adding a check for MDB_WRITEMAP in the if > statement that calls SetFilePointer: > > if (!(flags & MDB_WRITEMAP) && (SetFilePointer(env->me_fd, sizelo, &sizehi, 0) > != (DWORD)sizelo > || !SetEndOfFile(env->me_fd) > || SetFilePointer(env->me_fd, 0, NULL, 0) != 0)) > return ErrCode(); > > The attached URL has the change as a patch/diff as applied to our node package. > > I am certainly happy to just keep this change on our own branches. There may be > nuances of this that I might not be aware of, but it seems to be working great > for us and I have tested this with MDB_WRITEMAP enabled and disabled. So I > thought I would offer/suggest this change, as it seems like it is > straightforward change to improve stability. Thank you! > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#8975) Calling mdb_env_set_mapsize on a shared DB fails sometimes
by kriszyp＠gmail.com 08 Feb '19

08 Feb '19

Full_Name: Kristopher William Zyp Version: LMDB 0.9.23 OS: Windows Server 2012 R2, Windows 10 URL: https://github.com/kriszyp/node-lmdb/commit/6df903907f5516320e9a8afce45bd32… Submission from: (NULL) (71.199.6.148) Calling mdb_env_set_mapsize to increase the map size, when a DB is also in use by other processes, when MDB_WRITEMAP is enabled (and the db file has been opened with PAGE_READWRITE access), on Windows, will occasionally (seems like about 1/100 attempts fails) produce an error "The requested operation cannot be performed on a file with a user-mapped section open", or segfaults. The error occurs in the SetFilePointer (or SetEndOfFile) call in mdb_env_map that is performed to increase the allocated file size to the map size, prior to CreateFileMapping. As it turns out this is pretty easy to solve, because manually expanding the file size is not necessary when calling CreateFileMapping with PAGE_READWRITE access, as Windows will automatically expand the file size for us, when opened with the page write access enabled. Of course, this means all processes must be consistent in use of MDB_WRITEMAP, but the documentation already makes this explicit and clear. I believe this can be fixed by simply adding a check for MDB_WRITEMAP in the if statement that calls SetFilePointer: if (!(flags & MDB_WRITEMAP) && (SetFilePointer(env->me_fd, sizelo, &sizehi, 0) != (DWORD)sizelo || !SetEndOfFile(env->me_fd) || SetFilePointer(env->me_fd, 0, NULL, 0) != 0)) return ErrCode(); The attached URL has the change as a patch/diff as applied to our node package. I am certainly happy to just keep this change on our own branches. There may be nuances of this that I might not be aware of, but it seems to be working great for us and I have tested this with MDB_WRITEMAP enabled and disabled. So I thought I would offer/suggest this change, as it seems like it is straightforward change to improve stability. Thank you!

1 0

RE: (ITS#8972) Performance/freezing issues using NTDLL onWindowsServer on Azure
by kriszyp＠gmail.com 08 Feb '19

08 Feb '19

--_FEED8D1F-4CE2-4DE8-A6B1-1703AFFCC4FC_ Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" To follow up to this, after switching to the stable release in node-ldmb, a= nd running on our servers for several days, I can confirm this definitely d= oes fix our issues (and ticket can be closed). I=E2=80=99ve submitted a PR = for getting the correct stable version in node-lmdb, to ensure in this does= n=E2=80=99t happen for other NodeJS users: https://github.com/Venemo/node-l= mdb/pull/141=20 I did find a minor issue with setting up the file mapping in the release ve= rsion, but I will file that separately. And if the NTDLL mapping isn=E2=80= =99t intended to be merged into a release branch, sounds like there shouldn= =E2=80=99t be any issues for Windows users that are using LMDB releases. And to be clear, I am a happy windows user, sorting out versions hardly den= ts my enthusiasm for LMDB. And thank you again for the help. Thanks, Kris --_FEED8D1F-4CE2-4DE8-A6B1-1703AFFCC4FC_ Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset="utf-8" <html xmlns:o=3D"urn:schemas-microsoft-com:office:office" xmlns:w=3D"urn:sc= hemas-microsoft-com:office:word" xmlns:m=3D"http://schemas.microsoft.com/of= fice/2004/12/omml" xmlns=3D"http://www.w3.org/TR/REC-html40"><head><meta ht= tp-equiv=3DContent-Type content=3D"text/html; charset=3Dutf-8"><meta name= =3DGenerator content=3D"Microsoft Word 15 (filtered medium)"><style></style></head><body lang=3DEN-CA link=3Dblue vlink=3D"#954F72"><div cla= ss=3DWordSection1>To follow up to this, after switchin= g to the stable release in node-ldmb, and running on our servers for severa= l days, I can confirm this definitely does fix our issues (and ticket can b= e closed). I=E2=80=99ve submitted a PR for getting the correct stable versi= on in node-lmdb, to ensure in this doesn=E2=80=99t happen for other NodeJS = users: <a href=3D"https://github.com/Venemo/node-lmdb/pull/141">https://git= hub.com/Venemo/node-lmdb/pull/141</a> <o:p> <= /o:p>I did find a minor issue with setting up the = file mapping in the release version, but I will file that separately. And i= f the NTDLL mapping isn=E2=80=99t intended to be merged into a release bran= ch, sounds like there shouldn=E2=80=99t be any issues for Windows users tha= t are using LMDB releases.<o:p> </o:p>And to be clear, I am a happy windows user, sorting out v= ersions hardly dents my enthusiasm for LMDB. And thank you again for the he= lp.<o:p></o:p><o:p> </o:p>Thanks, Kris<o:p></o:p></div></body></html>= --_FEED8D1F-4CE2-4DE8-A6B1-1703AFFCC4FC_--

1 0

Re: (ITS#8974) olcAttributeOptions cannot be modified once set
by hyc＠symas.com 07 Feb '19

07 Feb '19

skoranda(a)gmail.com wrote: > Full_Name: Scott Koranda > Version: 2.4.47 > OS: Debian Linux 9.6 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (104.231.255.140) > > > Once the option olcAttributeOptions has been configured it cannot be modified. New options can be added; existing options cannot be deleted. Feel free to submit a patch adding the required support. Note the FIXME comment in slapd/bconfig.c. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#8974) olcAttributeOptions cannot be modified once set
by skoranda＠gmail.com 07 Feb '19

07 Feb '19

Full_Name: Scott Koranda Version: 2.4.47 OS: Debian Linux 9.6 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (104.231.255.140) Once the option olcAttributeOptions has been configured it cannot be modified. To reproduce begin with the following state: #ldapsearch -LLL -Y EXTERNAL -H ldapi:/// -b cn=config -s base olcAttributeOptions 2>/dev/null dn: cn=config olcAttributeOptions: lang- app- Create an LDIF file to modify the value for olcAttributeOptions: # cat /root/olcAttributeOptions.ldif dn: cn=config changetype: modify replace: olcAttributeOptions olcAttributeOptions: lang- app- internal prior role- scope- time- Use ldapmodify to attempt to modify the configuration: # ldapmodify -Y EXTERNAL -H ldapi:/// -f /root/olcAttributeOptions.ldif SASL/EXTERNAL authentication started SASL username: gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth SASL SSF: 0 modifying entry "cn=config" ldap_modify: Other (e.g., implementation specific) error (80) The slapd process running with -d -1 emits this output: 5c5c2e01 slapd: line 0: option "lang-" is already defined 5c5c2e01 olcAttributeOptions: value #0: <olcAttributeOptions> handler exited with 1! This behavior happens regardless of whether any directory has records that include options with attributes.

1 0

Re: (ITS#8973) Error with domainScope control when using C# LDAP API
by clement.oudot＠worteks.com 06 Feb '19

06 Feb '19

Le 06/02/2019 à 12:57, hyc(a)symas.com a écrit : > hyc(a)symas.com wrote: >> clement.oudot(a)worteks.com wrote: >>> Full_Name: Clement OUDOT >>> Version: 2.4.47 >>> OS: GNU/Linux >>> URL: ftp://ftp.openldap.org/incoming/ >>> Submission from: (NULL) (81.250.130.213) >>> >>> >>> Hello, >>> >>> after upgrading from 2.4.46 to 2.4.47, this C# code does not work anymore : >> The code in controls.c is correct. The DomainScope control does not take a value, >> so the value is required to be NULL. > Microsoft spec https://msdn.microsoft.com/en-us/library/cc223348.aspx > > "When sending this control to the DC, the controlValue field of the Control structure is omitted." > Thanks a lot for your feedback, I will try to understand with the C# code is sending a value inside the control. -- Clément Oudot | Identity Solutions Manager clement.oudot(a)worteks.com Worteks | https://www.worteks.com

1 0

Re: (ITS#8973) Error with domainScope control when using C# LDAP API
by hyc＠symas.com 06 Feb '19

06 Feb '19

hyc(a)symas.com wrote: > clement.oudot(a)worteks.com wrote: >> Full_Name: Clement OUDOT >> Version: 2.4.47 >> OS: GNU/Linux >> URL: ftp://ftp.openldap.org/incoming/ >> Submission from: (NULL) (81.250.130.213) >> >> >> Hello, >> >> after upgrading from 2.4.46 to 2.4.47, this C# code does not work anymore : > > The code in controls.c is correct. The DomainScope control does not take a value, > so the value is required to be NULL. Microsoft spec https://msdn.microsoft.com/en-us/library/cc223348.aspx "When sending this control to the DC, the controlValue field of the Control structure is omitted." -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8973) Error with domainScope control when using C# LDAP API
by hyc＠symas.com 06 Feb '19

06 Feb '19

clement.oudot(a)worteks.com wrote: > Full_Name: Clement OUDOT > Version: 2.4.47 > OS: GNU/Linux > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (81.250.130.213) > > > Hello, > > after upgrading from 2.4.46 to 2.4.47, this C# code does not work anymore : The code in controls.c is correct. The DomainScope control does not take a value, so the value is required to be NULL. > ---- > $auth = [System.DirectoryServices.AuthenticationTypes]::FastBind > $de = New-Object System.DirectoryServices.DirectoryEntry($DN,$userReferentiel, > $passwordReferentiel, $auth) > ---- > > The log in OpenLDAP 2.4.47 : > ---- > Feb 5 18:21:03 anteros slapd[30762]: conn=1577 fd=17 ACCEPT from > IP=10.10.2.163:47304 (IP=0.0.0.0:390) > Feb 5 18:21:03 anteros slapd[30762]: conn=1577 op=0 BIND > dn="uid=ADPassword,ou=configuration,dc=referentiel,dc=local" method=128 > Feb 5 18:21:03 anteros slapd[30762]: conn=1577 op=0 BIND > dn="uid=ADPassword,ou=configuration,dc=referentiel,dc=local" mech=SIMPLE ssf=0 > Feb 5 18:21:03 anteros slapd[30762]: conn=1577 op=0 RESULT tag=97 err=0 text= > Feb 5 18:21:03 anteros slapd[30762]: conn=1577 op=1 SRCH > base="dc=referentiel,dc=local" scope=0 deref=0 filter="(objectClass=*)" > Feb 5 18:21:03 anteros slapd[30762]: conn=1577 op=1 SRCH attr=objectClass > Feb 5 18:21:03 anteros slapd[30762]: conn=1577 op=1 SEARCH RESULT tag=101 err=0 > nentries=1 text= > Feb 5 18:21:03 anteros slapd[30762]: conn=1577 op=2 SRCH base="" scope=0 > deref=0 filter="(objectClass=*)" > Feb 5 18:21:03 anteros slapd[30762]: conn=1577 op=2 SRCH attr=subschemaSubentry > dsServiceName namingContexts defaultNamingContext schemaNamingContext > configurationNamingContext rootDomainNamingContext supportedControl > supportedLDAPVersion supportedLDAPPolicies supportedSASLMechanisms dnsHostName > ldapServiceName serverName supportedCapabilities > Feb 5 18:21:03 anteros slapd[30762]: conn=1577 op=2 SEARCH RESULT tag=101 err=0 > nentries=1 text= > Feb 5 18:21:03 anteros slapd[30762]: conn=1577 op=3 SEARCH RESULT tag=101 err=2 > nentries=0 text=domainScope control value not absent > Feb 5 18:21:03 anteros slapd[30762]: conn=1577 op=3 do_search: get_ctrls > failed > ---- > > The behavior change is related to the fix of ITS#8840 : > www.openldap.org/its/index.cgi/Software Bugs?id=8840 > > I did not see how to change the C# code, are we sure that the fix of ITS#8840 is > valid? > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#8973) Error with domainScope control when using C# LDAP API
by clement.oudot＠worteks.com 06 Feb '19

06 Feb '19

Full_Name: Clement OUDOT Version: 2.4.47 OS: GNU/Linux URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (81.250.130.213) Hello, after upgrading from 2.4.46 to 2.4.47, this C# code does not work anymore : ---- $auth = [System.DirectoryServices.AuthenticationTypes]::FastBind $de = New-Object System.DirectoryServices.DirectoryEntry($DN,$userReferentiel, $passwordReferentiel, $auth) ---- The log in OpenLDAP 2.4.47 : ---- Feb 5 18:21:03 anteros slapd[30762]: conn=1577 fd=17 ACCEPT from IP=10.10.2.163:47304 (IP=0.0.0.0:390) Feb 5 18:21:03 anteros slapd[30762]: conn=1577 op=0 BIND dn="uid=ADPassword,ou=configuration,dc=referentiel,dc=local" method=128 Feb 5 18:21:03 anteros slapd[30762]: conn=1577 op=0 BIND dn="uid=ADPassword,ou=configuration,dc=referentiel,dc=local" mech=SIMPLE ssf=0 Feb 5 18:21:03 anteros slapd[30762]: conn=1577 op=0 RESULT tag=97 err=0 text= Feb 5 18:21:03 anteros slapd[30762]: conn=1577 op=1 SRCH base="dc=referentiel,dc=local" scope=0 deref=0 filter="(objectClass=*)" Feb 5 18:21:03 anteros slapd[30762]: conn=1577 op=1 SRCH attr=objectClass Feb 5 18:21:03 anteros slapd[30762]: conn=1577 op=1 SEARCH RESULT tag=101 err=0 nentries=1 text= Feb 5 18:21:03 anteros slapd[30762]: conn=1577 op=2 SRCH base="" scope=0 deref=0 filter="(objectClass=*)" Feb 5 18:21:03 anteros slapd[30762]: conn=1577 op=2 SRCH attr=subschemaSubentry dsServiceName namingContexts defaultNamingContext schemaNamingContext configurationNamingContext rootDomainNamingContext supportedControl supportedLDAPVersion supportedLDAPPolicies supportedSASLMechanisms dnsHostName ldapServiceName serverName supportedCapabilities Feb 5 18:21:03 anteros slapd[30762]: conn=1577 op=2 SEARCH RESULT tag=101 err=0 nentries=1 text= Feb 5 18:21:03 anteros slapd[30762]: conn=1577 op=3 SEARCH RESULT tag=101 err=2 nentries=0 text=domainScope control value not absent Feb 5 18:21:03 anteros slapd[30762]: conn=1577 op=3 do_search: get_ctrls failed ---- The behavior change is related to the fix of ITS#8840 : www.openldap.org/its/index.cgi/Software Bugs?id=8840 I did not see how to change the C# code, are we sure that the fix of ITS#8840 is valid?

1 0

RE: (ITS#8972) Performance/freezing issues using NTDLL on WindowsServer on Azure
by kriszyp＠gmail.com 05 Feb '19

05 Feb '19

--_22B17C00-48BC-46BE-A758-3013CC9A68A3_ Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Oh wow, this is just a matter of me using the wrong LMDB version? I had no = idea I wasn=E2=80=99t using the release version. I=E2=80=99ve been using ht= tps://github.com/Venemo/node-lmdb, which includes LMDB directly in it, and = had assumed that the version number in the code implied a release. I can pu= t together a PR for that project to get it on the right version. Thank you so much the pointer, I appreciate! And thank you so much for the = great software, its a fantastic package! Thanks, Kris From: Howard Chu Sent: February 5, 2019 12:03 PM To: kriszyp(a)gmail.com; openldap-its(a)OpenLDAP.org Subject: Re: (ITS#8972) Performance/freezing issues using NTDLL on WindowsS= erver on Azure kriszyp(a)gmail.com wrote: > Full_Name: Kristopher William Zyp > Version: LMDB 0.9.70 > OS: Windows Server 2012 R2 > URL:=20 > Submission from: (NULL) (71.199.6.148) I guess Windows users are never happy. Note that you're using the LMDB work-in-progress code. You ought to be usin= g the LMDB 0.9 release branch, which just uses Win32. The fact is that you get optimal performance by using LMDB as originally de= signed and documented - choose as large a mapsize as possible and then forget abou= t it. Preallocation is always more efficient than incremental growth. Clearly Windows will struggle to allocate disk space to grow files and addr= ess space to grow maps while there's competing system activity occurring, and t= his problem is worse when running under a hypervisor. This is the reason we nev= er merged the NTDLL code into the release branch; we knew from the outset that= it would destroy performance. https://openldap.org/lists/openldap-technical/201511/msg00107.html As a general rule, we're opposed to adding platform-specific API options to= LMDB. The API is supposed to be uniform across all supported platforms. If you wa= nt to submit a patch for compile-time selection of this feature, I guess that cou= ld work. As it is, LMDB is clearly not going to be a preinstalled Windows DLL = so it should be being built privately for every app that uses it anyway, so th= is shouldn't cause compatibility issues. > We have experienced significant performance issues in the form of strange= pauses > or freezing of any processes that are using our LMDB databases. These pau= ses are > periodic, often every few minutes, and typically completely freeze a proc= ess for > about 5 seconds, sometimes less, sometimes as much as a minute. They ofte= n occur > when the process is completely idle (we have a sleep timer to monitor for= them), > but we also see a lot of unusually large pauses on commits too. We have n= ever > been able to reproduce any of these issues with our local machines runnin= g > Windows 10, only on the virtual Azure Windows servers. >=20 > After weeks of investigation, we finally realized that all of these stran= ge > performance issues seemed to be due to the use of NTDLL for memory mappin= g. I > swapped out the NtCreateSection/NtMapViewOfSection calls for standard Win= 32 > CreateFileMapping/MapViewOfFile calls to create the memory map, and sudde= nly > these issues went away, and performance dramatically improved. Making thi= s > switch does mean you lose the progressive file allocation functionality o= f > NtCreateSection, but this is a small price to pay for the vast performanc= e > benefit of using standard file mapping. >=20 > Anyway, my request is to simply to provide an option to use CreateFileMap= ping > API instead of NtCreateSection. I recognize that the NTDLL based APIs may= be > more convenient for many users, but an option to use the standard APIs, e= ven if > it means the entire map is pre-allocated in the file size, is well worth = it for > the gains. I also recognize that Windows isn't the first choice of LMDB, = but > performance with these standard APIs is very good. >=20 > I'd be happy to work on putting together a branch/pull request to address= this. > However, your repository is a little different than typical github reposi= tories, > so I wanted to know if that was doable, and if there was a proper way to = submit > a PR, or if it needs to be submitted through this ticket system as a patc= h. http://www.openldap.org/devel/contributing.html --=20 -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/ --_22B17C00-48BC-46BE-A758-3013CC9A68A3_ Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset="utf-8" <html xmlns:o=3D"urn:schemas-microsoft-com:office:office" xmlns:w=3D"urn:sc= hemas-microsoft-com:office:word" xmlns:m=3D"http://schemas.microsoft.com/of= fice/2004/12/omml" xmlns=3D"http://www.w3.org/TR/REC-html40"><head><meta ht= tp-equiv=3DContent-Type content=3D"text/html; charset=3Dutf-8"><meta name= =3DGenerator content=3D"Microsoft Word 15 (filtered medium)"><style></style></head><body lang=3DEN-CA link=3Dblue vlink=3D"#954F72"><div cla= ss=3DWordSection1>Oh wow, this is just a matter of me = using the wrong LMDB version? I had no idea I wasn=E2=80=99t using the rele= ase version. I=E2=80=99ve been using <a href=3D"https://github.com/Venemo/n= ode-lmdb">https://github.com/Venemo/node-lmdb</a>, which includes LMDB dire= ctly in it, and had assumed that the version number in the code implied a r= elease. I can put together a PR for that project to get it on the right ver= sion.Thank you so much the pointer, I appreciate! = And thank you so much for the great software, its a fantastic package!<= p class=3DMsoNormal><o:p> </o:p>Thanks, Kr= is<o:p> </o:p><div style=3D'mso-element:p= ara-border-div;border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm= 0cm 0cm'>From: <= /b><a href=3D"mailto:hyc@symas.com">Howard Chu</a> Sent: February= 5, 2019 12:03 PM To: <a href=3D"mailto:kriszyp@gmail.com">kriszy= p(a)gmail.com</a>; <a href=3D"mailto:openldap-its@OpenLDAP.org">openldap-its@= OpenLDAP.org</a> Subject: Re: (ITS#8972) Performance/freezing iss= ues using NTDLL on WindowsServer on Azure</div><o:= p> </o:p>kriszyp(a)gmail.com wrote:> Full_Name: Kristopher William Zyp> Version: LMDB 0.9.70> OS: Windows Server = 2012 R2> URL: > Sub= mission from: (NULL) (71.199.6.148)<o:p> </o:= p>I guess Windows users are never happy.<o:p> </o:p>Note that you're u= sing the LMDB work-in-progress code. You ought to be using theLMDB 0.9 release branch, which just uses Win32.<o:p> </o:p>The fact is that you ge= t optimal performance by using LMDB as originally designedand documented - choose as large a mapsize as possible and then for= get about it.Preallocation is always more efficien= t than incremental growth.<o:p> </o:p>Clearly Windows will struggle to allocate disk space to g= row files and addressspace to grow maps while ther= e's competing system activity occurring, and thisp= roblem is worse when running under a hypervisor. This is the reason we neve= rmerged the NTDLL code into the release branch; we= knew from the outset that itwould destroy perform= ance.<o:p> </o:p>htt= ps://openldap.org/lists/openldap-technical/201511/msg00107.html<o:p> </o:p>As a general rule, w= e're opposed to adding platform-specific API options to LMDB.The API is supposed to be uniform across all supported platfor= ms. If you want tosubmit a patch for compile-time = selection of this feature, I guess that couldwork.= As it is, LMDB is clearly not going to be a preinstalled Windows DLL so</p= >it should be being built privately for every app that= uses it anyway, so thisshouldn't cause compatibil= ity issues.<o:p> </o:p>> We have experienced significant performance issues in the form of s= trange pauses> or freezing of any processes tha= t are using our LMDB databases. These pauses are&g= t; periodic, often every few minutes, and typically completely freeze a pro= cess for> about 5 seconds, sometimes less, some= times as much as a minute. They often occur> wh= en the process is completely idle (we have a sleep timer to monitor for the= m),> but we also see a lot of unusually large p= auses on commits too. We have never> been able = to reproduce any of these issues with our local machines running> Windows 10, only on the virtual Azure Windows servers.</= p>> > After weeks of in= vestigation, we finally realized that all of these strange> performance issues seemed to be due to the use of NTDLL for me= mory mapping. I> swapped out the NtCreateSectio= n/NtMapViewOfSection calls for standard Win32> = CreateFileMapping/MapViewOfFile calls to create the memory map, and suddenl= y> these issues went away, and performance dram= atically improved. Making this> switch does mea= n you lose the progressive file allocation functionality of> NtCreateSection, but this is a small price to pay for the vas= t performance> benefit of using standard file m= apping.> > Anyway, = my request is to simply to provide an option to use CreateFileMapping<p= class=3DMsoNormal>> API instead of NtCreateSection. I recognize that th= e NTDLL based APIs may be> more convenient for = many users, but an option to use the standard APIs, even if> it means the entire map is pre-allocated in the file size, is= well worth it for> the gains. I also recognize= that Windows isn't the first choice of LMDB, but&= gt; performance with these standard APIs is very good.> > I'd be happy to work on putting toge= ther a branch/pull request to address this.> Ho= wever, your repository is a little different than typical github repositori= es,> so I wanted to know if that was doable, an= d if there was a proper way to submit> a PR, or= if it needs to be submitted through this ticket system as a patch.<o:p> </o:p>http://www.openl= dap.org/devel/contributing.html<o:p> </o:p></= p>-- =C2=A0=C2=A0-- Howard Ch= u=C2=A0 CTO, Symas Corp.=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 http://www.symas.com=C2=A0 Director, Highland Sun=C2=A0=C2=A0=C2=A0=C2=A0 http://highlan= dsun.com/hyc/ =C2=A0Chief Architect, OpenLDAP=C2= =A0 http://www.openldap.org/project/<o:p> </o= :p></div></body></html>= --_22B17C00-48BC-46BE-A758-3013CC9A68A3_--

1 0

Re: (ITS#8972) Performance/freezing issues using NTDLL on Windows Server on Azure
by hyc＠symas.com 05 Feb '19

05 Feb '19

kriszyp(a)gmail.com wrote: > Full_Name: Kristopher William Zyp > Version: LMDB 0.9.70 > OS: Windows Server 2012 R2 > URL: > Submission from: (NULL) (71.199.6.148) I guess Windows users are never happy. Note that you're using the LMDB work-in-progress code. You ought to be using the LMDB 0.9 release branch, which just uses Win32. The fact is that you get optimal performance by using LMDB as originally designed and documented - choose as large a mapsize as possible and then forget about it. Preallocation is always more efficient than incremental growth. Clearly Windows will struggle to allocate disk space to grow files and address space to grow maps while there's competing system activity occurring, and this problem is worse when running under a hypervisor. This is the reason we never merged the NTDLL code into the release branch; we knew from the outset that it would destroy performance. https://openldap.org/lists/openldap-technical/201511/msg00107.html As a general rule, we're opposed to adding platform-specific API options to LMDB. The API is supposed to be uniform across all supported platforms. If you want to submit a patch for compile-time selection of this feature, I guess that could work. As it is, LMDB is clearly not going to be a preinstalled Windows DLL so it should be being built privately for every app that uses it anyway, so this shouldn't cause compatibility issues. > We have experienced significant performance issues in the form of strange pauses > or freezing of any processes that are using our LMDB databases. These pauses are > periodic, often every few minutes, and typically completely freeze a process for > about 5 seconds, sometimes less, sometimes as much as a minute. They often occur > when the process is completely idle (we have a sleep timer to monitor for them), > but we also see a lot of unusually large pauses on commits too. We have never > been able to reproduce any of these issues with our local machines running > Windows 10, only on the virtual Azure Windows servers. > > After weeks of investigation, we finally realized that all of these strange > performance issues seemed to be due to the use of NTDLL for memory mapping. I > swapped out the NtCreateSection/NtMapViewOfSection calls for standard Win32 > CreateFileMapping/MapViewOfFile calls to create the memory map, and suddenly > these issues went away, and performance dramatically improved. Making this > switch does mean you lose the progressive file allocation functionality of > NtCreateSection, but this is a small price to pay for the vast performance > benefit of using standard file mapping. > > Anyway, my request is to simply to provide an option to use CreateFileMapping > API instead of NtCreateSection. I recognize that the NTDLL based APIs may be > more convenient for many users, but an option to use the standard APIs, even if > it means the entire map is pre-allocated in the file size, is well worth it for > the gains. I also recognize that Windows isn't the first choice of LMDB, but > performance with these standard APIs is very good. > > I'd be happy to work on putting together a branch/pull request to address this. > However, your repository is a little different than typical github repositories, > so I wanted to know if that was doable, and if there was a proper way to submit > a PR, or if it needs to be submitted through this ticket system as a patch. http://www.openldap.org/devel/contributing.html -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#8972) Performance/freezing issues using NTDLL on Windows Server on Azure
by kriszyp＠gmail.com 05 Feb '19

05 Feb '19

Full_Name: Kristopher William Zyp Version: LMDB 0.9.70 OS: Windows Server 2012 R2 URL: Submission from: (NULL) (71.199.6.148) We have experienced significant performance issues in the form of strange pauses or freezing of any processes that are using our LMDB databases. These pauses are periodic, often every few minutes, and typically completely freeze a process for about 5 seconds, sometimes less, sometimes as much as a minute. They often occur when the process is completely idle (we have a sleep timer to monitor for them), but we also see a lot of unusually large pauses on commits too. We have never been able to reproduce any of these issues with our local machines running Windows 10, only on the virtual Azure Windows servers. After weeks of investigation, we finally realized that all of these strange performance issues seemed to be due to the use of NTDLL for memory mapping. I swapped out the NtCreateSection/NtMapViewOfSection calls for standard Win32 CreateFileMapping/MapViewOfFile calls to create the memory map, and suddenly these issues went away, and performance dramatically improved. Making this switch does mean you lose the progressive file allocation functionality of NtCreateSection, but this is a small price to pay for the vast performance benefit of using standard file mapping. Anyway, my request is to simply to provide an option to use CreateFileMapping API instead of NtCreateSection. I recognize that the NTDLL based APIs may be more convenient for many users, but an option to use the standard APIs, even if it means the entire map is pre-allocated in the file size, is well worth it for the gains. I also recognize that Windows isn't the first choice of LMDB, but performance with these standard APIs is very good. I'd be happy to work on putting together a branch/pull request to address this. However, your repository is a little different than typical github repositories, so I wanted to know if that was doable, and if there was a proper way to submit a PR, or if it needs to be submitted through this ticket system as a patch.

1 0

Re: (ITS#8958) 2nd cn=config update blocks slapd while adding subordinate index
by h.b.furuseth＠usit.uio.no 04 Feb '19

04 Feb '19

The reason I like handling the problem in tpool is that tpools is fairly isolated, no worries about which bconfig parts are safe from each other. But by all means attack the problem in bconfig too:-) I've pushed a better version to <git://git.uio.no/u/hbf/openldap.git> branch "its8958-re24-task-speeds-v2". And I see how to port this to master after all, if anyone wants this included in OpenLDAP. RE24 tpool is simpler when discusissing the basics, though. -- Hallvard

1 0

RE: (ITS#8970) bind fails when length value is multi-byte
by lit＠CBORD.com 04 Feb '19

04 Feb '19

SSBqdXN0IG5vdGljZWQgdGhhdCBteSB0d28gdGVzdHMgZGlmZmVyIG9uIHRoZSBjYXNpbmVzcyBv ZiB0aGUgdXNlciBuYW1lLCBidXQgdGhhdCdzIG5vdCBzaWduaWZpY2FudC4gIEkgdGVzdGVkIGFn YWluIHRvIGJlIHN1cmUuIA0KDQoNCi0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQpGcm9tOiBI b3dhcmQgQ2h1IDxoeWNAc3ltYXMuY29tPiANClNlbnQ6IFN1bmRheSwgRmVicnVhcnkgMywgMjAx OSAxMDoyNSBQTQ0KVG86IExlbyBUb2hpbGwgPGxpdEBDQk9SRC5jb20+OyBvcGVubGRhcC1pdHNA T3BlbkxEQVAub3JnDQpTdWJqZWN0OiBSZTogKElUUyM4OTcwKSBiaW5kIGZhaWxzIHdoZW4gbGVu Z3RoIHZhbHVlIGlzIG11bHRpLWJ5dGUNCg0KbGl0QGNib3JkLmNvbSB3cm90ZToNCj4gRnVsbF9O YW1lOiBMZW8gVG9oaWxsDQo+IFZlcnNpb246IDIuNC4zMA0KPiBPUzogV2luZG93cyAxMA0KPiBV Ukw6IA0KPiBodHRwczovL2RvY3MuZ29vZ2xlLmNvbS9kb2N1bWVudC9kLzEwdUtnOU5oM0hMaXVP elRiTGZpNlo3YkNmVWI4eF9BaTZXDQo+IEs1THFOemJ3QS9lZGl0P3VzcD1zaGFyaW5nDQo+IFN1 Ym1pc3Npb24gZnJvbTogKE5VTEwpICg3NC43OS44LjQxKQ0KPiANCj4gDQo+IFN1bW1hcnk6IG9w ZW5sZGFwIDIuNC4zMCBkb2VzIG5vdCBhY2NvbW1vZGF0ZSBtdWx0aS1ieXRlIGxlbmd0aCB2YWx1 ZSANCj4gb24gYmluZCByZXF1ZXN0Lg0KPiANCj4gDQo+IEZpcnN0LCBJJ2xsIGFkbWl0IHRoYXQg SSdtIG91dCBvZiBteSBkZXB0aCBoZXJlLCBJJ20gcnVubmluZyBhIG9sZGVyIA0KPiB2ZXJzaW9u LCBJJ20gb24gV2luZG93cywgYW5kIG15IHBhY2thZ2Ugd2FzIGJ1aWx0IGJ5IEkgZG9uJ3Qga25v dy4gIA0KPiBCdXQgSSB3b3JrZWQgaGFyZCBlbm91Z2ggdG8gdHJhY2sgdGhpcyBkb3duIHRoYXQg SSB3YW50IHlvdSB0byBrbm93IA0KPiB3aGF0IEkgZm91bmQuICBJIG1pZ2h0IHVwZ3JhZGUsIGJ1 dCB0aGF0J3MgcHJvYmxlbWF0aWMuDQo+IA0KPiBBdCBzb21lIHBvaW50IG15IGJpbmRpbmdzIGZy b20gLm5ldCBiZWdhbiBmYWlsaW5nIHdpdGggInRoZSB1c2VybmFtZSANCj4gb3IgcGFzc3dvcmQg aXMgaW5jb3JyZWN0IiBCdXQgdGhleSB3ZXJlIGNvcnJlY3QuICBJIGNvdWxkIGNvbmZpcm0gd2l0 aCB2YXJpb3VzIG90aGVyIHRvb2xzLg0KPiBJIGNhcHR1cmVkIHRoZSB3aXJlIHRyYWZmaWMgdG8g aXNvbGF0ZSB0aGUgcHJvYmxlbS4gSXQgdHVybnMgb3V0IHRoYXQgDQo+IFdpbmRvd3MgZm9ybXMg dGhlIGJpbmRpbmcgcmVxdWVzdCB1c2luZyAgYSBtdWx0aS1ieXRlIGxlbmd0aCBpbmRpY2F0b3Ig aW4gdGhlIHJlcXVlc3QuDQo+IE9wZW5MZGFwIGFwcGFyZW50bHkgZG9lcyBub3QgYWNjb21tb2Rh dGUgdGhpcy4gIEkgY29tcGFyZWQgdG8gYSANCj4gcmVxdWVzdCBnZW5lcmF0ZWQgYnkgIGxkYXBz ZWFyY2guZXhlLiAgVGhhdCByZXF1ZXN0LCB3aGljaCBzdWNjZWVkcywgIA0KPiB2YXJpZXMgb25s eSBieSB1c2luZyBhIHNpbmdsZS1ieXRlIGxlbmd0aCBpbmRpY2F0b3IuDQo+IA0KPiBUaGUgbXVs dGktYnl0ZSBsZW5ndGggdmFsdWUgc2hvdWxkIGJlIGFsbG93ZWQsIHJpZ2h0PyAgSXNuJ3QgaXQg DQo+IHBvc3NpYmxlIHRvIGhhdmUgYSBiaW5kIHJlcXVlc3QgZGF0YSBwYWNrZXQgb2YgbGVuZ3Ro ID4gMTI3PyAgV2hpY2ggDQo+IHdvdWxkIHJlcXVpcmUgYSBtdWx0aS1ieXRlIGxlbmd0aCB2YWx1 ZS4gIFBlcmhhcHMgdGhpcyB3YXMgZml4ZWQgaW4gYSBsYXRlciB2ZXJzaW9uLg0KPiANCj4gU2Ny ZWVuc2hvdHMgb2YgdGhlIHdpcmUgY2FwdHVyZSBoZXJlOiANCj4gDQo+IGh0dHBzOi8vZG9jcy5n b29nbGUuY29tL2RvY3VtZW50L2QvMTB1S2c5TmgzSExpdU96VGJMZmk2WjdiQ2ZVYjh4X0FpNlcN Cj4gSzVMcU56YndBL2VkaXQ/dXNwPXNoYXJpbmcNCg0KVGhpcyBVUkwgaXMgaW5hY2Nlc3NpYmxl LCBwZXJtaXNzaW9uIGRlbmllZC4NCg0KSnVzdCBjb3B5IGhleCBkdW1wcyBvZiBib3RoIHJlcXVl c3RzIGhlcmUgaW4gdGV4dC4NCg0KLS0NCiAgLS0gSG93YXJkIENodQ0KICBDVE8sIFN5bWFzIENv cnAuICAgICAgICAgICBodHRwOi8vd3d3LnN5bWFzLmNvbQ0KICBEaXJlY3RvciwgSGlnaGxhbmQg U3VuICAgICBodHRwOi8vaGlnaGxhbmRzdW4uY29tL2h5Yy8NCiAgQ2hpZWYgQXJjaGl0ZWN0LCBP cGVuTERBUCAgaHR0cDovL3d3dy5vcGVubGRhcC5vcmcvcHJvamVjdC8NCg==

1 0

RE: (ITS#8970) bind fails when length value is multi-byte
by lit＠CBORD.com 04 Feb '19

04 Feb '19

ZnJhbWUgY2FwdHVyZXMgZm9sbG93LiAgTGRhcCBtZXNzYWdlIGJlZ2lucyBhdCBvZmZzZXQgMHgz NiAoYWZ0ZXIgZXRoZXJuZXQsIGlwLCB0Y3AgaGVhZGVycykNCnN1Y2Nlc3M6DQowMCBGRiAwNiAx NiBFNiBFMSAwMCBGRiAwNSAxNiBFNiBFMSAwOCAwMCA0NSAwMCAwMCA1MyA2QiBGRSA0MCAwMCA4 MCAwNiA0RiBGNCAwQSAwMSAyOCAyQSAwQSA3OCAwMiAxMCBENSAwOCAwMSA4NSAzMiBGQiAyQiBF NiA1MCAxNSA0NyBGMSA1MCAxOCAwMSAwMCA5MCA0OCAwMCAwMCAzMCAyOSAwMiAwMSAwMSA2MCAy NCAwMiAwMSAwMyAwNCAxOSA2MyA2RSAzRCA1NSA3MyA2NSA3MiAzMSAyQyA2NCA2MyAzRCA2RCA2 MSA3OCA2MyA3MiA2MyAyQyA2NCA2MyAzRCA2MyA2RiA2RCA4MCAwNCA2OSA2RSA2OSA3NCANCg0K ZmFpbHVyZToNCjAwIEZGIDA2IDE2IEU2IEUxIDAwIEZGIDA1IDE2IEU2IEUxIDA4IDAwIDQ1IDAw IDAwIDVCIDA1IDNBIDQwIDAwIDgwIDA2IEI3IEUyIDBBIDAxIDI4IDJBIDBBIDAxIDAxIDU1IEQ2 IDBBIDAxIDg1IDRGIDREIEYxIEYwIDU3IDJGIDUxIDNDIDUwIDE4IDAxIDAwIDE5IDA2IDAwIDAw IDMwIDg0IDAwIDAwIDAwIDJEIDAyIDAxIDAxIDYwIDg0IDAwIDAwIDAwIDI0IDAyIDAxIDAzIDA0 IDE5IDYzIDZFIDNEIDc1IDczIDY1IDcyIDMxIDJDIDY0IDYzIDNEIDZEIDYxIDc4IDYzIDcyIDYz IDJDIDY0IDYzIDNEIDYzIDZGIDZEIDgwIDA0IDY5IDZFIDY5IDc0IA0KDQpGb3IgeW91ciBjb252 ZW5pZW5jZSAobm90IG5lZWRlZD8pLCBoZXJlIGlzIHRoZSBuZXR3b3JrIG1vbml0b3IgVUkgbGFi ZWxpbmcgZm9yIGVhY2ggZnJhbWUgKHN0YXJ0aW5nIGF0IGxkYXAgbWVzc2FnZSkNCg0Kc3VjY2Vz czoNCiAgRnJhbWU6IE51bWJlciA9IDExOCwgQ2FwdHVyZWQgRnJhbWUgTGVuZ3RoID0gOTcsIE1l ZGlhVHlwZSA9IEVUSEVSTkVUDQorIEV0aGVybmV0OiBFdHlwZSA9IEludGVybmV0IElQIChJUHY0 KSxEZXN0aW5hdGlvbkFkZHJlc3M6WzAwLUZGLTA2LTE2LUU2LUUxXSxTb3VyY2VBZGRyZXNzOlsw MC1GRi0wNS0xNi1FNi1FMV0NCisgSXB2NDogU3JjID0gMTAuMS40MC40MiwgRGVzdCA9IDEwLjEy MC4yLjE2LCBOZXh0IFByb3RvY29sID0gVENQLCBQYWNrZXQgSUQgPSAyNzY0NiwgVG90YWwgSVAg TGVuZ3RoID0gODMNCisgVGNwOiBGbGFncz0uLi5BUC4uLiwgU3JjUG9ydD01NDUzNiwgRHN0UG9y dD1MREFQKDM4OSksIFBheWxvYWRMZW49NDMsIFNlcT04NTUzMjE1NzQgLSA4NTUzMjE2MTcsIEFj az0xMzQzNTcxOTUzLCBXaW49MjU2IChzY2FsZSBmYWN0b3IgMHg4KSA9IDY1NTM2DQogIExkYXA6 IEJpbmQgUmVxdWVzdCwgTWVzc2FnZUlEOiAxLCBWZXJzaW9uOiAzDQotIExEQVBNZXNzYWdlOiBC aW5kIFJlcXVlc3QsIE1lc3NhZ2VJRDogMQ0KICAtIFBhcnNlckhlYWRlcjogDQogICAtIEFzbklk OiBTZXF1ZW5jZSBhbmQgU2VxdWVuY2VPZiB0eXBlcyAoVW5pdmVyc2FsIDE2KQ0KICAgIC0gTG93 VGFnOiANCiAgICAgICBDbGFzczogICAgKDAwLi4uLi4uKSBVbml2ZXJzYWwgKDApDQogICAgICAg VHlwZTogICAgICguLjEuLi4uLikgQ29uc3RydWN0ZWQNCiAgICAgICBUYWdWYWx1ZTogKC4uLjEw MDAwKSAxNg0KICAgLSBBc25MZW46IExlbmd0aCA9IDQxLCBMZW5ndGhPZkxlbmd0aCA9IDANCiAg ICAgIExlbmd0aDogNDEgYnl0ZXMsIExlbmd0aE9mTGVuZ3RoID0gMA0KICAtIE1lc3NhZ2VJRDog MQ0KICAgLSBBc25JbnRlZ2VySGVhZGVyOiANCiAgICAtIEFzbklkOiBJbnRlZ2VyIHR5cGUgKFVu aXZlcnNhbCAyKQ0KICAgICAtIExvd1RhZzogDQogICAgICAgIENsYXNzOiAgICAoMDAuLi4uLi4p IFVuaXZlcnNhbCAoMCkNCiAgICAgICAgVHlwZTogICAgICguLjAuLi4uLikgUHJpbWl0aXZlDQog ICAgICAgIFRhZ1ZhbHVlOiAoLi4uMDAwMTApIDINCiAgICAtIEFzbkxlbjogTGVuZ3RoID0gMSwg TGVuZ3RoT2ZMZW5ndGggPSAwDQogICAgICAgTGVuZ3RoOiAxIGJ5dGVzLCBMZW5ndGhPZkxlbmd0 aCA9IDANCiAgICAgQXNuSW50OiAxICgweDEpDQogIC0gT3BlcmF0aW9uSGVhZGVyOiBCaW5kIFJl cXVlc3QsIDAoMCkNCiAgIC0gQXNuSWQ6IEFwcGxpY2F0aW9uIENvbnN0cnVjdGVkIFRhZyAoMCkN CiAgICAtIExvd1RhZzogDQogICAgICAgQ2xhc3M6ICAgICgwMS4uLi4uLikgQXBwbGljYXRpb24g KDEpDQogICAgICAgVHlwZTogICAgICguLjEuLi4uLikgQ29uc3RydWN0ZWQNCiAgICAgICBUYWdW YWx1ZTogKC4uLjAwMDAwKSAwDQogICAtIEFzbkxlbjogTGVuZ3RoID0gMzYsIExlbmd0aE9mTGVu Z3RoID0gMA0KICAgICAgTGVuZ3RoOiAzNiBieXRlcywgTGVuZ3RoT2ZMZW5ndGggPSAwDQogIC0g QmluZFJlcXVlc3Q6IFZlcnNpb246MywgTmFtZTpjbj1Vc2VyMSxkYz1tYXhjcmMsZGM9Y29tLCBV c2VyTmFtZTogaW5pdCwgQXV0aGVudGljYXRpb24gdHlwZSA9IHNpbXBsZQ0KICAgLSBWZXJzaW9u OiAzDQogICAgLSBBc25JbnRlZ2VySGVhZGVyOiANCiAgICAgLSBBc25JZDogSW50ZWdlciB0eXBl IChVbml2ZXJzYWwgMikNCiAgICAgIC0gTG93VGFnOiANCiAgICAgICAgIENsYXNzOiAgICAoMDAu Li4uLi4pIFVuaXZlcnNhbCAoMCkNCiAgICAgICAgIFR5cGU6ICAgICAoLi4wLi4uLi4pIFByaW1p dGl2ZQ0KICAgICAgICAgVGFnVmFsdWU6ICguLi4wMDAxMCkgMg0KICAgICAtIEFzbkxlbjogTGVu Z3RoID0gMSwgTGVuZ3RoT2ZMZW5ndGggPSAwDQogICAgICAgIExlbmd0aDogMSBieXRlcywgTGVu Z3RoT2ZMZW5ndGggPSAwDQogICAgICBBc25JbnQ6IDMgKDB4MykNCiAgIC0gTmFtZTogY249VXNl cjEsZGM9bWF4Y3JjLGRjPWNvbQ0KICAgIC0gQXNuT2N0ZXRTdHJpbmdIZWFkZXI6IA0KICAgICAt IEFzbklkOiBPY3RldFN0cmluZyB0eXBlIChVbml2ZXJzYWwgNCkNCiAgICAgIC0gTG93VGFnOiAN CiAgICAgICAgIENsYXNzOiAgICAoMDAuLi4uLi4pIFVuaXZlcnNhbCAoMCkNCiAgICAgICAgIFR5 cGU6ICAgICAoLi4wLi4uLi4pIFByaW1pdGl2ZQ0KICAgICAgICAgVGFnVmFsdWU6ICguLi4wMDEw MCkgNA0KICAgICAtIEFzbkxlbjogTGVuZ3RoID0gMjUsIExlbmd0aE9mTGVuZ3RoID0gMA0KICAg ICAgICBMZW5ndGg6IDI1IGJ5dGVzLCBMZW5ndGhPZkxlbmd0aCA9IDANCiAgICAgIE9jdGV0U3Ry ZWFtOiBjbj1Vc2VyMSxkYz1tYXhjcmMsZGM9Y29tDQogICAtIEF1dGhlbnRpY2F0aW9uOiBVc2Vy TmFtZTogaW5pdCwgQXV0aGVudGljYXRpb24gdHlwZSA9IHNpbXBsZQ0KICAgIC0gQXV0aGVudGlj YXRpb25UeXBlSGVhZGVyOiBBdXRoZW50aWNhdGlvbiB0eXBlID0gc2ltcGxlDQogICAgIC0gQXNu SWQ6IENvbnRleHQgU3BlY2lmaWMgUHJpbWl0aXZlIFRhZyAoMCkNCiAgICAgIC0gTG93VGFnOiAN CiAgICAgICAgIENsYXNzOiAgICAoMTAuLi4uLi4pIENvblRleHQgc3BlY2lmaWMgKDIpDQogICAg ICAgICBUeXBlOiAgICAgKC4uMC4uLi4uKSBQcmltaXRpdmUNCiAgICAgICAgIFRhZ1ZhbHVlOiAo Li4uMDAwMDApIDANCiAgICAgLSBBc25MZW46IExlbmd0aCA9IDQsIExlbmd0aE9mTGVuZ3RoID0g MA0KICAgICAgICBMZW5ndGg6IDQgYnl0ZXMsIExlbmd0aE9mTGVuZ3RoID0gMA0KICAgICAgU2lt cGxlQXV0aGVudGljYXRpb246IGluaXQNCg0KDQpmYWlsdXJlOg0KDQogIEZyYW1lOiBOdW1iZXIg PSAzMCwgQ2FwdHVyZWQgRnJhbWUgTGVuZ3RoID0gMTA1LCBNZWRpYVR5cGUgPSBFVEhFUk5FVA0K KyBFdGhlcm5ldDogRXR5cGUgPSBJbnRlcm5ldCBJUCAoSVB2NCksRGVzdGluYXRpb25BZGRyZXNz OlswMC1GRi0wNi0xNi1FNi1FMV0sU291cmNlQWRkcmVzczpbMDAtRkYtMDUtMTYtRTYtRTFdDQor IElwdjQ6IFNyYyA9IDEwLjEuNDAuNDIsIERlc3QgPSAxMC4xLjEuODUsIE5leHQgUHJvdG9jb2wg PSBUQ1AsIFBhY2tldCBJRCA9IDEzMzgsIFRvdGFsIElQIExlbmd0aCA9IDkxDQorIFRjcDogRmxh Z3M9Li4uQVAuLi4sIFNyY1BvcnQ9NTQ3OTQsIERzdFBvcnQ9TERBUCgzODkpLCBQYXlsb2FkTGVu PTUxLCBTZXE9MTMzMDUwODI3MiAtIDEzMzA1MDgzMjMsIEFjaz0xNDYyNzE4NzgwLCBXaW49MjU2 IChzY2FsZSBmYWN0b3IgMHg4KSA9IDY1NTM2DQogIExkYXA6IEJpbmQgUmVxdWVzdCwgTWVzc2Fn ZUlEOiAxLCBWZXJzaW9uOiAzDQotIExEQVBNZXNzYWdlOiBCaW5kIFJlcXVlc3QsIE1lc3NhZ2VJ RDogMQ0KICAtIFBhcnNlckhlYWRlcjogDQogICAtIEFzbklkOiBTZXF1ZW5jZSBhbmQgU2VxdWVu Y2VPZiB0eXBlcyAoVW5pdmVyc2FsIDE2KQ0KICAgIC0gTG93VGFnOiANCiAgICAgICBDbGFzczog ICAgKDAwLi4uLi4uKSBVbml2ZXJzYWwgKDApDQogICAgICAgVHlwZTogICAgICguLjEuLi4uLikg Q29uc3RydWN0ZWQNCiAgICAgICBUYWdWYWx1ZTogKC4uLjEwMDAwKSAxNg0KICAgLSBBc25MZW46 IExlbmd0aCA9IDQ1LCBMZW5ndGhPZkxlbmd0aCA9IDQNCiAgICAgIExlbmd0aFR5cGU6IExlbmd0 aE9mTGVuZ3RoID0gNA0KICAgICAgTGVuZ3RoOiA0NSBieXRlcw0KICAtIE1lc3NhZ2VJRDogMQ0K ICAgLSBBc25JbnRlZ2VySGVhZGVyOiANCiAgICAtIEFzbklkOiBJbnRlZ2VyIHR5cGUgKFVuaXZl cnNhbCAyKQ0KICAgICAtIExvd1RhZzogDQogICAgICAgIENsYXNzOiAgICAoMDAuLi4uLi4pIFVu aXZlcnNhbCAoMCkNCiAgICAgICAgVHlwZTogICAgICguLjAuLi4uLikgUHJpbWl0aXZlDQogICAg ICAgIFRhZ1ZhbHVlOiAoLi4uMDAwMTApIDINCiAgICAtIEFzbkxlbjogTGVuZ3RoID0gMSwgTGVu Z3RoT2ZMZW5ndGggPSAwDQogICAgICAgTGVuZ3RoOiAxIGJ5dGVzLCBMZW5ndGhPZkxlbmd0aCA9 IDANCiAgICAgQXNuSW50OiAxICgweDEpDQogIC0gT3BlcmF0aW9uSGVhZGVyOiBCaW5kIFJlcXVl c3QsIDAoMCkNCiAgIC0gQXNuSWQ6IEFwcGxpY2F0aW9uIENvbnN0cnVjdGVkIFRhZyAoMCkNCiAg ICAtIExvd1RhZzogDQogICAgICAgQ2xhc3M6ICAgICgwMS4uLi4uLikgQXBwbGljYXRpb24gKDEp DQogICAgICAgVHlwZTogICAgICguLjEuLi4uLikgQ29uc3RydWN0ZWQNCiAgICAgICBUYWdWYWx1 ZTogKC4uLjAwMDAwKSAwDQogICAtIEFzbkxlbjogTGVuZ3RoID0gMzYsIExlbmd0aE9mTGVuZ3Ro ID0gNA0KICAgICAgTGVuZ3RoVHlwZTogTGVuZ3RoT2ZMZW5ndGggPSA0DQogICAgICBMZW5ndGg6 IDM2IGJ5dGVzDQogIC0gQmluZFJlcXVlc3Q6IFZlcnNpb246MywgTmFtZTpjbj11c2VyMSxkYz1t YXhjcmMsZGM9Y29tLCBVc2VyTmFtZTogaW5pdCwgQXV0aGVudGljYXRpb24gdHlwZSA9IHNpbXBs ZQ0KICAgLSBWZXJzaW9uOiAzDQogICAgLSBBc25JbnRlZ2VySGVhZGVyOiANCiAgICAgLSBBc25J ZDogSW50ZWdlciB0eXBlIChVbml2ZXJzYWwgMikNCiAgICAgIC0gTG93VGFnOiANCiAgICAgICAg IENsYXNzOiAgICAoMDAuLi4uLi4pIFVuaXZlcnNhbCAoMCkNCiAgICAgICAgIFR5cGU6ICAgICAo Li4wLi4uLi4pIFByaW1pdGl2ZQ0KICAgICAgICAgVGFnVmFsdWU6ICguLi4wMDAxMCkgMg0KICAg ICAtIEFzbkxlbjogTGVuZ3RoID0gMSwgTGVuZ3RoT2ZMZW5ndGggPSAwDQogICAgICAgIExlbmd0 aDogMSBieXRlcywgTGVuZ3RoT2ZMZW5ndGggPSAwDQogICAgICBBc25JbnQ6IDMgKDB4MykNCiAg IC0gTmFtZTogY249dXNlcjEsZGM9bWF4Y3JjLGRjPWNvbQ0KICAgIC0gQXNuT2N0ZXRTdHJpbmdI ZWFkZXI6IA0KICAgICAtIEFzbklkOiBPY3RldFN0cmluZyB0eXBlIChVbml2ZXJzYWwgNCkNCiAg ICAgIC0gTG93VGFnOiANCiAgICAgICAgIENsYXNzOiAgICAoMDAuLi4uLi4pIFVuaXZlcnNhbCAo MCkNCiAgICAgICAgIFR5cGU6ICAgICAoLi4wLi4uLi4pIFByaW1pdGl2ZQ0KICAgICAgICAgVGFn VmFsdWU6ICguLi4wMDEwMCkgNA0KICAgICAtIEFzbkxlbjogTGVuZ3RoID0gMjUsIExlbmd0aE9m TGVuZ3RoID0gMA0KICAgICAgICBMZW5ndGg6IDI1IGJ5dGVzLCBMZW5ndGhPZkxlbmd0aCA9IDAN CiAgICAgIE9jdGV0U3RyZWFtOiBjbj11c2VyMSxkYz1tYXhjcmMsZGM9Y29tDQogICAtIEF1dGhl bnRpY2F0aW9uOiBVc2VyTmFtZTogaW5pdCwgQXV0aGVudGljYXRpb24gdHlwZSA9IHNpbXBsZQ0K ICAgIC0gQXV0aGVudGljYXRpb25UeXBlSGVhZGVyOiBBdXRoZW50aWNhdGlvbiB0eXBlID0gc2lt cGxlDQogICAgIC0gQXNuSWQ6IENvbnRleHQgU3BlY2lmaWMgUHJpbWl0aXZlIFRhZyAoMCkNCiAg ICAgIC0gTG93VGFnOiANCiAgICAgICAgIENsYXNzOiAgICAoMTAuLi4uLi4pIENvblRleHQgc3Bl Y2lmaWMgKDIpDQogICAgICAgICBUeXBlOiAgICAgKC4uMC4uLi4uKSBQcmltaXRpdmUNCiAgICAg ICAgIFRhZ1ZhbHVlOiAoLi4uMDAwMDApIDANCiAgICAgLSBBc25MZW46IExlbmd0aCA9IDQsIExl bmd0aE9mTGVuZ3RoID0gMA0KICAgICAgICBMZW5ndGg6IDQgYnl0ZXMsIExlbmd0aE9mTGVuZ3Ro ID0gMA0KICAgICAgU2ltcGxlQXV0aGVudGljYXRpb246IGluaXQNCiAgDQoNCi0tLS0tT3JpZ2lu YWwgTWVzc2FnZS0tLS0tDQpGcm9tOiBIb3dhcmQgQ2h1IDxoeWNAc3ltYXMuY29tPiANClNlbnQ6 IFN1bmRheSwgRmVicnVhcnkgMywgMjAxOSAxMDoyNSBQTQ0KVG86IExlbyBUb2hpbGwgPGxpdEBD Qk9SRC5jb20+OyBvcGVubGRhcC1pdHNAT3BlbkxEQVAub3JnDQpTdWJqZWN0OiBSZTogKElUUyM4 OTcwKSBiaW5kIGZhaWxzIHdoZW4gbGVuZ3RoIHZhbHVlIGlzIG11bHRpLWJ5dGUNCg0KbGl0QGNi b3JkLmNvbSB3cm90ZToNCj4gRnVsbF9OYW1lOiBMZW8gVG9oaWxsDQo+IFZlcnNpb246IDIuNC4z MA0KPiBPUzogV2luZG93cyAxMA0KPiBVUkw6IA0KPiBodHRwczovL2RvY3MuZ29vZ2xlLmNvbS9k b2N1bWVudC9kLzEwdUtnOU5oM0hMaXVPelRiTGZpNlo3YkNmVWI4eF9BaTZXDQo+IEs1THFOemJ3 QS9lZGl0P3VzcD1zaGFyaW5nDQo+IFN1Ym1pc3Npb24gZnJvbTogKE5VTEwpICg3NC43OS44LjQx KQ0KPiANCj4gDQo+IFN1bW1hcnk6IG9wZW5sZGFwIDIuNC4zMCBkb2VzIG5vdCBhY2NvbW1vZGF0 ZSBtdWx0aS1ieXRlIGxlbmd0aCB2YWx1ZSANCj4gb24gYmluZCByZXF1ZXN0Lg0KPiANCj4gDQo+ IEZpcnN0LCBJJ2xsIGFkbWl0IHRoYXQgSSdtIG91dCBvZiBteSBkZXB0aCBoZXJlLCBJJ20gcnVu bmluZyBhIG9sZGVyIA0KPiB2ZXJzaW9uLCBJJ20gb24gV2luZG93cywgYW5kIG15IHBhY2thZ2Ug d2FzIGJ1aWx0IGJ5IEkgZG9uJ3Qga25vdy4gIA0KPiBCdXQgSSB3b3JrZWQgaGFyZCBlbm91Z2gg dG8gdHJhY2sgdGhpcyBkb3duIHRoYXQgSSB3YW50IHlvdSB0byBrbm93IA0KPiB3aGF0IEkgZm91 bmQuICBJIG1pZ2h0IHVwZ3JhZGUsIGJ1dCB0aGF0J3MgcHJvYmxlbWF0aWMuDQo+IA0KPiBBdCBz b21lIHBvaW50IG15IGJpbmRpbmdzIGZyb20gLm5ldCBiZWdhbiBmYWlsaW5nIHdpdGggInRoZSB1 c2VybmFtZSANCj4gb3IgcGFzc3dvcmQgaXMgaW5jb3JyZWN0IiBCdXQgdGhleSB3ZXJlIGNvcnJl Y3QuICBJIGNvdWxkIGNvbmZpcm0gd2l0aCB2YXJpb3VzIG90aGVyIHRvb2xzLg0KPiBJIGNhcHR1 cmVkIHRoZSB3aXJlIHRyYWZmaWMgdG8gaXNvbGF0ZSB0aGUgcHJvYmxlbS4gSXQgdHVybnMgb3V0 IHRoYXQgDQo+IFdpbmRvd3MgZm9ybXMgdGhlIGJpbmRpbmcgcmVxdWVzdCB1c2luZyAgYSBtdWx0 aS1ieXRlIGxlbmd0aCBpbmRpY2F0b3IgaW4gdGhlIHJlcXVlc3QuDQo+IE9wZW5MZGFwIGFwcGFy ZW50bHkgZG9lcyBub3QgYWNjb21tb2RhdGUgdGhpcy4gIEkgY29tcGFyZWQgdG8gYSANCj4gcmVx dWVzdCBnZW5lcmF0ZWQgYnkgIGxkYXBzZWFyY2guZXhlLiAgVGhhdCByZXF1ZXN0LCB3aGljaCBz dWNjZWVkcywgIA0KPiB2YXJpZXMgb25seSBieSB1c2luZyBhIHNpbmdsZS1ieXRlIGxlbmd0aCBp bmRpY2F0b3IuDQo+IA0KPiBUaGUgbXVsdGktYnl0ZSBsZW5ndGggdmFsdWUgc2hvdWxkIGJlIGFs bG93ZWQsIHJpZ2h0PyAgSXNuJ3QgaXQgDQo+IHBvc3NpYmxlIHRvIGhhdmUgYSBiaW5kIHJlcXVl c3QgZGF0YSBwYWNrZXQgb2YgbGVuZ3RoID4gMTI3PyAgV2hpY2ggDQo+IHdvdWxkIHJlcXVpcmUg YSBtdWx0aS1ieXRlIGxlbmd0aCB2YWx1ZS4gIFBlcmhhcHMgdGhpcyB3YXMgZml4ZWQgaW4gYSBs YXRlciB2ZXJzaW9uLg0KPiANCj4gU2NyZWVuc2hvdHMgb2YgdGhlIHdpcmUgY2FwdHVyZSBoZXJl OiANCj4gDQo+IGh0dHBzOi8vZG9jcy5nb29nbGUuY29tL2RvY3VtZW50L2QvMTB1S2c5TmgzSExp dU96VGJMZmk2WjdiQ2ZVYjh4X0FpNlcNCj4gSzVMcU56YndBL2VkaXQ/dXNwPXNoYXJpbmcNCg0K VGhpcyBVUkwgaXMgaW5hY2Nlc3NpYmxlLCBwZXJtaXNzaW9uIGRlbmllZC4NCg0KSnVzdCBjb3B5 IGhleCBkdW1wcyBvZiBib3RoIHJlcXVlc3RzIGhlcmUgaW4gdGV4dC4NCg0KLS0NCiAgLS0gSG93 YXJkIENodQ0KICBDVE8sIFN5bWFzIENvcnAuICAgICAgICAgICBodHRwOi8vd3d3LnN5bWFzLmNv bQ0KICBEaXJlY3RvciwgSGlnaGxhbmQgU3VuICAgICBodHRwOi8vaGlnaGxhbmRzdW4uY29tL2h5 Yy8NCiAgQ2hpZWYgQXJjaGl0ZWN0LCBPcGVuTERBUCAgaHR0cDovL3d3dy5vcGVubGRhcC5vcmcv cHJvamVjdC8NCg==

1 0

Re: (ITS#8970) bind fails when length value is multi-byte
by hyc＠symas.com 04 Feb '19

04 Feb '19

lit(a)cbord.com wrote: > Full_Name: Leo Tohill > Version: 2.4.30 > OS: Windows 10 > URL: https://docs.google.com/document/d/10uKg9Nh3HLiuOzTbLfi6Z7bCfUb8x_Ai6WK5LqN… > Submission from: (NULL) (74.79.8.41) > > > Summary: openldap 2.4.30 does not accommodate multi-byte length value on bind > request. > > > First, I'll admit that I'm out of my depth here, I'm running a older version, > I'm on Windows, and my package was built by I don't know. But I worked hard > enough to track this down that I want you to know what I found. I might > upgrade, but that's problematic. > > At some point my bindings from .net began failing with "the username or password > is incorrect" But they were correct. I could confirm with various other tools. > I captured the wire traffic to isolate the problem. It turns out that Windows > forms the binding request using a multi-byte length indicator in the request. > OpenLdap apparently does not accommodate this. I compared to a request > generated by ldapsearch.exe. That request, which succeeds, varies only by > using a single-byte length indicator. > > The multi-byte length value should be allowed, right? Isn't it possible to have > a bind request data packet of length > 127? Which would require a multi-byte > length value. Perhaps this was fixed in a later version. > > Screenshots of the wire capture here: > > https://docs.google.com/document/d/10uKg9Nh3HLiuOzTbLfi6Z7bCfUb8x_Ai6WK5LqN… This URL is inaccessible, permission denied. Just copy hex dumps of both requests here in text. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8971) slapo-accesslog hits assert
by hyc＠symas.com 02 Feb '19

02 Feb '19

michael(a)stroeder.com wrote: > Full_Name: > Version: RE24 branch > OS: openSUSE Linux > URL: > Submission from: (NULL) (212.68.198.84) > > > For the records an issue tested with Howard today: > > slapo-accesslog hits an assert checking for empty 'reqDN' after processing a > password modify extended operation. > > More information in the ITS upon request. > > Fixed in git master -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#8971) slapo-accesslog hits assert
by michael＠stroeder.com 02 Feb '19

02 Feb '19

Full_Name: Version: RE24 branch OS: openSUSE Linux URL: Submission from: (NULL) (212.68.198.84) For the records an issue tested with Howard today: slapo-accesslog hits an assert checking for empty 'reqDN' after processing a password modify extended operation. More information in the ITS upon request.

1 0

Re: (ITS#8963) Start TLS failed errors on back-ldap binds / short timeout, config option request
by janne.peltonen＠helsinki.fi 01 Feb '19

01 Feb '19

On Thu, Jan 31, 2019 at 11:41:07PM +0000, Howard Chu wrote: > Thanks for the report. Note that if you simply set a "timeout" with no specific > qualifiers, it would have worked for you as well. Oh dear. Of course. And it says so cleary on the man page, too. Thank you! > At this point, the "extended" timeout was meant for the other exops that back-ldap > supports, e.g. whoami and passwordModify. Since 2.4 is feature frozen we are not > going to implement the extended timeout feature here. > > Instead, I've changed the start_tls code to use the "bind" timeout, since it will > always be used as part of a bind operation anyway. OK, thanks a lot! --Janne / Helsinki Uni

1 0

(ITS#8970) bind fails when length value is multi-byte
by lit＠cbord.com 01 Feb '19

01 Feb '19

Full_Name: Leo Tohill Version: 2.4.30 OS: Windows 10 URL: https://docs.google.com/document/d/10uKg9Nh3HLiuOzTbLfi6Z7bCfUb8x_Ai6WK5LqN… Submission from: (NULL) (74.79.8.41) Summary: openldap 2.4.30 does not accommodate multi-byte length value on bind request. First, I'll admit that I'm out of my depth here, I'm running a older version, I'm on Windows, and my package was built by I don't know. But I worked hard enough to track this down that I want you to know what I found. I might upgrade, but that's problematic. At some point my bindings from .net began failing with "the username or password is incorrect" But they were correct. I could confirm with various other tools. I captured the wire traffic to isolate the problem. It turns out that Windows forms the binding request using a multi-byte length indicator in the request. OpenLdap apparently does not accommodate this. I compared to a request generated by ldapsearch.exe. That request, which succeeds, varies only by using a single-byte length indicator. The multi-byte length value should be allowed, right? Isn't it possible to have a bind request data packet of length > 127? Which would require a multi-byte length value. Perhaps this was fixed in a later version. Screenshots of the wire capture here: https://docs.google.com/document/d/10uKg9Nh3HLiuOzTbLfi6Z7bCfUb8x_Ai6WK5LqN… Thanks for any comments.

1 0

Re: (ITS#8963) Start TLS failed errors on back-ldap binds / short timeout, config option request
by hyc＠symas.com 01 Feb '19

01 Feb '19

janne.peltonen(a)helsinki.fi wrote: > Full_Name: Janne Peltonen > Version: 2.4.44 > OS: Red Hat Enterprise Linux 7 > URL: https://www.openldap.org/lists/openldap-technical/201901/msg00038.html > Submission from: (NULL) (2001:708:110:1820:3664:a9ff:fe04:7363) > > > Proxied BINDs on back-ldap sometimes create Start TLS failed errors on the > connection from proxy to backend, and the client of the proxy receives the error > 52. We had a look at the code, and it appears that in back-ldap/bind.c, the > timeout argument of the function ldap_back_start_tls always ends up as zero, so > the timeout tv.sec and tv.usec end up being set as 0 and 100000, respectively, > by the macro LDAP_BACK_TV_SET defined in back-ldap/back-ldap.h. This means that > the actual timeout waiting for starttls operation to complete will be 0.1 > seconds, which is sometimes too little if there are latencies. We did a > quick&dirty hack to just set tv.sec to 3 and tv.usec to 0, and were no longer > able to get any Start TLS failed errors on the proxy bind. > > We did find that the back-ldap has an "extended" timeout defined in > back-ldap/config.c - except that is commented out as "not implemented". Seems to > us that if that were implemented, you could set the timeout in your ldap > database section by something like "timeout extended=" and then you could have a > longer timeout. Thanks for the report. Note that if you simply set a "timeout" with no specific qualifiers, it would have worked for you as well. At this point, the "extended" timeout was meant for the other exops that back-ldap supports, e.g. whoami and passwordModify. Since 2.4 is feature frozen we are not going to implement the extended timeout feature here. Instead, I've changed the start_tls code to use the "bind" timeout, since it will always be used as part of a bind operation anyway. > > Steps to reproduce: > 1. Configure an openldap proxy-backend pair, the proxy using the "ldap" database > and using ldap urls to connect to the backend (using ldaps urls might actually > be a workaround). Configure the binds to be proxied to the backend, too. > 2. Create a script that creates multiple - say, 200 - simultaneous connections > to the proxy, using an ldaps url to connect to the proxy, and using the same DN > to bind as. (We haven't tried to connect to the frontend using ldap+starttls, > but I would suppose that would have a similar result, since the client > establishes TLS anyway before trying to bind to the proxy.) > 3. Observe some of the binds fail (we typically get 5-6 fails for 200 > simultaneous connection), with the client of the proxy getting a "Server is > unavailable: 52" type of error, and the proxy log containing lines such as > "conn=7716 op=0 RESULT tag=97 err=52 text=Start TLS failed" after the BIND from > the client. > > Possible workaround: Configure the proxy to use ldaps urls to the backend. > > Suggested fix: Implement the "extended" timeout type in back-ldap/config.c > timeout_table, so that people can set the StartTLS timeout to a larger value > than the default 0.1 seconds. > > References: discussion thread on the OpenLDAP-Technical mailing list. > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8968) Async connect mode does not work on Solaris
by quanah＠symas.com 01 Feb '19

01 Feb '19

--On Wednesday, January 30, 2019 11:19 PM +0000 vsmith(a)interlinknetworks.com wrote: > Full_Name: Vernon Smith > Version: 2.4.47 > OS: Solaris 10 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (2601:40d:4300:679a:c0c9:ced:d06f:39a4) > > > I have managed to upgrade to version 2.4.47 libldap libraries using the > Linux OS and async connect mode works fine. But when I tried to upgrade > to 2.4.47 using Solaris OS and use async connect mode, no connections > could be established to the ldap server. The libldap code did not even > send the ldap request to the ldap server. The ldap debug trace showed it > just freeing the request and returned an unable to connect. I traced the > issue to the routine ldap_int_flush_request which checks the sock_errno > return code for EAGAIN but Solaris 10 returns ENOTCONN. This error code > seems to mean the same thing as EAGAIN. I changed the code to test for > either return code and the async connect works for both Linux and Solaris > now. Here is may patch. Thanks for the report, this is now fixed in master (and will be a part of the next RE24 release). Regards, Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

Re: (ITS#8957)
by quanah＠symas.com 01 Feb '19

01 Feb '19

--On Wednesday, January 30, 2019 3:15 PM +0000 quanah(a)symas.com wrote: > --On Wednesday, January 30, 2019 1:44 AM -0500 Vernon Smith > <vsmith(a)interlinknetworks.com> wrote: > >> I Tried to upload it to the openldap ftp server but was not successful so >> I am trying the patch again here. Thanks for the report, this has been committed to git master (and will be part of the next RE24 release). Regards, Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

Re: (ITS#8959) duplicate symbols
by quanah＠symas.com 31 Jan '19

31 Jan '19

--On Thursday, January 31, 2019 9:00 AM +0000 ruga(a)protonmail.com wrote: > This is a multi-part message in MIME format. Lets try this a different way. No one is disputing you've found an issue. However, as has been noted repeatedly, there's no indication that the issue you have found is caused by the OpenLDAP software. As has been demonstrated numerous times in this thread, the software builds correctly without issue, using the flags you provided, with both gcc and clang, on multiple OSes and compiler verions. I.e., it would appear the problem surfaces with either your very specific build environment or the very specific set of compiler tools that you've installed. As you've noted, your particular environment is able to compile other software without hitting this issue. That's great. That however does not mean that the issue you're encountering is a bug with the OpenLDAP software. The fact that other compiler chain combinations do not hit this problem would appear to back this up. At this point, the onus is on you to clearly and unequivocally demonstrate the issue is specific to OpenLDAP as the issue more appears to be on the toolchain side. And that happens as well. I once hit a problem that was due to a bug in GCC's optimization strategies with values > -O2 (I.e., -O3, -O4, etc). It only occurred with one particular piece of software I was compiling out of several hundred software packages. And only with that particular version of GCC. But it was also clearly *not* the fault of the software. Regards, Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

Re: (ITS#8575) Argon2 Password hashing contrib module
by simon＠slevermann.de 31 Jan '19

31 Jan '19

Hi, I have essentially given up on doing this, because I no longer work for the employer that had me work on this, and at the time I did, I never got the thing done. The code itself worked when I tried it, but it has the caveat of not being configurable. I never quite found out how to properly implement configuration of a module that isn't an overlay, so I never got that done. Feel free to adjust the existing code, it should be adaptable to libsodium relatively easily. Cheers, Simon Am 31.01.2019 um 16:17 schrieb Raphael Geissert: > Hi, > > Is there any progress on a libsodium-based module? Or is there any > chance that the current version be integrated as-is ("one is better > than none")? > > Regards,

1 0

Re: (ITS#8575) Argon2 Password hashing contrib module
by atomo64＠gmail.com 31 Jan '19

31 Jan '19

Hi, Is there any progress on a libsodium-based module? Or is there any chance that the current version be integrated as-is ("one is better than none")? Regards, -- Raphael Geissert

1 0

Re: (ITS#8959) duplicate symbols
by quanah＠symas.com 31 Jan '19

31 Jan '19

--On Thursday, January 31, 2019 8:13 AM +0000 hyc(a)symas.com wrote: > Still, nothing in the OpenLDAP build setup does this. You did it yourself, > most likely the same way you set "-D_FORTIFY_SOURCE=2" 3 times in your > CPPFLAGS. That was actually me when I inaverdently exported the variable more than once when playing with the flags. ;) Just to note. --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

Re: (ITS#8959) duplicate symbols
by ruga＠protonmail.com 31 Jan '19

31 Jan '19

This is a multi-part message in MIME format. --b1_ce770359f41de57b3c63a57268b4224b Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: base64 WW91IGFkZGVkIGl0IHRocmVlIHRpbWVzLCBieSBoYW5kLCBhcyBldmlkZW50IGluIHlvdXIgZm9s bG93dXAgMTIuCgpZb3Ugc2hvdWxkIGZpeCB5b3VyIE1ha2VmaWxlIGluc3RlYWQgb2YgYmxhbWlu ZyBsbHZtIG9yIG15IHBlcmZlY3RseSBmaW5lIHNldHRpbmcuCgpPbiBUaHUsIEphbiAzMSwgMjAx OSBhdCAwOToxMywgSG93YXJkIENodSA8aHljQHN5bWFzLmNvbT4gd3JvdGU6Cgo+IHJ1Z2FAcHJv dG9ubWFpbC5jb20gd3JvdGU6Cj4+IFRoaXMgaXMgYSBtdWx0aS1wYXJ0IG1lc3NhZ2UgaW4gTUlN RSBmb3JtYXQuCj4KPiBQbGVhc2Ugc3RvcCB1c2luZyBIVE1MIGVtYWlsLgo+Pgo+PiAtLWIxX2Q4 NjM3ZGJiMTIwYTUyOWY0ZjI2ZDQzZDRiMDg5Y2VlCj4+IENvbnRlbnQtVHlwZTogdGV4dC9wbGFp bjsgY2hhcnNldD1VVEYtOAo+PiBDb250ZW50LVRyYW5zZmVyLUVuY29kaW5nOiBiYXNlNjQKPj4K Pj4gU1dZZ1NTQmhaR1FnZEdobElDSXRkaUlnYjNCMGFXOXVJSFJ2SUdOc1lXNW5JR2x1SUM0dmJH bGljbUZ5YVdWekwwMWhhMlZtYVd4bAo+Cj4+IElmIEkgYWRkIHRoZSAiLXYiIG9wdGlvbiB0byBj bGFuZyBpbiAuL2xpYnJhcmllcy9NYWtlZmlsZSwgdGhlIHJlc3VsdGluZyBsb2cgc2hvd3MgdGhh dCB0aGUgbGlicmFyeSB3YXMgbGlua2VkIHR3aWNlLCBjYXVzaW5nIHN5bWJvbCBkdXBsaWNhdGlv bi4+Cj4+IFRoaXMgaG9sZHMgb24gbWFjb3MgMTAuMTMgd2l0aCBBcHBsZSB0b29sa2l0IDEwLjE0 IGFuZCB0aGUgb3JpZ2luYWwgbGx2bSA3LjAuMCB3aXRoIHRoZSBnaXZlbiBzZXR0aW5ncy4KPgo+ IFN0aWxsLCBub3RoaW5nIGluIHRoZSBPcGVuTERBUCBidWlsZCBzZXR1cCBkb2VzIHRoaXMuIFlv dSBkaWQgaXQgeW91cnNlbGYsCj4gbW9zdCBsaWtlbHkgdGhlIHNhbWUgd2F5IHlvdSBzZXQgIi1E X0ZPUlRJRllfU09VUkNFPTIiIDMgdGltZXMgaW4geW91ciBDUFBGTEFHUy4KPgo+IFRoaXMgSVRT IGlzIGNsb3NlZC4gVGhlcmUgaXMgbm8gT3BlbkxEQVAgU29mdHdhcmUgaXNzdWUgaGVyZSwgeW91 J3ZlIG1pc2NvbmZpZ3VyZWQKPiB5b3VyIGJ1aWxkLgo+Pgo+PiBPbiBNb24sIEphbiAyOCwgMjAx OSBhdCAxNzoxMSwgUXVhbmFoIEdpYnNvbi1Nb3VudCA8cXVhbmFoQHN5bWFzLmNvbT4gd3JvdGU6 Cj4+Cj4+PiAtLU9uIE1vbmRheSwgSmFudWFyeSAyOCwgMjAxOSAzOjM1IFBNICswMDAwIFJ1cGVy dCBHYWxsYWdoZXIKPj4+IDxydWdhQHByb3Rvbm1haWwuY29tPiB3cm90ZToKPj4+Cj4+Pj4gQXMg eW91IGNhbiBzZWUsIEkgYW0gdXNpbmcgdGhlIGRlZmF1bHQgQXBwbGUgbGlua2VyLCBhbmQgaGFy ZGVuZWQKPj4+PiBzZXR0aW5ncyBmb3IgdGhlIGNvbXBpbGVyLiBUcnkgdXNpbmcgdGhvc2Ugc2V0 dGluZ3MgeW91cnNlbGYsIGFuZCBzZWUKPj4+PiB3aGF0IGNvbWVzIG91dC4KPj4+Cj4+PiBJIHRl c3RlZCBvbiBGcmVlQlNEMTIgKHNpbmNlIGl0IHVzZXMgY2xhbmcgcmF0aGVyIHRoYW4gZ2NjKSB3 aXRoIHRoZQo+Pj4gZm9sbG93aW5nIGZsYWdzIGFuZCB0aGUgYnVpbGQgY29tcGxldGVkIHcvbyBp c3N1ZToKPj4+Cj4+PiBDRkxBR1M9IC1nIC1mc3RhY2stcHJvdGVjdG9yLWFsbCAtZm5vLW9taXQt ZnJhbWUtcG9pbnRlciAtZlBJQyAtZlBJRSAtTzIKPj4+IC1XZm9ybWF0IC1XZm9ybWF0LXNlY3Vy aXR5IC1XZXJyb3I9Zm9ybWF0LXNlY3VyaXR5Cj4+PiBMREZMQUdTPSAtV2wsLXoscmVscm8sLXos bm93IC1waWUKPj4+IENQUEZMQUdTPSAtRF9GT1JUSUZZX1NPVVJDRT0yIC1EX0ZPUlRJRllfU09V UkNFPTIgLURfRk9SVElGWV9TT1VSQ0U9Mgo+Cj4gLS0KPiAtLSBIb3dhcmQgQ2h1Cj4gQ1RPLCBT eW1hcyBDb3JwLiBodHRwOi8vd3d3LnN5bWFzLmNvbQo+IERpcmVjdG9yLCBIaWdobGFuZCBTdW4g aHR0cDovL2hpZ2hsYW5kc3VuLmNvbS9oeWMvCj4gQ2hpZWYgQXJjaGl0ZWN0LCBPcGVuTERBUCBo dHRwOi8vd3d3Lm9wZW5sZGFwLm9yZy9wcm9qZWN0Lw== --b1_ce770359f41de57b3c63a57268b4224b Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: base64 ICAgPGRpdj5Zb3UgYWRkZWQgaXQgdGhyZWUgdGltZXMsIGJ5IGhhbmQsIGFzIGV2aWRlbnQgaW4g eW91ciBmb2xsb3d1cCAxMi48L2Rpdj48ZGl2Pjxicj48L2Rpdj48ZGl2PllvdSBzaG91bGQgZml4 IHlvdXIgTWFrZWZpbGUgaW5zdGVhZCBvZiBibGFtaW5nIGxsdm0gb3IgbXkgcGVyZmVjdGx5IGZp bmUgc2V0dGluZy48L2Rpdj48ZGl2Pjxicj48L2Rpdj48ZGl2Pjxicj48L2Rpdj48ZGl2Pjxicj48 L2Rpdj5PbiBUaHUsIEphbiAzMSwgMjAxOSBhdCAwOToxMywgSG93YXJkIENodSAmbHQ7PGEgaHJl Zj0ibWFpbHRvOmh5Y0BzeW1hcy5jb20iIGNsYXNzPSIiPmh5Y0BzeW1hcy5jb208L2E+Jmd0OyB3 cm90ZTo8YmxvY2txdW90ZSBjbGFzcz0icHJvdG9ubWFpbF9xdW90ZSIgdHlwZT0iY2l0ZSI+ICBy dWdhQHByb3Rvbm1haWwuY29tIHdyb3RlOjxicj4mZ3Q7IFRoaXMgaXMgYSBtdWx0aS1wYXJ0IG1l c3NhZ2UgaW4gTUlNRSBmb3JtYXQuPGJyPjxicj5QbGVhc2Ugc3RvcCB1c2luZyBIVE1MIGVtYWls Ljxicj4mZ3Q7PGJyPiZndDsgLS1iMV9kODYzN2RiYjEyMGE1MjlmNGYyNmQ0M2Q0YjA4OWNlZTxi cj4mZ3Q7IENvbnRlbnQtVHlwZTogdGV4dC9wbGFpbjsgY2hhcnNldD1VVEYtODxicj4mZ3Q7IENv bnRlbnQtVHJhbnNmZXItRW5jb2Rpbmc6IGJhc2U2NDxicj4mZ3Q7PGJyPiZndDsgU1dZZ1NTQmha R1FnZEdobElDSXRkaUlnYjNCMGFXOXVJSFJ2SUdOc1lXNW5JR2x1SUM0dmJHbGljbUZ5YVdWekww MWhhMlZtYVd4bDxicj48YnI+Jmd0OyBJZiBJIGFkZCB0aGUgIi12IiBvcHRpb24gdG8gY2xhbmcg aW4gLi9saWJyYXJpZXMvTWFrZWZpbGUsIHRoZSByZXN1bHRpbmcgbG9nIHNob3dzIHRoYXQgdGhl IGxpYnJhcnkgd2FzIGxpbmtlZCB0d2ljZSwgY2F1c2luZyBzeW1ib2wgZHVwbGljYXRpb24uJmd0 Ozxicj4mZ3Q7IFRoaXMgaG9sZHMgb24gbWFjb3MgMTAuMTMgd2l0aCBBcHBsZSB0b29sa2l0IDEw LjE0IGFuZCB0aGUgb3JpZ2luYWwgbGx2bSA3LjAuMCB3aXRoIHRoZSBnaXZlbiBzZXR0aW5ncy48 YnI+PGJyPlN0aWxsLCBub3RoaW5nIGluIHRoZSBPcGVuTERBUCBidWlsZCBzZXR1cCBkb2VzIHRo aXMuIFlvdSBkaWQgaXQgeW91cnNlbGYsPGJyPm1vc3QgbGlrZWx5IHRoZSBzYW1lIHdheSB5b3Ug c2V0ICItRF9GT1JUSUZZX1NPVVJDRT0yIiAzIHRpbWVzIGluIHlvdXIgQ1BQRkxBR1MuPGJyPjxi cj5UaGlzIElUUyBpcyBjbG9zZWQuIFRoZXJlIGlzIG5vIE9wZW5MREFQIFNvZnR3YXJlIGlzc3Vl IGhlcmUsIHlvdSd2ZSBtaXNjb25maWd1cmVkPGJyPnlvdXIgYnVpbGQuPGJyPiZndDs8YnI+Jmd0 OyBPbiBNb24sIEphbiAyOCwgMjAxOSBhdCAxNzoxMSwgUXVhbmFoIEdpYnNvbi1Nb3VudCAmbHQ7 cXVhbmFoQHN5bWFzLmNvbSZndDsgd3JvdGU6PGJyPiZndDs8YnI+Jmd0OyZndDsgLS1PbiBNb25k YXksIEphbnVhcnkgMjgsIDIwMTkgMzozNSBQTSArMDAwMCBSdXBlcnQgR2FsbGFnaGVyPGJyPiZn dDsmZ3Q7ICZsdDtydWdhQHByb3Rvbm1haWwuY29tJmd0OyB3cm90ZTo8YnI+Jmd0OyZndDs8YnI+ Jmd0OyZndDsmZ3Q7IEFzIHlvdSBjYW4gc2VlLCBJIGFtIHVzaW5nIHRoZSBkZWZhdWx0IEFwcGxl IGxpbmtlciwgYW5kIGhhcmRlbmVkPGJyPiZndDsmZ3Q7Jmd0OyBzZXR0aW5ncyBmb3IgdGhlIGNv bXBpbGVyLiBUcnkgdXNpbmcgdGhvc2Ugc2V0dGluZ3MgeW91cnNlbGYsIGFuZCBzZWU8YnI+Jmd0 OyZndDsmZ3Q7IHdoYXQgY29tZXMgb3V0Ljxicj4mZ3Q7Jmd0Ozxicj4mZ3Q7Jmd0OyBJIHRlc3Rl ZCBvbiBGcmVlQlNEMTIgKHNpbmNlIGl0IHVzZXMgY2xhbmcgcmF0aGVyIHRoYW4gZ2NjKSB3aXRo IHRoZTxicj4mZ3Q7Jmd0OyBmb2xsb3dpbmcgZmxhZ3MgYW5kIHRoZSBidWlsZCBjb21wbGV0ZWQg dy9vIGlzc3VlOjxicj4mZ3Q7Jmd0Ozxicj4mZ3Q7Jmd0OyBDRkxBR1M9IC1nIC1mc3RhY2stcHJv dGVjdG9yLWFsbCAtZm5vLW9taXQtZnJhbWUtcG9pbnRlciAtZlBJQyAtZlBJRSAtTzI8YnI+Jmd0 OyZndDsgLVdmb3JtYXQgLVdmb3JtYXQtc2VjdXJpdHkgLVdlcnJvcj1mb3JtYXQtc2VjdXJpdHk8 YnI+Jmd0OyZndDsgTERGTEFHUz0gLVdsLC16LHJlbHJvLC16LG5vdyAtcGllPGJyPiZndDsmZ3Q7 IENQUEZMQUdTPSAtRF9GT1JUSUZZX1NPVVJDRT0yIC1EX0ZPUlRJRllfU09VUkNFPTIgLURfRk9S VElGWV9TT1VSQ0U9Mjxicj48YnI+PGJyPi0tPGJyPiAgLS0gSG93YXJkIENodTxicj4gIENUTywg U3ltYXMgQ29ycC4gICAgICAgICAgIGh0dHA6Ly93d3cuc3ltYXMuY29tPGJyPiAgRGlyZWN0b3Is IEhpZ2hsYW5kIFN1biAgICAgaHR0cDovL2hpZ2hsYW5kc3VuLmNvbS9oeWMvPGJyPiAgQ2hpZWYg QXJjaGl0ZWN0LCBPcGVuTERBUCAgaHR0cDovL3d3dy5vcGVubGRhcC5vcmcvcHJvamVjdC88YnI+ PC9ibG9ja3F1b3RlPjxkaXY+PGJyPjwvZGl2PjxkaXY+PGJyPjwvZGl2Pg== --b1_ce770359f41de57b3c63a57268b4224b--

1 0

Re: (ITS#8959) duplicate symbols
by hyc＠symas.com 31 Jan '19

31 Jan '19

ruga(a)protonmail.com wrote: > This is a multi-part message in MIME format. Please stop using HTML email. > > --b1_d8637dbb120a529f4f26d43d4b089cee > Content-Type: text/plain; charset=UTF-8 > Content-Transfer-Encoding: base64 > > SWYgSSBhZGQgdGhlICItdiIgb3B0aW9uIHRvIGNsYW5nIGluIC4vbGlicmFyaWVzL01ha2VmaWxl > If I add the "-v" option to clang in ./libraries/Makefile, the resulting log shows that the library was linked twice, causing symbol duplication.> > This holds on macos 10.13 with Apple toolkit 10.14 and the original llvm 7.0.0 with the given settings. Still, nothing in the OpenLDAP build setup does this. You did it yourself, most likely the same way you set "-D_FORTIFY_SOURCE=2" 3 times in your CPPFLAGS. This ITS is closed. There is no OpenLDAP Software issue here, you've misconfigured your build. > > On Mon, Jan 28, 2019 at 17:11, Quanah Gibson-Mount <quanah(a)symas.com> wrote: > >> --On Monday, January 28, 2019 3:35 PM +0000 Rupert Gallagher >> <ruga(a)protonmail.com> wrote: >> >>> As you can see, I am using the default Apple linker, and hardened >>> settings for the compiler. Try using those settings yourself, and see >>> what comes out. >> >> I tested on FreeBSD12 (since it uses clang rather than gcc) with the >> following flags and the build completed w/o issue: >> >> CFLAGS= -g -fstack-protector-all -fno-omit-frame-pointer -fPIC -fPIE -O2 >> -Wformat -Wformat-security -Werror=format-security >> LDFLAGS= -Wl,-z,relro,-z,now -pie >> CPPFLAGS= -D_FORTIFY_SOURCE=2 -D_FORTIFY_SOURCE=2 -D_FORTIFY_SOURCE=2 -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs