openldap-bugs March 2016

openldap-bugs@openldap.org

20 participants
38 discussions

Re: (ITS#8385) Use After Free of struct ldap_common in slap_client_connect
by hyc＠symas.com 12 Mar '16

12 Mar '16

mkp37215(a)gmail.com wrote: > Full_Name: Maciej Puzio > Version: 2.4.44 and current sources from git repo > OS: Ubuntu 15.10 amd64 > URL: > Submission from: (NULL) (129.112.109.41) > > > While configuring two servers in a dual-master setup I encountered an issue > causing replication failures, when replication is done on a TLS connection. In > this particular setup the issue manifested itself by replication retries > failing, even when the cause of the original failure no longer existed. For > example, let's consider that dual-master setup consists of servers A and B. If > server A was started before server B, its on-startup replication would fail (as > expected, since B is down). Server B would then be started and its replication > attempt would succeed (since A is already running). However A's subsequent > retries would fail, even though B was now accessible. If I then restarted server > A, its replication will succeed, but B will now lose connection to A and all B's > subsequent retries would fail. This issue has ~50% reproducibility, but if it > occurs once, then it would continue failing. On occasion both servers would > replicate fine, but obviously the whole situation was unacceptable on servers > being prepared for production use. > > I tested this issue thoroughly eliminating various hypotheses, and ended up > debugging OpenLDAP code and finding a bug causing the above behavior. In short, > on failure slap_client_connect deallocates LDAP object ld and objects pointed by > it, including ldap_common which contains ld_options. Pointer to ld_options is > held in sb->sb_tls_ctx and reused in subsequent slap_client_connect calls, at > this point referencing freed memory. > > This issue occurs in OpenLDAP newest stable version 2.4.44, as well as in > current source code from git repo (as of March 11, 2016), obtained from > git://git.openldap.org/openldap.git > Here, instead of freeing ld->ldc (struct ldap_common), I overwrite its contents > with a magic debug value 0xdeadbeef. I overwrite ldo_tls_require_cert, as an > incorrect value of this particular variable turned out to be the direct cause of > replication failures mentioned at the beginning (more about it later). However, > other fields of ldap_common could be used as well. > > File ldap-int.h is included to be able to examine contents of *ld, while the > declaration of struct tlsg_ctx has been copied verbatim from > libraries/libldap/tls_g.c, in order to examine the contents of > ld->ld_options.ldo_tls_ctx. I use GnuTLS library following Ubuntu build options. > But the issue in question is not GnuTLS-related, though it may cause different > symptoms with OpenSSL or Mozilla NSS (I did not test these options). Thanks for the report. Inspection shows that the issue only exists in libldap's GnuTLS support. As always, the Project recommends you use OpenSSL. It also looks like only the ldo_tls_require_cert field is being used so we can probably just copy that flag instead of keeping a pointer to the ldap options structure. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#8385) Use After Free of struct ldap_common in slap_client_connect
by mkp37215＠gmail.com 11 Mar '16

11 Mar '16

Full_Name: Maciej Puzio Version: 2.4.44 and current sources from git repo OS: Ubuntu 15.10 amd64 URL: Submission from: (NULL) (129.112.109.41) While configuring two servers in a dual-master setup I encountered an issue causing replication failures, when replication is done on a TLS connection. In this particular setup the issue manifested itself by replication retries failing, even when the cause of the original failure no longer existed. For example, let's consider that dual-master setup consists of servers A and B. If server A was started before server B, its on-startup replication would fail (as expected, since B is down). Server B would then be started and its replication attempt would succeed (since A is already running). However A's subsequent retries would fail, even though B was now accessible. If I then restarted server A, its replication will succeed, but B will now lose connection to A and all B's subsequent retries would fail. This issue has ~50% reproducibility, but if it occurs once, then it would continue failing. On occasion both servers would replicate fine, but obviously the whole situation was unacceptable on servers being prepared for production use. I tested this issue thoroughly eliminating various hypotheses, and ended up debugging OpenLDAP code and finding a bug causing the above behavior. In short, on failure slap_client_connect deallocates LDAP object ld and objects pointed by it, including ldap_common which contains ld_options. Pointer to ld_options is held in sb->sb_tls_ctx and reused in subsequent slap_client_connect calls, at this point referencing freed memory. This issue occurs in OpenLDAP newest stable version 2.4.44, as well as in current source code from git repo (as of March 11, 2016), obtained from git://git.openldap.org/openldap.git In order to demonstrate that the issue is indeed use-after-free, I made the following debug modifications to the code. (The diffs are against the source code from git repo as of 2016/03/11) --- servers/slapd/config.c.orig 2016-03-11 14:52:14.000000000 -0600 +++ servers/slapd/config.c 2016-03-11 17:20:55.216170535 -0600 @@ -60,6 +60,26 @@ #define ARGS_STEP 512 + +/* DEBUG */ +#include <syslog.h> +#include <gnutls/gnutls.h> +#include "../../libraries/libldap/ldap-int.h" + +typedef struct tlsg_ctx { + struct ldapoptions *lo; + gnutls_certificate_credentials_t cred; + gnutls_dh_params_t dh_params; + unsigned long verify_depth; + int refcount; + gnutls_priority_t prios; +#ifdef LDAP_R_COMPILE + ldap_pvt_thread_mutex_t ref_mutex; +#endif +} tlsg_ctx; +/* END DEBUG */ + + /* * defaults for various global variables */ @@ -2001,9 +2021,11 @@ #ifdef HAVE_TLS if ( sb->sb_tls_do_init ) { + syslog(LOG_DEBUG, "DEBUG: slap_client_connect sb->sb_tls_do_init branch\n"); rc = bindconf_tls_set( sb, ld ); } else if ( sb->sb_tls_ctx ) { + syslog(LOG_DEBUG, "DEBUG: slap_client_connect sb->sb_tls_ctx branch\n"); rc = ldap_set_option( ld, LDAP_OPT_X_TLS_CTX, sb->sb_tls_ctx ); } @@ -2015,6 +2037,7 @@ sb->sb_uri.bv_val, rc, 0 ); goto done; } + syslog(LOG_DEBUG, "DEBUG: slap_client_connect ldo_tls_require_cert=%X\n", ((tlsg_x*x*)ld->ld_options.ldo_tls_ctx)->lo->ldo_tls_require_cert); #endif --- libraries/libldap/unbind.c.orig 2016-03-11 14:52:14.000000000 -0600 +++ libraries/libldap/unbind.c 2016-03-11 15:49:18.603447766 -0600 @@ -27,6 +27,8 @@ #include "ldap-int.h" +#include <syslog.h> /* DEBUG */ + /* An Unbind Request looks like this: * * UnbindRequest ::= [APPLICATION 2] NULL @@ -208,6 +210,8 @@ #ifdef HAVE_TLS ldap_int_tls_destroy( &ld->ld_options ); + syslog(LOG_DEBUG, "DEBUG: ldap_ld_destroy\n"); + ld->ld_options.ldo_tls_require_cert = 0xdeadbeef; /* DEBUG */ #endif if ( ld->ld_options.ldo_sctrls != NULL ) { @@ -233,7 +237,7 @@ #ifndef NDEBUG LDAP_TRASH(ld); #endif - LDAP_FREE( (char *) ld->ldc ); + //LDAP_FREE( (char *) ld->ldc ); /* DEBUG */ LDAP_FREE( (char *) ld ); return( err ); Here, instead of freeing ld->ldc (struct ldap_common), I overwrite its contents with a magic debug value 0xdeadbeef. I overwrite ldo_tls_require_cert, as an incorrect value of this particular variable turned out to be the direct cause of replication failures mentioned at the beginning (more about it later). However, other fields of ldap_common could be used as well. File ldap-int.h is included to be able to examine contents of *ld, while the declaration of struct tlsg_ctx has been copied verbatim from libraries/libldap/tls_g.c, in order to examine the contents of ld->ld_options.ldo_tls_ctx. I use GnuTLS library following Ubuntu build options. But the issue in question is not GnuTLS-related, though it may cause different symptoms with OpenSSL or Mozilla NSS (I did not test these options). One would expect that the only effect of these modifications would be a mere memory leak, but in addition to that we see that the magic value gets logged. As you can see, the second replication attempts uses memory that, if executing non-debug code, would be already freed: The results are as follows (only one server started): Mar 11 17:42:21 srvA slapd[14360]: slapd starting Mar 11 17:42:21 srvA slapd[14360]: =>do_syncrepl rid=001 Mar 11 17:42:21 srvA slapd[14360]: DEBUG: slap_client_connect sb->sb_tls_do_init branch Mar 11 17:42:21 srvA slapd[14355]: ...done. Mar 11 17:42:21 srvA slapd[14360]: DEBUG: slap_client_connect ldo_tls_require_cert=0 Mar 11 17:42:21 srvA slapd[14360]: DEBUG: ldap_ld_destroy Mar 11 17:42:21 srvA slapd[14360]: do_syncrepl: rid=001 rc -1 retrying (9 retries left) Mar 11 17:42:51 srvA slapd[14360]: =>do_syncrepl rid=001 Mar 11 17:42:51 srvA slapd[14360]: DEBUG: slap_client_connect sb->sb_tls_ctx branch Mar 11 17:42:51 srvA slapd[14360]: DEBUG: slap_client_connect ldo_tls_require_cert=DEADBEEF Mar 11 17:42:51 srvA slapd[14360]: DEBUG: ldap_ld_destroy Mar 11 17:42:51 srvA slapd[14360]: do_syncrepl: rid=001 rc -1 retrying (8 retries left) When running unmodified non-debug code, on replication retry ld_options contains either (A) previous values, (B) zeros or (C) random values. Occasionally a segfault occurs (although I observed them only during debugging). Replication fails much more readily if, in addition to the issue discussed here, there is a TLS certificate problem. In this case connection tends to fail in tlsg_cert_verify, even if tls_reqcert is set to 'allow' or 'never'. This is because random value of ldo_tls_require_cert will cause the certificate to be checked regardless of tls_reqcert setting. If certificates are correct, the issue may well remain hidden, as TLS connection will be established regardless of whether A, B or C occurs. In this case an occasional segfault may be the only symptom. Of course, if by chance B occurs, certificate verification will be silently skipped and this may lead to a security problem. A quick and dirty fix is to comment out few lines in servers/slapd/config.c so that "rc = bindconf_tls_set( sb, ld )" is always executed, which avoids the reuse of sb->sb_tls_ctx. However, I think a better fix would involve changing the handling of ld->ldoptions and related objects. Given the complexity of interactions between various objects involved, and my limited familiarity with OpenLDAP design, I feel that I am not in position to propose a patch. For reference, OpenLDAP was build with this configuration: ./configure --prefix=/opt/openldap --enable-debug --enable-dynamic --enable-syslog --enable-local --enable-slapd --enable-crypt --enable-modules --enable-backends --enable-ndb=no --enable-shell=no --enable-perl=no --enable-sql=no --enable-wt=no --enable-overlays --with-tls=gnutls CPPFLAGS="-Wno-format-extra-args" The relevant parts of slapd.conf are [redacted to anonymize]: loglevel 1 serverID 001 moduleload syncprov TLSCipherSuite SECURE256:-VERS-SSL3.0 TLSCACertificateFile /etc/ldap/ssl/ca.pem TLSCertificateFile /etc/ldap/ssl/srvA.pem TLSCertificateKeyFile /etc/ldap/ssl/srvA.key syncrepl rid=001 provider=ldaps://10.0.0.2 type=refreshAndPersist retry="30 10 300 +" searchbase="dc=test" attrs="*,+" bindmethod=simple binddn="cn=root,dc=test" credentials="plaintext-password" tls_reqcert=never keepalive="240:5:10" mirrormode TRUE overlay syncprov syncprov-checkpoint 10 1440 Test machine hardware: CPU: Intel Atom C2558 @ 2.40GHz Motherboard: Supermicro A1SRM-2558F NIC: Intel Corporation Ethernet Connection I354 (rev 03) Software used: OS: Ubuntu 15.10 Kernel: 4.2.0-30-generic #36-Ubuntu SMP x86_64 OpenLDAP: slapd 2.4.44 and slapd 2.X (from git repo) GnuTLS: 3.3.15-5ubuntu2 Please let me know if you need further information. Thank you. Maciej Puzio

1 0

Re: (ITS#8384) LDAP Servers in Multi-master configuration locked up and unresponsive
by hyc＠symas.com 11 Mar '16

11 Mar '16

mhonek(a)redhat.com wrote: > Hello, > > this seems to be a RHEL/CentOS-specific issue. Please contact Red Hat Support. > I think this ITS ticket may be closed. Sounds good, thanks. If it's not RHEL/CentOS-specific, we can re-open it but investigating would require updating from 2.4.39 to current. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8384) LDAP Servers in Multi-master configuration locked up and unresponsive
by mhonek＠redhat.com 11 Mar '16

11 Mar '16

Hello, this seems to be a RHEL/CentOS-specific issue. Please contact Red Hat Support. I think this ITS ticket may be closed. Regards. On 03/11/2016 04:01 PM, kevin4sullivan(a)gmail.com wrote: > Full_Name: Kevin Sullivan > Version: 2.4.39-8 > OS: RHEL 6.4 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (38.127.164.138) > > > Setup: > > I have two OpenLDAP servers running on separate hosts and replicating via delta > syncrepl in mirrormode. > > Problem: > > On rare occasions when one of my servers comes online, both servers will lock up > and be unresponsive. Any query to either server will time out. The servers will > stay in this state indefinitely. > > Workaround: > > Killing either of the servers will resolve this problem. > > Observations: > > - I don't know how to reproduce this reliably. > - Both servers have a thread that is stuck in a do_syncrep1() call. > - Netstat shows that each server has unacknowledged data in one of their > socket's receive queue. > > > GDB on host1: > > (gdb) info threads > 18 Thread 0x7f8b7b80d700 (LWP 31620) 0x00007f8ba7fa6f03 in epoll_wait () from > /lib64/libc.so.6 > 17 Thread 0x7f8b7b00c700 (LWP 31621) 0x00007f8ba846a054 in __lll_lock_wait () > from /lib64/libpthread.so.0 > 16 Thread 0x7f8b7a80b700 (LWP 31622) 0x00007f8ba846a054 in __lll_lock_wait () > from /lib64/libpthread.so.0 > 15 Thread 0x7f8b7a00a700 (LWP 31623) 0x00007f8ba846a054 in __lll_lock_wait () > from /lib64/libpthread.so.0 > 14 Thread 0x7f8b79809700 (LWP 31625) 0x00007f8ba846a54d in read () from > /lib64/libpthread.so.0 > 13 Thread 0x7f8b79008700 (LWP 31626) 0x00007f8ba846a054 in __lll_lock_wait () > from /lib64/libpthread.so.0 > 12 Thread 0x7f8b59ffd700 (LWP 25459) 0x00007f8ba846a054 in __lll_lock_wait () > from /lib64/libpthread.so.0 > 11 Thread 0x7f8b597fc700 (LWP 10195) 0x00007f8ba846a054 in __lll_lock_wait () > from /lib64/libpthread.so.0 > 10 Thread 0x7f8b70b37700 (LWP 28996) 0x00007f8ba846a054 in __lll_lock_wait () > from /lib64/libpthread.so.0 > 9 Thread 0x7f8b58ffb700 (LWP 8763) 0x00007f8ba846a054 in __lll_lock_wait () > from /lib64/libpthread.so.0 > 8 Thread 0x7f8b4bfff700 (LWP 28614) 0x00007f8ba846a054 in __lll_lock_wait () > from /lib64/libpthread.so.0 > 7 Thread 0x7f8b4b7fe700 (LWP 23367) 0x00007f8ba846a054 in __lll_lock_wait () > from /lib64/libpthread.so.0 > 6 Thread 0x7f8b4affd700 (LWP 13359) 0x00007f8ba846a054 in __lll_lock_wait () > from /lib64/libpthread.so.0 > 5 Thread 0x7f8b4a7fc700 (LWP 29064) 0x00007f8ba846a054 in __lll_lock_wait () > from /lib64/libpthread.so.0 > 4 Thread 0x7f8b49ffb700 (LWP 2184) 0x00007f8ba846a054 in __lll_lock_wait () > from /lib64/libpthread.so.0 > 3 Thread 0x7f8b497fa700 (LWP 31175) 0x00007f8ba846a054 in __lll_lock_wait () > from /lib64/libpthread.so.0 > 2 Thread 0x7f8b48ff9700 (LWP 2316) 0x00007f8ba846a054 in __lll_lock_wait () > from /lib64/libpthread.so.0 > * 1 Thread 0x7f8baa8db700 (LWP 31615) 0x00007f8ba84640ad in pthread_join () > from /lib64/libpthread.so.0 > (gdb) bt > #0 0x00007f8ba84640ad in pthread_join () from /lib64/libpthread.so.0 > #1 0x00007f8baa93c969 in slapd_daemon () at > ../../%./servers/slapd/daemon.c:2929 > #2 0x00007f8baa927733 in main (argc=6, argv=<value optimized out>) at > ../../../servers/slapd/main.c:1012 > (gdb) thread 2 > [Switching to thread 2 (Thread 0x7f8b48ff9700 (LWP 2316))]#0 0x00007f8ba846a054 > in __lll_lock_wait (929 from /lib64/libpthread.so.0 > (gdb) bt > #0 0x00007f8ba846a054 in __lll_lock_wait () from /lib64/libpthread.so.0 > #1 0x00007f8ba8465388 in _L_lock_854 () from /lib64/libpthread.so.0 > #2 0x00007f8ba8465257 in pthread_mutex_lock () from /lib64/libpthread.so.0 > #3 0x00007f8baa4b96a0 in tlsm_session_accept_or_connect > (session=0x7f8b2c104110, is_accept=<value optimized out>) at tls_m.c:2656 > #4 0x00007f8baa4b8787 in ldap_pvt_tls_accept (sb=0x7f8b2c104150, > ctx_arg=0x7f8bac6f8230) at tls2.c:425 > #5 0x00007f8baa943f23 in connection_read (ctx=0x7f8b48ff8b70, argv=0x2f) at > ../../../servers/slapd/connection.c:1372 > #6 connection_read_thread (ctx=0x7f8b48ff8b70, argv=0x2f) at > ../../../servers/slapd/connection.c:1284 > #7 0x00007f8baa490a98 in ldap_int_thread_pool_wrapper (xpool=0x7f8bac5c8fb0) at > ../../../libraries/libldap_r/tpool.c:688 > #8 0x00007f8ba8463851 in start_thread () from /lib64/libpthread.so.0 > #9 0x00007f8ba7fa690d in clone () from /lib64/libc.so.6 > (gdb) thread 14 > [Switching to thread 14 (Thread 0x7f8b79809700 (LWP 31625))]#0 > 0x00007f8ba846a54d in read () from /lib64/libpthread.so.0 > (gdb) bt > #0 0x00007f8ba846a54d in read () from /lib64/libpthread.so.0 > #1 0x00007f8baa278b1c in sb_debug_read (sbiod=0x7f8b681def80, > buf=0x7f8b6819c830, len=5) at ../../../libraries/liblber/sockbuf.c:829 > #2 0x00007f8baa4b9984 in tlsm_PR_Recv (fd=<value optimized out>, > buf=0x7f8b6819c830, len=5, flags=<value optimized out>, timeout=<value optimized > out>) at tls_m.c:3007 > #3 0x00007f8ba946c1ed in ?? () from /usr/lib64/libssl3.so > #4 0000007f8ba9467480 in ?? () from /usr/lib64/libssl3.so > #5 0x00007f8ba9469ed2 in ?? () from /usr/lib64/libssl3.so > #6 0x00007f8ba9470135 in ?? () from /usr/lib64/libssl3.so > #7 0x00007f8ba947196f in SSL_ForceHandshake () from /usr/lib64/libssl3o%o > #8 0x00007f8baa4b96a8 in tlsm_session_accept_or_connect > (session=0x7f8b681b0f20, is_accept=<value optimized out>) at tls_m.c:2658 > #9 0x00007f8baa4b82e2 in ldap_int_tls_connect (ld=0x7f8b681d9250, conn=<value > optimized out>) at tls2.c:362 > #10 0x00007f8baa4b857inin ldap_int_tls_start (ld=0x7f8b681d9250, > conn=0x7f8b681b0ba0, srv=<value optimized out>) at tls2.c:860 > #11 0x00007f8baa4b86ce in ldap_start_tls_s (ld=0x7f8b681d9250, serverctrls=0x0, > clientctrls=0x0) at tls2.c:1040 > #12 0x00007f8baa937ff6 in slap_client_connect (ldp=0x7f8bac6d35a8, > sb=0x7f8bac6d3380) at ../../../servers/slapd/config.c:2012 > #13 0x00007f8baa9ae3f5 in do_syncrep1 (ctx=<value optimized out>, > arg=0x7f8bac6cc430) at ../../../servers/slapd/syncrepl.c:613 > #14 do_syncrepl (ctx=<value optimized out>, arg=0x7f8bac6cc430) at > ../../../servers/slapd/syncrepl.c:1527 > #15 0x00007f8baa490a98 in ldap_int_thread_pool_wrapper (xpool=0x7f8bac5c8fb0) at > ../../../libraries/libldap_r/tpool.c:688 > #16 0x00007f8ba8463851 in start_thread () from /lib64/libpthread.so.0 > #17 0x00007f8ba7fa690d in clone () from /lib64/libc.so.6 > (gdb) thread 18 > [Switching to thread 18 (Thread 0x7f8b7b80d700 (LWP 31620))]#0 > 0x00007f8ba7fa6f03 in epoll_wait () from /lib64/libc.so.6 > (gdb) bt > #0 0x00007f8ba7fa6f03 in epoll_wait () from /lib64/libc.so.6 > #1 0x00007f8baa93d9e2 in slapd_daemon_task (ptr=<value optimized out>) at > ../../../servers/slapd/daemon.c:2536 > #2 0x00007f8ba8463851 in start_thread () from /lib64/libpthread.so.0 > #3 0x00007f8ba7fa690d in clone () from /lib64/libc.so.6 > > > GDB on host2: > > (gdb) info threads > 7 Thread 0x7f5c942d5700 (LWP 3869) 0x00007f5cc0797f03 in epoll_wait () from > /lib64/libc.so.6 > 6 Thread 0x7f5c93ad4700 (LWP 3870) 0x00007f5cc0c5b54d in read () from > /lib64/libpthread.so.0 > 5 Thread 0x7f5c932d3700 (LWP 3871) 0x00007f5cc0c5b054 in __lll_lock_wait () > from /lib64/libpthread.so.0 > 4 Thread 0x7f5c92ad2700 (LWP 3872) 0x00007f5cc0c5843c in > pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 > 3 Thread 0x7f5c922d1700 (LWP 3873) 0x00007f5cc0c5843c in > pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 > 2 Thread 0x7f5c91ad0700 (LWP 3874) 0x00007f5cc0c5843c in > pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 > * 1 Thread 0x7f5cc30cc700 (LWP 3868) 0x00007f5cc0c550ad in pthread_join () from > /lib64/libpthread.so.0 > (gdb) bt > #0 0x00007f5cc0c550ad in pthread_join () from /lib64/libpthread.so.0 > #1 0x00007f5cc312d969 in slapd_daemon () at > ../../../servers/slapd/daemon.c:2929 > #2 0x00007f5cc3118733 in main (argc=6, argv=<value optimized out>) at > ../../../servers/slapd/main.c:1012 > (gdb) thread 2 > [Switching to thread 2 (Thread 0x7f5c91ad0700 (LWP 3874))]#0 0x00007f5cc0c5843c > in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 > (gdb) bt > #0 0x00007f5cc0c5843c in pthread_cond_wait@@GLIBC_2.3.2 () from > /lib64/libpthread.so.0 > #1 0x00007f5cc2c81af5 in ldap_int_thread_pool_wrapper (xpool=0x7f5cc3d18030) at > ../../../libraries/libldap_r/tpool.c:675 > #2 0x00007f5cc0c54851 in start_thread () from /lib64/libpthread.so.0 > #3 0x00007f5cc079790d in clone () from /lib64/libc.so.6 > (gdb) thread 5 > [Switching to thread 5 (Thread 0x7f5c932d3700 (LWP 3871))]#0 0x00007f5cc0c5b054 > in __lll_lock_wait () from /lib64/libpthread.so.0 > (gdb) bt > #0 0x00007f5cc0c5b054 in __lll_lock_wait () from /lib64/libpthread.so.0 > #1 0x00007f5cc0c56388 in _L_lock_854 () from /lib64/libpthread.so.0 > #2 0x00007f5cc0c56257 in pthread_mutex_lock () from /lib64/libpthread.so.0 > #3 0x00007f5cc2caa6a0 in tlsm_session_accept_or_connect > (session=0x7f5c841165c0, is_accept=<value optimized out>) at tls_m.c:2656 > #4 0x00007f5cc2ca9787 in ldap_pvt_tls_accept (sb=0x7f5c800008c0, > ctx_arg=0x7f5cc3e3b240) at tls2.c:425 > #5 0x00007f5cc3134f23 in connection_read (ctx=0x7f5c932d2b70, argv=0x10) at > ../../../servers/slapd/connection.c:1372 > #6 connection_read_thread (ctx=0x7f5c932d2b70, argv=0x10) at > ../../../servers/slapd/connection.c:1284 > #7 0x00007f5cc2c81a98 in ldap_int_thread_pool_wrapper (xpool=0x7f5cc3d18030) at > ../../../libraries/libldap_r%tptpool.c:688 > #8 0x00007f5cc0c54851 in start_thread () from /lib64/libpthread.so.0 > #9 0x00007f5cc079790d in clone () from /lib64/libc.so.6 > (gdb) thread 6 > [Switching to thread 6 (Thread 0x7f5c93ad4700 (LWP 3870))]#0 0x00007f5cc0c5b54d > in read () from /lib64/libpthread.so.0 > (gdb) bt > #0 0x00007f5cc0c5b54d in read () from /lib64/libpthread.so.0 > #1 0x00007f5cc2a69b1c in sb_debug_read (sbiod=0x7f5c7c185e10, > buf=0x7f5c7c18a7f0, len=5) at ../../../libraries/liblber/sockbuf.c:829 > #2 0x00007f5cc2caa984 in tlsm_PR_Recv (fd=<value optimized out>, > buf=0x7f5c7c18a7f0, len=5, flags=<value optimized out>, timeout=<value optimized > out>) at tls_m.c:3007 > #3 0x00007f5cc1c5d1ed in ?? () from /usr/lib64/libssl3.so > #4 0x00007f5cc1c58480 in ?? () from /usr/lib64/libssl3.so > #5 0x00007f5cc1c5aed2 in ?? () from /usr/lib64/libssl3.so > #6 0x00007f5cc1c61135 in ?? () from /usr/lib64/libssl3.so > #7 0x00007f5cc1c6296f in SSL_ForceHandshake () from /usr/lib64/libssl3.so > #8 0x00007f5cc2caa6a8 in tlsm_session_accept_or_connect > (session=0x7f5c7c182100, is_accept=<value optimized out>) at tls_m.c:2658 > #9 0x00007f5cc2ca92e2 in ldap_int_tls_connect (ld=0x7f5c7c100910, conn=<value > optimized out>) at tls2.c:362 > #10 0x00007f5cc2ca9574 in ldap_int_tls_start (ld=0x7f5c7c100910, > conn=0x7f5c7c10a020, srv=<value optimized out>) at tls2.c:860 > #11 0x00007f5cc2ca96ce in ldap_start_tls_s (ld=0x7f5c7c100910, serverctrls=0x0, > clientctrls=0x0) at tls2.c:1040 > #12 0x00007f5cc3128ff6 in slap_client_connect (ldp=0x7f5cc3e22558, > sb=0x7f5cc3e22330) at ../../../servers/slapd/config.c:2012 > #13 0x00007f5cc319f3f5 in do_syncrep1 (ctx=<value optimized out>, > arg=0x7f5cc3e1f410) at ../../../servers/slapd/syncrepl.c:613 > #14 do_syncrepl (ctx=<value optimized out>, arg=0x7f5cc3e1f410) at > ../../../servers/slapd/syncrepl.c:1527 > #15 0x00007f5cc2c81a98 in ldap_int_thread_pool_wrapper (xpool=0x7f5cc3d18030) at > ../../../libraries/libldap_r/tpool.c:688 > #16 0x00007f5cc0c54851 in start_thread () from /lib64/libpthread.so.0 > #17 0x00007f5cc079790d in clone () from /lib64/libc.so.6 > (gdb) thread 7 > [Switching to thread 7 (Thread 0x7f5c942d5700 (LWP 3869))]#0 0x00007f5cc0797f03 > in epoll_wait () from /lib64/libc.so.6 > (gdb) bt > #0 0x00007f5cc0797f03 in epoll_wait () from /lib64/libc.so.6 > #1 0x00007f5cc312e9e2 in slapd_daemon_task (ptr=<value optimized out>) at > ../../../servers/slapd/daemon.c:2536 > #2 0x00007f5cc0c54851 in start_thread () from /lib64/libpthread.so.0 > #3 0x00007f5cc079790d in clone () from /lib64/libc.so.6 > > > Relevant configuration information (from host2): > > syncrepl rid=001 > provider=ldap://host1/ > type=refreshAndPersist > retry="10 +" > searchbase="dc=example,dc=com" > starttls=critical > tls_cacert=/etc/openldap/ssl-certs/cacert.pem > tls_cert=/etc/openldap/ssl-certs/host1.crt > tls_key=/etc/openldap/ssl-certs/host1.key > tls_reqcert=demand > bindmethod=sasl > saslmech=EXTERNAL > sizelimit=unlimited > keepalive="10:2:10" > logfilter="(&(objectClass=auditWriteObject)(reqResult=0))" > logbase="cn=accesslog" > syncdata=accesslog > > > Netstat of host1: > > [root@host1 ~]# netstat -anp | grep slapd > tcp 0 0 0.0.0.0:389 0.0.0.0:* > LISTEN 31615/slapd > tcp 56 0 192.168.1.1:389 192.168.1.2:57276 > ESTABLISHED 31615/slapd > tcp 0 0 192.168.1.1:45569 192.168.1.2:389 > ESTABLISHED 31615/slapd > unix 2 [ ] STREAM CONNECTED 45346152 31615/slapd > unix 2 [ ] DGRAM 45346144 31615/slapd > > Netstat of host2: > > [root@host2 ~]# netstat -anp | grep slapd > tcp 0 0 0.0.0.0:389 0.0.0.0:* > LISTEN 3868/slapd > tcp 56 0 192.168.1.2:389 192.168.1.1:45569 > ESTABLISHED 3868/slapd > tcp 0 0 192.168.1.2:57276 192.168.1.1:389 > ESTABLISHED 3868/slapd > unix 2 [ ] STREAM CONNECTED 19144 3868/slapd > unix [ ] DGRAM 19136 3868/slapd > > -- Matúš Honěk

1 0

(ITS#8384) LDAP Servers in Multi-master configuration locked up and unresponsive
by kevin4sullivan＠gmail.com 11 Mar '16

11 Mar '16

Full_Name: Kevin Sullivan Version: 2.4.39-8 OS: RHEL 6.4 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (38.127.164.138) Setup: I have two OpenLDAP servers running on separate hosts and replicating via delta syncrepl in mirrormode. Problem: On rare occasions when one of my servers comes online, both servers will lock up and be unresponsive. Any query to either server will time out. The servers will stay in this state indefinitely. Workaround: Killing either of the servers will resolve this problem. Observations: - I don't know how to reproduce this reliably. - Both servers have a thread that is stuck in a do_syncrep1() call. - Netstat shows that each server has unacknowledged data in one of their socket's receive queue. GDB on host1: (gdb) info threads 18 Thread 0x7f8b7b80d700 (LWP 31620) 0x00007f8ba7fa6f03 in epoll_wait () from /lib64/libc.so.6 17 Thread 0x7f8b7b00c700 (LWP 31621) 0x00007f8ba846a054 in __lll_lock_wait () from /lib64/libpthread.so.0 16 Thread 0x7f8b7a80b700 (LWP 31622) 0x00007f8ba846a054 in __lll_lock_wait () from /lib64/libpthread.so.0 15 Thread 0x7f8b7a00a700 (LWP 31623) 0x00007f8ba846a054 in __lll_lock_wait () from /lib64/libpthread.so.0 14 Thread 0x7f8b79809700 (LWP 31625) 0x00007f8ba846a54d in read () from /lib64/libpthread.so.0 13 Thread 0x7f8b79008700 (LWP 31626) 0x00007f8ba846a054 in __lll_lock_wait () from /lib64/libpthread.so.0 12 Thread 0x7f8b59ffd700 (LWP 25459) 0x00007f8ba846a054 in __lll_lock_wait () from /lib64/libpthread.so.0 11 Thread 0x7f8b597fc700 (LWP 10195) 0x00007f8ba846a054 in __lll_lock_wait () from /lib64/libpthread.so.0 10 Thread 0x7f8b70b37700 (LWP 28996) 0x00007f8ba846a054 in __lll_lock_wait () from /lib64/libpthread.so.0 9 Thread 0x7f8b58ffb700 (LWP 8763) 0x00007f8ba846a054 in __lll_lock_wait () from /lib64/libpthread.so.0 8 Thread 0x7f8b4bfff700 (LWP 28614) 0x00007f8ba846a054 in __lll_lock_wait () from /lib64/libpthread.so.0 7 Thread 0x7f8b4b7fe700 (LWP 23367) 0x00007f8ba846a054 in __lll_lock_wait () from /lib64/libpthread.so.0 6 Thread 0x7f8b4affd700 (LWP 13359) 0x00007f8ba846a054 in __lll_lock_wait () from /lib64/libpthread.so.0 5 Thread 0x7f8b4a7fc700 (LWP 29064) 0x00007f8ba846a054 in __lll_lock_wait () from /lib64/libpthread.so.0 4 Thread 0x7f8b49ffb700 (LWP 2184) 0x00007f8ba846a054 in __lll_lock_wait () from /lib64/libpthread.so.0 3 Thread 0x7f8b497fa700 (LWP 31175) 0x00007f8ba846a054 in __lll_lock_wait () from /lib64/libpthread.so.0 2 Thread 0x7f8b48ff9700 (LWP 2316) 0x00007f8ba846a054 in __lll_lock_wait () from /lib64/libpthread.so.0 * 1 Thread 0x7f8baa8db700 (LWP 31615) 0x00007f8ba84640ad in pthread_join () from /lib64/libpthread.so.0 (gdb) bt #0 0x00007f8ba84640ad in pthread_join () from /lib64/libpthread.so.0 #1 0x00007f8baa93c969 in slapd_daemon () at ../../%./servers/slapd/daemon.c:2929 #2 0x00007f8baa927733 in main (argc=6, argv=<value optimized out>) at ../../../servers/slapd/main.c:1012 (gdb) thread 2 [Switching to thread 2 (Thread 0x7f8b48ff9700 (LWP 2316))]#0 0x00007f8ba846a054 in __lll_lock_wait (929 from /lib64/libpthread.so.0 (gdb) bt #0 0x00007f8ba846a054 in __lll_lock_wait () from /lib64/libpthread.so.0 #1 0x00007f8ba8465388 in _L_lock_854 () from /lib64/libpthread.so.0 #2 0x00007f8ba8465257 in pthread_mutex_lock () from /lib64/libpthread.so.0 #3 0x00007f8baa4b96a0 in tlsm_session_accept_or_connect (session=0x7f8b2c104110, is_accept=<value optimized out>) at tls_m.c:2656 #4 0x00007f8baa4b8787 in ldap_pvt_tls_accept (sb=0x7f8b2c104150, ctx_arg=0x7f8bac6f8230) at tls2.c:425 #5 0x00007f8baa943f23 in connection_read (ctx=0x7f8b48ff8b70, argv=0x2f) at ../../../servers/slapd/connection.c:1372 #6 connection_read_thread (ctx=0x7f8b48ff8b70, argv=0x2f) at ../../../servers/slapd/connection.c:1284 #7 0x00007f8baa490a98 in ldap_int_thread_pool_wrapper (xpool=0x7f8bac5c8fb0) at ../../../libraries/libldap_r/tpool.c:688 #8 0x00007f8ba8463851 in start_thread () from /lib64/libpthread.so.0 #9 0x00007f8ba7fa690d in clone () from /lib64/libc.so.6 (gdb) thread 14 [Switching to thread 14 (Thread 0x7f8b79809700 (LWP 31625))]#0 0x00007f8ba846a54d in read () from /lib64/libpthread.so.0 (gdb) bt #0 0x00007f8ba846a54d in read () from /lib64/libpthread.so.0 #1 0x00007f8baa278b1c in sb_debug_read (sbiod=0x7f8b681def80, buf=0x7f8b6819c830, len=5) at ../../../libraries/liblber/sockbuf.c:829 #2 0x00007f8baa4b9984 in tlsm_PR_Recv (fd=<value optimized out>, buf=0x7f8b6819c830, len=5, flags=<value optimized out>, timeout=<value optimized out>) at tls_m.c:3007 #3 0x00007f8ba946c1ed in ?? () from /usr/lib64/libssl3.so #4 0000007f8ba9467480 in ?? () from /usr/lib64/libssl3.so #5 0x00007f8ba9469ed2 in ?? () from /usr/lib64/libssl3.so #6 0x00007f8ba9470135 in ?? () from /usr/lib64/libssl3.so #7 0x00007f8ba947196f in SSL_ForceHandshake () from /usr/lib64/libssl3o%o #8 0x00007f8baa4b96a8 in tlsm_session_accept_or_connect (session=0x7f8b681b0f20, is_accept=<value optimized out>) at tls_m.c:2658 #9 0x00007f8baa4b82e2 in ldap_int_tls_connect (ld=0x7f8b681d9250, conn=<value optimized out>) at tls2.c:362 #10 0x00007f8baa4b857inin ldap_int_tls_start (ld=0x7f8b681d9250, conn=0x7f8b681b0ba0, srv=<value optimized out>) at tls2.c:860 #11 0x00007f8baa4b86ce in ldap_start_tls_s (ld=0x7f8b681d9250, serverctrls=0x0, clientctrls=0x0) at tls2.c:1040 #12 0x00007f8baa937ff6 in slap_client_connect (ldp=0x7f8bac6d35a8, sb=0x7f8bac6d3380) at ../../../servers/slapd/config.c:2012 #13 0x00007f8baa9ae3f5 in do_syncrep1 (ctx=<value optimized out>, arg=0x7f8bac6cc430) at ../../../servers/slapd/syncrepl.c:613 #14 do_syncrepl (ctx=<value optimized out>, arg=0x7f8bac6cc430) at ../../../servers/slapd/syncrepl.c:1527 #15 0x00007f8baa490a98 in ldap_int_thread_pool_wrapper (xpool=0x7f8bac5c8fb0) at ../../../libraries/libldap_r/tpool.c:688 #16 0x00007f8ba8463851 in start_thread () from /lib64/libpthread.so.0 #17 0x00007f8ba7fa690d in clone () from /lib64/libc.so.6 (gdb) thread 18 [Switching to thread 18 (Thread 0x7f8b7b80d700 (LWP 31620))]#0 0x00007f8ba7fa6f03 in epoll_wait () from /lib64/libc.so.6 (gdb) bt #0 0x00007f8ba7fa6f03 in epoll_wait () from /lib64/libc.so.6 #1 0x00007f8baa93d9e2 in slapd_daemon_task (ptr=<value optimized out>) at ../../../servers/slapd/daemon.c:2536 #2 0x00007f8ba8463851 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f8ba7fa690d in clone () from /lib64/libc.so.6 GDB on host2: (gdb) info threads 7 Thread 0x7f5c942d5700 (LWP 3869) 0x00007f5cc0797f03 in epoll_wait () from /lib64/libc.so.6 6 Thread 0x7f5c93ad4700 (LWP 3870) 0x00007f5cc0c5b54d in read () from /lib64/libpthread.so.0 5 Thread 0x7f5c932d3700 (LWP 3871) 0x00007f5cc0c5b054 in __lll_lock_wait () from /lib64/libpthread.so.0 4 Thread 0x7f5c92ad2700 (LWP 3872) 0x00007f5cc0c5843c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 3 Thread 0x7f5c922d1700 (LWP 3873) 0x00007f5cc0c5843c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 2 Thread 0x7f5c91ad0700 (LWP 3874) 0x00007f5cc0c5843c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 * 1 Thread 0x7f5cc30cc700 (LWP 3868) 0x00007f5cc0c550ad in pthread_join () from /lib64/libpthread.so.0 (gdb) bt #0 0x00007f5cc0c550ad in pthread_join () from /lib64/libpthread.so.0 #1 0x00007f5cc312d969 in slapd_daemon () at ../../../servers/slapd/daemon.c:2929 #2 0x00007f5cc3118733 in main (argc=6, argv=<value optimized out>) at ../../../servers/slapd/main.c:1012 (gdb) thread 2 [Switching to thread 2 (Thread 0x7f5c91ad0700 (LWP 3874))]#0 0x00007f5cc0c5843c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 (gdb) bt #0 0x00007f5cc0c5843c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f5cc2c81af5 in ldap_int_thread_pool_wrapper (xpool=0x7f5cc3d18030) at ../../../libraries/libldap_r/tpool.c:675 #2 0x00007f5cc0c54851 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f5cc079790d in clone () from /lib64/libc.so.6 (gdb) thread 5 [Switching to thread 5 (Thread 0x7f5c932d3700 (LWP 3871))]#0 0x00007f5cc0c5b054 in __lll_lock_wait () from /lib64/libpthread.so.0 (gdb) bt #0 0x00007f5cc0c5b054 in __lll_lock_wait () from /lib64/libpthread.so.0 #1 0x00007f5cc0c56388 in _L_lock_854 () from /lib64/libpthread.so.0 #2 0x00007f5cc0c56257 in pthread_mutex_lock () from /lib64/libpthread.so.0 #3 0x00007f5cc2caa6a0 in tlsm_session_accept_or_connect (session=0x7f5c841165c0, is_accept=<value optimized out>) at tls_m.c:2656 #4 0x00007f5cc2ca9787 in ldap_pvt_tls_accept (sb=0x7f5c800008c0, ctx_arg=0x7f5cc3e3b240) at tls2.c:425 #5 0x00007f5cc3134f23 in connection_read (ctx=0x7f5c932d2b70, argv=0x10) at ../../../servers/slapd/connection.c:1372 #6 connection_read_thread (ctx=0x7f5c932d2b70, argv=0x10) at ../../../servers/slapd/connection.c:1284 #7 0x00007f5cc2c81a98 in ldap_int_thread_pool_wrapper (xpool=0x7f5cc3d18030) at ../../../libraries/libldap_r%tptpool.c:688 #8 0x00007f5cc0c54851 in start_thread () from /lib64/libpthread.so.0 #9 0x00007f5cc079790d in clone () from /lib64/libc.so.6 (gdb) thread 6 [Switching to thread 6 (Thread 0x7f5c93ad4700 (LWP 3870))]#0 0x00007f5cc0c5b54d in read () from /lib64/libpthread.so.0 (gdb) bt #0 0x00007f5cc0c5b54d in read () from /lib64/libpthread.so.0 #1 0x00007f5cc2a69b1c in sb_debug_read (sbiod=0x7f5c7c185e10, buf=0x7f5c7c18a7f0, len=5) at ../../../libraries/liblber/sockbuf.c:829 #2 0x00007f5cc2caa984 in tlsm_PR_Recv (fd=<value optimized out>, buf=0x7f5c7c18a7f0, len=5, flags=<value optimized out>, timeout=<value optimized out>) at tls_m.c:3007 #3 0x00007f5cc1c5d1ed in ?? () from /usr/lib64/libssl3.so #4 0x00007f5cc1c58480 in ?? () from /usr/lib64/libssl3.so #5 0x00007f5cc1c5aed2 in ?? () from /usr/lib64/libssl3.so #6 0x00007f5cc1c61135 in ?? () from /usr/lib64/libssl3.so #7 0x00007f5cc1c6296f in SSL_ForceHandshake () from /usr/lib64/libssl3.so #8 0x00007f5cc2caa6a8 in tlsm_session_accept_or_connect (session=0x7f5c7c182100, is_accept=<value optimized out>) at tls_m.c:2658 #9 0x00007f5cc2ca92e2 in ldap_int_tls_connect (ld=0x7f5c7c100910, conn=<value optimized out>) at tls2.c:362 #10 0x00007f5cc2ca9574 in ldap_int_tls_start (ld=0x7f5c7c100910, conn=0x7f5c7c10a020, srv=<value optimized out>) at tls2.c:860 #11 0x00007f5cc2ca96ce in ldap_start_tls_s (ld=0x7f5c7c100910, serverctrls=0x0, clientctrls=0x0) at tls2.c:1040 #12 0x00007f5cc3128ff6 in slap_client_connect (ldp=0x7f5cc3e22558, sb=0x7f5cc3e22330) at ../../../servers/slapd/config.c:2012 #13 0x00007f5cc319f3f5 in do_syncrep1 (ctx=<value optimized out>, arg=0x7f5cc3e1f410) at ../../../servers/slapd/syncrepl.c:613 #14 do_syncrepl (ctx=<value optimized out>, arg=0x7f5cc3e1f410) at ../../../servers/slapd/syncrepl.c:1527 #15 0x00007f5cc2c81a98 in ldap_int_thread_pool_wrapper (xpool=0x7f5cc3d18030) at ../../../libraries/libldap_r/tpool.c:688 #16 0x00007f5cc0c54851 in start_thread () from /lib64/libpthread.so.0 #17 0x00007f5cc079790d in clone () from /lib64/libc.so.6 (gdb) thread 7 [Switching to thread 7 (Thread 0x7f5c942d5700 (LWP 3869))]#0 0x00007f5cc0797f03 in epoll_wait () from /lib64/libc.so.6 (gdb) bt #0 0x00007f5cc0797f03 in epoll_wait () from /lib64/libc.so.6 #1 0x00007f5cc312e9e2 in slapd_daemon_task (ptr=<value optimized out>) at ../../../servers/slapd/daemon.c:2536 #2 0x00007f5cc0c54851 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f5cc079790d in clone () from /lib64/libc.so.6 Relevant configuration information (from host2): syncrepl rid=001 provider=ldap://host1/ type=refreshAndPersist retry="10 +" searchbase="dc=example,dc=com" starttls=critical tls_cacert=/etc/openldap/ssl-certs/cacert.pem tls_cert=/etc/openldap/ssl-certs/host1.crt tls_key=/etc/openldap/ssl-certs/host1.key tls_reqcert=demand bindmethod=sasl saslmech=EXTERNAL sizelimit=unlimited keepalive="10:2:10" logfilter="(&(objectClass=auditWriteObject)(reqResult=0))" logbase="cn=accesslog" syncdata=accesslog Netstat of host1: [root@host1 ~]# netstat -anp | grep slapd tcp 0 0 0.0.0.0:389 0.0.0.0:* LISTEN 31615/slapd tcp 56 0 192.168.1.1:389 192.168.1.2:57276 ESTABLISHED 31615/slapd tcp 0 0 192.168.1.1:45569 192.168.1.2:389 ESTABLISHED 31615/slapd unix 2 [ ] STREAM CONNECTED 45346152 31615/slapd unix 2 [ ] DGRAM 45346144 31615/slapd Netstat of host2: [root@host2 ~]# netstat -anp | grep slapd tcp 0 0 0.0.0.0:389 0.0.0.0:* LISTEN 3868/slapd tcp 56 0 192.168.1.2:389 192.168.1.1:45569 ESTABLISHED 3868/slapd tcp 0 0 192.168.1.2:57276 192.168.1.1:389 ESTABLISHED 3868/slapd unix 2 [ ] STREAM CONNECTED 19144 3868/slapd unix [ ] DGRAM 19136 3868/slapd

1 0

Re: (ITS#8382) Activation of idlcache leads to wrong ldapsearch results
by fdopheide＠iponweb.net 09 Mar '16

09 Mar '16

Hi, I am also adding the guide to replicate the error as plain text:=20 Step-by-step guide The following guide was created on an Ubuntu Linux V. 14.04.3 LTS, but = it should be executeable on an Debian based Linux operating systems. You = should be logged in as root user or adopt all commands with sudo. 1. Add hostname echo 'ldap-debug.example.com' > /etc/hostname && hostname `cat = /etc/hostname` && hostname echo '52.49.174.211 ldap-debug.example.com ldap-debug' >> /etc/hosts 2. Download latest Sources wget = http://www.openldap.org/software/download/OpenLDAP/openldap-release/openld= ap-2.4.44.tgz 3. Validate Sources(compare md5 provided on website with generated md5 = hash) md5sum openldap-2.4.44.tgz 4. Make archive executable and extract sources and change into = extracted directory chmod +x openldap-2.4.44.tgz && tar xzf openldap-2.4.44.tgz && cd = openldap-2.4.44 5. Fullfil most important software requirements (Dependencies) apt-get update apt-get install libdb-dev apt-get install openssl apt-get install gnutls-bin apt-get install gcc apt-get install make 6. Run configure with default settings and add a target dir only ./configure --prefix=3D/usr/local/openldap 7. Run make and install sources to target directory make depend make make test make install 8. Stop the daemon, because slapd was already started by make. pkill slapd 9. Edit main config file in /usr/local/openldap/etc/openldap/slapd.conf = and add the following lines: include /usr/local/openldap/etc/openldap/schema/core.schema pidfile /usr/local/openldap/var/run/slapd.pid argsfile /usr/local/openldap/var/run/slapd.args modulepath /usr/lib/ldap moduleload back_hdb backend hdb database hdb suffix "dc=3Dexample,dc=3Dcom" rootdn "cn=3DManager,dc=3Dexample,dc=3Dcom" rootpw secret directory /var/lib/ldap index objectClass eq 10. Call slappasswd and set rootpw to secret slappasswd 11. Create data directory for hdb files mkdir /var/lib/ldap 12. We will add a symbolic link to ease slapd commands ln -s /usr/local/openldap/libexec/slapd /usr/local/bin/slapd 13. Add the path to OpenLDAP bin and sbin dirs and activate the new path echo "export = PATH=3D\$PATH:/usr/local/openldap/bin/:/usr/local/openldap/sbin/" >> = /root/.bashrc && source /root/.bashrc 14. Start the slapd daemon with -f flag for location of the config file slapd -f /usr/local/openldap/etc/openldap/slapd.conf 15. Create initial DIT file with an editor of your choice, i.e. vi = /root/create_dit.ldif and copy paste the following lines into this file. dn: dc=3Dexample,dc=3Dcom changetype: add objectClass: dcObject objectClass: organization objectClass: top dc: example o: example.com =20 dn: ou=3DProjects,dc=3Dexample,dc=3Dcom changetype: add ou: Projects objectClass: organizationalUnit =20 dn: ou=3DTrash,dc=3Dexample,dc=3Dcom changetype: add ou: Trash objectClass: organizationalUnit =20 dn: ou=3Dtest1,ou=3DProjects,dc=3Dexample,dc=3Dcom changetype: add ou: test1 objectClass: organizationalUnit =20 dn: ou=3Dtest2,ou=3Dtest1,ou=3DProjects,dc=3Dexample,dc=3Dcom changetype: add ou: test2 objectClass: organizationalUnit 16. Add the initial DIT ldapadd -H 'ldap://localhost' -D 'cn=3DManager,dc=3Dexample,dc=3Dcom' = -w secret -f /root/create_dit.ldif 17. Execute modrdn and assign directory Trash as a new superior of test1 = directory ldapmodrdn -x -w secret -D "cn=3DManager,dc=3Dexample,dc=3Dcom" -s = "ou=3DTrash,dc=3Dexample,dc=3Dcom" -r = "ou=3Dtest1,ou=3DProjects,dc=3Dexample,dc=3Dcom" "ou=3Dtest1" 18. Execute following ldapsearch and save the result somewhere for i in {,{,{,ou=3Dtest2\,}ou=3Dtest1\,}ou=3DTrash\,}dc=3Dexample,dc=3D= com; do echo -e "\n=3D=3D=3D=3D=3D> basedn: $i <=3D=3D=3D=3D=3D\n"; = ldapsearch -LLL -s 'sub' -H 'ldap://localhost:389' -D = 'cn=3DManager,dc=3Dexample,dc=3Dcom' -w secret -b "$i" '(ou=3Dtest*)' = dn; done Result: =3D=3D=3D=3D=3D> basedn: dc=3Dexample,dc=3Dcom <=3D=3D=3D=3D=3D dn: ou=3Dtest1,ou=3DTrash,dc=3Dexample,dc=3Dcom dn: ou=3Dtest2,ou=3Dtest1,ou=3DTrash,dc=3Dexample,dc=3Dcom =3D=3D=3D=3D=3D> basedn: ou=3DTrash,dc=3Dexample,dc=3Dcom <=3D=3D=3D=3D= =3D dn: ou=3Dtest1,ou=3DTrash,dc=3Dexample,dc=3Dcom dn: ou=3Dtest2,ou=3Dtest1,ou=3DTrash,dc=3Dexample,dc=3Dcom =3D=3D=3D=3D=3D> basedn: ou=3Dtest1,ou=3DTrash,dc=3Dexample,dc=3Dcom = <=3D=3D=3D=3D=3D dn: ou=3Dtest1,ou=3DTrash,dc=3Dexample,dc=3Dcom dn: ou=3Dtest2,ou=3Dtest1,ou=3DTrash,dc=3Dexample,dc=3Dcom =3D=3D=3D=3D=3D> basedn: = ou=3Dtest2,ou=3Dtest1,ou=3DTrash,dc=3Dexample,dc=3Dcom <=3D=3D=3D=3D=3D dn: ou=3Dtest2,ou=3Dtest1,ou=3DTrash,dc=3Dexample,dc=3Dcom 19. We see until here everything is working as expected, but now we will = enable cache and idlcache as recommended in OpenLDAP docs and = Performance Guides and problems will appear soon. echo -e "cachesize\t10000\nidlcachesize\t30000" >> = /usr/local/openldap/etc/openldap/slapd.conf 20. Remove existing DIT ldapdelete -H 'ldap://localhost' -D 'cn=3DManager,dc=3Dexample,dc=3Dco= m' -w secret -r "dc=3Dexample,dc=3Dcom" -v 21. Restart daemon and recreate initial DIT pkill slapd slapd -f /usr/local/openldap/etc/openldap/slapd.conf ldapadd -H 'ldap://localhost' -D 'cn=3DManager,dc=3Dexample,dc=3Dcom' = -w secret -f /root/create_dit.ldif 22. Execute modrdn ldapmodrdn -x -w secret -D "cn=3DManager,dc=3Dexample,dc=3Dcom" -s = "ou=3DTrash,dc=3Dexample,dc=3Dcom" -r = "ou=3Dtest1,ou=3DProjects,dc=3Dexample,dc=3Dcom" "ou=3Dtest1" 23. Check result again with ldapsearch for i in {,{,{,ou=3Dtest2\,}ou=3Dtest1\,}ou=3DTrash\,}dc=3Dexample,dc=3D= com; do echo -e "\n=3D=3D=3D=3D=3D> basedn: $i <=3D=3D=3D=3D=3D\n"; = ldapsearch -LLL -s 'sub' -H 'ldap://localhost:389' -D = 'cn=3DManager,dc=3Dexample,dc=3Dcom' -w secret -b "$i" '(ou=3Dtest*)' = dn; done Result: =3D=3D=3D=3D=3D> basedn: dc=3Dexample,dc=3Dcom <=3D=3D=3D=3D=3D dn: ou=3Dtest1,ou=3DTrash,dc=3Dexample,dc=3Dcom dn: ou=3Dtest2,ou=3Dtest1,ou=3DTrash,dc=3Dexample,dc=3Dcom =3D=3D=3D=3D=3D> basedn: ou=3DTrash,dc=3Dexample,dc=3Dcom <=3D=3D=3D=3D= =3D dn: ou=3Dtest1,ou=3DTrash,dc=3Dexample,dc=3Dcom dn: ou=3Dtest2,ou=3Dtest1,ou=3DTrash,dc=3Dexample,dc=3Dcom =3D=3D=3D=3D=3D> basedn: ou=3Dtest1,ou=3DTrash,dc=3Dexample,dc=3Dcom = <=3D=3D=3D=3D=3D dn: ou=3Dtest1,ou=3DTrash,dc=3Dexample,dc=3Dcom dn: ou=3Dtest2,ou=3Dtest1,ou=3DTrash,dc=3Dexample,dc=3Dcom =3D=3D=3D=3D=3D> basedn: = ou=3Dtest2,ou=3Dtest1,ou=3DTrash,dc=3Dexample,dc=3Dcom <=3D=3D=3D=3D=3D dn: ou=3Dtest2,ou=3Dtest1,ou=3DTrash,dc=3Dexample,dc=3Dcom 24. Once again the result was fine and as expected. Keep in mind cache = and idlcache were empty as we've restarted the daemon. Now we are = performing same actions again while we have cached results. 25. Remove directories in ou =E2=80=9ETrash=E2=80=9C ldapdelete -w secret -D "cn=3DManager,dc=3Dexample,dc=3Dcom" -r = "ou=3Dtest1,ou=3DTrash,dc=3Dexample,dc=3Dcom" 26. Add directories to ou =E2=80=9EProjects=E2=80=9C again ldapmodify -D "cn=3DManager,dc=3Dexample,dc=3Dcom" -W Type in password Enter LDAP Password: Copy paste following ldif and be sure emtpy lines are copied, too! dn: ou=3Dtest1,ou=3DProjects,dc=3Dexample,dc=3Dcom changetype: add objectClass: organizationalUnit =20 dn: ou=3Dtest2,ou=3Dtest1,ou=3DProjects,dc=3Dexample,dc=3Dcom changetype: add objectClass: organizationalUnit 27. Now we have same DIT structure again and can rerun modrdn command ldapmodrdn -x -w secret -D "cn=3DManager,dc=3Dexample,dc=3Dcom" -s = "ou=3DTrash,dc=3Dexample,dc=3Dcom" -r = "ou=3Dtest1,ou=3DProjects,dc=3Dexample,dc=3Dcom" "ou=3Dtest1" 28. If we now check result with ldapsearch again the idlcache error will = appear for i in {,{,{,ou=3Dtest2\,}ou=3Dtest1\,}ou=3DTrash\,}dc=3Dexample,dc=3D= com; do echo -e "\n=3D=3D=3D=3D=3D> basedn: $i <=3D=3D=3D=3D=3D\n"; = ldapsearch -LLL -s 'sub' -H 'ldap://localhost:389' -D = 'cn=3DManager,dc=3Dexample,dc=3Dcom' -w secret -b "$i" '(ou=3Dtest*)' = dn; done Result: =3D=3D=3D=3D=3D> basedn: dc=3Dexample,dc=3Dcom <=3D=3D=3D=3D=3D dn: ou=3Dtest1,ou=3DTrash,dc=3Dexample,dc=3Dcom dn: ou=3Dtest2,ou=3Dtest1,ou=3DTrash,dc=3Dexample,dc=3Dcom =3D=3D=3D=3D=3D> basedn: ou=3DTrash,dc=3Dexample,dc=3Dcom <=3D=3D=3D=3D= =3D dn: ou=3Dtest1,ou=3DTrash,dc=3Dexample,dc=3Dcom =3D=3D=3D=3D=3D> basedn: ou=3Dtest1,ou=3DTrash,dc=3Dexample,dc=3Dcom = <=3D=3D=3D=3D=3D dn: ou=3Dtest1,ou=3DTrash,dc=3Dexample,dc=3Dcom dn: ou=3Dtest2,ou=3Dtest1,ou=3DTrash,dc=3Dexample,dc=3Dcom =3D=3D=3D=3D=3D> basedn: = ou=3Dtest2,ou=3Dtest1,ou=3DTrash,dc=3Dexample,dc=3Dcom <=3D=3D=3D=3D=3D dn: ou=3Dtest2,ou=3Dtest1,ou=3DTrash,dc=3Dexample,dc=3Dcom 29. The search result of basedn: ou=3DTrash,dc=3Dexample,dc=3Dcom is = wrong! The line of dn: ou=3Dtest2,ou=3Dtest1,ou=3DTrash,dc=3Dexample,dc=3D= com is missing within the resultset. 30. After a restart of the daemon the missing line will appear again. = Actually the one and only solution is to switch of idlcachesize and set = it explicitely to "0" > Am 07.03.2016 um 11:45 schrieb fdopheide(a)iponweb.net: >=20 > Full_Name: Frank Dopheide > Version: slapd 2.4.44 (Mar 3 2016 10:27:05) > OS: Ubuntu 14.04.3 LTS > URL: http://52.48.8.167/idlcache_error.pdf > Submission from: (NULL) (87.139.29.59) >=20 >=20 > Good morning Gentlemen, >=20 > we've found a quite serious bug in slapd 2.4.44 on an Ubuntu 14.04.3 = LTS server. > As soon as we enable the idlcache the ldapsearch results will be wrong = after > moddn and modrdn commands. First result after idlcachesize activation = is still > correct as the cache is still empty and the daemon reads from db, but = as soon as > it starts reading from idlcache the latest modifications are missing = until slapd > will be restarted! >=20 > We've created a small howto to replicatehe e error. The howto is = available via > http: http://52.48.8.167/idlcache_error.pdf >=20 > Please let us now if you require any additional information! >=20 > Thx, > Frank >=20 >=20 >=20

1 0

(ITS#8383) msys build error "portable.h:1116:19: error: two or more data types in declaration specifiers"
by fixed-term.Asal.Mirzaieva＠boschrexroth.de 08 Mar '16

08 Mar '16

Full_Name: Asal Mirzaieva Version: 2.4.44 OS: Windows 7 URL: http://paste.ubuntu.com/15326640/, http://paste.ubuntu.com/15326646/, http://paste.ubuntu.com/15326655/ Submission from: (NULL) (194.39.218.10) When building openldap on windows, using msys and mingw (mingw32 v4.9.3) the following error appears: In file included from base64.c:38:0:../../include/portable.h:1116:19: error: two or more data types in declaration specifiers #define socklen_t int Steps to reproduce the bug: 1. Install Mingw32 with mingw-get-setup.exe. 2. Launch msys.bat in msys folder to open msys console. 3. Download and install Spencer's regex in /usr/local 4. Download openldap-2.4.44 5. Unzip and untar the openldap and go to the openldap-2.4.44's directory 6. Run configure as follows ./configure CFLAGS="-I/usr/local/include/rxspencer" LDFLAGS=2-2-L/usr/local/lib" LIBS="-static -lrxspencer" --enable-static --disable-shared --disable-slapd --disable-bdb --disable-hdb 7. Run make depend 8. Run make 9. Get the error, that was described above. I haven't tried to reproduce this bug with other PCs/OS versions/MinGW versions. Workaround: after commenting the line 1116 in include/portable.h the error was gone and I was able to successfully build and use openldap. Well, "successfully" means, I could link it statically and run ldap_initialize(), but not ldap_sals_bind(), because I haven't installed SASL.

1 0

(ITS#8382) Activation of idlcache leads to wrong ldapsearch results
by fdopheide＠iponweb.net 07 Mar '16

07 Mar '16

Full_Name: Frank Dopheide Version: slapd 2.4.44 (Mar 3 2016 10:27:05) OS: Ubuntu 14.04.3 LTS URL: http://52.48.8.167/idlcache_error.pdf Submission from: (NULL) (87.139.29.59) Good morning Gentlemen, we've found a quite serious bug in slapd 2.4.44 on an Ubuntu 14.04.3 LTS server. As soon as we enable the idlcache the ldapsearch results will be wrong after moddn and modrdn commands. First result after idlcachesize activation is still correct as the cache is still empty and the daemon reads from db, but as soon as it starts reading from idlcache the latest modifications are missing until slapd will be restarted! We've created a small howto to replicatehe e error. The howto is available via http: http://52.48.8.167/idlcache_error.pdf Please let us now if you require any additional information! Thx, Frank

1 0

(ITS#8381) Proposing new mirror for OpenLDAP sources
by dcoutadeur＠linagora.com 07 Mar '16

07 Mar '16

Full_Name: David Coutadeur Version: 2.4.44 OS: Debian 8 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (92.103.166.6) Linagora (France) is now hosting a new mirror for OpenLDAP sources. It is available at: http://repository.linagora.org/OpenLDAP/ http://repository.linagora.com/OpenLDAP/ The latter (.com) will not be supported in the long term. Please update MIRRORS file accordingly. Regards, David

1 0

Re: (ITS#8380) Feature request: make a plugin like smbk5pwd for the HA1 and HA1b hashes used in DIGEST and HMAC
by daniel＠pocock.pro 07 Mar '16

07 Mar '16

On 06/03/16 23:02, Howard Chu wrote: > Daniel Pocock wrote: >> >> >> On 06/03/16 20:49, Howard Chu wrote: >>> Daniel Pocock wrote: >>>> >>>> >>>> On 06/03/16 20:00, Howard Chu wrote: >>>>> daniel(a)pocock.pro wrote: >>>>>> Full_Name: Daniel Pocock >>>>>> Version: >>>>>> OS: Debian >>>>>> URL: ftp://ftp.openldap.org/incoming/ >>>>>> Submission from: (NULL) (2001:1620:b22::2042) >>>>>> >>>>>> >>>>>> There are a few protocols that use a HA1[1] password hash, such as >>>>>> HTTP >>>>>> DIGEST[1], SIP DIGEST[2] and TURN[3] (which uses HMAC rather than >>>>>> DIGEST) >>>>>> >>>>>> Is there a standard LDAP attribute name for storing a HA1 value or >>>>>> should it be stored in a regular userPassword attribute as >>>>>> described in >>>>>> the manual[4]? >>>>> >>>>> The ITS is not for usage questions. You already asked this and were >>>>> answered on the discussion mailing list. >>>>> >>>>> http://www.openldap.org/lists/openldap-technical/201507/msg00073.html >>>>> >>>>> There is nothing here that requires any OpenLDAP development activity. >>>>> It's all already handled by the SASL Digest mechanism, as I already >>>>> noted in the above email. >>>>> >>>>> Closing this ITS. >>>> >>>> >>>> I didn't open this feature request to ask for somebody to implement it, >>>> I'm simply trying to track a number of items that I'm working on >>>> myself. >>>> Normally I open a bug/feature request in anything I work on in case >>>> somebody else wants to work on the same thing, it helps avoid >>>> duplication. >>>> >>>> The email thread doesn't fully resolve the issue, it does appear to >>>> require some plugin to be created for the server side, especially if >>>> the >>>> LDAP server doesn't keep plain text passwords. Given the fairly >>>> generic >>>> nature of the DIGEST algorithm, I also felt that when implemented, this >>>> code should be contributed to the OpenLDAP repository and not hosted >>>> elsewhere. >>> >>> Take the hint: RTF SASL Digest code. All the code you're asking for has >>> already been implemented in Cyrus SASL and is of zero concern to >>> OpenLDAP. >>> >>> The most important skill of a programmer is being able to *read* - not >>> being able to write. Any fool can spew code. >>> >> >> I'm not sure if you've seen my reply to the list, it looks like it got >> stuck in moderation >> >> I understand your point about DIGEST and that may well work for HTTP and >> SIP. TURN, however, uses HMAC-SHA1 and that involves sending a copy of >> the entire message body to the authentication server for use in the >> algorithm: >> https://tools.ietf.org/html/rfc5766#page-51 > > You continue to fail to read. > > Pasting this again > > http://www.openldap.org/lists/openldap-technical/201507/msg00073.html > > The required parameters cannot be passed in a Simple Bind request. The > only way to be able to pass what you want and have the server > authenticate it is to use a SASL mechanism. For TURN you may need to > define your own SASL mechanism. > > None of this has anything to do with OpenLDAP. I fully agree with that - the HMAC-SHA1 message verification for TURN, like the DIGEST-MD5 code, would be in Cyrus SASL or in the TURN server (LDAP client). The OpenLDAP manual, s15.2.3 about DIGEST-MD5 explains "it needs access to the plaintext password". The DIGEST-MD5 code that uses the password is here: https://cgit.cyrus.foundation/cyrus-sasl/tree/plugins/digestmd5.c#n440 and it takes the plaintext password and transforms it into a HA1 string. Notice that: HA1 = MD5(user:realm:password) This is a repetitive calculation that does not involve the nonce, salt or any other transient value. HA1 is always the same value for any given user/realm/password permutation. Therefore, OpenLDAP could store HA1 values instead of plaintext values. In fact, Apache already does the same thing with a flat file maintained by the htdigest utility: $ htdigest -c /tmp/test.ha1 example.org daniel instead of storing plaintext values in a file. The HA1 values, not being salted, are still sensitive but some sites may prefer to store the HA1 values instead of plaintext values. So section 15.2.3 could actually be changed: "it needs access to the plaintext password or a password that has already been hashed in HA1" An alternative is that the TURN server process could bind to the LDAP server and then query the HA1 attribute for any user or the HA1 values could be periodically extract into a flat file similar to that created by the htdigest utility. I agree that people who have plaintext passwords don't need to store any HA1 attribute, but if somebody does not have plaintext passwords and does not want to have plaintext passwords, it appears that the HA1 hashing needs to be done in OpenLDAP on password changes. This would also appear to require a change to the Cyrus code so that it can accept either the plaintext or HA1 from the LDAP entry and the slapd auxprop plugin would need to be able to return either attribute. > >> CRAM-MD5 from SASL does HMAC but it does not appear to transfer entire >> message bodies in the manner required for TURN. >> >> DIGEST-MD5 and HMAC-SHA1 both use a HA1 value (or a cleartext password) >> as the lowest common denominator but otherwise they are not the same. >> >>> Your mention of smbk5pwd is totally off base as well. The reason the >>> smbk5pwd module was needed was because Samba 3 and the Kerberos5 KDC >>> both stored their secrets in separate and incompatible formats but >>> everyone wanted central coordinated administration for these separate >>> attributes. If you're writing something from scratch there is no reason >>> to use your own separate and incompatible attribute, and thus there is >>> no reason to require any special synchronization or coordination. >>> >> >> The reason I mentioned that is because it was the closest thing I could >> find to the concept of storing multiple password hashes, but I'm not >> locked in to that strategy. If there are cleartext passwords in LDAP >> then they can be used for all of these algorithms. If the administrator >> does not want to store cleartext values, however, then the salted >> password strings used for UNIX logins are not interchangeable with HA1 >> hashed values, in that case, isn't it necessary to store and synchronize >> multiple values, hashed with each algorithm? > > Once again you're back to software usage questions which is not what the > ITS is for. >

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs March 2016