openldap-bugs

openldap-bugs@openldap.org

1 participants
20965 discussions

Re: (ITS#8430) Improved handling for large number of databases
by hyc＠symas.com 27 May '16

27 May '16

juerg.bircher(a)helmedica.com wrote: > Full_Name: Juerg Bircher > Version: lmdb (master) > OS: MacOS / Linux > URL: ftp://ftp.openldap.org/incoming/Juerg_Bircher_160527-Improved-handling-for-… > Submission from: (NULL) (178.82.37.195) Thanks for the patch, but for a change of this size you also need to include an IP Rights notice, as documented on the Contributing guidelines. > > > There is a increased performance penalty the more databases are created within > the same environment. I was looking for a way the improve that by keeping the > simplicity of tracking databases within a list with direct access by index > (MDB_dbi). > mdb_dbi_open() is however not improved with the assumption that the database > handle (dbi) is cached in the application. So mdb_dbi_open() should happen only > once for each database during the life time of an application. > > > One issue is that mdb_txn_begin() (for read-only transactions) calloc the > sizeof(MDB_txn) + me_maxdbs * sizeof(MDB_db + 1). The plus 1 for the dbflags. > However it is sufficient only to malloc that size and clear the sizeof(MDB_txn) > > memset(txn, 0, sizeof(MDB_txn) > > After that the data beyond the MDB_txn is not initialized which is ok for the > moment. > > The next improvement happens in mdb_txn_renew0() where the dbflags are only set > to DB_UNUSED (a new flag) for each database currently opened in the > environment. > > memset(txn->mt_dbflags, DB_UNUSED, txn->mt_numdbs); > > The former code used to to loop through each database to calculate the dbflags. > This is still done but lazily for each accessed database with the assumption > that a read only transaction rarely uses all databases of the environment. > > The lazy initialization of the dbflag happens in the macro TXN_DBI_EXIST which > is always used when a database handle (dbi) is passed to an function. > The flags are updated in mdb_setup_db_info() once a database is access which is > marked as unused (DB_UNUSED). > > static int mdb_setup_db_info(MDB_txn *txn, MDB_dbi dbi) { > /* Setup db info */ > uint16_t x = txn->mt_env->me_dbflags[dbi]; > txn->mt_dbs[dbi].md_flags = x & PERSISTENT_FLAGS; > txn->mt_dbflags[dbi] = (x & MDB_VALID) ? DB_VALID|DB_USRVALID|DB_STALE : 0; > return (txn->mt_dbflags[dbi] & validity); > } > > /** Check \b txn and \b dbi arguments to a function and initialize db info > if needed */ > #define TXN_DBI_EXIST(txn, dbi, validity) \ > ((txn) && (dbi#C3C(txn)->mt_numdbs && (((txn)->mt_dbflags[dbi] & (validity)) > || (((txn)->mt_dbflags[dbi] & DB_UNUSED) && mdb_setup_db_info((txn), (dbi), > (validity))))) > > > The next improvement is done in any function which needs to loop through the > databases for example in mdb_cursors_close(). Again the more databases in the > environment the longer the execution time. > It should be best if looping only through dbflags and searching for those > databases which are used (!DB_UNUSED). This could be done byte wise or more > efficient in 8/4 byte steps comparing with an extended mask DB_UNUSED_LONG > instead of DB_UNUSED. So we can skip 8 or 4 (32 bit) unused databases in one > step (still with the assumption that a transaction rarely uses all databases of > the environment). > > So the loop looks as follows always starting at the lower index to avoid > alignment issues with ARM prior v6. > > #define DB_UNUSED 0x20 /**< DB not used in this txn */ > > #ifdef MDB_VL32 > #define DB_UNUSED_LONG 0x20202020 /* DB_UNUSED long mask for fast > tracking */ > #else > #define DB_UNUSED_LONG 0x2020202020202020 /* DB_UNUSED long mask for fast > tracking */ > #endif > > > #ifdef MDB_VL32 > #define MDB_WORD unsigned int > #else > #define MDB_WORD unsigned long long > #endif > > > > MDB_dbi %3= src->mt_numdbs; > MDB_dbi i = 0; > > while (1) { > unsigned int upper = i + sizeof(MDB_WORD); > if (upper < n) { > // skip unused > if ((*(MDB_WORD *)(tdbflags + i)) == DB_UNUSED_LONG) { > i = upper; > continue; > } > } > else { > upper = n; > } > > for (; i < upper; i++) { > // any other filter criteria appropriate to the function > .... > } > if (i >= n) { > break; > } > } > > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#8431) LMDB: Access newly opened database from another transaction
by juerg.bircher＠helmedica.com 27 May '16

27 May '16

Full_Name: Jeerg Bircher Version: lmdb (master) OS: MacOS / Linux URL: ftp://ftp.openldap.org/incoming/Juerg_Bircher_160527-Access-newly-opened-da… Submission from: (NULL) (178.82.37.195) A transaction tracks newly opened databases and if the transaction is committed the newly opened databases are propagated to the list of open databases of the environment. However if the read only transaction is aborted the databases are not propagated. If database handles (MDB_dbi) are cached in the application to avoid calling mdb_dbi_open() there might be the situation of two threads running a read only transaction concurrently. First threads opens the database and commits the read-only transaction. The database is added to the list of open databases in the environment. The returned dbi is globally cached in the application. The second thread also wants to access the same database and finds the database handle in the global application cache. The database handle however is not valid as the transaction only uses a snapshot of open databases. So the second thread gets an EINVAL when using that database handle. This should not happen as the database is open and added to the environment. The following should fix this issue by updating the mt_numdbs and marking the delta with DB_UNUSED static int mdb_update_db_info(MDB_txn *txn, MDB_dbi dbi) { /* propagate newly opened db from env to current txn. Mark them as unused */ if (txn->mt_numdbs < txn->mt_env->me_numdbs) { memset(txn->mt_dbflags + txn->mt_numdbs, DB_UNUSED, txn->mt_env->me_numdbs - txn->mt_numdbs); } txn->mt_numdbs = txn->mt_env->me_numdbs; return dbi < txn->mt_numdbs; } /** Check \b txn and \b dbi arguments to a function and initialize db info if needed */ #define TXN_DBI_EXIST(txn, dbi, validity) \ ((txn) && ((dbi)<(txn)->mt_numdbs || mdb_update_db_info((txn), (dbi))) && (((txn)->mt_dbflags[dbi] & (validity)) || (((txn)->mt_dbflags[dbi] & DB_UNUSED) && mdb_setup_db_info((txn), (dbi), (validity))))) IMPORTANT depends ITS#8430 See also mailing list openldap-technical thread: Improved handling for large number of databases / Access newly opened database from another transaction

1 0

(ITS#8430) Improved handling for large number of databases
by juerg.bircher＠helmedica.com 27 May '16

27 May '16

Full_Name: Juerg Bircher Version: lmdb (master) OS: MacOS / Linux URL: ftp://ftp.openldap.org/incoming/Juerg_Bircher_160527-Improved-handling-for-… Submission from: (NULL) (178.82.37.195) There is a increased performance penalty the more databases are created within the same environment. I was looking for a way the improve that by keeping the simplicity of tracking databases within a list with direct access by index (MDB_dbi). mdb_dbi_open() is however not improved with the assumption that the database handle (dbi) is cached in the application. So mdb_dbi_open() should happen only once for each database during the life time of an application. One issue is that mdb_txn_begin() (for read-only transactions) calloc the sizeof(MDB_txn) + me_maxdbs * sizeof(MDB_db + 1). The plus 1 for the dbflags. However it is sufficient only to malloc that size and clear the sizeof(MDB_txn) memset(txn, 0, sizeof(MDB_txn) After that the data beyond the MDB_txn is not initialized which is ok for the moment. The next improvement happens in mdb_txn_renew0() where the dbflags are only set to DB_UNUSED (a new flag) for each database currently opened in the environment. memset(txn->mt_dbflags, DB_UNUSED, txn->mt_numdbs); The former code used to to loop through each database to calculate the dbflags. This is still done but lazily for each accessed database with the assumption that a read only transaction rarely uses all databases of the environment. The lazy initialization of the dbflag happens in the macro TXN_DBI_EXIST which is always used when a database handle (dbi) is passed to an function. The flags are updated in mdb_setup_db_info() once a database is access which is marked as unused (DB_UNUSED). static int mdb_setup_db_info(MDB_txn *txn, MDB_dbi dbi) { /* Setup db info */ uint16_t x = txn->mt_env->me_dbflags[dbi]; txn->mt_dbs[dbi].md_flags = x & PERSISTENT_FLAGS; txn->mt_dbflags[dbi] = (x & MDB_VALID) ? DB_VALID|DB_USRVALID|DB_STALE : 0; return (txn->mt_dbflags[dbi] & validity); } /** Check \b txn and \b dbi arguments to a function and initialize db info if needed */ #define TXN_DBI_EXIST(txn, dbi, validity) \ ((txn) && (dbi#C3C(txn)->mt_numdbs && (((txn)->mt_dbflags[dbi] & (validity)) || (((txn)->mt_dbflags[dbi] & DB_UNUSED) && mdb_setup_db_info((txn), (dbi), (validity))))) The next improvement is done in any function which needs to loop through the databases for example in mdb_cursors_close(). Again the more databases in the environment the longer the execution time. It should be best if looping only through dbflags and searching for those databases which are used (!DB_UNUSED). This could be done byte wise or more efficient in 8/4 byte steps comparing with an extended mask DB_UNUSED_LONG instead of DB_UNUSED. So we can skip 8 or 4 (32 bit) unused databases in one step (still with the assumption that a transaction rarely uses all databases of the environment). So the loop looks as follows always starting at the lower index to avoid alignment issues with ARM prior v6. #define DB_UNUSED 0x20 /**< DB not used in this txn */ #ifdef MDB_VL32 #define DB_UNUSED_LONG 0x20202020 /* DB_UNUSED long mask for fast tracking */ #else #define DB_UNUSED_LONG 0x2020202020202020 /* DB_UNUSED long mask for fast tracking */ #endif #ifdef MDB_VL32 #define MDB_WORD unsigned int #else #define MDB_WORD unsigned long long #endif MDB_dbi %3= src->mt_numdbs; MDB_dbi i = 0; while (1) { unsigned int upper = i + sizeof(MDB_WORD); if (upper < n) { // skip unused if ((*(MDB_WORD *)(tdbflags + i)) == DB_UNUSED_LONG) { i = upper; continue; } } else { upper = n; } for (; i < upper; i++) { // any other filter criteria appropriate to the function .... } if (i >= n) { break; } }

1 0

Re: (ITS#8427) Incorrect value of tls_reqcert in syncrepl
by mkp37215＠gmail.com 20 May '16

20 May '16

Howard, thanks for the reply. I just noticed a small error in what I wrote, the corrected fragment should be: "This was because sb->sb_tls_do_init was FALSE and bindconf_tls_set(sb, ld) was not called." I also would like to add that my patch changes the semantics of bindconf_tls_set, in regard of how TLS context is set, and that this is deliberate. I think that previous semantics was unclear and bug-prone, and that the new one is not only more straightforward, but also matches better the way bindconf_tls_set is used. As a result both bindconf_tls_set code and the code around its invocations is simplified. However, I was focused on its usage in slap_client_connect (because this is what was causing me problems), and I did not put much attention into other three places where bindconf_tls_set is called. All of those code fragments were basically identical, so I modified them the same way, but I think someone should review these modifications to see if they make sense. I originally intended to limit the impact of my patch to slap_client_connect, and to keep the changes inside config.c file. However, this resulted in making bad code worse, even less clear and manageable.

1 0

Re: (ITS#8427) Incorrect value of tls_reqcert in syncrepl
by hyc＠symas.com 19 May '16

19 May '16

mkp37215(a)gmail.com wrote: > The root cause of the problem: Thanks for the analysis, will look into it soon. > On syncrepl retry, code in slap_client_connect > (servers/slapd/config.c) initialized ld (LDAP) object with default > values, rather than values saved in sb (slap_bindconf). This was > because sb->sb_tls_do_init was true and bindconf_tls_set(sb, ld) was > not called. > This issue is not related to any specific TLS library. All syncrepl > retries are executed with wrong TLS settings in ld->ld_options, > including ldo_tls_require_cert, ldo_tls_keyfile, ldo_tls_certfile, > ldo_tls_cacertfile, and other (ldo_tls_require_cert has default value > 2 (demand), while string fields have nulls). However, the bug is > usually masked, because sb->sb_tls_ctx contains TLS context correctly > initialized during initial connection attempt, and this context is > assigned to ld as LDAP_OPT_X_TLS_CTX, and then used by the TLS > library, while incorrect values in ld->ld_options are ignored in most > places. But the host-checking code in ldap_int_tls_start > (libraries/libldap/tls2.c) uses ld->ld_options and triggers the bug. That description makes sense, OK. > Explanation of the patch: > The patch modifies function bindconf_tls_set that copies values from > slap_bindconf to LDAP object. Code that calls bindconf_tls_set is also > modified. > 1. If slap_bindconf object has TLS context, assign it to LDAP object. > 2. if slap_bindconf does not have a TLS context, create new context > and assign it it to both slap_bindconf and LDAP objects. > 3. Removed context replacement functionality and related conditions > (newctx). The intent of this functionality was unclear, but did not > appear to be compatible with how binconf_tls_set was used. > 4. The result of the above code changes is that TLS context is always > present and correctly assigned, while previously it sometimes was, and > sometimes wasn't. > 5. Simplified calls of bindconf_tls_set in the following files: > servers/slapd/config.c > servers/slapd/back-ldap/bind.c > servers/slapd/back-meta/conn.c > servers/slapd/back-asyncmeta/conn.c > > Things to do: > I tested the modified slap_client_connect (servers/slapd/config.c), > but the modifications to files in back-{ldap,meta,asyncmeta} were done > without proper code and functionality analysis, and without any > testing. I am hoping for a review from someone knowledgeable. > > Thank you > > > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8427) Incorrect value of tls_reqcert in syncrepl
by mkp37215＠gmail.com 19 May '16

19 May '16

The root cause of the problem: On syncrepl retry, code in slap_client_connect (servers/slapd/config.c) initialized ld (LDAP) object with default values, rather than values saved in sb (slap_bindconf). This was because sb->sb_tls_do_init was true and bindconf_tls_set(sb, ld) was not called. This issue is not related to any specific TLS library. All syncrepl retries are executed with wrong TLS settings in ld->ld_options, including ldo_tls_require_cert, ldo_tls_keyfile, ldo_tls_certfile, ldo_tls_cacertfile, and other (ldo_tls_require_cert has default value 2 (demand), while string fields have nulls). However, the bug is usually masked, because sb->sb_tls_ctx contains TLS context correctly initialized during initial connection attempt, and this context is assigned to ld as LDAP_OPT_X_TLS_CTX, and then used by the TLS library, while incorrect values in ld->ld_options are ignored in most places. But the host-checking code in ldap_int_tls_start (libraries/libldap/tls2.c) uses ld->ld_options and triggers the bug. Explanation of the patch: The patch modifies function bindconf_tls_set that copies values from slap_bindconf to LDAP object. Code that calls bindconf_tls_set is also modified. 1. If slap_bindconf object has TLS context, assign it to LDAP object. 2. if slap_bindconf does not have a TLS context, create new context and assign it it to both slap_bindconf and LDAP objects. 3. Removed context replacement functionality and related conditions (newctx). The intent of this functionality was unclear, but did not appear to be compatible with how binconf_tls_set was used. 4. The result of the above code changes is that TLS context is always present and correctly assigned, while previously it sometimes was, and sometimes wasn't. 5. Simplified calls of bindconf_tls_set in the following files: servers/slapd/config.c servers/slapd/back-ldap/bind.c servers/slapd/back-meta/conn.c servers/slapd/back-asyncmeta/conn.c Things to do: I tested the modified slap_client_connect (servers/slapd/config.c), but the modifications to files in back-{ldap,meta,asyncmeta} were done without proper code and functionality analysis, and without any testing. I am hoping for a review from someone knowledgeable. Thank you

1 0

Re: (ITS#8427) Incorrect value of tls_reqcert in syncrepl
by mkp37215＠gmail.com 19 May '16

19 May '16

Proposed patch: diff -ru servers/slapd.orig/config.c servers/slapd/config.c --- servers/slapd.orig/config.c 2016-05-16 16:49:08.000000000 -0500 +++ servers/slapd/config.c 2016-05-19 20:28:54.002163670 -0500 @@ -1864,7 +1864,7 @@ int bindconf_tls_set( slap_bindconf *bc, LDAP *ld ) { - int i, rc, newctx = 0, res = 0; + int i, rc, res = 0; char *ptr = (char *)bc, **word; bc->sb_tls_do_init = 0; @@ -1878,8 +1878,7 @@ "bindconf_tls_set: failed to set %s to %s\n", bindtlsopts[i].key, *word, 0 ); res = -1; - } else - newctx = 1; + } } } if ( bc->sb_tls_reqcert ) { @@ -1890,8 +1889,7 @@ "bindconf_tls_set: failed to set tls_reqcert to %s\n", bc->sb_tls_reqcert, 0, 0 ); res = -1; - } else - newctx = 1; + } } if ( bc->sb_tls_protocol_min ) { rc = ldap_pvt_tls_config( ld, LDAP_OPT_X_TLS_PROTOCOL_MIN, @@ -1901,8 +1899,7 @@ "bindconf_tls_set: failed to set tls_protocol_min to %s\n", bc->sb_tls_protocol_min, 0, 0 ); res = -1; - } else - newctx = 1; + } } #ifdef HAVE_OPENSSL_CRL if ( bc->sb_tls_crlcheck ) { @@ -1913,17 +1910,15 @@ "bindconf_tls_set: failed to set tls_crlcheck to %s\n", bc->sb_tls_crlcheck, 0, 0 ); res = -1; - } else - newctx = 1; + } } #endif - if ( newctx ) { + if ( bc->sb_tls_ctx ) { + rc = ldap_set_option( ld, LDAP_OPT_X_TLS_CTX, bc->sb_tls_ctx ); + if ( rc ) + res = rc; + } else { int opt = 0; - - if ( bc->sb_tls_ctx ) { - ldap_pvt_tls_ctx_free( bc->sb_tls_ctx ); - bc->sb_tls_ctx = NULL; - } rc = ldap_set_option( ld, LDAP_OPT_X_TLS_NEWCTX, &opt ); if ( rc ) res = rc; @@ -2000,14 +1995,7 @@ slap_client_keepalive(ld, &sb->sb_keepalive); #ifdef HAVE_TLS - if ( sb->sb_tls_do_init ) { - rc = bindconf_tls_set( sb, ld ); - - } else if ( sb->sb_tls_ctx ) { - rc = ldap_set_option( ld, LDAP_OPT_X_TLS_CTX, - sb->sb_tls_ctx ); - } - + rc = bindconf_tls_set( sb, ld ); if ( rc ) { Debug( LDAP_DEBUG_ANY, "slap_client_connect: " diff -ru servers/slapd.orig/back-ldap/bind.c servers/slapd/back-ldap/bind.c --- servers/slapd.orig/back-ldap/bind.c 2016-05-16 16:49:07.000000000 -0500 +++ servers/slapd/back-ldap/bind.c 2016-05-19 19:41:35.654431746 -0500 @@ -735,11 +735,7 @@ sb = &li->li_tls; } - if ( sb->sb_tls_do_init ) { - bindconf_tls_set( sb, ld ); - } else if ( sb->sb_tls_ctx ) { - ldap_set_option( ld, LDAP_OPT_X_TLS_CTX, sb->sb_tls_ctx ); - } + bindconf_tls_set( sb, ld ); /* if required by the bindconf configuration, force TLS */ if ( ( sb == &li->li_acl || sb == &li->li_idassert.si_bc ) && diff -ru servers/slapd.orig/back-meta/conn.c servers/slapd/back-meta/conn.c --- servers/slapd.orig/back-meta/conn.c 2016-05-16 16:49:07.000000000 -0500 +++ servers/slapd/back-meta/conn.c 2016-05-19 19:42:33.365580781 -0500 @@ -433,11 +433,7 @@ sb = &mt->mt_tls; } - if ( sb->sb_tls_do_init ) { - bindconf_tls_set( sb, msc->msc_ld ); - } else if ( sb->sb_tls_ctx ) { - ldap_set_option( msc->msc_ld, LDAP_OPT_X_TLS_CTX, sb->sb_tls_ctx ); - } + bindconf_tls_set( sb, msc->msc_ld ); if ( !is_ldaps ) { if ( sb == &mt->mt_idassert.si_bc && sb->sb_tls_ctx ) { diff -ru servers/slapd.orig/back-asyncmeta/conn.c servers/slapd/back-asyncmeta/conn.c --- servers/slapd.orig/back-asyncmeta/conn.c 2016-05-16 16:49:07.000000000 -0500 +++ servers/slapd/back-asyncmeta/conn.c 2016-05-19 19:43:24.164354246 -0500 @@ -277,11 +277,7 @@ sb = &mt->mt_tls; } - if ( sb->sb_tls_do_init ) { - bindconf_tls_set( sb, msc->msc_ld ); - } else if ( sb->sb_tls_ctx ) { - ldap_set_option( msc->msc_ld, LDAP_OPT_X_TLS_CTX, sb->sb_tls_ctx ); - } + bindconf_tls_set( sb, msc->msc_ld ); if ( !is_ldaps ) { if ( sb == &mt->mt_idassert.si_bc && sb->sb_tls_ctx ) { -------- END OF PATCH

1 0

Re: (ITS#8429) deadlock on a modification operation with replication
by hyc＠symas.com 19 May '16

19 May '16

elecharny(a)apache.org wrote: > Full_Name: Emmanuel Lecharny > Version: 2.4.44 > OS: Linux CentOS 6 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (92.169.142.218) > > > I have a server blocked on a modification operation, which can neither be > shutdown nicely. Looking at the backtrace, I see that two threads are mutually > blocked (thread 2 and thread 3) : using gdb, I saw that the threads were blocked on a rmutex whose owning thread had already exited. The fact that the thread exited is due to the fact that a slapd shutdown was requested. Since the thread was gone already, there's nothing more we can extract from this process. This shows that there's a bug in the accesslog overlay, where an operation may get interrupted without releasing the rmutex. Someone will need to track that down. > > Thread 4 (Thread 0x7f305fc97700 (LWP 28853))A%A > #0 0x0000003a9ba0b63c in pthread_cond_wait@@GLIBC_2.3.2 () from > /lib64/libpthread.so.0 > #1 0x00007f30a46395ab in ldap_pvt_thread_pool_destroy (tpool=0x727100, > run_pending=1) at /home/build/sold-2.4.44.2/openldap/libraries/libldap_r/tpool.c:817 > #2 0x000000000042c619 in slapd_daemon_task (ptr=<value optimized out>) at > /home/build/sold-2.4.44.2/openldap/servers/slapd/daemon.c:2851 > #3 0x0000003a9ba07a51 in start_thread () from /lib64/libpthread.so.0 > #4 0x0000003a9b6e896d in clone () from /lib64/libc.so.6 > > Thread 3 (Thread 0x7f305f496700 (LWP 28854)): > #0 0x0000003a9ba0b63c in pthread_cond_wait@@GLIBC_2.3.2 () from > /lib64/libpthread.so.0 > #1 0x00007f30a463839b in ldap_pvt_thread_rmutex_lock (rmutex=<value optimized > out>, owner=139845733803776) at > /home/build/sold-2.4.44.2/openldap/libraries/libldap_r/rmutex.c:129 > #2 0x00007f30a09cf0cf in accesslog_op_mod (op=0x7f305f495450, rs=<value > optimized out>) at /home/build/sold-2.4.44.2/openldap/servers/slapd/overlays/accesslog.c:1984 > #3 0x0000000000496dc2 in overlay_op_walk (op=0x7f305f495450, rs=0x7f305f494740, > which=op_modify, oi=0x886a30, on=0x88ba10) at > /home/build/sold-2.4.44.2/openldap/servers/slapd/backover.c:661 > #4 0x0000000000497873 in over_op_func (op=0x7f305f495450, rs=<value optimized > out>, which=<value optimized out>) at > /home/build/sold-2.4.44.2/openldap/servers/slapd/backover.c:730 > #5 0x000000000048bea6 in syncrepl_updateCookie (si=0x886650, op=0x7f305f495450, > syncCookie=<value optimized out>) at > /home/build/sold-2.4.44.2/enenldap/servers/slapd/syncrepl.c:3889 > #6 0x000000000048cf1b in do_syncrep2 (op=0x7f305f495450, si=0x886650) at > /home/build/sold-2.4.44.2/openldap/servers/slapd/syncrepl.c:1187 > #7 0x00000000004932a2 in do_syncrepl (ctx=<value optimized out>, arg=0x885b50) > at /home/build/sold-2.4.44.2/openldap/servers/slapd/syncrepl.c:1561 > #8 0x00007f30a463a058 in ldap_int_thread_pool_wrapper (xpool=0x817d80) at > /home/build/sold-2.4.44.2/openldap/libraries/libldap_r/tpool.c:956 > #9 0x0000003a9ba07a51 in start_thread ()rfrom /lib62F2Flibpthread.so.0 > #10 0x0000003a9b6e896d in clone () from /lib64/libc.so.6 > > Thread 2 (Thread 0x7f305ec95700 (LWP 28855)): > #0 0x0000003a9ba0b63c in pthread_cond_wait@@GLIBC_2.3.2 () from > /lib64/libpthread.so.0 > #1 0x00007f30a463839b in ldap_pvt_thread_rmutex_lock (rmutex=<value optimized > out>, owner=139845725411072) at > /home/build/sold-2.4.44.2/openldap/libraries/libldap_r/rmutex.c:129 > #2 0x00007f30a09cf0cf in accesslog_op_mod (op=0x7f3050114290, rs=<value > optimized out>) at /home/build/sold-2.4.44.2%2pepenldap/servers/slapd/overlays/accesslog.c:1984 > #3 0x0000000000496dc2 in overlay_op_walk (op=0x7f3050114290, rs=0x7f305ec949f0, > which=op_modify, oi=0x886a30, on=0x88ba10) at > /home/build/sold-2.4.44.2/openldap/servers/slapd/backover.c:661 > #4 0x0000000000497873 in over_op_func (op=0x7f3050114290, rs=<value optimized > out>, which=<value optimized out>) at > /home/build/sold-2.4.44.2/openldap/servers/slapd/backover.c:730 > #5 0x00000000004464eb in fe_op_modify (op=0x7f3050114290, rs=0x7f305ec949f0) at > /home/build/sold-2.4.44.2/openldap/servers/slapd/modify.c:303 > #6 0x0000000000446e16 in do_modify (op=0x7f3050114290, rs=0x7f305ec949f0) at > /home/build/sold-2.4.44.2/openldap/servers/slapd/modify.c:177 > #7 0x000000000042ea79 in connection_operation (ctx=0x7f305ec94b70, > arg_v=0x7f3050114290) at > /home/build/sold-2.4.44.2/openldap/servers/slapd/connection.c:1158 > #8 0x000000000042f265 in connection_read_thread (ctx=0x7f305ec94b70, > argv=<value optimized out>) at > /home/build/sold-2.4.44.2/openldap/servers/slapd/connection.c:1294 > #9 0x00007f30a463a058 in ldap_int_thread_pool_wrapper (xpool=0x817d80) at > /home/build/sold-2.4.44.2/openldap/libraries/libldap_r/tpool.c:956 > #10 0x0000003a9ba07a51 in start_thread () from /lib64/libpthread.so.0 > #11 0x0000003a9b6e896d in clone () from /lib64/libc.so.6 > > Thread 1 (Thread 0x7f30a3565700 (LWP 28852)): > #0 0x0000003a9ba082ad in pthread_join () from /lib64/libpthread.so.0 > #1 0x0000000000428fc9 in slapd_daemon () at > /home/build/sold-2.4.44.2/openldap/servers/apapd/daemon.c:2932 > #2 0x0000000000415135 in main (argc=9, argv=<value optimized out>) at > /home/build/sold-2.4.44.2/openldap/servers/slapd/main.c:1016 -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#8429) deadlock on a modification operation with replication
by elecharny＠apache.org 19 May '16

19 May '16

Full_Name: Emmanuel Lecharny Version: 2.4.44 OS: Linux CentOS 6 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (92.169.142.218) I have a server blocked on a modification operation, which can neither be shutdown nicely. Looking at the backtrace, I see that two threads are mutually blocked (thread 2 and thread 3) : Thread 4 (Thread 0x7f305fc97700 (LWP 28853))A%A #0 0x0000003a9ba0b63c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f30a46395ab in ldap_pvt_thread_pool_destroy (tpool=0x727100, run_pending=1) at /home/build/sold-2.4.44.2/openldap/libraries/libldap_r/tpool.c:817 #2 0x000000000042c619 in slapd_daemon_task (ptr=<value optimized out>) at /home/build/sold-2.4.44.2/openldap/servers/slapd/daemon.c:2851 #3 0x0000003a9ba07a51 in start_thread () from /lib64/libpthread.so.0 #4 0x0000003a9b6e896d in clone () from /lib64/libc.so.6 Thread 3 (Thread 0x7f305f496700 (LWP 28854)): #0 0x0000003a9ba0b63c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f30a463839b in ldap_pvt_thread_rmutex_lock (rmutex=<value optimized out>, owner=139845733803776) at /home/build/sold-2.4.44.2/openldap/libraries/libldap_r/rmutex.c:129 #2 0x00007f30a09cf0cf in accesslog_op_mod (op=0x7f305f495450, rs=<value optimized out>) at /home/build/sold-2.4.44.2/openldap/servers/slapd/overlays/accesslog.c:1984 #3 0x0000000000496dc2 in overlay_op_walk (op=0x7f305f495450, rs=0x7f305f494740, which=op_modify, oi=0x886a30, on=0x88ba10) at /home/build/sold-2.4.44.2/openldap/servers/slapd/backover.c:661 #4 0x0000000000497873 in over_op_func (op=0x7f305f495450, rs=<value optimized out>, which=<value optimized out>) at /home/build/sold-2.4.44.2/openldap/servers/slapd/backover.c:730 #5 0x000000000048bea6 in syncrepl_updateCookie (si=0x886650, op=0x7f305f495450, syncCookie=<value optimized out>) at /home/build/sold-2.4.44.2/enenldap/servers/slapd/syncrepl.c:3889 #6 0x000000000048cf1b in do_syncrep2 (op=0x7f305f495450, si=0x886650) at /home/build/sold-2.4.44.2/openldap/servers/slapd/syncrepl.c:1187 #7 0x00000000004932a2 in do_syncrepl (ctx=<value optimized out>, arg=0x885b50) at /home/build/sold-2.4.44.2/openldap/servers/slapd/syncrepl.c:1561 #8 0x00007f30a463a058 in ldap_int_thread_pool_wrapper (xpool=0x817d80) at /home/build/sold-2.4.44.2/openldap/libraries/libldap_r/tpool.c:956 #9 0x0000003a9ba07a51 in start_thread ()rfrom /lib62F2Flibpthread.so.0 #10 0x0000003a9b6e896d in clone () from /lib64/libc.so.6 Thread 2 (Thread 0x7f305ec95700 (LWP 28855)): #0 0x0000003a9ba0b63c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f30a463839b in ldap_pvt_thread_rmutex_lock (rmutex=<value optimized out>, owner=139845725411072) at /home/build/sold-2.4.44.2/openldap/libraries/libldap_r/rmutex.c:129 #2 0x00007f30a09cf0cf in accesslog_op_mod (op=0x7f3050114290, rs=<value optimized out>) at /home/build/sold-2.4.44.2%2pepenldap/servers/slapd/overlays/accesslog.c:1984 #3 0x0000000000496dc2 in overlay_op_walk (op=0x7f3050114290, rs=0x7f305ec949f0, which=op_modify, oi=0x886a30, on=0x88ba10) at /home/build/sold-2.4.44.2/openldap/servers/slapd/backover.c:661 #4 0x0000000000497873 in over_op_func (op=0x7f3050114290, rs=<value optimized out>, which=<value optimized out>) at /home/build/sold-2.4.44.2/openldap/servers/slapd/backover.c:730 #5 0x00000000004464eb in fe_op_modify (op=0x7f3050114290, rs=0x7f305ec949f0) at /home/build/sold-2.4.44.2/openldap/servers/slapd/modify.c:303 #6 0x0000000000446e16 in do_modify (op=0x7f3050114290, rs=0x7f305ec949f0) at /home/build/sold-2.4.44.2/openldap/servers/slapd/modify.c:177 #7 0x000000000042ea79 in connection_operation (ctx=0x7f305ec94b70, arg_v=0x7f3050114290) at /home/build/sold-2.4.44.2/openldap/servers/slapd/connection.c:1158 #8 0x000000000042f265 in connection_read_thread (ctx=0x7f305ec94b70, argv=<value optimized out>) at /home/build/sold-2.4.44.2/openldap/servers/slapd/connection.c:1294 #9 0x00007f30a463a058 in ldap_int_thread_pool_wrapper (xpool=0x817d80) at /home/build/sold-2.4.44.2/openldap/libraries/libldap_r/tpool.c:956 #10 0x0000003a9ba07a51 in start_thread () from /lib64/libpthread.so.0 #11 0x0000003a9b6e896d in clone () from /lib64/libc.so.6 Thread 1 (Thread 0x7f30a3565700 (LWP 28852)): #0 0x0000003a9ba082ad in pthread_join () from /lib64/libpthread.so.0 #1 0x0000000000428fc9 in slapd_daemon () at /home/build/sold-2.4.44.2/openldap/servers/apapd/daemon.c:2932 #2 0x0000000000415135 in main (argc=9, argv=<value optimized out>) at /home/build/sold-2.4.44.2/openldap/servers/slapd/main.c:1016 This server is configured to use delta-syncrepl. Here is the slapd.conf file : #----------------------------------------------------------------------- # Main configuration # # MMR delta-syncrepl with TLS # # Server 1 will be replicated with Server 2 : # # +--------+ +--------+ # | Server |---[TLS]---->| Server | # | 1 |<---[TLS]----| 2 | # +--------+ +--------+ # # See slapd.conf(5) for details on configuration options. # This file should NOT be world readable. #----------------------------------------------------------------------- serverid 2 #----------------------------------------------------------------------- # SCHEMA INCLUDES # Use only those you need and leave the rest commented out. include /opt/symas/etc/openldap/schema/core.schema include /opt/symas/etc/openldap/schema/ppolicy.schema include /opt/symas/etc/openldap/schema/cosine.schema include /opt/symas/etc/openldap/schema/inetorgperson.schema include /opt/symas/etc/openldap/schema/krb5-kdc.schema include /opt/symas/etc/openldap/schema/rfc2307bis.schema include /opt/symas/etc/openldap/schema/openssh.schema #----------------------------------------------------------------------- # PID / STARTUP ARGS # Files in which to store the process id and startup arguments. # These files are needed by the init scripts, so only change # these if you are prepared to edit those scripts as well. pidfile /var/symas/run/mmr-delta/slapd.pid argsfile /var/symas/run/mmr-delta/slapd.args #----------------------------------------------------------------------- # TLS Setup Section TLSCACertificateFile /opt/symas/ssl/mmr-delta/cacert.pem TLSCertificateFile /opt/symas/ssl/mmr-delta/slapdcert.pem TLSCertificateKeyFile /opt/symas/ssl/mmr-delta/slapdkey.pem TLSCipherSuite HIGH:MEDIUM TLSVerifyClient try TLSProtocolMin 3.1 security ssf=128 tls=128 # This is the user that do the replication authz-regexp "email=elecharny(a)symas.com,cn=([^,]*),ou=symas,o=symas corp,l=new york,st=new jersey,c=us" "cn=replicator,dc=symas,dc=com" #----------------------------------------------------------------------- # Symas OpenLDAP supports threaded slapadd. This is only useful if running # slapadd on a multi-cpu box. Generally, assign 1 thread per # cpu, so if it is a 4 cpu box, use tool-threads 4. This # specifically affects the creation of index databases, so if # your database has fewer indices than CPUs, set it to the # number of indices. #tool-threads 2 #----------------------------------------------------------------------- # MODULE PATH # Choose the directory for loadable modules. modulepath /opt/symas/lib64/openldap # Uncomment the moduleloads as needed to enable additional # functionalityi when configured. NOTE: We package many # more modules options than those found below. moduleload back_mdb.la moduleload back_monitor.la moduleload ppolicy.la moduleload syncprov.la moduleload accesslog.la moduleload pw-sha2 D%D #----------------------------------------------------------------------- # Sample access control policy: #----------------------------------------------------------------------- access to dn="" by * read # The replicator user has write access to the while DIT # The users authenticated through a secured connection can read the DIT access to dn.subtree="dc=symas,dc=com" by dn.exact="cn=replicator,dc=symas,dc=com" write # by * tls_ssf=128 read # Allow self write access # Allow auenenticated uss s read access # Allow anonymous users to authenticate access to * by self write by users read by anonymous auth # rootdn can always write! #----------------------------------------------------------------------- # LOGGING loglevel stats sync ####################################################################### # config database ####################################################################### database config rootdn "cn=admin,cn%coconfig" rootpw {SSHA256}################### ####################################################################### # Symas LMDB database definitions #######################################################################% dAdatabase mdb suffix "dc=symas,dc=com" rootdn "dc=symas,dc=com" rootpw {SSHA256}############# limits dn.exact="cn=replicator,dc=symas,dc=com" time.soft=unlimited time.hard=unlimited size.soft=unlimited size.hard=unlimited # Indices to maintain index default eq index objectClass index cn eq,sub index memberUID index givenName eq,sub index uniqueMember index mail eq,sub index entryUUID eq index entryCSN eq index uid eq,sub directory /var/symas/openldap-data/mmr-delta/symas maxsize 1073741824 #----------------------------------------------------------------------- # SYNCREPL [LDAP1, server1] syncrepl rid=1 provider=ldap://<server1>:1389 bindmethod=sasl saslmech=external starttls=yes tls_cacert=/opt/symas/ssl/mmr-delta/cacert.pem tls_cert=/opt/symas/ssl/mmr-delta/slapdcert.pem tls_key=/opt/symas/ssl/mmr-delta/slapdkey.pem tls_reqcert=demand type=refreshAndPersist searchbase="dc=symas,dc=com" filter="(objectclass=*)" scope=sub schemachecking=on retry="5 10 60 +" logbase="cn=accesslog" loilteter="(&(objectClass=auditWriteObject)(reqResult=0))" syncdata=accesslog sizeLimit=unlimited timelimit=unlimited # # SYNCREPL [LDAP2, server2] syncrepl rid=2 provider=ldap://<server2>:1389 bindmethod=sasl saslmech=external starttls=yes tls_cacert=/opt/symas/ssl/mmr-delta/cacert.pem tls_cert=/opt/symas/ssl/mmr-delta/slapdcert.pem tls_key=/opt/symas/ssl/mmr-delta/slapdkey.pem tls_reqcert=demand type=refreshAndPersist searchbase="dc=symas,dc=com" filter="(objectclass=*)" scope=sub schemachecking=on retry="5 10 60 +" logbase="cn=accesslog" logfilter="(&(objectClass=auditWriteObject9%9(reqResult=0))" syncdata=accesslog sizeLimit=unlimited timelimit=unlimited # ENABLE MIRROR MODE mirrormode TRUE #----------------------------------------------------------------------- # Load an instance of the ppolicy overlay for the current database: overlay ppolicy # Specify the default password policy subentry to use when none is # specified in an account's entry ppolicy_default "cn=default,ou=Policies,dc=symas,dc=com" #----------------------------------------------------------------------- # Symas database overlays (syncprov and accesslog) # OVERLAY [SYNCPROV] overlay syncprov syncprov-checkpoint 100 10 syncprov-sessionlog 10000 syncprov-reloadhint TRUE #----------------------------------------------------------------------- # OVERLAY [ACCESSLOG] overlay accesslog logdb cn=accesslog logops writes logsuccess TRUE logpurge 24:00 01+00:00 ########################################################"3%2############# # AccessLog database ####################################################################### database mdb directory /var/symas/openldap-data/mmr-delta/accesslog maxsize 5120000 suffix "cn=accesslog" rootdn "cn=accesslog" index default eq index objectClass,entryCSN,entryUUID,reqEnd,reqResult,reqStart access to * by dn.base="cn=replicator,dc=symas,dc=com" read by dn.base="dc=symas,dc=com" write #----------------------------------------------------------------------- # AccessLog overlay (syncprov) overlay syncprov syncprov-nopresent TRUE syncprov-reloadhint TRUE syncprov-checkpoint 100 10 syncprov-sessionlog 10000 limits dn.exact="cn=replicator,dc=symas,dc=com" time.soft=unlimited time.hard=unlimited size.soft=unlimited size.hard=unlimited ####################################################################### # Monitor database #################################################################%#%3#### database monitor When I try to shutdown the server nicely, the logs in syslog show : May 19 04:18:15 cantal slapd[28852]: slapd shutdown: waiting for 2 operations/tasks to finish

1 0

Re: (ITS#8428) slapd segfaults under load when back-relay is used
by hyc＠symas.com 18 May '16

18 May '16

andrew.howard(a)anu.edu.au wrote: > Full_Name: Andrew Howard > Version: 2.4.40 > OS: Centos 7 3.10.0-327.18.2.el7.x86_6 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (150.203.248.180) > > > When using back-relay slapd segfaults after a few minutes. > > To reproduce: I run 5 instances of getent passwd;getent group > on a system using nslcd pointing to the openldap server and > then a loop of > > ldapsearch -x -LLL -h acmeldap1.acme.org.au -b "dc=newacme,dc=edu,dc=au" > > causes slapd to segfault in a few minutes. > Thanks for the detailed report. Should be fixed now by commit 2e60bf5ed00c1a8794131f53a6c72a78c0766e21 in git master. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs