Full_Name: Howard Chu
Version: 2.4
OS:
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (110.49.207.42)
Submitted by: hyc
Add a config keyword to specify DN subtrees to be processed by the overlay (thus
ignoring anything that doesn't match)
richton(a)nbcs.rutgers.edu wrote:
> Full_Name: Aaron Richton
> Version: RE24
> OS: Fedora 20
> URL: ftp://ftp.openldap.org/incoming/
> Submission from: (NULL) (128.6.31.135)
>
>
> While performing RE24 testing observed an issue in slapd-mtread:
>
> ==28795== Process terminating with default action of signal 11 (SIGSEGV):
> dumping core
> ==28795== Access not within mapped region at address 0x2FE9F9D0
> ==28795== at 0x3A45E0A1%3: pthread_join (pthread_join.c:47)
> ==28795== by 0x4A09005: ??? (in
> /usr/lib64/valgrind/vgpreload_helgrind-amd64-linux.so)
> ==28795== by 0x4A0BBA5: pthread_join (in
> /usr/lib64/valgrind/vgpreload_helgrind-amd64-linux.so)
> ==28795== by 0 0x409A2A: ldap_pvt_thread_join (thr_posix.c:197)
> ==28795== by 0x405540: main (slapd-mtread.c:361)
>
> Hmm, not particularly helpful. Gave this a run through helgrind, report below.
> So far as I can tell we had one thread in ldap_dup protected by ldcmutex:
>
> 542 LDAP_MUTEX_LOCK( &old->ld_ldcmutex );
> 543 ld->ldc = old->ldc;
> 544 old->ld_ldcrefcnt++;
> 545 LDAP_MUTEX_UNLOCK( &old->ld_ldcmutex );
>
> while ldap_get_option(LDAP_OPT_SESSION_REFCNT) is protected by ldo_mutex only.
> As a quick attempt:
Thanks. I see your patch is this way to avoid lock ordering issues. The
original code in libldap/unbind.c seems to hold ldcmutex for the entire
duration of the function, without any good reason. I suspect we can just
move the mutex lock/unlock to just around the actual option fetch.
> --- a/libraries/libldap/options.c
> +++ b/libraries/libldap/options.c
> @@ -123,6 +123,10 @@ ldap_get_option(
> return LDAP_OPT_ERROR;
> }
>
> + if( option == LDAP_OPT_SESSION_REFCNT ) {
> + LDAP_MUTEX_LOCK( &ld->ld_ldcmutex );
> + }
> +
> LDAP_MUTEX_LOCK( &lo->ldo_mutex );
>
> switch(option) {
> @@ -414,6 +418,11 @@ ldap_get_option(
> }
>
> LDAP_MUTEX_UNLOCK( &lo->ldo_mutex );
> +
> + if( option == LDAP_OPT_SESSION_REFCNT ) {
> + LDAP_MUTEX_UNLOCK( &ld->ld_ldcmutex );
> + }
> +
> return ( rc );
> }
>
> helgrind looks happy with this. The if() is obviously optional but is hopefully
> a small performance gain for ldap_get_option(others).
>
> helgrind report:
>
> ==28795== Lock at 0x4C393D0 was first observed
> ==28795== at 0x4A0BCCF: pthread_mutex_init (in
> /usr/lib64/valgrind/vgpreload_helgrind-amd64-linux.so)
> ==28795== by 0x409B08: ldap_pvt_thread_mutex_init (thr_posix.c:383)
> ==28795== by 0x40A25A: ldap_create (open.c:180)
> ==28795== by 0x40A3E4: ldap_initialize (open.c:240)
> ==28795== by 0x4062B7: do_conn (slapd-mtread.c:584)
> ==28795== by 0x40530A: main (slapd-mtread.c:331)
> ==28795==
> ==28795== Lock at 0x4C393A8 was first observed
> ==28795== at 0x4A0BCCF: pthread_mutex_init (in
> /usr/lib64/valgrind/vgpreload_helgrind-amd64-linux.so)
> ==28795== by 0x409B08: ldap_pvt_thread_mutex_init (thr_posix.c:283)
> ==28795== by 0x409FBF: ldap_create (open.c:129)
> ==28795%%3= by 0xA3A3E4: ldap_initialize (open.c:240)
> ==28795== by 0x4062B7: do_conn (slapd-mtread.c:584)
> ==28795== by 0x40530A: main (slapd-mtread.c:331)
> ==28795==
> ==28795== Possible data race during write of size 4 at 0x4C39238 by thread #%2
> ==28795== Locks held: 1, at address 0x4C393D0
> ==28795== at 0x412AC1: ldap_ld_free (unbind.c:87)
> ==28795== by 0x4130EE: ldap_destroy (unbind.c:245)
> ==28795== by 0x405BEF: do_onethread (slapd-mtread.c:455)
> ==28795== by 0x4A0AECD: ??? (in
> /usr/lib64/valgrind/vgpreload_helgrind-amd64-linux.so)
> ==28795== by 0x3A45E07EE4: start_thread (pthread_create.c:309)
> ==28795== by 0x3A45AF4B8C: clone (clone.S:111)
> ==28795==
> ==28795== This conflicts with a previous read of size 4 by thread #29
> ==28795%%3= Locks held: 1, at address 0x4C393A8
> ==28795== at 0x426957: ldap_get_option (options.c:374)
> ==28795== by 0x405914: do_onethread (slapd-mtread.c:423)
> ==28795== by 0x4A0AECD: ??? (in
> /usr/lib64/valgrind/vgpreload_helgrind-amd64-linux.so)
> ==28795== by 0x3A45E07EE4: start_thread (pthread_create.c:309)
> ==28795== by 0x3A45AF4B8C: clone (clone.S:111)
> ==28795==
> ==28795== Address 0x4C39238 is 88 bytes inside a block of size 736 alloc'd
> ==28795== at 0x4A083F4: calloc (in
> /usr/lib64/valgrind/vgpreload_helgrind-amd64-linux.so)
> ==28795== by 0x43E4B1: ber_memcalloc_x (memory.c:283)
> ==28795== by 0x409F4B: ldap_create (open.c:119)
> ==28795== by 0x40A3E4: ldap_initialize (open.c:240)
> ==28795== by 0x4062B7: do_conn (slapd-mtread.c:584)
> ==28795== by 0x40530A: main (slapd-mtread.c:331)
>
>
>
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
Full_Name: Aaron Richton
Version: RE24
OS: Fedora 20
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (128.6.31.135)
While performing RE24 testing observed an issue in slapd-mtread:
==28795== Process terminating with default action of signal 11 (SIGSEGV):
dumping core
==28795== Access not within mapped region at address 0x2FE9F9D0
==28795== at 0x3A45E0A1%3: pthread_join (pthread_join.c:47)
==28795== by 0x4A09005: ??? (in
/usr/lib64/valgrind/vgpreload_helgrind-amd64-linux.so)
==28795== by 0x4A0BBA5: pthread_join (in
/usr/lib64/valgrind/vgpreload_helgrind-amd64-linux.so)
==28795== by 0 0x409A2A: ldap_pvt_thread_join (thr_posix.c:197)
==28795== by 0x405540: main (slapd-mtread.c:361)
Hmm, not particularly helpful. Gave this a run through helgrind, report below.
So far as I can tell we had one thread in ldap_dup protected by ldcmutex:
542 LDAP_MUTEX_LOCK( &old->ld_ldcmutex );
543 ld->ldc = old->ldc;
544 old->ld_ldcrefcnt++;
545 LDAP_MUTEX_UNLOCK( &old->ld_ldcmutex );
while ldap_get_option(LDAP_OPT_SESSION_REFCNT) is protected by ldo_mutex only.
As a quick attempt:
--- a/libraries/libldap/options.c
+++ b/libraries/libldap/options.c
@@ -123,6 +123,10 @@ ldap_get_option(
return LDAP_OPT_ERROR;
}
+ if( option == LDAP_OPT_SESSION_REFCNT ) {
+ LDAP_MUTEX_LOCK( &ld->ld_ldcmutex );
+ }
+
LDAP_MUTEX_LOCK( &lo->ldo_mutex );
switch(option) {
@@ -414,6 +418,11 @@ ldap_get_option(
}
LDAP_MUTEX_UNLOCK( &lo->ldo_mutex );
+
+ if( option == LDAP_OPT_SESSION_REFCNT ) {
+ LDAP_MUTEX_UNLOCK( &ld->ld_ldcmutex );
+ }
+
return ( rc );
}
helgrind looks happy with this. The if() is obviously optional but is hopefully
a small performance gain for ldap_get_option(others).
helgrind report:
==28795== Lock at 0x4C393D0 was first observed
==28795== at 0x4A0BCCF: pthread_mutex_init (in
/usr/lib64/valgrind/vgpreload_helgrind-amd64-linux.so)
==28795== by 0x409B08: ldap_pvt_thread_mutex_init (thr_posix.c:383)
==28795== by 0x40A25A: ldap_create (open.c:180)
==28795== by 0x40A3E4: ldap_initialize (open.c:240)
==28795== by 0x4062B7: do_conn (slapd-mtread.c:584)
==28795== by 0x40530A: main (slapd-mtread.c:331)
==28795==
==28795== Lock at 0x4C393A8 was first observed
==28795== at 0x4A0BCCF: pthread_mutex_init (in
/usr/lib64/valgrind/vgpreload_helgrind-amd64-linux.so)
==28795== by 0x409B08: ldap_pvt_thread_mutex_init (thr_posix.c:283)
==28795== by 0x409FBF: ldap_create (open.c:129)
==28795%%3= by 0xA3A3E4: ldap_initialize (open.c:240)
==28795== by 0x4062B7: do_conn (slapd-mtread.c:584)
==28795== by 0x40530A: main (slapd-mtread.c:331)
==28795==
==28795== Possible data race during write of size 4 at 0x4C39238 by thread #%2
==28795== Locks held: 1, at address 0x4C393D0
==28795== at 0x412AC1: ldap_ld_free (unbind.c:87)
==28795== by 0x4130EE: ldap_destroy (unbind.c:245)
==28795== by 0x405BEF: do_onethread (slapd-mtread.c:455)
==28795== by 0x4A0AECD: ??? (in
/usr/lib64/valgrind/vgpreload_helgrind-amd64-linux.so)
==28795== by 0x3A45E07EE4: start_thread (pthread_create.c:309)
==28795== by 0x3A45AF4B8C: clone (clone.S:111)
==28795==
==28795== This conflicts with a previous read of size 4 by thread #29
==28795%%3= Locks held: 1, at address 0x4C393A8
==28795== at 0x426957: ldap_get_option (options.c:374)
==28795== by 0x405914: do_onethread (slapd-mtread.c:423)
==28795== by 0x4A0AECD: ??? (in
/usr/lib64/valgrind/vgpreload_helgrind-amd64-linux.so)
==28795== by 0x3A45E07EE4: start_thread (pthread_create.c:309)
==28795== by 0x3A45AF4B8C: clone (clone.S:111)
==28795==
==28795== Address 0x4C39238 is 88 bytes inside a block of size 736 alloc'd
==28795== at 0x4A083F4: calloc (in
/usr/lib64/valgrind/vgpreload_helgrind-amd64-linux.so)
==28795== by 0x43E4B1: ber_memcalloc_x (memory.c:283)
==28795== by 0x409F4B: ldap_create (open.c:119)
==28795== by 0x40A3E4: ldap_initialize (open.c:240)
==28795== by 0x4062B7: do_conn (slapd-mtread.c:584)
==28795== by 0x40530A: main (slapd-mtread.c:331)
Full_Name: Cl.ment OUDOT
Version: 2.4.40
OS: GNU/Linux
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (88.173.78.196)
This is a minor issue, but I post it anyway.
clement@ader-xps:~/Projets/openldap$ grep -ri olcdbmaxsize
servers/slapd/back-mdb/*
servers/slapd/back-mdb/config.c: mdb_cf_ge2C2C "( OLcfgDbAt:12.2
NAME 'olcDbMaxSize' "
servers/slapd/back-mdb/config.c: "olcDbNoSync $ olcDbIndex $
olcDbMaxReaders $ olcDbMaxsize $ "
The first time we have a capital S, and not the second time.
This is a multi-part message in MIME format.
--------------030107090007020809070602
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
On 31.01.2015. 14:09, hyc(a)symas.com wrote:
>> A real fix would involve modifying the persistent search to include reqControls
>> in its attribute list and activating Relax Rules on the consumer only if it had
>> been active on the provider when the modification occurred.
>
> Right. This is also important for a few other controls. E.g. LDAP
> Transactions.
Attached is a patch implementing the outlined approach. Notes:
- I've invented the attribute name for the CHANGELOG format, for symmetry.
- I've structured the check so it shouldn't be difficult to handle other
controls. Other approaches are certainly possible.
- The patch has been generated against OPENLDAP_REL_ENG_2_4, not HEAD.
If an IPR statement is needed, I'm including it here.
The attached patch file is derived from OpenLDAP Software. All of the
modifications to OpenLDAP Software represented in the following patch were
developed by Ivan Nejgebauer <ian(a)uns.ac.rs>. I have not assigned rights
and/or interest in this work to any party.
I, Ivan Nejgebauer, hereby place the following modifications to OpenLDAP
Software (and only these modifications) into the public domain. Hence, these
modifications may be freely used and/or redistributed for any purpose with or
without attribution and/or other notice.
--
Ivan Nejgebauer +381 21 485 2025
Glavni sistem inženjer ian(a)uns.ac.rs
CIT-UNS/ARMUNS http://www.uns.ac.rs
Univerzitet u Novom Sadu $ Dr Zorana Đinđića 1 $ 21000 Novi Sad $ Srbija
--------------030107090007020809070602
Content-Type: text/x-patch;
name="0001-Add-relax-control-handling-for-delta-syncrepl.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename="0001-Add-relax-control-handling-for-delta-syncrepl.patch"
>From 52e08a1c2c63d527b8e2b65d3a63d9a12e2b6edc Mon Sep 17 00:00:00 2001
From: Ivan Nejgebauer <ian(a)uns.ac.rs>
Date: Tue, 3 Feb 2015 13:35:38 +0100
Subject: [PATCH] Add relax control handling for delta-syncrepl
---
servers/slapd/syncrepl.c | 28 +++++++++++++++++++++++-----
1 file changed, 23 insertions(+), 5 deletions(-)
diff --git a/servers/slapd/syncrepl.c b/servers/slapd/syncrepl.c
index 4e42a47..93b2723 100644
--- a/servers/slapd/syncrepl.c
+++ b/servers/slapd/syncrepl.c
@@ -165,6 +165,7 @@ typedef struct logschema {
struct berval ls_newRdn;
struct berval ls_delRdn;
struct berval ls_newSup;
+ struct berval ls_controls;
} logschema;
static logschema changelog_sc = {
@@ -173,7 +174,8 @@ static logschema changelog_sc = {
BER_BVC("changes"),
BER_BVC("newRDN"),
BER_BVC("deleteOldRDN"),
- BER_BVC("newSuperior")
+ BER_BVC("newSuperior"),
+ BER_BVC("controls"),
};
static logschema accesslog_sc = {
@@ -182,7 +184,8 @@ static logschema accesslog_sc = {
BER_BVC("reqMod"),
BER_BVC("reqNewRDN"),
BER_BVC("reqDeleteOldRDN"),
- BER_BVC("reqNewSuperior")
+ BER_BVC("reqNewSuperior"),
+ BER_BVC("reqControls")
};
static const char *
@@ -398,7 +401,7 @@ ldap_sync_search(
int rc;
int rhint;
char *base;
- char **attrs, *lattrs[8];
+ char **attrs, *lattrs[9];
char *filter;
int attrsonly;
int scope;
@@ -427,8 +430,9 @@ ldap_sync_search(
lattrs[3] = ls->ls_newRdn.bv_val;
lattrs[4] = ls->ls_delRdn.bv_val;
lattrs[5] = ls->ls_newSup.bv_val;
- lattrs[6] = slap_schema.si_ad_entryCSN->ad_cname.bv_val;
- lattrs[7] = NULL;
+ lattrs[6] = ls->ls_controls.bv_val;
+ lattrs[7] = slap_schema.si_ad_entryCSN->ad_cname.bv_val;
+ lattrs[8] = NULL;
rhint = 0;
base = si->si_logbase.bv_val;
@@ -2291,6 +2295,20 @@ syncrepl_message_to_op(
}
} else if ( !ber_bvstrcasecmp( &bv, &ls->ls_newSup ) ) {
sup = bvals[0];
+ } else if ( !ber_bvstrcasecmp( &bv, &ls->ls_controls ) ) {
+ int i;
+ struct berval rel_ctrl_bv;
+
+ (void)ber_str2bv( "{" LDAP_CONTROL_RELAX, 0, 0, &rel_ctrl_bv );
+ for ( i = 0; bvals[i].bv_val; i++ ) {
+ struct berval cbv, tmp;
+
+ ber_bvchr_post( &cbv, &bvals[i], '}' );
+ ber_bvchr_post( &tmp, &cbv, '{' );
+ ber_bvchr_pre( &cbv, &tmp, '}' );
+ if ( !ber_bvcmp( &cbv, &rel_ctrl_bv ) )
+ op->o_relax = SLAP_CONTROL_CRITICAL;
+ }
} else if ( !ber_bvstrcasecmp( &bv,
&slap_schema.si_ad_entryCSN->ad_cname ) )
{
--
1.8.1.2
--------------030107090007020809070602--
Full_Name: Howard Chu
Version: 2.4
OS:
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (110.49.207.42)
Submitted by: hyc
The result parser is checking for CONTINUE in the wrong place.
--On Tuesday, February 03, 2015 12:08:36 AM +0000 Howard Chu <hyc(a)symas.com> wrote:
> whm(a)stanford.edu wrote:
>> Full_Name: Bill MacAllister
>> Version: 2.4.40
>> OS: Debian Wheezy
>> URL: ftp://ftp.openldap.org/incoming/
>> Submission from: (NULL) (171.64.19.165)
>>
>> I have a perl script that uses Net::LDAPapi to report data from our
>> OpenLDAP servers. I have used the script on an off for years.
>> This morning I created a new report this morning that is causing
>> slapd on the servers to core dump. When I do the same query using
>> ldapsearch the query returns normally.
>
> What is the query? The filter code where this occurs hasn't changed
> in 4 years.
>
> Provide the slapd -d7 output for the query via your script, as well
> as via ldapsearch.
The system exhibiting this problem was running a beta release of
2.4.40. When I installed from a build of the current stable the
problem disappeared. Apologies for the bother, I didn't realize
the system had not been updated.
I think that documenting the query would be useful anyway, but I
want to hold off on that because I know the problem exists in the
build that is in debian backports. I would like to give Ryan a
chance to fix it before I publish it. I was able to reproduce the
problem with ldapsearch and it is a trival and very effective
denial of service attack.
Bill
--
Bill MacAllister
Systems Programmer, Stanford University
jsynacek(a)redhat.com wrote:
> Full_Name: Jan Synacek
> Version: 2.4.40
> OS: GNU/Linux
> URL: ftp://ftp.openldap.org/incoming/
> Submission from: (NULL) (213.175.37.10)
>
>
> TIMEOUT and NETWORK_TIMEOUT are not applied when trying to connect to a stalled
> server using SSL. The same scenario works when using an unencrypted connection.
This is a known issue - we don't have async connect/handshake APIs for
these crypto libraries.
>
> Reproducer:
> 1) set up a server for use with SSL (localhost connection is enough)
> 2) set NETWORK_TIMEOUT and TIMEOUT in ldap.conf
> 3) slapd -u ldap -g ldap -h "ldapi:/// ldaps://localhost" -d1
> 4) verify that connection works
> ldapsearch -x -H ldaps://localhost
> 5) kill -STOP <server pid>
> 6) ldapsearch -x -H ldaps://localhost
> At this point, the client hangs and doesn't properly time out.
>
> For more information including a packet capture, see the original bug report:
> https://bugzilla.redhat.com/show_bug.cgi?id=1186562#c4
>
> This bug doesn't seem to be crypto library specific. I reproduced it with both
> moznss and openssl.
>
>
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
Full_Name: Jan Synacek
Version: 2.4.40
OS: GNU/Linux
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (213.175.37.10)
TIMEOUT and NETWORK_TIMEOUT are not applied when trying to connect to a stalled
server using SSL. The same scenario works when using an unencrypted connection.
Reproducer:
1) set up a server for use with SSL (localhost connection is enough)
2) set NETWORK_TIMEOUT and TIMEOUT in ldap.conf
3) slapd -u ldap -g ldap -h "ldapi:/// ldaps://localhost" -d1
4) verify that connection works
ldapsearch -x -H ldaps://localhost
5) kill -STOP <server pid>
6) ldapsearch -x -H ldaps://localhost
At this point, the client hangs and doesn't properly time out.
For more information including a packet capture, see the original bug report:
https://bugzilla.redhat.com/show_bug.cgi?id=1186562#c4
This bug doesn't seem to be crypto library specific. I reproduced it with both
moznss and openssl.