(ITS#8820) ldap_get_attribute_ber() return NULL pointer while OK
by daniel＠haxx.se
Full_Name: Daniel Stenberg
Version: any
OS: Linux
URL:
Submission from: (NULL) (178.174.211.173)
The function ldap_get_attribute_ber() is called to get attributes, but it turns
out that it can return LDAP_SUCCESS and still return a NULL pointer in the
result pointer when getting a particularly crafted response.
This was a surprise to us and to curl, as this caused us a security
vulnerability. See https://curl.haxx.se/docs/adv_2018-97a2.html
1. There's no man page nor online resource to read the docs for this function so
its really hard to figure out this fact.
2. This behavior is surprising, and this flaw was even written by someone very
familiar with OpenLDAP, indicating it is unintended or at least not the normal
path.
3. Due to the above two points, I believe there's a risk curl is not the only
application in the world that had this bad assumption and thus this might be a
lurking security issue in more projects.
/ Daniel
2 years, 10 months
Re: (ITS#8819) LMDB seg fault with MDB_DUPSORT on -O3
by hyc＠symas.com
hyc(a)symas.com wrote:
> github(a)nicwatson.org wrote:
>> Full_Name: Nic Watson
>> Version: LMDB v 0.9.21
>> OS: Ubuntu 17.04
>> URL: ftp://ftp.openldap.org/incoming/
>> Submission from: (NULL) (108.56.136.246)
>>
>>
>> I'm getting a seg fault in using LMDB on a database opened with MDB_DUPSORT.
>>
>> Here's a minimal set of operations that will cause the problem:
>
>> It will *not* crash under debug. In fact, -O3 -fvect-cost-model=cheap will
>> *not* crash. This makes some sense since it is crashing on an SSE instruction.
>>
>> I tried gcc versions (Ubuntu 7.2.0-8ubuntu3.2) and (Ubuntu 6.4.0-8ubuntu1) with
>> the same result. I also tried with the mdb.master branch (0.9.70) with the same
>> result.
>>
>> I'm not convinced this a fault in your code. It may be a gcc bug.
>
> Interesting. I've got gcc 5.4.0 on Ubuntu 16.04 here, and no crash. Also
> rather puzzled that there's anything vectorizable in this code, that seems
> pretty unlikely.
>
I take it back - was able to reproduce the crash with gcc 5.4.0.
The crash is on this instruction
0x000000000040f26b <+4235>: add $0x1,%edi
=> 0x000000000040f26e <+4238>: movdqa (%rax,%rdx,1),%xmm1
0x000000000040f273 <+4243>: mov (%rsp),%rax
0x000000000040f277 <+4247>: paddw %xmm0,%xmm1
movdqa's description is "Move aligned packed integer values" and the values
here are not aligned.
(gdb) bt
#0 0x000000000040f26e in mdb_cursor_put (mc=mc@entry=0x7fffffffdd00,
key=key@entry=0x7fffffffe0e0, data=data@entry=0x7fffffffe1c0,
flags=flags@entry=0) at mdb.c:7673
#1 0x0000000000411d64 in mdb_cursor_put (flags=0, data=0x7fffffffe1c0,
key=0x7fffffffe0e0, mc=0x7fffffffdd00) at mdb.c:9867
#2 mdb_put (txn=0x61e680, dbi=2, key=key@entry=0x7fffffffe0e0,
data=data@entry=0x7fffffffe1c0, flags=flags@entry=0) at mdb.c:9868
#3 0x0000000000401c13 in cause_crash () at its8819.c:53
#4 0x0000000000401991 in main (argc=<optimized out>, argv=<optimized out>) at
its8819.c:72
(gdb) l
7668 memcpy(METADATA(mp), METADATA(fp), NUMKEYS(fp) * fp->mp_pad);
7669 } else {
7670 memcpy((char *)mp + mp->mp_upper + PAGEBASE, (char *)fp +
fp->mp_upper + PAGEBASE,
7671 olddata.mv_size - fp->mp_upper - PAGEBASE);
7672 for (i=0; i<NUMKEYS(fp); i++)
7673 mp->mp_ptrs[i] = fp->mp_ptrs[i] + offset;
7674 }
7675 }
7676
7677 rdata = &xdata;
In particular, the fp pointer is on an odd address. Basically the movdqa
instruction is not valid for use here.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
2 years, 10 months
Re: (ITS#8819) LMDB seg fault with MDB_DUPSORT on -O3
by hyc＠symas.com
github(a)nicwatson.org wrote:
> Full_Name: Nic Watson
> Version: LMDB v 0.9.21
> OS: Ubuntu 17.04
> URL: ftp://ftp.openldap.org/incoming/
> Submission from: (NULL) (108.56.136.246)
>
>
> I'm getting a seg fault in using LMDB on a database opened with MDB_DUPSORT.
>
> Here's a minimal set of operations that will cause the problem:
> It will *not* crash under debug. In fact, -O3 -fvect-cost-model=cheap will
> *not* crash. This makes some sense since it is crashing on an SSE instruction.
>
> I tried gcc versions (Ubuntu 7.2.0-8ubuntu3.2) and (Ubuntu 6.4.0-8ubuntu1) with
> the same result. I also tried with the mdb.master branch (0.9.70) with the same
> result.
>
> I'm not convinced this a fault in your code. It may be a gcc bug.
Interesting. I've got gcc 5.4.0 on Ubuntu 16.04 here, and no crash. Also
rather puzzled that there's anything vectorizable in this code, that seems
pretty unlikely.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
2 years, 10 months
(ITS#8819) LMDB seg fault with MDB_DUPSORT on -O3
by github＠nicwatson.org
Full_Name: Nic Watson
Version: LMDB v 0.9.21
OS: Ubuntu 17.04
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (108.56.136.246)
I'm getting a seg fault in using LMDB on a database opened with MDB_DUPSORT.
Here's a minimal set of operations that will cause the problem:
#include <stdio.h>
#include "lmdb.h"
#define ARRAY_SIZE(array) (sizeof(array) / sizeof(*array))
void cause_crash()
{
MDB_env *env;
MDB_txn *txn;
MDB_dbi dbi;
int rc;
int line_fail = 0;
MDB_val key = {.mv_size = 1, .mv_data = "a"};
MDB_val vals[] = {
{ .mv_size = 8, .mv_data = "\x00\x00\x00\x00\x00\x00\x00\x00" },
{ .mv_size = 1, .mv_data = "\x05" },
{ .mv_size = 1, .mv_data = "\t" },
{ .mv_size = 1, .mv_data = "\r" },
{ .mv_size = 1, .mv_data = "\x11" },
{ .mv_size = 1, .mv_data = "\x15" },
{ .mv_size = 1, .mv_data = "\x19" },
{ .mv_size = 1, .mv_data = "\x1d" },
{ .mv_size = 1, .mv_data = "!" },
{ .mv_size = 1, .mv_data = "%" },
{ .mv_size = 1, .mv_data = ")" },
{ .mv_size = 1, .mv_data = "-" },
{ .mv_size = 1, .mv_data = "1" },
{ .mv_size = 1, .mv_data = "5" },
};
rc = mdb_env_create(&env);
if (rc) { line_fail = __LINE__; goto fail; }
rc = mdb_env_set_maxdbs(env, 2);
if (rc) { line_fail = __LINE__; goto fail; }
rc = mdb_env_open(env, "foo.lmdb", 0, 0777);
if (rc) { line_fail = __LINE__; goto fail; }
rc = mdb_txn_begin(env, NULL, 0, &txn);
if (rc) { line_fail = __LINE__; goto fail; }
rc = mdb_dbi_open(txn, "another_db", MDB_CREATE | MDB_DUPSORT, &dbi);
if (rc) { line_fail = __LINE__; goto fail; }
rc = mdb_txn_commit(txn);
if (rc) { line_fail = __LINE__; goto fail; }
rc = mdb_txn_begin(env, NULL, 0, &txn);
if (rc) { line_fail = __LINE__; goto fail; }
for (int i = 0; i < ARRAY_SIZE(vals); i++) {
rc = mdb_put(txn, dbi, &key, &vals[i], 0);
if (rc) { line_fail = __LINE__; goto fail; }
}
rc = mdb_txn_commit(txn);
if (rc) { line_fail = __LINE__; goto fail; }
mdb_dbi_close(env, dbi);
mdb_env_close(env);
return;
fail:
printf("Failed with error %d on line %d\n", rc, line_fail);
}
int main(int argc, char **argv)
{
printf("Running %s\n", MDB_VERSION_STRING);
cause_crash();
printf("Didn't crash!\n");
return 0;
}
Here's a simple sh make script:
LMDB_DIR=~/src/lmdb/libraries/liblmdb
gcc-6 -O3 -ggdb -std=c11 -Wall -I $LMDB_DIR lmdb_crash2.c
$LMDB_DIR/{mdb.c,midl.c} -lpthread -o mdb_c_exe
mkdir -p foo.lmdb
Here's a basic stack trace:
Program received signal SIGSEGV, Segmentation fault.
0x000055555555ee7c in mdb_cursor_put (mc=mc@entry=0x7fffffffcfa0,
key=key@entry=0x7fffffffd380, data=data@entry=0x7fffffffd460,
flags=flags@entry=0)
at /home/nic/src/lmdb/libraries/liblmdb/mdb.c:6792
6792 mp->mp_ptrs[i] =
fp->mp_ptrs[i] + offset;
(gdb) bt
#0 0x000055555555ee7c in mdb_cursor_put (mc=mc@entry=0x7fffffffcfa0,
key=key@entry=0x7fffffffd380, data=data@entry=0x7fffffffd460,
flags=flags@entry=0)
at /home/nic/src/lmdb/libraries/liblmdb/mdb.c:6792
#1 0x00005555555635d0 in mdb_cursor_put (flags=0, data=0x7fffffffd460,
key=0x7fffffffd380, mc=0x7fffffffcfa0) at
/home/nic/src/lmdb/libraries/liblmdb/mdb.c:8984
#2 mdb_put (txn=0x55555576e8d0,
It will *not* crash under debug. In fact, -O3 -fvect-cost-model=cheap will
*not* crash. This makes some sense since it is crashing on an SSE instruction.
I tried gcc versions (Ubuntu 7.2.0-8ubuntu3.2) and (Ubuntu 6.4.0-8ubuntu1) with
the same result. I also tried with the mdb.master branch (0.9.70) with the same
result.
I'm not convinced this a fault in your code. It may be a gcc bug.
2 years, 10 months
Re: (ITS#8813) MDB_VL32 causes MDB_TXN_FULL
by markus＠greenrobot.de
--001a114390bed95e010567661549
Content-Type: text/plain; charset="UTF-8"
Some more details:
Transaction read only pages are running out in mdb_rpage_get(). MDB_txn.
mt_rpages[0] is at 4096 and thus MDB_TXN_FULL is returned. This happens
after around 100K mdb_cursor_del() calls.
How can we get millions of deletes to work in a single transaction?
It's an essential feature to us and I am happy to contribute under some
guidance.
So, here's my hope: given that the cleanup involves only consecutive keys,
we could consider a bulk delete function (delete an entire range from
position 1 to position 2). This should be much more efficient. Basically,
for most of the data, there's no need to operate on the node level.
Deleting entire leaf pages should be relatively simple (rebalance on the
parent branch page)? Not sure if rebalancing could also cope with cutting
off entire branches (superefficient if that would work).
Thanks,
Markus
--001a114390bed95e010567661549
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
<html><head><meta http-equiv=3D"Content-Type" content=3D"text/html; charset=
=3Dus-ascii"><meta name=3D"Generator" content=3D"Microsoft Word 14 (filtere=
d medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";
mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.E-MailFormatvorlage17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 70.85pt 2.0cm 70.85pt;}
div.WordSection1
{page:WordSection1;}
--></style></head><body lang=3D"DE" link=3D"blue" vlink=3D"purple"><div cla=
ss=3D"WordSection1"><p class=3D"MsoNormal"><span lang=3D"EN-US">Some more d=
etails:</span></p><p class=3D"MsoNormal"><span lang=3D"EN-US">Transaction r=
ead only pages are running out in mdb_rpage_get(). MDB_txn. mt_rpages[0] is=
at 4096 and thus MDB_TXN_FULL is returned. This happens after around 100K =
mdb_cursor_del() calls.</span></p><p class=3D"MsoNormal"><span lang=3D"EN-U=
S"> </span></p><p class=3D"MsoNormal"><span lang=3D"EN-US">How can we get m=
illions of deletes to work in a single transaction?</span></p><p class=3D"M=
soNormal"><span lang=3D"EN-US">=C2=A0</span></p><p class=3D"MsoNormal"><spa=
n lang=3D"EN-US">It's an essential feature to us and I am happy to cont=
ribute under some guidance.</span></p><p class=3D"MsoNormal"><span lang=3D"=
EN-US">=C2=A0</span></p><p class=3D"MsoNormal"><span lang=3D"EN-US">So, her=
e's my hope: given that the cleanup involves only consecutive keys, we =
could consider a bulk delete function (delete an entire range from position=
1 to position 2). This should be much more efficient. Basically, for most =
of the data, there's no need to operate on the node level. Deleting ent=
ire leaf pages should be relatively simple (rebalance on the parent branch =
page)? Not sure if rebalancing could also cope with cutting off entire bran=
ches (superefficient if that would work).</span></p><p class=3D"MsoNormal">=
<span lang=3D"EN-US">=C2=A0</span></p><p class=3D"MsoNormal"><span lang=3D"=
EN-US">Thanks,</span></p><p class=3D"MsoNormal"><span lang=3D"EN-US">Markus=
</span></p></div></body></html>
--001a114390bed95e010567661549--
2 years, 10 months
Re: (ITS#8818) SASL_MECH is wrongly documented as user option
by cheimes＠redhat.com
On 2018-03-13 18:49, Howard Chu wrote:
> It appears the doc is stale. The user-only attribute was removed from
> SASL_MECH in ITS#4327 (commit 86d10729) and apparently the docs weren't
> updated then.
Thanks for your confirmation and fixing the documentation in f183b81a!
2 years, 10 months
Re: (ITS#8816) OpenLDAP - Embedded Prodcut or Stand-alone product
by michael＠stroeder.com
muthamma.appaiah(a)wellsfargo.com wrote:
> Full_Name: Muthamma
> Version: OpenLDAP latest
> OS:
> URL: ftp://ftp.openldap.org/incoming/
> Submission from: (NULL) (2620:160:e708:6::a)
>
>
> Hi team,
>
> Could you confirm if the product OpenLDAP is a stand-alone product or embedded?
>
> If this comes bundled with any other product?
ITS is only used for reporting bugs. Please ask such a question on the
openldap-technical mailing list.
And please do not file the same question several times.
Ciao, Michael.
2 years, 10 months
