openldap-bugs

openldap-bugs@openldap.org

1 participants
21001 discussions

Re: (ITS#8819) LMDB seg fault with MDB_DUPSORT on -O3
by hyc＠symas.com 20 Mar '18

20 Mar '18

nic(a)nicwatson.org wrote: > That's news to me. Then I googled it. You're right. > >>From https://wiki.sei.cmu.edu/confluence/display/c/EXP36-C.+Do+not+cast+pointers… > > The C Standard, 6.3.2.3, paragraph 7 [ISO/IEC 9899:2011], states > > A pointer to an object or incomplete type may be converted to a > pointer to a different object or incomplete type. If the resulting > pointer is not correctly aligned for the referenced type, the behavior > is undefined. > > Nic > > On Tue, Mar 20, 2018 at 9:41 AM, Hallvard Breien Furuseth > <h.b.furuseth(a)usit.uio.no> wrote: >> Looks like another type aliasing problem to me. The data is accessed >> through an MDB_page* variable. This tells the compiler that the data >> is word-aligned, like struct MDB_page. Fix: Use a void/char pointer, >> don't lie to the compiler. Good catch. We once discussed padding odd-length keys to make sure the data was still word-aligned. Maybe should do that in LMDB 1.0. This particular crash is now fixed in mdb.master. I've left other derefs of *fp alone for the moment but may need to revisit that later; older ARM and SPARC would probably choke on them. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8819) LMDB seg fault with MDB_DUPSORT on -O3
by nic＠nicwatson.org 20 Mar '18

20 Mar '18

That's news to me. Then I googled it. You're right. >From https://wiki.sei.cmu.edu/confluence/display/c/EXP36-C.+Do+not+cast+pointers… The C Standard, 6.3.2.3, paragraph 7 [ISO/IEC 9899:2011], states A pointer to an object or incomplete type may be converted to a pointer to a different object or incomplete type. If the resulting pointer is not correctly aligned for the referenced type, the behavior is undefined. Nic On Tue, Mar 20, 2018 at 9:41 AM, Hallvard Breien Furuseth <h.b.furuseth(a)usit.uio.no> wrote: > Looks like another type aliasing problem to me. The data is accessed > through an MDB_page* variable. This tells the compiler that the data > is word-aligned, like struct MDB_page. Fix: Use a void/char pointer, > don't lie to the compiler. >

1 0

Re: (ITS#8819) LMDB seg fault with MDB_DUPSORT on -O3
by h.b.furuseth＠usit.uio.no 20 Mar '18

20 Mar '18

Looks like another type aliasing problem to me. The data is accessed through an MDB_page* variable. This tells the compiler that the data is word-aligned, like struct MDB_page. Fix: Use a void/char pointer, don't lie to the compiler.

1 0

Re: (ITS#8819) LMDB seg fault with MDB_DUPSORT on -O3
by nic＠nicwatson.org 20 Mar '18

20 Mar '18

--000000000000b2e0cc0567d7decb Content-Type: text/plain; charset="UTF-8" Summary of findings so far: Attached is a make script that applies the minimal set of optimizations (at least on gcc 7) that still triggers the seg fault. It fails the first time that address is executed, when NUMKEYS(fp) hits 10 or 11 (I *think* depending on the alignment of fp). It looks like the autovectorization logic is split in three parts: one unaligned non-SSE copy, an aligned copy that uses SSE, and then an unaligned non-SSE copy. It makes sense that the SSE logic path is only triggered when the size is greater than some threshold. We should coordinate filing a gcc bug report. Nic Watson On Fri, Mar 16, 2018 at 3:11 PM, <openldap-its(a)openldap.org> wrote: > > *** THIS IS AN AUTOMATICALLY GENERATED REPLY *** > > Thanks for your report to the OpenLDAP Issue Tracking System. Your > report has been assigned the tracking number ITS#8819. > > One of our support engineers will look at your report in due course. > Note that this may take some time because our support engineers > are volunteers. They only work on OpenLDAP when they have spare > time. > > If you need to provide additional information in regards to your > issue report, you may do so by replying to this message. Note that > any mail sent to openldap-its(a)openldap.org with (ITS#8819) > in the subject will automatically be attached to the issue report. > > mailto:openldap-its@openldap.org?subject=(ITS#8819) > > You may follow the progress of this report by loading the following > URL in a web browser: > http://www.OpenLDAP.org/its/index.cgi?findid=8819 > > Please remember to retain your issue tracking number (ITS#8819) > on any further messages you send to us regarding this report. If > you don't then you'll just waste our time and yours because we > won't be able to properly track the report. > > Please note that the Issue Tracking System is not intended to > be used to seek help in the proper use of OpenLDAP Software. > Such requests will be closed. > > OpenLDAP Software is user supported. > http://www.OpenLDAP.org/support/ > > -------------- > Copyright 1998-2007 The OpenLDAP Foundation, All Rights Reserved. > --000000000000b2e0cc0567d7decb Content-Type: application/x-shellscript; name="make.sh" Content-Disposition: attachment; filename="make.sh" Content-Transfer-Encoding: base64 X-Attachment-Id: f_jezny59e0 TE1EQl9ESVI9fi9zcmMvbG1kYi9saWJyYXJpZXMvbGlibG1kYgoKCk5PT1BUUzE9IgotZm5vLWF1 dG8taW5jLWRlYwotZm5vLWJyYW5jaC1jb3VudC1yZWcKLWZuby1jb21iaW5lLXN0YWNrLWFkanVz dG1lbnRzCi1mbm8tY29tcGFyZS1lbGltCi1mbm8tY3Byb3AtcmVnaXN0ZXJzCi1mbm8tZGNlCi1m bm8tZGVmZXItcG9wCi1mbm8tZGVsYXllZC1icmFuY2gKLWZuby1kc2UKLWZuby1mb3J3YXJkLXBy b3BhZ2F0ZQotZm5vLWd1ZXNzLWJyYW5jaC1wcm9iYWJpbGl0eQotZm5vLWlmLWNvbnZlcnNpb24y Ci1mbm8taWYtY29udmVyc2lvbgotZm5vLWlubGluZS1mdW5jdGlvbnMtY2FsbGVkLW9uY2UKLWZu by1pcGEtcHVyZS1jb25zdAotZm5vLWlwYS1wcm9maWxlCi1mbm8taXBhLXJlZmVyZW5jZQotZm5v LW1lcmdlLWNvbnN0YW50cwotZm5vLW1vdmUtbG9vcC1pbnZhcmlhbnRzCi1mbm8tb21pdC1mcmFt ZS1wb2ludGVyCi1mbm8tcmVvcmRlci1ibG9ja3MKLWZuby1zaHJpbmstd3JhcAotZm5vLXNocmlu ay13cmFwLXNlcGFyYXRlCi1mbm8tc3BsaXQtd2lkZS10eXBlcwotZm5vLXNzYS1iYWNrcHJvcAot Zm5vLXNzYS1waGlvcHQKLWZuby10cmVlLWJpdC1jY3AKLWZuby10cmVlLWNjcAotZm5vLXRyZWUt Y29hbGVzY2UtdmFycwotZm5vLXRyZWUtY29weS1wcm9wCi1mbm8tdHJlZS1kY2UKLWZuby10cmVl LWRvbWluYXRvci1vcHRzCi1mbm8tdHJlZS1kc2UKLWZuby10cmVlLWZvcndwcm9wCi1mbm8tdHJl ZS1waGlwcm9wCi1mbm8tdHJlZS1zaW5rCi1mbm8tdHJlZS1zbHNyCi1mbm8tdHJlZS1zcmEKLWZu by10cmVlLXB0YQotZm5vLXRyZWUtdGVyCi1mbm8tdW5pdC1hdC1hLXRpbWUKLWZ0cmVlLWZyZQot ZnRyZWUtY2gKIgojIE5vdGUgbGFzdCBlbnRyeSBpcyAqbm90KiBkaXNhYmxlZAoKT1BUUzI9Jwot ZnN0cmljdC1hbGlhc2luZwonCgpybSBtZGJfY19leGUKZ2NjIC1PICROT09QVFMxICRPUFRTMiAt ZnRyZWUtdmVjdG9yaXplIC1mdmVjdC1jb3N0LW1vZGVsPXVubGltaXRlZCAtZ2dkYiAtc3RkPWMx MSAtV2FsbCAtSSAkTE1EQl9ESVIgIGxtZGJfY3Jhc2gyLmMgJExNREJfRElSL3ttZGIuYyxtaWRs LmN9IC1scHRocmVhZCAtbyBtZGJfY19leGUKcm0gLXIgZm9vLmxtZGIKbWtkaXIgLXAgZm9vLmxt ZGIK --000000000000b2e0cc0567d7decb--

1 0

Re: (ITS#8820) ldap_get_attribute_ber() return NULL pointer while OK
by daniel＠haxx.se 18 Mar '18

18 Mar '18

On Sun, 18 Mar 2018, Howard Chu wrote: > Looking at the mitigation you've applied, I'm not sure it's correct. In > particular, you're terminating the loop when you receive a NULL value, but > there may actually be multiple attributes present (with no values on any of > them) and you ought to continue iterating through them all. ... which is why my issue here is about the lack of documentation for the function! I tried to read up on how its supposed to work but I couldn't find any docs anywhere. -- / daniel.haxx.se

1 0

Re: (ITS#8820) ldap_get_attribute_ber() return NULL pointer while OK
by hyc＠symas.com 18 Mar '18

18 Mar '18

daniel(a)haxx.se wrote: > Full_Name: Daniel Stenberg > Version: any > OS: Linux > URL: > Submission from: (NULL) (178.174.211.173) > > > The function ldap_get_attribute_ber() is called to get attributes, but it turns > out that it can return LDAP_SUCCESS and still return a NULL pointer in the > result pointer when getting a particularly crafted response. > > This was a surprise to us and to curl, as this caused us a security > vulnerability. See https://curl.haxx.se/docs/adv_2018-97a2.html > > 1. There's no man page nor online resource to read the docs for this function so > its really hard to figure out this fact. > > 2. This behavior is surprising, and this flaw was even written by someone very > familiar with OpenLDAP, indicating it is unintended or at least not the normal > path. It's actually normal; if you issue a search and specify attrsonly, the results will only contain attribute names and no values. (e.g. using ldapsearch -A) As such, returning LDAP_SUCCESS with a NULL value is correct. Unfortunate oversight on my part when writing that curl patch. Looking at the mitigation you've applied, I'm not sure it's correct. In particular, you're terminating the loop when you receive a NULL value, but there may actually be multiple attributes present (with no values on any of them) and you ought to continue iterating through them all. > 3. Due to the above two points, I believe there's a risk curl is not the only > application in the world that had this bad assumption and thus this might be a > lurking security issue in more projects. > > / Daniel > > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#8820) ldap_get_attribute_ber() return NULL pointer while OK
by daniel＠haxx.se 17 Mar '18

17 Mar '18

Full_Name: Daniel Stenberg Version: any OS: Linux URL: Submission from: (NULL) (178.174.211.173) The function ldap_get_attribute_ber() is called to get attributes, but it turns out that it can return LDAP_SUCCESS and still return a NULL pointer in the result pointer when getting a particularly crafted response. This was a surprise to us and to curl, as this caused us a security vulnerability. See https://curl.haxx.se/docs/adv_2018-97a2.html 1. There's no man page nor online resource to read the docs for this function so its really hard to figure out this fact. 2. This behavior is surprising, and this flaw was even written by someone very familiar with OpenLDAP, indicating it is unintended or at least not the normal path. 3. Due to the above two points, I believe there's a risk curl is not the only application in the world that had this bad assumption and thus this might be a lurking security issue in more projects. / Daniel

1 0

Re: (ITS#8819) LMDB seg fault with MDB_DUPSORT on -O3
by hyc＠symas.com 17 Mar '18

17 Mar '18

hyc(a)symas.com wrote: > github(a)nicwatson.org wrote: >> Full_Name: Nic Watson >> Version: LMDB v 0.9.21 >> OS: Ubuntu 17.04 >> URL: ftp://ftp.openldap.org/incoming/ >> Submission from: (NULL) (108.56.136.246) >> >> >> I'm getting a seg fault in using LMDB on a database opened with MDB_DUPSORT. >> >> Here's a minimal set of operations that will cause the problem: > >> It will *not* crash under debug. In fact, -O3 -fvect-cost-model=cheap will >> *not* crash. This makes some sense since it is crashing on an SSE instruction. >> >> I tried gcc versions (Ubuntu 7.2.0-8ubuntu3.2) and (Ubuntu 6.4.0-8ubuntu1) with >> the same result. I also tried with the mdb.master branch (0.9.70) with the same >> result. >> >> I'm not convinced this a fault in your code. It may be a gcc bug. > > Interesting. I've got gcc 5.4.0 on Ubuntu 16.04 here, and no crash. Also > rather puzzled that there's anything vectorizable in this code, that seems > pretty unlikely. > I take it back - was able to reproduce the crash with gcc 5.4.0. The crash is on this instruction 0x000000000040f26b <+4235>: add $0x1,%edi => 0x000000000040f26e <+4238>: movdqa (%rax,%rdx,1),%xmm1 0x000000000040f273 <+4243>: mov (%rsp),%rax 0x000000000040f277 <+4247>: paddw %xmm0,%xmm1 movdqa's description is "Move aligned packed integer values" and the values here are not aligned. (gdb) bt #0 0x000000000040f26e in mdb_cursor_put (mc=mc@entry=0x7fffffffdd00, key=key@entry=0x7fffffffe0e0, data=data@entry=0x7fffffffe1c0, flags=flags@entry=0) at mdb.c:7673 #1 0x0000000000411d64 in mdb_cursor_put (flags=0, data=0x7fffffffe1c0, key=0x7fffffffe0e0, mc=0x7fffffffdd00) at mdb.c:9867 #2 mdb_put (txn=0x61e680, dbi=2, key=key@entry=0x7fffffffe0e0, data=data@entry=0x7fffffffe1c0, flags=flags@entry=0) at mdb.c:9868 #3 0x0000000000401c13 in cause_crash () at its8819.c:53 #4 0x0000000000401991 in main (argc=<optimized out>, argv=<optimized out>) at its8819.c:72 (gdb) l 7668 memcpy(METADATA(mp), METADATA(fp), NUMKEYS(fp) * fp->mp_pad); 7669 } else { 7670 memcpy((char *)mp + mp->mp_upper + PAGEBASE, (char *)fp + fp->mp_upper + PAGEBASE, 7671 olddata.mv_size - fp->mp_upper - PAGEBASE); 7672 for (i=0; i<NUMKEYS(fp); i++) 7673 mp->mp_ptrs[i] = fp->mp_ptrs[i] + offset; 7674 } 7675 } 7676 7677 rdata = &xdata; In particular, the fp pointer is on an odd address. Basically the movdqa instruction is not valid for use here. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#8819) LMDB seg fault with MDB_DUPSORT on -O3
by hyc＠symas.com 17 Mar '18

17 Mar '18

github(a)nicwatson.org wrote: > Full_Name: Nic Watson > Version: LMDB v 0.9.21 > OS: Ubuntu 17.04 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (108.56.136.246) > > > I'm getting a seg fault in using LMDB on a database opened with MDB_DUPSORT. > > Here's a minimal set of operations that will cause the problem: > It will *not* crash under debug. In fact, -O3 -fvect-cost-model=cheap will > *not* crash. This makes some sense since it is crashing on an SSE instruction. > > I tried gcc versions (Ubuntu 7.2.0-8ubuntu3.2) and (Ubuntu 6.4.0-8ubuntu1) with > the same result. I also tried with the mdb.master branch (0.9.70) with the same > result. > > I'm not convinced this a fault in your code. It may be a gcc bug. Interesting. I've got gcc 5.4.0 on Ubuntu 16.04 here, and no crash. Also rather puzzled that there's anything vectorizable in this code, that seems pretty unlikely. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#8819) LMDB seg fault with MDB_DUPSORT on -O3
by github＠nicwatson.org 16 Mar '18

16 Mar '18

Full_Name: Nic Watson Version: LMDB v 0.9.21 OS: Ubuntu 17.04 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (108.56.136.246) I'm getting a seg fault in using LMDB on a database opened with MDB_DUPSORT. Here's a minimal set of operations that will cause the problem: #include <stdio.h> #include "lmdb.h" #define ARRAY_SIZE(array) (sizeof(array) / sizeof(*array)) void cause_crash() { MDB_env *env; MDB_txn *txn; MDB_dbi dbi; int rc; int line_fail = 0; MDB_val key = {.mv_size = 1, .mv_data = "a"}; MDB_val vals[] = { { .mv_size = 8, .mv_data = "\x00\x00\x00\x00\x00\x00\x00\x00" }, { .mv_size = 1, .mv_data = "\x05" }, { .mv_size = 1, .mv_data = "\t" }, { .mv_size = 1, .mv_data = "\r" }, { .mv_size = 1, .mv_data = "\x11" }, { .mv_size = 1, .mv_data = "\x15" }, { .mv_size = 1, .mv_data = "\x19" }, { .mv_size = 1, .mv_data = "\x1d" }, { .mv_size = 1, .mv_data = "!" }, { .mv_size = 1, .mv_data = "%" }, { .mv_size = 1, .mv_data = ")" }, { .mv_size = 1, .mv_data = "-" }, { .mv_size = 1, .mv_data = "1" }, { .mv_size = 1, .mv_data = "5" }, }; rc = mdb_env_create(&env); if (rc) { line_fail = __LINE__; goto fail; } rc = mdb_env_set_maxdbs(env, 2); if (rc) { line_fail = __LINE__; goto fail; } rc = mdb_env_open(env, "foo.lmdb", 0, 0777); if (rc) { line_fail = __LINE__; goto fail; } rc = mdb_txn_begin(env, NULL, 0, &txn); if (rc) { line_fail = __LINE__; goto fail; } rc = mdb_dbi_open(txn, "another_db", MDB_CREATE | MDB_DUPSORT, &dbi); if (rc) { line_fail = __LINE__; goto fail; } rc = mdb_txn_commit(txn); if (rc) { line_fail = __LINE__; goto fail; } rc = mdb_txn_begin(env, NULL, 0, &txn); if (rc) { line_fail = __LINE__; goto fail; } for (int i = 0; i < ARRAY_SIZE(vals); i++) { rc = mdb_put(txn, dbi, &key, &vals[i], 0); if (rc) { line_fail = __LINE__; goto fail; } } rc = mdb_txn_commit(txn); if (rc) { line_fail = __LINE__; goto fail; } mdb_dbi_close(env, dbi); mdb_env_close(env); return; fail: printf("Failed with error %d on line %d\n", rc, line_fail); } int main(int argc, char **argv) { printf("Running %s\n", MDB_VERSION_STRING); cause_crash(); printf("Didn't crash!\n"); return 0; } Here's a simple sh make script: LMDB_DIR=~/src/lmdb/libraries/liblmdb gcc-6 -O3 -ggdb -std=c11 -Wall -I $LMDB_DIR lmdb_crash2.c $LMDB_DIR/{mdb.c,midl.c} -lpthread -o mdb_c_exe mkdir -p foo.lmdb Here's a basic stack trace: Program received signal SIGSEGV, Segmentation fault. 0x000055555555ee7c in mdb_cursor_put (mc=mc@entry=0x7fffffffcfa0, key=key@entry=0x7fffffffd380, data=data@entry=0x7fffffffd460, flags=flags@entry=0) at /home/nic/src/lmdb/libraries/liblmdb/mdb.c:6792 6792 mp->mp_ptrs[i] = fp->mp_ptrs[i] + offset; (gdb) bt #0 0x000055555555ee7c in mdb_cursor_put (mc=mc@entry=0x7fffffffcfa0, key=key@entry=0x7fffffffd380, data=data@entry=0x7fffffffd460, flags=flags@entry=0) at /home/nic/src/lmdb/libraries/liblmdb/mdb.c:6792 #1 0x00005555555635d0 in mdb_cursor_put (flags=0, data=0x7fffffffd460, key=0x7fffffffd380, mc=0x7fffffffcfa0) at /home/nic/src/lmdb/libraries/liblmdb/mdb.c:8984 #2 mdb_put (txn=0x55555576e8d0, It will *not* crash under debug. In fact, -O3 -fvect-cost-model=cheap will *not* crash. This makes some sense since it is crashing on an SSE instruction. I tried gcc versions (Ubuntu 7.2.0-8ubuntu3.2) and (Ubuntu 6.4.0-8ubuntu1) with the same result. I also tried with the mdb.master branch (0.9.70) with the same result. I'm not convinced this a fault in your code. It may be a gcc bug.

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs