openldap-bugs

openldap-bugs@openldap.org

1 participants
20963 discussions

Re: (ITS#8819) LMDB seg fault with MDB_DUPSORT on -O3
by hyc＠symas.com 17 Mar '18

17 Mar '18

github(a)nicwatson.org wrote: > Full_Name: Nic Watson > Version: LMDB v 0.9.21 > OS: Ubuntu 17.04 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (108.56.136.246) > > > I'm getting a seg fault in using LMDB on a database opened with MDB_DUPSORT. > > Here's a minimal set of operations that will cause the problem: > It will *not* crash under debug. In fact, -O3 -fvect-cost-model=cheap will > *not* crash. This makes some sense since it is crashing on an SSE instruction. > > I tried gcc versions (Ubuntu 7.2.0-8ubuntu3.2) and (Ubuntu 6.4.0-8ubuntu1) with > the same result. I also tried with the mdb.master branch (0.9.70) with the same > result. > > I'm not convinced this a fault in your code. It may be a gcc bug. Interesting. I've got gcc 5.4.0 on Ubuntu 16.04 here, and no crash. Also rather puzzled that there's anything vectorizable in this code, that seems pretty unlikely. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#8819) LMDB seg fault with MDB_DUPSORT on -O3
by github＠nicwatson.org 16 Mar '18

16 Mar '18

Full_Name: Nic Watson Version: LMDB v 0.9.21 OS: Ubuntu 17.04 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (108.56.136.246) I'm getting a seg fault in using LMDB on a database opened with MDB_DUPSORT. Here's a minimal set of operations that will cause the problem: #include <stdio.h> #include "lmdb.h" #define ARRAY_SIZE(array) (sizeof(array) / sizeof(*array)) void cause_crash() { MDB_env *env; MDB_txn *txn; MDB_dbi dbi; int rc; int line_fail = 0; MDB_val key = {.mv_size = 1, .mv_data = "a"}; MDB_val vals[] = { { .mv_size = 8, .mv_data = "\x00\x00\x00\x00\x00\x00\x00\x00" }, { .mv_size = 1, .mv_data = "\x05" }, { .mv_size = 1, .mv_data = "\t" }, { .mv_size = 1, .mv_data = "\r" }, { .mv_size = 1, .mv_data = "\x11" }, { .mv_size = 1, .mv_data = "\x15" }, { .mv_size = 1, .mv_data = "\x19" }, { .mv_size = 1, .mv_data = "\x1d" }, { .mv_size = 1, .mv_data = "!" }, { .mv_size = 1, .mv_data = "%" }, { .mv_size = 1, .mv_data = ")" }, { .mv_size = 1, .mv_data = "-" }, { .mv_size = 1, .mv_data = "1" }, { .mv_size = 1, .mv_data = "5" }, }; rc = mdb_env_create(&env); if (rc) { line_fail = __LINE__; goto fail; } rc = mdb_env_set_maxdbs(env, 2); if (rc) { line_fail = __LINE__; goto fail; } rc = mdb_env_open(env, "foo.lmdb", 0, 0777); if (rc) { line_fail = __LINE__; goto fail; } rc = mdb_txn_begin(env, NULL, 0, &txn); if (rc) { line_fail = __LINE__; goto fail; } rc = mdb_dbi_open(txn, "another_db", MDB_CREATE | MDB_DUPSORT, &dbi); if (rc) { line_fail = __LINE__; goto fail; } rc = mdb_txn_commit(txn); if (rc) { line_fail = __LINE__; goto fail; } rc = mdb_txn_begin(env, NULL, 0, &txn); if (rc) { line_fail = __LINE__; goto fail; } for (int i = 0; i < ARRAY_SIZE(vals); i++) { rc = mdb_put(txn, dbi, &key, &vals[i], 0); if (rc) { line_fail = __LINE__; goto fail; } } rc = mdb_txn_commit(txn); if (rc) { line_fail = __LINE__; goto fail; } mdb_dbi_close(env, dbi); mdb_env_close(env); return; fail: printf("Failed with error %d on line %d\n", rc, line_fail); } int main(int argc, char **argv) { printf("Running %s\n", MDB_VERSION_STRING); cause_crash(); printf("Didn't crash!\n"); return 0; } Here's a simple sh make script: LMDB_DIR=~/src/lmdb/libraries/liblmdb gcc-6 -O3 -ggdb -std=c11 -Wall -I $LMDB_DIR lmdb_crash2.c $LMDB_DIR/{mdb.c,midl.c} -lpthread -o mdb_c_exe mkdir -p foo.lmdb Here's a basic stack trace: Program received signal SIGSEGV, Segmentation fault. 0x000055555555ee7c in mdb_cursor_put (mc=mc@entry=0x7fffffffcfa0, key=key@entry=0x7fffffffd380, data=data@entry=0x7fffffffd460, flags=flags@entry=0) at /home/nic/src/lmdb/libraries/liblmdb/mdb.c:6792 6792 mp->mp_ptrs[i] = fp->mp_ptrs[i] + offset; (gdb) bt #0 0x000055555555ee7c in mdb_cursor_put (mc=mc@entry=0x7fffffffcfa0, key=key@entry=0x7fffffffd380, data=data@entry=0x7fffffffd460, flags=flags@entry=0) at /home/nic/src/lmdb/libraries/liblmdb/mdb.c:6792 #1 0x00005555555635d0 in mdb_cursor_put (flags=0, data=0x7fffffffd460, key=0x7fffffffd380, mc=0x7fffffffcfa0) at /home/nic/src/lmdb/libraries/liblmdb/mdb.c:8984 #2 mdb_put (txn=0x55555576e8d0, It will *not* crash under debug. In fact, -O3 -fvect-cost-model=cheap will *not* crash. This makes some sense since it is crashing on an SSE instruction. I tried gcc versions (Ubuntu 7.2.0-8ubuntu3.2) and (Ubuntu 6.4.0-8ubuntu1) with the same result. I also tried with the mdb.master branch (0.9.70) with the same result. I'm not convinced this a fault in your code. It may be a gcc bug.

1 0

Re: (ITS#8813) MDB_VL32 causes MDB_TXN_FULL
by markus＠greenrobot.de 14 Mar '18

14 Mar '18

--001a114390bed95e010567661549 Content-Type: text/plain; charset="UTF-8" Some more details: Transaction read only pages are running out in mdb_rpage_get(). MDB_txn. mt_rpages[0] is at 4096 and thus MDB_TXN_FULL is returned. This happens after around 100K mdb_cursor_del() calls. How can we get millions of deletes to work in a single transaction? It's an essential feature to us and I am happy to contribute under some guidance. So, here's my hope: given that the cleanup involves only consecutive keys, we could consider a bulk delete function (delete an entire range from position 1 to position 2). This should be much more efficient. Basically, for most of the data, there's no need to operate on the node level. Deleting entire leaf pages should be relatively simple (rebalance on the parent branch page)? Not sure if rebalancing could also cope with cutting off entire branches (superefficient if that would work). Thanks, Markus --001a114390bed95e010567661549 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <html><head><meta http-equiv=3D"Content-Type" content=3D"text/html; charset= =3Dus-ascii"><meta name=3D"Generator" content=3D"Microsoft Word 14 (filtere= d medium)"><style></style></head><body lang=3D"DE" link=3D"blue" vlink=3D"purple"><div cla= ss=3D"WordSection1">Some more d= etails:Transaction r= ead only pages are running out in mdb_rpage_get(). MDB_txn. mt_rpages[0] is= at 4096 and thus MDB_TXN_FULL is returned. This happens after around 100K = mdb_cursor_del() calls. How can we get m= illions of deletes to work in a single transaction?=C2=A0<spa= n lang=3D"EN-US">It's an essential feature to us and I am happy to cont= ribute under some guidance.=C2=A0So, her= e's my hope: given that the cleanup involves only consecutive keys, we = could consider a bulk delete function (delete an entire range from position= 1 to position 2). This should be much more efficient. Basically, for most = of the data, there's no need to operate on the node level. Deleting ent= ire leaf pages should be relatively simple (rebalance on the parent branch = page)? Not sure if rebalancing could also cope with cutting off entire bran= ches (superefficient if that would work).= =C2=A0Thanks,Markus= </div></body></html> --001a114390bed95e010567661549--

1 0

Re: (ITS#8818) SASL_MECH is wrongly documented as user option
by cheimes＠redhat.com 14 Mar '18

14 Mar '18

On 2018-03-13 18:49, Howard Chu wrote: > It appears the doc is stale. The user-only attribute was removed from > SASL_MECH in ITS#4327 (commit 86d10729) and apparently the docs weren't > updated then. Thanks for your confirmation and fixing the documentation in f183b81a!

1 0

Re: (ITS#8818) SASL_MECH is wrongly documented as user option
by hyc＠symas.com 13 Mar '18

13 Mar '18

cheimes(a)redhat.com wrote: > Full_Name: Christian Heimes > Version: 2.4.45 > OS: Fedora > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (2001:16b8:601e:1c00:18cd:bce4:2531:fe3) > > > SASL_MECH is documented as "This is a user-only option" in man ldap.conf, > https://www.openldap.org/software/man.cgi?query=ldap.conf . User-only options > are documented as "Such options are ignored if present in the ldap.conf". > Contrary to the documentation, libldap loads SASL_MECH from global ldap.conf. We > discussed the issue in > https://github.com/freeipa/freeipa/pull/1470#issuecomment-372602434 > > Is this a documentation bug or an implementation bug? It appears the doc is stale. The user-only attribute was removed from SASL_MECH in ITS#4327 (commit 86d10729) and apparently the docs weren't updated then. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#8818) SASL_MECH is wrongly documented as user option
by cheimes＠redhat.com 13 Mar '18

13 Mar '18

Full_Name: Christian Heimes Version: 2.4.45 OS: Fedora URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (2001:16b8:601e:1c00:18cd:bce4:2531:fe3) SASL_MECH is documented as "This is a user-only option" in man ldap.conf, https://www.openldap.org/software/man.cgi?query=ldap.conf . User-only options are documented as "Such options are ignored if present in the ldap.conf". Contrary to the documentation, libldap loads SASL_MECH from global ldap.conf. We discussed the issue in https://github.com/freeipa/freeipa/pull/1470#issuecomment-372602434 Is this a documentation bug or an implementation bug?

1 0

(ITS#8817) SQL scripts for Altibase Backend Database
by grace.bae＠altibase.com 08 Mar '18

08 Mar '18

Full_Name: Younghee Bae Version: 2.4.45 OS: Linux URL: ftp://ftp.openldap.org/incoming/altibase-180309.patch Submission from: (NULL) (222.112.181.231) I'd like to add SQL scripts that create metatables, test data, metadata for Altibase database to OpenLDAP. I copied rdbms_depend/oracle and modified them for Altibase database. And I fixed some meta-data that were not working.

1 0

Re: (ITS#8816) OpenLDAP - Embedded Prodcut or Stand-alone product
by michael＠stroeder.com 07 Mar '18

07 Mar '18

muthamma.appaiah(a)wellsfargo.com wrote: > Full_Name: Muthamma > Version: OpenLDAP latest > OS: > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (2620:160:e708:6::a) > > > Hi team, > > Could you confirm if the product OpenLDAP is a stand-alone product or embedded? > > If this comes bundled with any other product? ITS is only used for reporting bugs. Please ask such a question on the openldap-technical mailing list. And please do not file the same question several times. Ciao, Michael.

1 0

Re: (ITS#8812) OpenLDAP 2.4 Standalone or embedded
by michael＠stroeder.com 07 Mar '18

07 Mar '18

muthamma.appaiah(a)wellsfargo.com wrote: > Full_Name: Muthamma > Version: 2.4 and 2.3 > OS: > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (2620:160:e708:6::a) > > > Hi team, > > Could you confirm if the product OpenLDAP is a stand-alone product or embedded? ITS is only used for reporting bugs. Please ask such a question on the openldap-technical mailing list. Ciao, Michael.

1 0

(ITS#8816) OpenLDAP - Embedded Prodcut or Stand-alone product
by muthamma.appaiah＠wellsfargo.com 07 Mar '18

07 Mar '18

Full_Name: Muthamma Version: OpenLDAP latest OS: URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (2620:160:e708:6::a) Hi team, Could you confirm if the product OpenLDAP is a stand-alone product or embedded? If this comes bundled with any other product?

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs