openldap-bugs

openldap-bugs@openldap.org

1 participants
20960 discussions

(ITS#7828) LMDB crash in mdb_cursor_put(MDB_CURRENT)
by sbn＠tbricks.com 26 Mar '14

26 Mar '14

Full_Name: Dmitri Shubin Version: LMDB 0.9.11 (2727e97) OS: Linux (CentOS 6) URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (80.239.140.98) Hi! I got a crash inside LMDB in mdb_cursor_put() functions when I trying to modify the record under cursor. Simple test: $ cat test-mdb_current.c #include <stdio.h> #include <stdlib.h> #include "lmdb.h" #define E(expr) CHECK((rc = (expr)) == MDB_SUCCESS, #expr) #define CHECK(test, msg) ((test) ? (void)0 : ((void)fprintf(stderr, \ "%s:%d: %s: %s\n", __FILE__, __LINE__, msg, mdb_strerror(rc)), abort())) #define DBDIR "./testdb" int main() { int rc; MDB_env *env; MDB_dbi dbi; MDB_txn *txn; MDB_val key, data; MDB_cursor *cursor; int key_v = 17; char data_v[] = "0123456789"; E(system("rm -rf " DBDIR " && mkdir " DBDIR)); E(mdb_env_create(&env)); E(mdb_env_open(env, DBDIR, 0, 0664)); E(mdb_txn_begin(env, NULL, 0, &txn)); E(mdb_dbi_open(txn, NULL, 0, &dbi)); /* put some data */ key.mv_data = &key_v; key.mv_size = sizeof(key_v); data.mv_data = data_v; data.mv_size = sizeof(data_v); E(mdb_put(txn, dbi, &key, &data, 0)); E(mdb_txn_commit(txn)); /* replace using cursor */ E(mdb_txn_begin(env, NULL, 0, &txn)); E(mdb_cursor_open(txn, dbi, &cursor)); E(mdb_cursor_get(cursor, &key, &data, MDB_SET)); data.mv_data = data_v; data.mv_size = sizeof(data_v) - 1; E(mdb_cursor_put(cursor, &key, &data, MDB_CURRENT)); mdb_cursor_close(cursor); E(mdb_txn_commit(txn)); mdb_dbi_close(env, dbi); mdb_env_close(env); return 0; } $ gcc -g test-mdb_current.c -llmdb -L. -Wl,-rpath,. $ ./a.out Segmentation fault (core dumped) $ gdb a.out ... Program terminated with signal 11, Segmentation fault. #0 0x00007fefaba9895c in mdb_leaf_size (env=0xbd8010, key=0x0, data=0x7fff59e6a830) at mdb.c:6430 6430 sz = LEAFSIZE(key, data); (gdb) bt #0 0x00007fefaba9895c in mdb_leaf_size (env=0xbd8010, key=0x0, data=0x7fff59e6a830) at mdb.c:6430 #1 0x00007fefaba97c53 in mdb_cursor_put (mc=0xbda2f0, key=0x0, data=0x7fff59e6a830, flags=64) at mdb.c:6178 #2 0x0000000000400ee0 in main () at test-mdb_current.c:51 (gdb) l 6425 static size_t 6426 mdb_leaf_size(MDB_env *env, MDB_val *key, MDB_val *data) 6427 { 6428 size_t sz; 6429 6430 sz = LEAFSIZE(key, data); 6431 if (sz > env->me_nodemax) { 6432 /* put on overflow page */ 6433 sz -= data->mv_size - sizeof(pgno_t); 6434 } (gdb) p key $1 = (MDB_val *) 0x0 If I remove MDB_CURRENT flag from mdb_cursor_put() call it works fine. Thanks!

1 0

Re: (ITS#7827) Typo in slapacl can causes unclean database
by hyc＠symas.com 25 Mar '14

25 Mar '14

quanah(a)OpenLDAP.org wrote: > Full_Name: Quanah Gibson-Mount > Version: openldap master > OS: Linux 2.6 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (75.111.58.125) > > > As reported in https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=741248, slapacl > when used with a base that is not contained in the OpenLDAP configuration can > cause unclean DB messages. Fixed now in master > > To reproduce, I had to disable the monitor database in my configuration, so that > there was only the cn=config db and a primary BDB based backend. It also does > not occur if the suffix for the database is "" (as that contains everything). > > If the suffix of the DB is specific(such as "cn=zimbra"), then you can cause the > unclean shutdown status to trigger by running slapacl against a suffix that is > not contained in the slapd configuration: > > zimbra@zre-ldap001:~$ /opt/zimbra/openldap/sbin/slapacl -F > /opt/zimbra/data/ldap/config -b "cn=zimbraaaaa" -D > "uid=zimbra,cn=admins,cn=zimbra" entry > 5331d242 hdb_monitor_db_open: monitoring disabled; configure monitor database to > enable > 5331d242 hdb_db_open: database "cn=zimbra": unclean shutdown detected; > attempting recovery. > 5331d242 hdb_db_open: database "cn=zimbra": recovery skipped in read-only mode. > Run manual recovery if errors are encountered. > authcDN: "uid=zimbra,cn=admins,cn=zimbra" > cn=zimbraaaaa: no target database has been found for baseDN="slapacl"; you may > try with "-u" (dry run). > zimbra@zre-ldap001:~$ /opt/zimbra/openldap/sbin/slapacl -F > /opt/zimbra/data/ldap/config -b "cn=zimbra" -D "uid=zimbra,cn=admins,cn=zimbra" > entry > 5331d258 hdb_db_open: database "cn=zimbra": unclean shutdown detected; > attempting recovery. > 5331d258 hdb_db_open: database "cn=zimbra": recovery skipped in read-only mode. > Run manual recovery if errors are encountered. > 5331d258 hdb_monitor_db_open: monitoring disabled; configure monitor database to > enable > authcDN: "uid=zimbra,cn=admins,cn=zimbra" > entry: write(=wrscxd) > zimbra@zre-ldap001:~$ /opt/zimbra/openldap/sbin/slapacl -F > /opt/zimbra/data/ldap/config -b "cn=zimbra" -D "uid=zimbra,cn=admins,cn=zimbra" > entry > 5331d262 hdb_db_open: database "cn=zimbra": unclean shutdown detected; > attempting recovery. > 5331d262 hdb_db_open: database "cn=zimbra": recovery skipped in read-only mode. > Run manual recovery if errors are encountered. > 5331d262 hdb_monitor_db_open: monitoring disabled; configure monitor database to > enable > authcDN: "uid=zimbra,cn=admins,cn=zimbra" > entry: write(=wrscxd) > > Even running db_recover does not fix it: > > zimbra@zre-ldap001:~/data/ldap/hdb/db$ db_recover > zimbra@zre-ldap001:~/data/ldap/hdb/db$ cd > zimbra@zre-ldap001:~$ /opt/zimbra/openldap/sbin/slapacl -F > /opt/zimbra/data/ldap/config -b "cn=zimbra" -D "uid=zimbra,cn=admins,cn=zimbra" > entry > 5331d350 hdb_db_open: database "cn=zimbra": unclean shutdown detected; > attempting recovery. > 5331d350 hdb_db_open: database "cn=zimbra": recovery skipped in read-only mode. > Run manual recovery if errors are encountered. > 5331d350 hdb_monitor_db_open: monitoring disabled; configure monitor database to > enable > authcDN: "uid=zimbra,cn=admins,cn=zimbra" > entry: write(=wrscxd) > > After starting slapd, the db is properly cleaned up: > > zimbra@zre-ldap001:~$ ps -eaf | grep slapd > zimbra 1655 1 3 12:05 ? 00:00:00 /opt/zimbra/openldap/sbin/slapd > -l LOCAL0 -u zimbra -h ldap://zre-ldap001.eng.zimbra.com:389 ldapi:/// -F > /opt/zimbra/data/ldap/config > > zimbra@zre-ldap001:~$ /opt/zimbra/openldap/sbin/slapacl -F > /opt/zimbra/data/ldap/config -b "cn=zimbra" -D "uid=zimbra,cn=admins,cn=zimbra" > entry > authcDN: "uid=zimbra,cn=admins,cn=zimbra" > entry: write(=wrscxd) > > > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#7827) Typo in slapacl can causes unclean database
by quanah＠OpenLDAP.org 25 Mar '14

25 Mar '14

Full_Name: Quanah Gibson-Mount Version: openldap master OS: Linux 2.6 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (75.111.58.125) As reported in https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=741248, slapacl when used with a base that is not contained in the OpenLDAP configuration can cause unclean DB messages. To reproduce, I had to disable the monitor database in my configuration, so that there was only the cn=config db and a primary BDB based backend. It also does not occur if the suffix for the database is "" (as that contains everything). If the suffix of the DB is specific(such as "cn=zimbra"), then you can cause the unclean shutdown status to trigger by running slapacl against a suffix that is not contained in the slapd configuration: zimbra@zre-ldap001:~$ /opt/zimbra/openldap/sbin/slapacl -F /opt/zimbra/data/ldap/config -b "cn=zimbraaaaa" -D "uid=zimbra,cn=admins,cn=zimbra" entry 5331d242 hdb_monitor_db_open: monitoring disabled; configure monitor database to enable 5331d242 hdb_db_open: database "cn=zimbra": unclean shutdown detected; attempting recovery. 5331d242 hdb_db_open: database "cn=zimbra": recovery skipped in read-only mode. Run manual recovery if errors are encountered. authcDN: "uid=zimbra,cn=admins,cn=zimbra" cn=zimbraaaaa: no target database has been found for baseDN="slapacl"; you may try with "-u" (dry run). zimbra@zre-ldap001:~$ /opt/zimbra/openldap/sbin/slapacl -F /opt/zimbra/data/ldap/config -b "cn=zimbra" -D "uid=zimbra,cn=admins,cn=zimbra" entry 5331d258 hdb_db_open: database "cn=zimbra": unclean shutdown detected; attempting recovery. 5331d258 hdb_db_open: database "cn=zimbra": recovery skipped in read-only mode. Run manual recovery if errors are encountered. 5331d258 hdb_monitor_db_open: monitoring disabled; configure monitor database to enable authcDN: "uid=zimbra,cn=admins,cn=zimbra" entry: write(=wrscxd) zimbra@zre-ldap001:~$ /opt/zimbra/openldap/sbin/slapacl -F /opt/zimbra/data/ldap/config -b "cn=zimbra" -D "uid=zimbra,cn=admins,cn=zimbra" entry 5331d262 hdb_db_open: database "cn=zimbra": unclean shutdown detected; attempting recovery. 5331d262 hdb_db_open: database "cn=zimbra": recovery skipped in read-only mode. Run manual recovery if errors are encountered. 5331d262 hdb_monitor_db_open: monitoring disabled; configure monitor database to enable authcDN: "uid=zimbra,cn=admins,cn=zimbra" entry: write(=wrscxd) Even running db_recover does not fix it: zimbra@zre-ldap001:~/data/ldap/hdb/db$ db_recover zimbra@zre-ldap001:~/data/ldap/hdb/db$ cd zimbra@zre-ldap001:~$ /opt/zimbra/openldap/sbin/slapacl -F /opt/zimbra/data/ldap/config -b "cn=zimbra" -D "uid=zimbra,cn=admins,cn=zimbra" entry 5331d350 hdb_db_open: database "cn=zimbra": unclean shutdown detected; attempting recovery. 5331d350 hdb_db_open: database "cn=zimbra": recovery skipped in read-only mode. Run manual recovery if errors are encountered. 5331d350 hdb_monitor_db_open: monitoring disabled; configure monitor database to enable authcDN: "uid=zimbra,cn=admins,cn=zimbra" entry: write(=wrscxd) After starting slapd, the db is properly cleaned up: zimbra@zre-ldap001:~$ ps -eaf | grep slapd zimbra 1655 1 3 12:05 ? 00:00:00 /opt/zimbra/openldap/sbin/slapd -l LOCAL0 -u zimbra -h ldap://zre-ldap001.eng.zimbra.com:389 ldapi:/// -F /opt/zimbra/data/ldap/config zimbra@zre-ldap001:~$ /opt/zimbra/openldap/sbin/slapacl -F /opt/zimbra/data/ldap/config -b "cn=zimbra" -D "uid=zimbra,cn=admins,cn=zimbra" entry authcDN: "uid=zimbra,cn=admins,cn=zimbra" entry: write(=wrscxd)

1 0

Re: (ITS#7825) LMDB: misleading error message
by h.b.furuseth＠usit.uio.no 25 Mar '14

25 Mar '14

I wrote: > Let's see -- here are the options I can see (after an IRC chat): > (...) > 2) Catch this case and instead return a generic "something is > wrong" - MDB_BAD_TXN or MDB_INCOMPATIBLE. > (...) Duh, I forgot (6) Like (2) but create a new error code after all. Then we can document in its description that it's not caught reliably. That wasn't hard:-)

1 0

Re: (ITS#7825) LMDB: misleading error message
by h.b.furuseth＠usit.uio.no 25 Mar '14

25 Mar '14

On Sat, 2014-03-22 at 22:49 +0000, timur.kristof(a)gmail.com wrote: > When you operate on a dbi using a cursor and accidentally close the dbi before > committing the transaction of the cursor, mdb_txn_commit will return > MDB_BAD_VALSIZE. This is quite misleading because the problem doesn't actually > have anything to do with what the error message suggests. ("Too big key/data, > key is empty, or wrong DUPFIXED size") > > I suggest to either add a new kind of error code for this situation or to extend > the description of MDB_BAD_VALSIZE. Let's see -- here are the options I can see (after an IRC chat): 1) Extending the MDB_BAD_VALSIZE description. Simple. However: This is documented as a user error, and it's one which lmdb won't always catch. If you also called mdb_dbi_open() and the DBI got reused, lmdb silently updates the wrong named database. So I'm a bit wary about both of these fixes: Documenting this case can be misleading if it makes users expect it to be caught. Unless people know better - it corresponds to EBADF for using a closed file descriptor, not caught if open() reused it. 2) Catch this case and instead return a generic "something is wrong" - MDB_BAD_TXN or MDB_INCOMPATIBLE. And maybe assert(did not happen) to teach users to avoid this. Needs to be done where md_name is used: Commit/drop(named DB), page_search(stale named DB), cursor_touch(clean named DB). Or pass F_SUBDATA inwards and tweak the "return MDB_BAD_VALSIZE" statements to detect this. After playing around a bit, I guess this is my favorite. 3) Leave the misleading message alone, which can send the user off to a wild goose chase (as described on IRC). Or we can add code to also catch close followed by open: 4) Put hash(DB name) in MDB_db.md_pad and verify it before overwriting a named DB. Costs execution time even when there's no error. Must be done in commit, drop and maybe cursor_touch. (I've pushed a demo patch for just commit.) 5) Add field MDB_dbx.md_dbiseq = DBI usage sequence number, incremented when dbi_open creates (not reuses) a DBI and in dbi_close. Copy it to a new malloced array txn->mt_dbiseqs[] in write txns, verify it in at least commit and drop. This means close()-open(same DB) will also be caught, which seems a good thing since with (4) it will work unreliably: Only if open() reuses the same DBI for the same DB.

1 0

Re: (ITS#7616) syncrepl enhancement: defer requests when refreshing
by hyc＠symas.com 24 Mar '14

24 Mar '14

This is a multi-part message in MIME format. --------------050807020608030505070800 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit hyc(a)symas.com wrote: > Michael Ströder wrote: >> hyc(a)symas.com wrote: >>> I was considering returning LDAP_BUSY for this case, but it may make more >>> sense to return a REFERRAL to the provider instead. (Although again, if we >>> have two MMR servers pointed at each other starting at the same time, they >>> would just refer to each other and clients would get nowhere until one of them >>> finishes its refresh.) >> >> IMO referrals are harmful. >> Just returning LDAP_BUSY seems best to me because if reliable HA is really >> important for a deployment you have decent load-balancers in front of your >> LDAP server which should do a proper health-check. The attached patch should provide the desired behavior. It needs more testing though. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/ --------------050807020608030505070800 Content-Type: text/plain; charset=UTF-8; name="diff.txt" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="diff.txt" diff --git a/servers/slapd/syncrepl.c b/servers/slapd/syncrepl.c index 9b5c0f3..4ea8a70 100644 --- a/servers/slapd/syncrepl.c +++ b/servers/slapd/syncrepl.c @@ -148,6 +148,8 @@ static int syncrepl_add_glue_ancestors( /* delta-mmr overlay handler */ static int syncrepl_op_modify( Operation *op, SlapReply *rs ); +static int syncrepl_op_search( Operation *op, SlapReply *rs ); + /* callback functions */ static int dn_callback( Operation *, SlapReply * ); static int nonpresent_callback( Operation *, SlapReply * ); @@ -216,16 +218,14 @@ init_syncrepl(syncinfo_t *si) if ( !syncrepl_ov.on_bi.bi_type ) { syncrepl_ov.on_bi.bi_type = "syncrepl"; syncrepl_ov.on_bi.bi_op_modify = syncrepl_op_modify; + syncrepl_ov.on_bi.bi_op_search = syncrepl_op_search; overlay_register( &syncrepl_ov ); } - /* delta-MMR needs the overlay, nothing else does. - * This must happen before accesslog overlay is configured. - */ - if ( si->si_syncdata && - !overlay_is_inst( si->si_be, syncrepl_ov.on_bi.bi_type )) { + if (!overlay_is_inst( si->si_be, syncrepl_ov.on_bi.bi_type )) { overlay_config( si->si_be, syncrepl_ov.on_bi.bi_type, -1, NULL, NULL ); - if ( !ad_reqMod ) { + /* delta-MMR needs this. Must happen before accesslog overlay is configured. */ + if ( si->si_syncdata && !ad_reqMod ) { const char *text; logschema *ls = &accesslog_sc; @@ -1179,6 +1179,7 @@ do_syncrep2( } else { rc = -2; } + si->si_refreshDone = 1; goto done; case LDAP_RES_INTERMEDIATE: @@ -2164,6 +2165,31 @@ syncrepl_op_modify( Operation *op, SlapReply *rs ) } static int +syncrepl_op_search( Operation *op, SlapReply *rs ) +{ + slap_overinst *on = (slap_overinst *)op->o_bd->bd_info; + syncinfo_t *si; + + /* Allow syncrepl internal searches */ + if ( op->o_conn->c_conn_idx == -1 ) + return SLAP_CB_CONTINUE; + + /* Check if any of our consumers are refreshing */ + for ( si = op->o_bd->be_syncinfo; si; si = si->si_next ) { + /* If we have some state, allow other consumers to progress */ + if ( si->si_cookieState->cs_num > 0 && + op->o_sync > SLAP_CONTROL_IGNORED ) + return SLAP_CB_CONTINUE; + /* If we have any consumer refreshing, reject searches */ + if ( !si->si_refreshDone ) { + send_ldap_error( op, rs, LDAP_BUSY, "syncrepl refresh in progress" ); + return rs->sr_err; + } + } + return SLAP_CB_CONTINUE; +} + +static int syncrepl_message_to_op( syncinfo_t *si, Operation *op, --------------050807020608030505070800--

1 0

Re: (ITS#7336) Ldapmodify crashes slapd when updating olcTLSVerifyClient attribute via TLS authentication
by ck＠cksoft.de 23 Mar '14

23 Mar '14

Hi, I can confirm that openldap-2.4.39 still has in issue with this. When connecting via TLS I tried to modify olcTLSVerifyClient from never to try with following ldif: dn: cn=config changetype: modify replace: olcTLSVerifyClient olcTLSVerifyClient: try this caused slapd to hang indefinetely. I was able to successfully modify above when connecting without TLS. I need to complete my current task but will set up a small proof of concept later on in my lab. Greetings Christian -- Christian Kratzer CK Software GmbH Email: ck(a)cksoft.de Wildberger Weg 24/2 Phone: +49 7032 893 997 - 0 D-71126 Gaeufelden Fax: +49 7032 893 997 - 9 HRB 245288, Amtsgericht Stuttgart Mobile: +49 171 1947 843 Geschaeftsfuehrer: Christian Kratzer Web: http://www.cksoft.de/

1 0

(ITS#7825) LMDB: misleading error message
by timur.kristof＠gmail.com 22 Mar '14

22 Mar '14

Full_Name: Timur Kristóf Version: LMDB latest master OS: Fedora 20 URL: Submission from: (NULL) (94.248.245.60) Hi, I'm the author of the Node.js LMDB binding and I've found an interesting issue that might be useful to share with other people who develop bindings or applications on LMDB. When you operate on a dbi using a cursor and accidentally close the dbi before committing the transaction of the cursor, mdb_txn_commit will return MDB_BAD_VALSIZE. This is quite misleading because the problem doesn't actually have anything to do with what the error message suggests. ("Too big key/data, key is empty, or wrong DUPFIXED size") I suggest to either add a new kind of error code for this situation or to extend the description of MDB_BAD_VALSIZE. Here's a small test case which will demonstrate it: http://pastebin.com/hHwD1srf Best regards, Timur Kristóf

1 0

Re: (ITS#7822) Segmentation fault while modifying olcDbMaxSize
by quanah＠zimbra.com 22 Mar '14

22 Mar '14

--On Friday, March 21, 2014 10:28 AM +0000 frederic.poisson(a)admin.gmessaging.net wrote: > Full_Name: Frederic POISSON > Version: 2.4.39 > OS: Linux RHEL5 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (57.250.229.136) > > > Hello, > I have a Segmentation fault while i'm updating the value of olcDbMaxSize > parameter to a number inside olcDatabase={1}mdb,cn=config. The result of > a gdb backtrace show this error : > > Program received signal SIGSEGV, Segmentation fault. > [Switching to Thread 0x42499940 (LWP 31424)] > 0x0000000000553683 in ppolicy_connection_destroy (bd=0x424989d0, > conn=0x18ab2a50) at ppolicy.c:1260 > 1260 if ( !BER_BVISEMPTY( &pwcons[conn->c_conn_idx].dn )) { > > The ldif for the modification is : > dn: olcDatabase={1}mdb,cn=config > changetype: modify > replace: olcDbMaxSize > olcDbMaxSize: 20737418240 > > I will push onto FTP the full log of the backtrace, the cn=config.ldif > from slapcat. Hoping it could help for resolution. Please provide the full backtrace in the ITS --Quanah -- Quanah Gibson-Mount Architect - Server Zimbra, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

(ITS#7824) Segmentation fault while modifying olcDbMaxSize
by frederic.poisson＠admin.gmessaging.net 21 Mar '14

21 Mar '14

Full_Name: Frederic POISSON Version: 2.4.39 OS: Linux RHEL5 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (57.250.229.136) Hello, I have a Segmentation fault while i'm updating the value of olcDbMaxSize parameter to a number inside olcDatabase={1}mdb,cn=config. The result of a gdb backtrace show this error : Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x42499940 (LWP 31424)] 0x0000000000553683 in ppolicy_connection_destroy (bd=0x424989d0, conn=0x18ab2a50) at ppolicy.c:1260 1260 if ( !BER_BVISEMPTY( &pwcons[conn->c_conn_idx].dn )) { The ldif for the modification is : dn: olcDatabase={1}mdb,cn=config changetype: modify replace: olcDbMaxSize olcDbMaxSize: 20737418240 I will push onto FTP the full log of the backtrace, the cn=config.ldif from slapcat. Hoping it could help for resolution. Regards,

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs