openldap-bugs

openldap-bugs@openldap.org

1 participants
20959 discussions

(ITS#7894) LMDB: MDB_DUPSORT and mdb_cursor_put(.., MDB_CURRENT) can put DB in illegal state
by batterseapower＠hotmail.com 01 Aug '14

01 Aug '14

Full_Name: Max Bolingbroke Version: LMDB HEAD OS: OS X URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (149.254.58.67) It is possible to insert a (key, value) pair into an MDB_DUPSORTed LMDB database twice using mdb_cursor_put(..., MDB_CURRENT). If you attempt to insert the same (key, value) pair twice with mdb_put then the second insert has no effect. This seems like a bug in mdb_cursor_put. Furthermore, you can use mdb_cursor_put(..., MDB_CURRENT) to insert duplicates in the opposite of their correctly sorted order. This is certainly a bug. The following C program demonstrates both issues: #define _XOPEN_SOURCE 500 #include <stdio.h> #include <stdlib.h> #include <unistd.h> #include <stdarg.h> #include "lmdb.h" void panic(const char * restrict format, ...) { va_list argp; va_start(argp, format); vprintf%foformat, argp); va_end(argp); exit(1); } void mdb_check(char *context, int rc) { if (rc != 0) { panic("%s: %s", context, mdb_strerror(rc)); } } void mdb_check_notfound(char *context, int rc) { if (rc != MDB_NOTFOUND) { panic("%s: %s", context, mdb_strerror(rc)); } } int BE(int x) { return (x & 0x000000FF) << 24 | (x & 0x0000FF00) << 8 | (x & 0x00FF0000) >> 8 | (x & 0xFF000000) >> 24; } int main(int argc, char **argv) { srand(1337); MDB_e * *env; mdb_check("mdb_env_create", mdb_env_create(&env)); mdb_check("mdb_env_set_maxdbs", mdb_env_set_maxdbs(env, 1)); mdb_check("mdb_env_set_mapsize", mdb_env_set_mapsize(env, 1024 * 1024)); mdb_check("mdb_env_open", mdb_env_open(env, "testdb", MDB_NOSUBDIR, 0644)); MDB_txn *txn; mdb_check("mdb_txn_begin", mdb_txn_begin(env, NULL, 0, &txn)); MDB_dbi dbi; mdb_check("mdb_dbi_open", mdb_dbi_open(txn, "root", MDB_CREATE | MDB_DUPSORT, &dbi)); int key; int value; MDB_val keyBuffer; keyBuffer.mv_size = sizeof(int); keyBuffer.mv_data = &key; MDB_val valueBuffer; valueBuffer.mv_size = sizeof(int); valueBuffer.mv_data = &value; { key = BE(2); value = BE(200); mdb_check("mdb_put", mdb_put(txn, dbi, &keyBuffer, &valueBuffer, 0)); key = BE(2); value = BE(300); mdb_check("mdb_put", mdb_put(txn, dbi, &keyBuffer, &valueBuffer, 0)); // Inserting the same (key, value) pair twice: second insert is ignored key = BE(2); value = BE(300); mdb_check("mdb_put", mdb_put(txn, dbi, &keyBuffer, &valueBuffer, 0)); } { MDB_cursor *cursor; mdb_check("mdb_cursor_open", mdb_cursor_open(txn, dbi, &cursor)); size_t count; mdb_check("mdb_cursor_get", mdb_cursor_get(cursor, &keyBuffer, &valueBuffer, MDB_FIRST))0D0D mdb_check("mdb_cursor_count", mdb_cursor_count(cursor, &count)); if (count != 2) panic("count != 2"); if (0) { // Inserting the same (key, value) pair twice: second insert is not ignored. // While this is different to mdb_put, I can't tell from the docs whether this is a bug or not. keyBuffer.mv_size = sizeof(int); valueBuffer.mv_size = sizeof(int); keyBuffer.mv_data = &key; valueBuffer.mv_data = &value; key = BE(2); value = BE(300); mdb_check("mdb_cursor_put2,2C mdb_cursor_put(cursor, &keyBuffer, &valueBuffer, MDB_CURRENT)); mdb_check("mdb_cursor_count", mdb_cursor_count(cursor, &count)); if (count != 1) panic("count != 1"); } else { // This on the other hand is definitely a bug: after doing this the duplicates are not sorted. keyBuffer.mv_size = sizeof(int); valueBuffer.mv_size = sizeof(int); keyBuffer.mv_data = &key; valueBuffer.mv_data = &value; key = BE(2); value = BE(400); mdb_check("mdb_cursor_put", mdb_cursor_p(c8cursor, &keyBuffer, &valueBuffer, MDB_CURRENT)); mdb_check("mdb_cursor_count", mdb_cursor_count(cursor, &count)); if (count != 2) panic("count != 2"); mdb_check("mdb_cursor_get", mdb_cursor_get(cursor, &keyBuffer, &valueBuffer, MDB_FIRST)); printf("%d\n", BE(*((int*)valueBuffer.mv_data))); mdb_check("mdb_cursor_get", mdb_cursor_get(cursor, &keyBuffer, &valueBuffer, MDB_NEXT)); printf("%d\n", BE(*((int*)valueBuffer.mv_data))); mdb_check_notfound("mdb_cursor_get", mdb_cursor_get(cursor, &keyBuffer, &valueBuffer, MDB_NEXT)); } mdb_cursor_close(cursor); } mdb_check("mdb_txn_commit", mdb_txn_commit(txn)); return 0; }

1 0

Re: (ITS#7713) Segmentation fault if the pagesize of the Operating system is not equal to 4096.
by h.b.furuseth＠usit.uio.no 09 Jul '14

09 Jul '14

On 09/07/14 01:41, hyc(a)symas.com wrote: > The fix is now available in mdb.master; currently you must compile with > -DMDB_DEVEL to enable it. Since this is an incompatible DB format change, you > should backup any existing DBs before using this option. Well, there is a fix in LDMB 0.9.10: It uses pages <= 32k, doing less efficient write operations if that's less than the OS page size. -- Hallvard

1 0

Re: (ITS#7893) memberof: bad a_numvals count
by hyc＠symas.com 09 Jul '14

09 Jul '14

jsoula(a)univ-lille2.fr wrote: > Full_Name: julien soula > Version: 2.4.39 > OS: > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (194.254.117.28) > > > Hi, > > this is related to ITS#6508. I don't think that the supplied patch is enough to > fix the bug. > > When the last "member" entry is dangling then "( j - i == 1 )" is true and the > loop is broken without reducing a_numvals. Looks like you're right. Committed to master, thanks. > > So I propose this patch : > > --- servers/slapd/overlays/memberof.c.orig 2014-07-09 21:17:14.487296363 > +0200 > +++ servers/slapd/overlays/memberof.c 2014-07-09 21:31:44.071565223 +0200 > @@ -596,6 +596,7 @@ > ber_memfree( a->a_nvals[ i > ].bv_val ); > BER_BVZERO( &a->a_nvals[ i ] ); > } > + a->a_numvals--; > if ( j - i == 1 ) { > break; > } > @@ -607,7 +608,6 @@ > sizeof( struct berval ) > * ( j - i ) ); > } > i--; > - a->a_numvals--; > } > } > > > > Best regards, > Julien > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#7893) memberof: bad a_numvals count
by jsoula＠univ-lille2.fr 09 Jul '14

09 Jul '14

Full_Name: julien soula Version: 2.4.39 OS: URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (194.254.117.28) Hi, this is related to ITS#6508. I don't think that the supplied patch is enough to fix the bug. When the last "member" entry is dangling then "( j - i == 1 )" is true and the loop is broken without reducing a_numvals. So I propose this patch : --- servers/slapd/overlays/memberof.c.orig 2014-07-09 21:17:14.487296363 +0200 +++ servers/slapd/overlays/memberof.c 2014-07-09 21:31:44.071565223 +0200 @@ -596,6 +596,7 @@ ber_memfree( a->a_nvals[ i ].bv_val ); BER_BVZERO( &a->a_nvals[ i ] ); } + a->a_numvals--; if ( j - i == 1 ) { break; } @@ -607,7 +608,6 @@ sizeof( struct berval ) * ( j - i ) ); } i--; - a->a_numvals--; } } Best regards, Julien

1 0

Re: (ITS#7351) can not start slapd when enable accesslog relay
by hyc＠symas.com 09 Jul '14

09 Jul '14

bashift(a)gmail.com wrote: > Full_Name: ChenShuJun > Version: 2.4.32 > OS: RHEL5.5(2.6.18-194.el5) > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (218.246.127.3) > Due to lack of cooperation from Oracle, back-ndb is no longer supported. Some info here http://www.openldap.org/lists/openldap-technical/201203/msg00216.html http://www.openldap.org/lists/openldap-devel/201307/msg00018.html This ITS will be closed. > > ####################################################################### > # BDB database definitions > ####################################################################### > > database ndb > suffix "dc=bgctv,dc=net,dc=cn" > rootdn "cn=admin,dc=bgctv,dc=net,dc=cn" > rootpw ghwl > directory /usr/local/openldap/var/openldap-data > dbconnect 172.30.5.20:1186 > dbhost 172.30.5.20 > dbname bgctv > dbuser bgctv > dbpass ghwl_h0 > dbconnections 1 > overlay accesslog > logdb "cn=auditlog" > logops writes > logold (objectclass=*) > > #dbconnect 172.30.5.20:1186 > #dbhost 172.30.5.20 > #dbname auditlog > #dbuser bgctv > #dbpass ghwl_h0 > #dbconnections 1 > > access to attrs=userPassword,shadowLastChange > by anonymous auth > by self write > by * none > access to dn.sub="ou=cm,dc=bgctv,dc=net,dc=cn" > by dn="userid=cmadmin,dc=bgctv,dc=net,dc=cn" write > by anonymous auth > by * none > access to * > by dn="userid=cmadmin,dc=bgctv,dc=net,dc=cn" read > by * none > > database bdb > suffix "cn=auditlog" > index reqStart eq > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7825) LMDB: misleading error message
by hyc＠symas.com 09 Jul '14

09 Jul '14

h.b.furuseth(a)usit.uio.no wrote: > 5) Add field MDB_dbx.md_dbiseq = DBI usage sequence number, > incremented when dbi_open creates (not reuses) a DBI and in > dbi_close. Copy it to a new malloced array txn->mt_dbiseqs[] > in write txns, verify it in at least commit and drop. > This means close()-open(same DB) will also be caught, which > seems a good thing since with (4) it will work unreliably: > Only if open() reuses the same DBI for the same DB. This is now fixed in mdb.master, using basically this approach. There is a master array of DBI sequence numbers in the environment and another array in each write transaction. It is verified anywhere the dbx->md_name would get used. The sequence number is incremented in dbi_open and when a handle is closed. Note - it is only checked in write txns, and only a few places (drop, commit, cursor_touch, page_search) ever check. There is a new error code MDB_BAD_DBI as well. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7789) Unreliable mdb_env_set_mapsize()
by hyc＠symas.com 09 Jul '14

09 Jul '14

Hallvard Breien Furuseth wrote: > Should have enclosed a test program. Here: > http://folk.uio.no/hbf/testsize.c > Output: > process #0: open env > process #1: open env > process #0: write txn, mapsize 204800 > process #1: write txn, mapsize 40960 > process #2: open env, write txn, mapsize 40960 > process #3: open env, write txn, mapsize 204800 > > > Summarizing a reply which got sent privately instead of to ITS: > > On 2014-01-24 00:07, Howard Chu wrote: >> The doc says the caller of set_mapsize is required to make sure there >> are no active transactions when it is called. As such, X failed this >> requirement, and this sequence of events is explicitly unsupported. > > No, I'm talking about X changing the mapsize when there is no txn, > then afterwards doing a write txn so the mapsize gets written. > >> If Y doesn't start its write txn until after X finishes, then Y will >> see the new size. > > It doesn't, that's the point. Only txn_commit pays attention, > and it checks the size which existed in the wrong metapage. > > Come to think of it, set_mapsize(); env_open(); txn begin/end > should do the same. Y will either way not pay attention to > the new mapsize. > Fixed now in mdb.master -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7713) Segmentation fault if the pagesize of the Operating system is not equal to 4096.
by hyc＠symas.com 08 Jul '14

08 Jul '14

hyc(a)symas.com wrote: > sumanth wrote: >> As per ur suggestions,tried changing >> >> 1. MDB_PAGESIZE to 65536 >> 2. changing pb_lower and pb_upper to uint32_t. >> As of now it is running fine for the existing testcases in openldap-2.4.36. >> However there were problems in 2.4.35. Also saw the comments in >> openldap-devel. So whether this will make to openldap 2.4.36? > > The current plan is to fix this in OpenLDAP 2.5, and we are now halting new > development in 2.4. > The fix is now available in mdb.master; currently you must compile with -DMDB_DEVEL to enable it. Since this is an incompatible DB format change, you should backup any existing DBs before using this option. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7892) LMDB OS X build issue: ESECT section attribute
by hyc＠symas.com 08 Jul '14

08 Jul '14

batterseapower(a)hotmail.com wrote: > Full_Name: Max Bolingbroke > Version: HEAD > OS: OS X > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (81.111.197.81) > > > Basically this line of mdb.c doesn't work on OS X: Fixed in mdb.master > > #define ESECT __attribute__ ((section("text_env"))) > > You just get an error: "argument to 'section' attribute is not valid for this > target: mach-o section specifier requires a segment and section separated by a > comma" > > I think you need to use this instead on this platform: > > #define ESECT __attribute__ ((section("__TEXT,text_env"))) > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#7892) LMDB OS X build issue: ESECT section attribute
by batterseapower＠hotmail.com 08 Jul '14

08 Jul '14

Full_Name: Max Bolingbroke Version: HEAD OS: OS X URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (81.111.197.81) Basically this line of mdb.c doesn't work on OS X: #define ESECT __attribute__ ((section("text_env"))) You just get an error: "argument to 'section' attribute is not valid for this target: mach-o section specifier requires a segment and section separated by a comma" I think you need to use this instead on this platform: #define ESECT __attribute__ ((section("__TEXT,text_env")))

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs