openldap-bugs

openldap-bugs@openldap.org

1 participants
21065 discussions

Re: (ITS#7829) MDB mdb_cursor_del causes records to be skipped
by armon.dadgar＠gmail.com 01 Apr '14

01 Apr '14

--533b042f_327b23c6_a8c6 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline I agree that it would be great to have MDB=5FNOT=46OUND returned instead of EINVAL. =46or my purposes, I needed a work around to the issue, and this seemed to work. Additionally because I=E2=80=99m in Go an= d not C, I cannot easily make use of the system definition for EINVAL. Best Regards, Armon Dadgar =46rom:=C2=A0Hallvard Breien =46uruseth h.b.furuseth=40usit.uio.no Reply:=C2=A0Hallvard Breien =46uruseth h.b.furuseth=40usit.uio.no Date:=C2=A0April 1, 2014 at 8:36:43 AM To:=C2=A0armon.dadgar=40gmail.com armon.dadgar=40gmail.com Cc:=C2=A0openldap-its=40openldap.org openldap-its=40openldap.org Subject:=C2=A0 Re: (ITS=237829) MDB mdb=5Fcursor=5Fdel causes records to = be skipped =20 On =46ri, 2014-03-28 at 04:58 +0000, armon.dadgar=40gmail.com wrote: =20 > I have a gist demonstrating this: https://gist.github.com/armon/9825666= =20 Some notes about this code: =20 // LMDB will return EINVAL(22) for the GET=5FCURRENT op if =20 // there is no further keys. We treat this as no more =20 // keys being found. =20 if num, ok :=3D err.(mdb.Errno); ok && num =3D=3D 22 =7B =20 println(=22errno 22=22) =20 err =3D mdb.Not=46ound =20 =7D =20 The value of EINVAL is OS-dependent, it need not be 22. =20 EINVAL means you did something wrong, it's a bad idea to =20 assume it means any particular wrong thing. =46or one thing, =20 mdb might not keep catching that error. I'd follow up =20 with a test check that there are indeed no more keys. =20 That said, mdb=5Fcursor=5Fdel() followed by MDB=5FGET=5FCURRENT =20 seems to me a sensible thing to want to do. So I think =20 MDB=5FGET=5FCURRENT should return MDB=5FNOT=46OUND here. And maybe =20 also after cursor=5Fnext() returned MDB=5FNOT=46OUND, and on an =20 empty DB. OTOH maybe cursors would need to keep more state =20 to tell this from user errors which should get EINVAL. =20 --533b042f_327b23c6_a8c6 Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline <html><head><style>body=7Bfont-family:Helvetica,Arial;font-size:13px=7D</= style></head><body style=3D=22word-wrap: break-word; -webkit-nbsp-mode: s= pace; -webkit-line-break: after-white-space;=22><div id=3D=22bloop=5Fcust= omfont=22 style=3D=22font-family:Helvetica,Arial;font-size:13px; color: r= gba(0,0,0,1.0); margin: 0px; line-height: auto;=22>I agree that it would = be great to have MDB=5FNOT=46OUND returned</div><div id=3D=22bloop=5Fcust= omfont=22 style=3D=22font-family:Helvetica,Arial;font-size:13px; color: r= gba(0,0,0,1.0); margin: 0px; line-height: auto;=22>instead of EINVAL. =46= or my purposes, I needed a work around to the</div><div id=3D=22bloop=5Fc= ustomfont=22 style=3D=22font-family:Helvetica,Arial;font-size:13px; color= : rgba(0,0,0,1.0); margin: 0px; line-height: auto;=22>issue, and this see= med to work. Additionally because I=E2=80=99m in Go and not</div><div id=3D= =22bloop=5Fcustomfont=22 style=3D=22font-family:Helvetica,Arial;font-size= :13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;=22>C, I ca= nnot easily make use of the system definition for EINVAL.</div><div id=3D= =22bloop=5Fcustomfont=22 style=3D=22font-family:Helvetica,Arial;font-size= :13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;=22><br></d= iv> <div id=3D=22bloop=5Fsign=5F1396376480489799936=22 class=3D=22bloop=5F= sign=22><div style=3D=22font-family:helvetica,arial;font-size:13px=22>Bes= t Regards,</div><div style=3D=22font-family:helvetica,arial;font-size:13p= x=22>Armon Dadgar<br></div></div> <div style=3D=22color:black=22><br>=46r= om: <span style=3D=22color:black=22>Hallvard Breien =46uruseth</span= > <a href=3D=22mailto:h.b.furuseth=40usit.uio.no=22>h.b.furuseth=40usit.u= io.no</a><br>Reply: <span style=3D=22color:black=22>Hallvard Breien = =46uruseth</span> <a href=3D=22mailto:h.b.furuseth=40usit.uio.no=22>h.b.f= uruseth=40usit.uio.no</a><br>Date: <span style=3D=22color:black=22>A= pril 1, 2014 at 8:36:43 AM</span><br>To: <span style=3D=22color:blac= k=22>armon.dadgar=40gmail.com</span> <a href=3D=22mailto:armon.dadgar=40g= mail.com=22>armon.dadgar=40gmail.com</a><br>Cc: <span style=3D=22col= or:black=22>openldap-its=40openldap.org</span> <a href=3D=22mailto:openld= ap-its=40openldap.org=22>openldap-its=40openldap.org</a><br>Subject:&nbsp= ;<span style=3D=22color:black=22> Re: (ITS=237829) MDB mdb=5Fcursor=5Fdel= causes records to be skipped <br></span></div><br> <blockquote type=3D=22= cite=22 class=3D=22clean=5Fbq=22><span><div><div></div><div>On =46ri, 201= 4-03-28 at 04:58 +0000, armon.dadgar=40gmail.com wrote: <br>> I have a gist demonstrating this: https://gist.github.com/armon/= 9825666 <br> <br>Some notes about this code: <br> // LMDB will return EINVAL(22) for the GET=5FCURRENT op if <br> // there is no further keys. We treat this as no more <br> // keys being found. <br> if num, ok :=3D err.(mdb.Errno); ok && num =3D=3D 22 =7B <br> println(=22errno 22=22) <br> err =3D mdb.Not=46ound <br> =7D <br> <br>The value of EINVAL is OS-dependent, it need not be 22. <br> <br>EINVAL means you did something wrong, it's a bad idea to <br>assume it means any particular wrong thing. =46or one thing, <br>mdb might not keep catching that error. I'd follow up <br>with a test check that there are indeed no more keys. <br> <br> <br>That said, mdb=5Fcursor=5Fdel() followed by MDB=5FGET=5FCURRENT <br>seems to me a sensible thing to want to do. So I think <br>MDB=5FGET=5FCURRENT should return MDB=5FNOT=46OUND here. And maybe <br>also after cursor=5Fnext() returned MDB=5FNOT=46OUND, and on an <br>empty DB. OTOH maybe cursors would need to keep more state <br>to tell this from user errors which should get EINVAL. <br> <br></div></div></span></blockquote></body></html> --533b042f_327b23c6_a8c6--

1 0

Re: (ITS#7829) MDB mdb_cursor_del causes records to be skipped
by h.b.furuseth＠usit.uio.no 01 Apr '14

01 Apr '14

On Fri, 2014-03-28 at 04:58 +0000, armon.dadgar(a)gmail.com wrote: > I have a gist demonstrating this: https://gist.github.com/armon/9825666 Some notes about this code: // LMDB will return EINVAL(22) for the GET_CURRENT op if // there is no further keys. We treat this as no more // keys being found. if num, ok := err.(mdb.Errno); ok && num == 22 { println("errno 22") err = mdb.NotFound } The value of EINVAL is OS-dependent, it need not be 22. EINVAL means you did something wrong, it's a bad idea to assume it means any particular wrong thing. For one thing, mdb might not keep catching that error. I'd follow up with a test check that there are indeed no more keys. That said, mdb_cursor_del() followed by MDB_GET_CURRENT seems to me a sensible thing to want to do. So I think MDB_GET_CURRENT should return MDB_NOTFOUND here. And maybe also after cursor_next() returned MDB_NOTFOUND, and on an empty DB. OTOH maybe cursors would need to keep more state to tell this from user errors which should get EINVAL.

1 0

Re: (ITS#7829) MDB mdb_cursor_del causes records to be skipped
by hyc＠symas.com 31 Mar '14

31 Mar '14

Hallvard Breien Furuseth wrote: > Sorry, forgot the crash info. Thanks, confirmed. In a nested rebalance, doing a page merge with the neighbor on the left was discarding the rebalanced state. Fixed now in mdb.master > > mdb.c:7419: Assertion 'NUMKEYS(mc->mc_pg[ptop]) > 1' failed in mdb_rebalance() > > (gdb) p *mc->mc_pg[ptop] > $1 = { > mp_p = {p_pgno = 24, p_next = 0x18}, mp_pad = 0, mp_flags = 17, > mp_pb = {pb = {pb_lower = 18, pb_upper = 1016}, pb_pages = 66584594}, > mp_ptrs = {1016} > } > (gdb) info locals > node = 0x0 > rc = -268443168 > ptop = 0 > minkeys = 1 > mn = { > mc_next = 0x7ffff5ce0010, mc_backup = 0x7fffe41013a0, mc_xcursor = 0x0, > mc_txn = 0x7fffe42c9700, mc_dbi = 6, mc_db = 0x7fffe42c9890, > mc_dbx = 0x7fffe42c3fb0, mc_dbflag = 0x7fffe42c3fb0 "\002", > mc_snum = 2, mc_top = 1, mc_flags = 65, > mc_pg = {0x7fffe4102840, 0x7fffe8109670, 0x2efffe300, 0x7fffefffe0c0, 0x7fffefffe0a0, 0x7fffefffe2c0, > 0x7fffefffe2c0, 0x53e7cd, 0x7fffefffe1c0, 0x7fffe40019f0, 0x7fffe41013a0, 0x7fffefffe2c0, 0x30c3669909, > 0xfbad8001, 0x7fffefffe2c0, 0x7fffefffe2c0, 0x7fffefffe2c0, 0x7fffefffe130, 0x7fffefffe2d0, 0x7fffefffe180, > 0x49c7ca, 0x7fffefffe140, 0x7fffefffe1dc, 0x1100000000, 0x0, 0x7fffefffe2d0, 0x7fffe4101528, 0x7fffefffe180, 0x1, > 0x7fffe81099d4, 0x7fffefffe260, 0x100000008}, > mc_ki = {5416, 58384, 32767, 0, 39380, 59408, 32767, 0, 39364, 59408, 32767, 0, 39400, 59408, 32767, 0, 39332, > 59408, 32767, 0, 68, 0, 0, 0, 15369, 50024, 48, 0, 57792, 61439, 32767, 0} > } > oldki = 0 > __FUNCTION__ = "mdb_rebalance" > (gdb) p *mc > $2 = { > mc_next = 0x0, mc_backup = 0x0, mc_xcursor = 0x7fffe4101528, > mc_txn = 0x7fffe42c9700, mc_dbi = 6, mc_db = 0x7fffe42c9890, > mc_dbx = 0x7fffe42c3fb0, mc_dbflag = 0x7fffe42cb3e6 "\t\n\n", > mc_snum = 2, mc_top = 1, mc_flags = 65, > mc_pg = {0x7fffe4102840, 0x7fffe8109670, 0x0, 0x40, 0x44, 0x0, 0x6a206e616d726568, 0x6f65703d756f2c72, > 0x653d63642c656c70, 0x642c656c706d6178, 0x302e006d6f633d63, 0x30235a3632353631, 0x2d1, 0x7fffe4000338, 0x81, > 0x7fffe4102c40, 0x7fffe42c7860, 0x20, 0x64, 0x0, 0x4, 0x1a, 0x19, 0x10, 0xe, 0xd, 0xb, 0xa, 0x9, 0x140, 0x84, > 0x0}, > mc_ki = {0, 5, 0, 0, 22160, 63214, 32767, 0, 32769, 0, 0, 0, 64592, 70, 0, 0, 22160, 63214, 32767, 0, 32775, 0, 0, > 0, 63424, 70, 0, 0, 22160, 63214, 32767, 0} > } > > Backtrace: > #2 mdb_assert_fail () > #3 mdb_rebalance (mc=0x7fffe41013a0) at mdb.c:7419 > #4 mdb_cursor_del0 (mc=0x7fffe41013a0) at mdb.c:7496 > #5 mdb_cursor_del (mc=0x7fffe41013a0, flags=0) at mdb.c:6360 > #6 mdb_idl_delete_keys (be, cursor=0x7fffe41013a0, keys, id=7) at idl.c:608 > #7 indexer (op=0x7fffe4000950, txn, ai, ad=0x838180, atname=0x837fc8, vals=0x7fffe40019f0, id=7, opid=2, mask=1814) at index.c:258 > #8 index_at_values (op=0x7fffe4000950, txn=0x7fffe42c9700, ad, type=0x837f60, tags=0x8381a0, vals=0x7fffe40019f0, id=7, opid=2) at index.c:337 > #9 mdb_index_values (op, txn, desc, vals, id, opid) at index.c:386 > #10 mdb_index_entry (op=0x7fffe4000950, txn=0x7fffe42c9700, opid=2, e=0x7fffe40017d0) at index.c:558 > #11 mdb_delete (op=0x7fffe4000950, rs=0x7fffefffe910) at delete.c:347 > #12 overlay_op_walk (op=0x7fffe4000950, rs=0x7fffefffe910, which=op_delete, oi=0x7fffe81036a0, on=0x0) at backover.c:671 > #13 over_op_func (op=0x7fffe4000950, rs, which) at backover.c:723 > #14 fe_op_delete (op=0x7fffe4000950, rs=0x7fffefffe910) at delete.c:174 > #15 do_delete (op=0x7fffe4000950, rs=0x7fffefffe910) at delete.c:95 > #16 connection_operation (ctx=0x7fffefffea70, arg_v=0x7fffe4000950) at connection.c:1155 > #17 connection_read_thread (ctx=0x7fffefffea70, argv) at connection.c:1291 > #18 ldap_int_thread_pool_wrapper (xpool=0x83b3b0) at tpool.c:688 > > Test output: > Running ldapadd to build slapd config database... > Running ldapadd to build slapd database... > Search the entire database... > Running ldapmodify to add a member... > Re-search the entire database... > Running ldapmodify to rename a member... > Re-search the entire database... > Running ldapmodify to rename a group... > Re-search the entire database... > Running ldapmodify to add self... > Re-search the entire database... > Running ldapdelete to remove a member... > Re-search the entire database... > Running ldapdelete to remove a group... > Re-search the entire database... > Adding groups with MAY member type schemas... > > Log ends with: > 533898e1 => index_entry_del( 7, "cn=Jessica Rabbit,ou=People,dc=example,dc=com" ) > 533898e1 mdb_idl_delete_keys: 7 [860433ad] > 533898e1 mdb_idl_delete_keys: 7 [00000000] > 533898e1 mdb_idl_delete_keys: 7 [aaacba45] > 533898e1 mdb_idl_delete_keys: 7 [e10d2617] > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7829) MDB mdb_cursor_del causes records to be skipped
by h.b.furuseth＠usit.uio.no 30 Mar '14

30 Mar '14

Sorry, forgot the crash info. mdb.c:7419: Assertion 'NUMKEYS(mc->mc_pg[ptop]) > 1' failed in mdb_rebalance() (gdb) p *mc->mc_pg[ptop] $1 = { mp_p = {p_pgno = 24, p_next = 0x18}, mp_pad = 0, mp_flags = 17, mp_pb = {pb = {pb_lower = 18, pb_upper = 1016}, pb_pages = 66584594}, mp_ptrs = {1016} } (gdb) info locals node = 0x0 rc = -268443168 ptop = 0 minkeys = 1 mn = { mc_next = 0x7ffff5ce0010, mc_backup = 0x7fffe41013a0, mc_xcursor = 0x0, mc_txn = 0x7fffe42c9700, mc_dbi = 6, mc_db = 0x7fffe42c9890, mc_dbx = 0x7fffe42c3fb0, mc_dbflag = 0x7fffe42c3fb0 "\002", mc_snum = 2, mc_top = 1, mc_flags = 65, mc_pg = {0x7fffe4102840, 0x7fffe8109670, 0x2efffe300, 0x7fffefffe0c0, 0x7fffefffe0a0, 0x7fffefffe2c0, 0x7fffefffe2c0, 0x53e7cd, 0x7fffefffe1c0, 0x7fffe40019f0, 0x7fffe41013a0, 0x7fffefffe2c0, 0x30c3669909, 0xfbad8001, 0x7fffefffe2c0, 0x7fffefffe2c0, 0x7fffefffe2c0, 0x7fffefffe130, 0x7fffefffe2d0, 0x7fffefffe180, 0x49c7ca, 0x7fffefffe140, 0x7fffefffe1dc, 0x1100000000, 0x0, 0x7fffefffe2d0, 0x7fffe4101528, 0x7fffefffe180, 0x1, 0x7fffe81099d4, 0x7fffefffe260, 0x100000008}, mc_ki = {5416, 58384, 32767, 0, 39380, 59408, 32767, 0, 39364, 59408, 32767, 0, 39400, 59408, 32767, 0, 39332, 59408, 32767, 0, 68, 0, 0, 0, 15369, 50024, 48, 0, 57792, 61439, 32767, 0} } oldki = 0 __FUNCTION__ = "mdb_rebalance" (gdb) p *mc $2 = { mc_next = 0x0, mc_backup = 0x0, mc_xcursor = 0x7fffe4101528, mc_txn = 0x7fffe42c9700, mc_dbi = 6, mc_db = 0x7fffe42c9890, mc_dbx = 0x7fffe42c3fb0, mc_dbflag = 0x7fffe42cb3e6 "\t\n\n", mc_snum = 2, mc_top = 1, mc_flags = 65, mc_pg = {0x7fffe4102840, 0x7fffe8109670, 0x0, 0x40, 0x44, 0x0, 0x6a206e616d726568, 0x6f65703d756f2c72, 0x653d63642c656c70, 0x642c656c706d6178, 0x302e006d6f633d63, 0x30235a3632353631, 0x2d1, 0x7fffe4000338, 0x81, 0x7fffe4102c40, 0x7fffe42c7860, 0x20, 0x64, 0x0, 0x4, 0x1a, 0x19, 0x10, 0xe, 0xd, 0xb, 0xa, 0x9, 0x140, 0x84, 0x0}, mc_ki = {0, 5, 0, 0, 22160, 63214, 32767, 0, 32769, 0, 0, 0, 64592, 70, 0, 0, 22160, 63214, 32767, 0, 32775, 0, 0, 0, 63424, 70, 0, 0, 22160, 63214, 32767, 0} } Backtrace: #2 mdb_assert_fail () #3 mdb_rebalance (mc=0x7fffe41013a0) at mdb.c:7419 #4 mdb_cursor_del0 (mc=0x7fffe41013a0) at mdb.c:7496 #5 mdb_cursor_del (mc=0x7fffe41013a0, flags=0) at mdb.c:6360 #6 mdb_idl_delete_keys (be, cursor=0x7fffe41013a0, keys, id=7) at idl.c:608 #7 indexer (op=0x7fffe4000950, txn, ai, ad=0x838180, atname=0x837fc8, vals=0x7fffe40019f0, id=7, opid=2, mask=1814) at index.c:258 #8 index_at_values (op=0x7fffe4000950, txn=0x7fffe42c9700, ad, type=0x837f60, tags=0x8381a0, vals=0x7fffe40019f0, id=7, opid=2) at index.c:337 #9 mdb_index_values (op, txn, desc, vals, id, opid) at index.c:386 #10 mdb_index_entry (op=0x7fffe4000950, txn=0x7fffe42c9700, opid=2, e=0x7fffe40017d0) at index.c:558 #11 mdb_delete (op=0x7fffe4000950, rs=0x7fffefffe910) at delete.c:347 #12 overlay_op_walk (op=0x7fffe4000950, rs=0x7fffefffe910, which=op_delete, oi=0x7fffe81036a0, on=0x0) at backover.c:671 #13 over_op_func (op=0x7fffe4000950, rs, which) at backover.c:723 #14 fe_op_delete (op=0x7fffe4000950, rs=0x7fffefffe910) at delete.c:174 #15 do_delete (op=0x7fffe4000950, rs=0x7fffefffe910) at delete.c:95 #16 connection_operation (ctx=0x7fffefffea70, arg_v=0x7fffe4000950) at connection.c:1155 #17 connection_read_thread (ctx=0x7fffefffea70, argv) at connection.c:1291 #18 ldap_int_thread_pool_wrapper (xpool=0x83b3b0) at tpool.c:688 Test output: Running ldapadd to build slapd config database... Running ldapadd to build slapd database... Search the entire database... Running ldapmodify to add a member... Re-search the entire database... Running ldapmodify to rename a member... Re-search the entire database... Running ldapmodify to rename a group... Re-search the entire database... Running ldapmodify to add self... Re-search the entire database... Running ldapdelete to remove a member... Re-search the entire database... Running ldapdelete to remove a group... Re-search the entire database... Adding groups with MAY member type schemas... Log ends with: 533898e1 => index_entry_del( 7, "cn=Jessica Rabbit,ou=People,dc=example,dc=com" ) 533898e1 mdb_idl_delete_keys: 7 [860433ad] 533898e1 mdb_idl_delete_keys: 7 [00000000] 533898e1 mdb_idl_delete_keys: 7 [aaacba45] 533898e1 mdb_idl_delete_keys: 7 [e10d2617] -- Hallvard

1 0

Re: (ITS#7829) MDB mdb_cursor_del causes records to be skipped
by h.b.furuseth＠usit.uio.no 30 Mar '14

30 Mar '14

hyc(a)symas.com writes: > Thanks for the report. Fixed now in mdb.master. That fix breaks test052-memberof with pagesize 1024. Unrealistic, but it's not a MAXKEYSIZE issue so it should not have made a difference. Tested with RE24 9e22d8f2cd5ed10cd1dd22acbd729a49e5ecc1ab + mdb.master~ or mdb.master ac3acc121855e4297d597a4d71e9863e1e1e2450 + this patch: diff --git a/libraries/liblmdb/lmdb.h b/libraries/liblmdb/lmdb.h index 2ebd43f..3f4a167 100644 --- a/libraries/liblmdb/lmdb.h +++ b/libraries/liblmdb/lmdb.h @@ -411,4 +411,4 @@ typedef enum MDB_cursor_op { /** Too big key/data, key is empty, or wrong DUPFIXED size */ -#define MDB_BAD_VALSIZE (-30781) -#define MDB_LAST_ERRCODE MDB_BAD_VALSIZE +#define MDB_BAD_VALSIZE (abort(),-30781) /* we don't hit MDB_MAXKEYSIZE */ +#define MDB_LAST_ERRCODE (-30781) /** @} */ diff --git a/libraries/liblmdb/mdb.c b/libraries/liblmdb/mdb.c index 7e29058..ad9bfc6 100644 --- a/libraries/liblmdb/mdb.c +++ b/libraries/liblmdb/mdb.c @@ -405,3 +405,3 @@ static txnid_t mdb_debug_start; #ifndef MDB_MAXKEYSIZE -#define MDB_MAXKEYSIZE 511 +#define MDB_MAXKEYSIZE 400 /* max-max is 446 for psize=1024 on 64-bit hosts */ #endif @@ -3536,3 +3536,3 @@ mdb_env_open2(MDB_env *env) newenv = 1; - env->me_psize = env->me_os_psize; + env->me_psize = 1024; if (env->me_psize > MAX_PAGESIZE) -- Hallvard

1 0

Re: (ITS#7829) MDB mdb_cursor_del causes records to be skipped
by hyc＠symas.com 28 Mar '14

28 Mar '14

armon.dadgar(a)gmail.com wrote: > Full_Name: Armon Dadgar > Version: > OS: > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (50.78.97.10) > > > I have an issue where when iterating over data and deleting keys, the cursor > will incorrectly skip rows, and then returns an early NOTFOUND even though more > records still exist. > > I have a gist demonstrating this: https://gist.github.com/armon/9825666 > > I am using small Go wrapper around LMDB, version 0.9.11. > > The expected behavior is to numbers 1..91 printed. However, when I run it, > it gets to 16 and returns no further rows. > > Thanks for the report. Fixed now in mdb.master. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7825) LMDB: misleading error message
by timur.kristof＠gmail.com 28 Mar '14

28 Mar '14

On Fri Mar 28 2014 18:36:11 GMT+0100 (CET), Howard Chu wrote: > timur.kristof(a)gmail.com wrote: > > > Here's a small test case which will demonstrate it: > > http://pastebin.com/hHwD1srf > > As an aside, this is some pretty horrible code. Why are you using > MDB_GET_CURRENT after all of your cursor_get calls? It truly is horrendous. I wrote it in an attempt to (roughly) reproduce what one of my examples did in javascript. Then when I discovered what the issue really was, I forgot to clean it up before pastebinning. I hope it's still okay as a testcase for this bugreport :) > -- Howard Chu > CTO, Symas Corp. http://www.symas.com > Director, Highland Sun http://highlandsun.com/hyc/ > Chief Architect, OpenLDAP http://www.openldap.org/project/ > -- Timur Sent from my Jolla

1 0

Re: (ITS#7825) LMDB: misleading error message
by hyc＠symas.com 28 Mar '14

28 Mar '14

timur.kristof(a)gmail.com wrote: > Here's a small test case which will demonstrate it: > http://pastebin.com/hHwD1srf As an aside, this is some pretty horrible code. Why are you using MDB_GET_CURRENT after all of your cursor_get calls? -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#7830) MDB Size grown a lot during massive updates and concurrent read access
by antonin.meunier＠cgi.com 28 Mar '14

28 Mar '14

Full_Name: Antonin Meunier Version: 2.4.39 OS: Red Hat 6.5 URL: Submission from: (NULL) (195.6.127.108) Ldap is set to a maximum MDB database size of 80GB. Ldap uses 20GB of effective Size. There are approximately 60 index , 1500000 peoples and 4500000 groups. The LDAP is accessed in read / write by about 20 tomcat. In parallel to this, update treatment (inject ldif) are started performing lots of massive upates (750 000, 500 000 groups). When processing the massives updates, the MDB size increases to the 80max and therefore the treatment plant updates. If I stop all tomcat, the problem no longer occurs. So it seems that the problem comes from locks that are placed on the pages of indexes and data by readers and force to create new pages for changes with conservation of all intermediate pages. After that, the only way to find a "normal" size is to export the ldap , delete it and re-import the backup. The LDAP database is then again 20GB

1 0

Re: (ITS#7828) LMDB crash in mdb_cursor_put(MDB_CURRENT)
by sbn＠tbricks.com 27 Mar '14

27 Mar '14

Works after reverting commit (checked f537429) Thanks!

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs