openldap-bugs

openldap-bugs@openldap.org

1 participants
20967 discussions

Re: (ITS#9037) observing crash in mdb_cursor_put()
by hyc＠symas.com 19 Jun '19

19 Jun '19

myk(a)mykzilla.org wrote: > hyc(a)symas.com wrote on 2019-06-18 01:30: >> There is no issue with mdb_cursor_put then. The question is why didn't mdb_page_search find the DBI's root node? > Would it be useful for mdb_cursor_init to return the result code of > mdb_page_search, and for mdb_put to then return that code if it isn't > MDB_SUCCESS? Maybe. For testing purposes you could try something like this diff --git a/libraries/liblmdb/mdb.c b/libraries/liblmdb/mdb.c index 692feaa38b..e41f3bc36a 100644 --- a/libraries/liblmdb/mdb.c +++ b/libraries/liblmdb/mdb.c @@ -7620,7 +7620,8 @@ mdb_cursor_init(MDB_cursor *mc, MDB_txn *txn, MDB_dbi dbi, MDB_xcursor *mx) mc->mc_xcursor = NULL; } if (*mc->mc_dbflag & DB_STALE) { - mdb_page_search(mc, NULL, MDB_PS_ROOTONLY); + int rc = mdb_page_search(mc, NULL, MDB_PS_ROOTONLY); + mdb_cassert(mc, !rc); } } I suspect the problem will simply be that you've used an invalid DBI. > > That wouldn't resolve the problem, but it might avoid the crash and > provide some insight into the failure (if mdb_page_search is actually > returning a failure code when it doesn't find the DBI's root node). > > -myk > > > > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#9037) observing crash in mdb_cursor_put()
by myk＠mykzilla.org 19 Jun '19

19 Jun '19

hyc(a)symas.com wrote on 2019-06-18 01:30: > There is no issue with mdb_cursor_put then. The question is why didn't mdb_page_search find the DBI's root node? Would it be useful for mdb_cursor_init to return the result code of mdb_page_search, and for mdb_put to then return that code if it isn't MDB_SUCCESS? That wouldn't resolve the problem, but it might avoid the crash and provide some insight into the failure (if mdb_page_search is actually returning a failure code when it doesn't find the DBI's root node). -myk

1 0

Re: (ITS#9037) observing crash in mdb_cursor_put()
by grobins＠pulsesecure.net 18 Jun '19

18 Jun '19

--_000_MN2PR06MB55017A7261FA27380778753BAFEA0MN2PR06MB5501namp_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable <repost since the previous message came as 64 encoded, apologies> Hello Howard, we are not able to reproduce the issue consistently so test code is not ava= ilable at this time, and I do believe this issue will be present in the lat= est code as well since Myk also reported the same.. Attaching analysis fro= m our side, hope it helps. The call stack what we have seen is, #0 0x0011b62e in mdb_cursor_put (mc=3D0xffe52d18, key=3D0xffe52e74, dat= a=3D0xffe52e6c, flags=3D0) at mdb.c:6688 #1 0x0011c8ec in mdb_put (txn=3D0xdc13f008, dbi=3D2, key=3D0xffe52e74, = data=3D0xffe52e6c, flags=3D0) at mdb.c:8771 #2 0x00b792b8 in LMDB::LMDBContext::createSession (this=3D0x80ca418, si= d=3D0x8297be4 "sidebf3708f0fd9fcb8e062529da47adf51402dfc4600000000+") at lm= dbint.cc:460 #3 0x08053564 in updateLMDB (request=3D..., forward=3D@0xffe5315f) at s= essionserver.cc:1075 #4 processRequestWithoutResponse (request=3D..., forward=3D@0xffe5315f)= at sessionserver.cc:1160 #5 0x08056a80 in processRequest (this=3D0x80b2ac0, fd=3D33) at sessions= erver.cc:1189 #6 readFromSock (this=3D0x80b2ac0, fd=3D33) at sessionserver.cc:1342 #7 SockHandler::ioReady (this=3D0x80b2ac0, fd=3D33) at sessionserver.cc= :1455 #8 0x00ba2592 in runCoreDispatcher (default_t=3D<value optimized out>, = flags=3D-1) at fds.cc:889 #9 0x00ba2ff7 in DSEvntFds::runDispatcher () at fds.cc:945 #10 0x080559c6 in main () at sessionserver.cc:1739 Here it is helpful to examine frames #0 and #1 in detail in relation to the= relevant source code /libraries/liblmdb/mdb.c. Frame #1 #1 0x0011c8ec in mdb_put (txn=3D0xdc13f008, dbi=3D2, key=3D0xffe52e74, dat= a=3D0xffe52e6c, flags=3D0) at mdb.c:8771 mc =3D {mc_next =3D 0x0, mc_backup =3D 0x0, mc_xcursor =3D 0x0, mc_= txn =3D 0xdc13f008, mc_dbi =3D 2, mc_db =3D 0xdc13f088, mc_dbx =3D 0xe12150= 38, mc_dbflag =3D 0xdc1cf09a "\033", '\032' <repeats 51 times>, mc_snum =3D= 0, mc_top =3D 0, mc_flags =3D 0, mc_pg =3D {0x0, 0x80b2ac0, 0xffe52d68, 0x= 28afce, 0xdc13f008, 0x80ce9b0, 0xffe52dc8, 0x4c5ff4, 0x2b, 0x80b2ac0, 0xffe= 52d98, 0x49209a, 0x2b, 0x4c5ff4, 0x2121cb, 0x21576c, 0x8297d34, 0x3497af, 0= x21576c, 0x21316a, 0x8297d34, 0x80ac7b0, 0x1e, 0x46eed6, 0x2b, 0x0, 0x80ce9= b0, 0x4c5ff4, 0x80ac7b0, 0x8297d34, 0xffe52df8, 0x46fb96}, mc_ki =3D {0, 20= 89, 51120, 2058, 30, 0, 16796, 17, 11760, 65509, 3, 0, 32040, 2089, 30, 0, = 0, 0, 0, 0, 3, 0, 24564, 76, 51120, 2058, 10944, 2059, 11800, 65509, 64758,= 70}} <-------- mx =3D {mx_cursor =3D {mc_next =3D 0x38, mc_backup =3D 0x3, mc_xcur= sor =3D 0x0, mc_txn =3D 0x0, mc_dbi =3D 0, mc_db =3D 0x28, mc_dbx =3D 0x0, = mc_dbflag =3D 0x0, mc_snum =3D 3, mc_top =3D 0, mc_flags =3D 17, mc_pg =3D = {0x3a93d8, 0x10, 0x0, 0x18, 0x3a93a0, 0x0, 0x3a93d0, 0x289abe, 0x3a93a0, 0x= ffe52d3f, 0xffe52c68, 0x28afce, 0xffe52dbc, 0xffe52db4, 0x3, 0x4c5ff4, 0x11= , 0x36c99b, 0x6e, 0x77, 0x7c, 0x5b, 0x38, 0x289abe, 0x0, 0x0, 0x0, 0x28, 0x= 0, 0x0, 0x3, 0x10}, mc_ki =3D {37848, 58, 51611, 54, 110, 0, 119, 0, 124, 0= , 91, 0, 58, 0, 39614, 40, 0, 0, 0, 0, 0, 0, 160, 0, 0, 0, 2, 0, 18, 0, 136= , 0}}, mx_db =3D {md_pad =3D 3838936, md_flags =3D 51611, md_depth =3D 54, = md_branch_pages =3D 110, md_leaf_pages =3D 119, md_overflow_pages =3D 124, = md_entries =3D 91, md_root =3D 56}, mx_dbx =3D {md_name =3D {mv_size =3D 6,= mv_data =3D 0x0}, md_cmp =3D 0, md_dcmp =3D 0, md_rel =3D 0x40, md_relctx = =3D 0x0}, mx_dbflag =3D 0 '\000'} rc =3D 0 And source code for mdb_put() is from mdb.c int mdb_put(MDB_txn *txn, MDB_dbi dbi, MDB_val *key, MDB_val *data, unsigned int flags) { MDB_cursor mc; MDB_xcursor mx; int rc; if (!key || !data || !TXN_DBI_EXIST(txn, dbi, DB_USRVALID)) return EINVAL; if (flags & ~(MDB_NOOVERWRITE|MDB_NODUPDATA|MDB_RESERVE|MDB= _APPEND|MDB_APPENDDUP)) return EINVAL; if (txn->mt_flags & (MDB_TXN_RDONLY|MDB_TXN_BLOCKED)) return (txn->mt_flags & MDB_TXN_RDONLY) ? E= ACCES : MDB_BAD_TXN; mdb_cursor_init(&mc, txn, dbi, &mx); <-------- see source c= ode below mc.mc_next =3D txn->mt_cursors[dbi]; txn->mt_cursors[dbi] =3D &mc; rc =3D mdb_cursor_put(&mc, key, data, flags); <-------- see= Frame #0 txn->mt_cursors[dbi] =3D mc.mc_next; return rc; } Source Code For mdb_cursor_init() From mdb.c /** Initialize a cursor for a given transaction and database. */ static voi= d mdb_cursor_init(MDB_cursor *mc, MDB_txn *txn, MDB_dbi dbi, MDB_xcursor *m= x) { mc->mc_next =3D NULL; mc->mc_backup =3D NULL; mc->mc_dbi =3D dbi; mc->mc_txn =3D txn; mc->mc_db =3D &txn->mt_dbs[dbi]; mc->mc_dbx =3D &txn->mt_dbxs[dbi]; mc->mc_dbflag =3D &txn->mt_dbflags[dbi]; mc->mc_snum =3D 0; <-------- mc->mc_top =3D 0; <-------- mc->mc_pg[0] =3D 0; <-------- mc->mc_ki[0] =3D 0; mc->mc_flags =3D 0; if (txn->mt_dbs[dbi].md_flags & MDB_DUPSORT) { mdb_tassert(txn, mx !=3D NULL); mc->mc_xcursor =3D mx; mdb_xcursor_init0(mc); } else { mc->mc_xcursor =3D NULL; } if (*mc->mc_dbflag & DB_STALE) { <-------- false *mc->mc_db= flag =3D 27, DB_STALE =3D 0x02 /**< Named-DB record is old= er than txnID */ mdb_page_search(mc, NULL, MDB_PS_ROOTONLY); } } >From this we know that when mdb_cursor_put() is called, the following value= s remain assigned: mc->mc_snum =3D 0; <-------- mc->mc_top =3D 0; <-------- mc->mc_pg[0] =3D 0; <-------- and its noted these two values still have their initial values at the time = of the segmentation fault. (gdb) fr 0 #0 0x0011b62e in mdb_cursor_put (mc=3D0xffe52d18, key=3D0xffe52e74, data=3D= 0xffe52e6c, flags=3D0) at mdb.c:6688 6688 in mdb.c (gdb) p mc->mc_top $1 =3D 0 (gdb) p mc->mc_pg[mc->mc_top] $19 =3D (MDB_page *) 0x0 Let us consider how the path taken through mdb_cursor_put() could account f= or these null variable values which result in a segmentation fault. Frame #0 #0 0x0011b62e in mdb_cursor_put (mc=3D0xffe52d18, key=3D0xffe52e74, data= =3D0xffe52e6c, flags=3D0) at mdb.c:6688 <-------- flags =3D 0 env =3D 0x80ce9b0 leaf =3D <value optimized out> fp =3D <value optimized out> mp =3D 0x38 sub_root =3D 0x0 fp_flags =3D <value optimized out> xdata =3D {mv_size =3D 3838880, mv_data =3D 0xdc1cf09a} rdata =3D 0xffe52e6c dkey =3D {mv_size =3D 0, mv_data =3D 0x3a93cc} olddata =3D {mv_size =3D 22, mv_data =3D 0x36c820} dummy =3D {md_pad =3D 22, md_flags =3D 11128, md_depth =3D 65509, m= d_branch_pages =3D 16, md_leaf_pages =3D 1507329, md_overflow_pages =3D 0, = md_entries =3D 3838928, md_root =3D 3833844} do_sub =3D 0 insert_key =3D -30798 <-------- #define MDB_NOTFOUND (-307= 98) insert_data =3D -30798 <-------- #define MDB_NOTFOUND (-3079= 8) mcount =3D 0 dcount =3D 0 nospill =3D 0 <-------- nsize =3D <value optimized out> rc =3D 0 rc2 =3D <value optimized out> nflags =3D 0 <-------- __func__ =3D "mdb_cursor_put" We know that mc->mc_top and mc->mc_pg[0] were initialized to 0 in mdb_curso= r_init(). As my annotations of the source code below suggest, careful analy= sis reveals that the paths actually taken through mdb_cursor_put() do not m= odify these assignments. Excerpted Source Code For mdb_cursor_put() From mdb.c int mdb_cursor_put(MDB_cursor *mc, MDB_val *key, MDB_val *data, unsigned int flags) { MDB_env *env; MDB_node *leaf =3D NULL; MDB_page *fp, *mp, *sub_root =3D NULL; uint16_t fp_flags; MDB_val xdata, *rdata, dkey, o= lddata; MDB_db dummy; int do_sub =3D 0, insert_key, insert_data; unsigned int mcount =3D 0, dcount =3D 0, nospill; size_t nsize; int rc, rc2; unsigned int nflags; DKBUF; if (mc =3D=3D NULL || key =3D=3D NULL) return EINVAL; env =3D mc->mc_txn->mt_env; /* Check this first so counter will always be zero on any * early failures. */ if (flags & MDB_MULTIPLE) { dcount =3D data[1].mv_size; data[1].mv_size =3D 0; if (!F_ISSET(mc->mc_db->md_flags, MDB_DUPFI= XED)) return MDB_INCOMPATIBLE; } nospill =3D flags & MDB_NOSPILL; flags &=3D ~MDB_NOSPILL; if (mc->mc_txn->mt_flags & (MDB_TXN_RDONLY|MDB_TXN_BLOCKED)= ) return (mc->mc_txn->mt_flags & MDB_TXN_RDON= LY) ? EACCES : MDB_BAD_TXN; if (key->mv_size-1 >=3D ENV_MAXKEY(env)) return MDB_BAD_VALSIZE; #if SIZE_MAX > MAXDATASIZE if (data->mv_size > ((mc->mc_db->md_flags & MDB_DUPSORT) ? = ENV_MAXKEY(env) : MAXDATASIZE)) return MDB_BAD_VALSIZE; #else if ((mc->mc_db->md_flags & MDB_DUPSORT) && data->mv_size > = ENV_MAXKEY(env)) return MDB_BAD_VALSIZE; #endif DPRINTF(("=3D=3D> put db %d key [%s], size %"Z"u, data size= %"Z"u", DDBI(mc), DKEY(key), key ? key->mv_size : 0= , data->mv_size)); dkey.mv_size =3D 0; if (flags =3D=3D MDB_CURRENT) { <-------- false flags =3D 0 if (!(mc->mc_flags & C_INITIALIZED)) return EINVAL; rc =3D MDB_SUCCESS; } else if (mc->mc_db->md_root =3D=3D P_INVALID) { <--------= false, mc->mc_db->md_root =3D 68 /* new database, cursor has nothing to poin= t to */ mc->mc_snum =3D 0; mc->mc_top =3D 0; mc->mc_flags &=3D ~C_INITIALIZED; rc =3D MDB_NO_ROOT; } else { <-------- executed int exact =3D 0; MDB_val d2; if (flags & MDB_APPEND) { <-------- false f= lags =3D 0 MDB_val k2; rc =3D mdb_cursor_last(mc, = &k2, &d2); if (rc =3D=3D 0) { rc =3D mc->= mc_dbx->md_cmp(key, &k2); if (rc > 0)= { = rc =3D MDB_NOTFOUND; = mc->mc_ki[mc->mc_top]++; } else { = /* new key is <=3D last key */ = rc =3D MDB_KEYEXIST; } } } else {<-------- executed rc =3D mdb_cursor_set(mc, k= ey, &d2, MDB_SET, &exact); <-------- returns -30798 which is MDB_NOTFOUND }<-------- executed if ((flags & MDB_NOOVERWRITE) && rc =3D=3D = 0) { <-------- false flags =3D 0 DPRINTF(("duplicate key [%s= ]", DKEY(key))); *data =3D d2; return MDB_KEYEXIST; } if (rc && rc !=3D MDB_NOTFOUND) <-------- f= alse rc =3D MDB_NOTFOUND return rc; }<-------- executed if (mc->mc_flags & C_DEL) mc->mc_flags ^=3D C_DEL; /* Cursor is positioned, check for room in the dirty list *= / if (!nospill) { <-------- true if (flags & MDB_MULTIPLE) { rdata =3D &xdata; xdata.mv_size =3D data->mv_= size * dcount; } else { rdata =3D data; <-------- e= xecuted } if ((rc2 =3D mdb_page_spill(mc, key, rdata)= )) <-------- returned 0 or would have returned return rc2; }<-------- executed if (rc =3D=3D MDB_NO_ROOT) { <-------- false MDB_page *np; /* new database, write a root leaf page */ DPUTS("allocating new root leaf page"); if ((rc2 =3D mdb_page_new(mc, P_LEAF, 1, &n= p))) { return rc2; } mdb_cursor_push(mc, np); mc->mc_db->md_root =3D np->mp_pgno; mc->mc_db->md_depth++; *mc->mc_dbflag |=3D DB_DIRTY; if ((mc->mc_db->md_flags & (MDB_DUPSORT|MDB= _DUPFIXED)) =3D=3D MDB_DUPFIXED) np->mp_flags |=3D P_LEAF2; mc->mc_flags |=3D C_INITIALIZED; } else { /* make sure all cursor pages are writable = */ rc2 =3D mdb_cursor_touch(mc); <-------- is = this called and returns 0 because it does not return if (rc2) return rc2; } <-------- executed insert_key =3D insert_data =3D rc; <-------- rc =3D -30798 = or MDB_NOTFOUND! if (insert_key) { <-------- true /* The key does not exist */ DPRINTF(("inserting key at index %i", mc->m= c_ki[mc->mc_top])); if ((mc->mc_db->md_flags & MDB_DUPSORT) && LEAFSIZE(key, data) > env->= me_nodemax) <-------- false since mc->mc_db->md_flags =3D 0 { /* Too big for a node, inse= rt in sub-DB. Set up an empty * "old sub-page" for prep_s= ubDB to expand to a full page. */ fp_flags =3D P_LEAF|P_DIRTY= ; fp =3D env->me_pbuf; fp->mp_pad =3D data->mv_siz= e; /* used if MDB_DUPFIXED */ fp->mp_lower =3D fp->mp_upp= er =3D (PAGEHDRSZ-PAGEBASE); olddata.mv_size =3D PAGEHDR= SZ; goto prep_subDB; } <-------- this was not executed, and the = very long external else contingency below should be skipped! } else { /* there's only a key anyway, so this is a = no-op */ if (IS_LEAF2(mc->mc_pg[mc->mc_top])) { char *ptr; unsigned int ksize =3D mc->= mc_db->md_pad; if (key->mv_size !=3D ksize= ) return MDB_= BAD_VALSIZE; ptr =3D LEAF2KEY(mc->mc_pg[= mc->mc_top], mc->mc_ki[mc->mc_top], ksize); memcpy(ptr, key->mv_data, k= size); fix_parent: /* if overwriting slot 0 of= leaf, need to * update branch key if ther= e is a parent page */ if (mc->mc_top && !mc->mc_k= i[mc->mc_top]) { unsigned sh= ort dtop =3D 1; mc->mc_top-= -; /* slot 0 i= s always an empty key, find real slot */ while (mc->= mc_top && !mc->mc_ki[mc->mc_top]) { = mc->mc_top--; = dtop++; } if (mc->mc_= ki[mc->mc_top]) = rc2 =3D mdb_update_key(mc, key); else = rc2 =3D MDB_SUCCESS; mc->mc_top = +=3D dtop; if (rc2) = return rc2; } return MDB_SUCCESS; } more: leaf =3D NODEPTR(mc->mc_pg[mc->mc_top], mc-= >mc_ki[mc->mc_top]); olddata.mv_size =3D NODEDSZ(leaf); olddata.mv_data =3D NODEDATA(leaf); /* DB has dups? */ if (F_ISSET(mc->mc_db->md_flags, MDB_DUPSOR= T)) { /* Prepare (sub-)page/sub-D= B to accept the new item, * if needed. fp: old sub-p= age or a header faking * it. mp: new (sub-)page. = offset: growth in page * size. xdata: node data w= ith new page or DB. */ unsigned i, off= set =3D 0; mp =3D fp =3D xdata.mv_data= =3D env->me_pbuf; mp->mp_pgno =3D mc->mc_pg[m= c->mc_top]->mp_pgno; /* Was a single item before= , must convert now */ if (!F_ISSET(leaf->mn_flags= , F_DUPDATA)) { MDB_cmp_fun= c *dcmp; /* Just ove= rwrite the current item */ if (flags = =3D=3D MDB_CURRENT) = goto current; dcmp =3D mc= ->mc_dbx->md_dcmp; #if UINT_MAX < SIZE_MAX if (dcmp = =3D=3D mdb_cmp_int && olddata.mv_size =3D=3D sizeof(size_t)) = dcmp =3D mdb_cmp_clong; #endif /* does dat= a match? */ if (!dcmp(d= ata, &olddata)) { = if (flags & (MDB_NODUPDATA|MDB_APPENDDUP)) = return MDB_KEYEXIST; = /* overwrite it */ = goto current; } /* Back up = original data item */ dkey.mv_siz= e =3D olddata.mv_size; dkey.mv_dat= a =3D memcpy(fp+1, olddata.mv_data, olddata.mv_size); /* Make sub= -page header for the dup items, with dummy body */ fp->mp_flag= s =3D P_LEAF|P_DIRTY|P_SUBP; fp->mp_lowe= r =3D (PAGEHDRSZ-PAGEBASE); xdata.mv_si= ze =3D PAGEHDRSZ + dkey.mv_size + data->mv_size; if (mc->mc_= db->md_flags & MDB_DUPFIXED) { = fp->mp_flags |=3D P_LEAF2; = fp->mp_pad =3D data->mv_size; = xdata.mv_size +=3D 2 * data->mv_size; /* leave space for 2 more *= / } else { = xdata.mv_size +=3D 2 * (sizeof(indx_t) + NODESIZE) + = (dkey.mv_size & 1) + (data->mv_size & 1); } fp->mp_uppe= r =3D xdata.mv_size - PAGEBASE; olddata.mv_= size =3D xdata.mv_size; /* pretend olddata is fp */ } else if (leaf->mn_flags &= F_SUBDATA) { /* Data is = on sub-DB, just store it */ flags |=3D = F_DUPDATA|F_SUBDATA; goto put_su= b; } else { /* Data is = on sub-page */ fp =3D oldd= ata.mv_data; switch (fla= gs) { default: = if (!(mc->mc_db->md_flags & MDB_DUPFIXED)) { = offset =3D EVEN(NODESIZE + sizeof(indx_t) + = data->mv_size); = break; = } = offset =3D fp->mp_pad; = if (SIZELEFT(fp) < offset) { = offset *=3D 4; /* space for 4 more */ = break; = } = /* FALLTHRU: Big enough MDB_DUPFIXED sub-page */ case MDB_CU= RRENT: = fp->mp_flags |=3D P_DIRTY; = COPY_PGNO(fp->mp_pgno, mp->mp_pgno); = mc->mc_xcursor->mx_cursor.mc_pg[0] =3D fp; = flags |=3D F_DUPDATA; = goto put_sub; } xdata.mv_si= ze =3D olddata.mv_size + offset; } fp_flags =3D fp->mp_flags; if (NODESIZE + NODEKSZ(leaf= ) + xdata.mv_size > env->me_nodemax) { = /* Too big for a sub-page, convert to sub-DB */ = fp_flags &=3D ~P_SUBP; prep_subDB: = if (mc->mc_db->md_flags & MDB_DUPFIXED) { = fp_flags |=3D P_LEAF2; = dummy.md_pad =3D fp->mp_pad; = dummy.md_flags =3D MDB_DUPFIXED; = if (mc->mc_db->md_flags & MDB_INTEGERDUP) = dummy.md_flags |=3D MDB_INTEGERKEY; = } else { = dummy.md_pad =3D 0; = dummy.md_flags =3D 0; = } = dummy.md_depth =3D 1; = dummy.md_branch_pages =3D 0; = dummy.md_leaf_pages =3D 1; = dummy.md_overflow_pages =3D 0; = dummy.md_entries =3D NUMKEYS(fp); = xdata.mv_size =3D sizeof(MDB_db); = xdata.mv_data =3D &dummy; = if ((rc =3D mdb_page_alloc(mc, 1, &mp))) = return rc; = offset =3D env->me_psize - olddata.mv_size; = flags |=3D F_DUPDATA|F_SUBDATA; = dummy.md_root =3D mp->mp_pgno; = sub_root =3D mp; } if (mp !=3D fp) { mp->mp_flag= s =3D fp_flags | P_DIRTY; mp->mp_pad = =3D fp->mp_pad; mp->mp_lowe= r =3D fp->mp_lower; mp->mp_uppe= r =3D fp->mp_upper + offset; if (fp_flag= s & P_LEAF2) { = memcpy(METADATA(mp), METADATA(fp), NUMKEYS(fp) * fp->mp_pad); } else { = memcpy((char *)mp + mp->mp_upper + PAGEBASE, (char *)fp + fp->mp_upper= + PAGEBASE, = olddata.mv_size - fp->mp_upper - PAGEBASE); = for (i=3D0; i<NUMKEYS(fp); i++) = mp->mp_ptrs[i] =3D fp->mp_ptrs[i] + offset; } } rdata =3D &xdata; flags |=3D F_DUPDATA; do_sub =3D 1; if (!insert_key) mdb_node_de= l(mc, 0); goto new_sub; } current: /* LMDB passes F_SUBDATA in 'flags' to writ= e a DB record */ if ((leaf->mn_flags ^ flags) & F_SUBDATA) return MDB_INCOMPATIBLE; /* overflow page overwrites need special ha= ndling */ if (F_ISSET(leaf->mn_flags, F_BIGDATA)) { <= -------- is this taken? MDB_page *omp; pgno_t pg; int level, ovpages, dpages = =3D OVPAGES(data->mv_size, env->me_psize); memcpy(&pg, olddata.mv_data= , sizeof(pg)); if ((rc2 =3D mdb_page_get(m= c->mc_txn, pg, &omp, &level)) !=3D 0) return rc2; ovpages =3D omp->mp_pages; /* Is the ov page large eno= ugh? */ if (ovpages >=3D dpages) { if (!(omp->mp_flags & P_D= IRTY) && (level ||= (env->me_flags & MDB_WRITEMAP))) { rc =3D mdb_= page_unspill(mc->mc_txn, omp, &omp); <-------- is this taken? if (rc) = return rc; level =3D 0= ; /* dirty in this txn or clean */ } /* Is it dirty? */ if (omp->mp_flags & P_DIR= TY) { /* yes, ove= rwrite it. Note in this case we don't * bother to= try shrinking the page if the new data * is smalle= r than the overflow threshold. */ if (level >= 1) { = /* It is writable only in a parent txn */ = size_t sz =3D (size_t) env->me_psize * ovpages, off; = MDB_page *np =3D mdb_page_malloc(mc->mc_txn, ovpages); = MDB_ID2 id2; = if (!np) = return ENOMEM; = id2.mid =3D pg; = id2.mptr =3D np; = /* Note - this page is already counted in parent's dirty_room */ = rc2 =3D mdb_mid2l_insert(mc->mc_txn->mt_u.dirty_list, &id2); = mdb_cassert(mc, rc2 =3D=3D 0); = if (!(flags & MDB_RESERVE)) { = /* Copy end of page, adjusting alignment so = * compiler may copy words instead of bytes. = */ = off =3D (PAGEHDRSZ + data->mv_size) & -sizeof(size_t); = memcpy((size_t *)((char *)np + off), = (size_t *)((char *)omp + off), sz - of= f); = sz =3D PAGEHDRSZ; = } = memcpy(np, omp, sz); /* Copy beginning of page */ = omp =3D np; } SETDSZ(leaf= , data->mv_size); if (F_ISSET= (flags, MDB_RESERVE)) = data->mv_data =3D METADATA(omp); else = memcpy(METADATA(omp), data->mv_data, data->mv_size); return MDB_= SUCCESS; } } if ((rc2 =3D mdb_ovpage_fre= e(mc, omp)) !=3D MDB_SUCCESS) return rc2; } else if (data->mv_size =3D=3D olddata.mv_= size) { /* same size, just replace = it. Note that we could * also reuse this node if t= he new data is smaller, * but instead we opt to shr= ink the node in that case. */ if (F_ISSET(flags, MDB_RESE= RVE)) data->mv_da= ta =3D olddata.mv_data; else if (!(mc->mc_flags & C= _SUB)) memcpy(oldd= ata.mv_data, data->mv_data, data->mv_size); else { memcpy(NODE= KEY(leaf), key->mv_data, key->mv_size); goto fix_pa= rent; } return MDB_SUCCESS; } mdb_node_del(mc, 0); } <-------- This entire else contingency was likely skipped= !! rdata =3D data; new_sub: nflags =3D flags & NODE_ADD_FLAGS; nsize =3D IS_LEAF2(mc->mc_pg[mc->mc_top]) ? key->mv_size : = mdb_leaf_size(env, key, rdata); <-------- SEGV ... This may be an identifiable bug in mdb_cursor_put() as It does not safely h= andle the case when the noted call to mdb_cursor_set() returns MDB_NOTFOUND= (-30798)! It explicitly fails to preemptively exit by returning an error in this case= . The value of -30798 is in turn assigned to insert_key, which is therefore n= on-null in the condition of the subsequent if statement. insert_key =3D insert_data =3D rc; <-------- insert_key =3D= -30798 or MDB_NOTFOUND! if (insert_key) { <-------- true /* The key does not exist */ DPRINTF(("inserting key at index %i", mc->m= c_ki[mc->mc_top])); if ((mc->mc_db->md_flags & MDB_DUPSORT) && LEAFSIZE(key, data) > env->= me_nodemax) <-------- false since mc->mc_db->md_flags =3D 0 { /* Too big for a node, inse= rt in sub-DB. Set up an empty * "old sub-page" for prep_s= ubDB to expand to a full page. */ fp_flags =3D P_LEAF|P_DIRTY= ; fp =3D env->me_pbuf; fp->mp_pad =3D data->mv_siz= e; /* used if MDB_DUPFIXED */ fp->mp_lower =3D fp->mp_upp= er =3D (PAGEHDRSZ-PAGEBASE); olddata.mv_size =3D PAGEHDR= SZ; goto prep_subDB; } <-------- this was not executed, and the = very long else contingency below is skipped! } else { Since mc->mc_db->md_flags =3D 0, the contingency in the embedded if stateme= nt fails. The segmentation fault occurs due to the call of IS_LEAF2 on the null point= er mc->mc_pg[0]. rdata =3D data; new_sub: nflags =3D flags & NODE_ADD_FLAGS; nsize =3D IS_LEAF2(mc->mc_pg[mc->mc_top]) ? key->mv_size : = mdb_leaf_size(env, key, rdata); <-------- SEGV Regards, Robins. --_000_MN2PR06MB55017A7261FA27380778753BAFEA0MN2PR06MB5501namp_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable <html xmlns:v=3D"urn:schemas-microsoft-com:vml" xmlns:o=3D"urn:schemas-micr= osoft-com:office:office" xmlns:w=3D"urn:schemas-microsoft-com:office:word" = xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" xmlns=3D"http:= //www.w3.org/TR/REC-html40"> <head> <meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii"= > <meta name=3D"Generator" content=3D"Microsoft Word 15 (filtered medium)"> <style></style> </head> <body lang=3D"EN-US" link=3D"#0563C1" vlink=3D"#954F72"> <div class=3D"WordSection1"> <repost since the previous message came as 64 = encoded, apologies><o:p></o:p> <o:p> </o:p> Hello Howard,<o:p></o:p> we are not able to reproduce the issue consistent= ly so test code is not available at this time, and I do believe this issue = will be present in the latest code as well since Myk also reported the same= ..  Attaching analysis from our side, hope it helps.<o:p></o:p> <o:p> </o:p> The call stack what we have seen is,<o:p></o:p></= p> <o:p> </o:p>     #0 0x0011b62e in mdb_cursor_pu= t (mc=3D0xffe52d18, key=3D0xffe52e74, data=3D0xffe52e6c, flags=3D0) at mdb.= c:6688<o:p></o:p>     #1 0x0011c8ec in mdb_put (txn= =3D0xdc13f008, dbi=3D2, key=3D0xffe52e74, data=3D0xffe52e6c, flags=3D0) at = mdb.c:8771<o:p></o:p>     #2 0x00b792b8 in LMDB::LMDBCon= text::createSession (this=3D0x80ca418, sid=3D0x8297be4 "sidebf3708f0fd= 9fcb8e062529da47adf51402dfc4600000000+") at lmdbint.cc:460<o:p></o= :p>     #3 0x08053564 in updateLMDB (r= equest=3D..., forward=3D@0xffe5315f) at sessionserver.cc:1075<o:p></o:p></p= >     #4 processRequestWithoutRespon= se (request=3D..., forward=3D@0xffe5315f) at sessionserver.cc:1160<o:p></o:= p>     #5 0x08056a80 in processReques= t (this=3D0x80b2ac0, fd=3D33) at sessionserver.cc:1189<o:p></o:p>     #6 readFromSock (this=3D0x80b2= ac0, fd=3D33) at sessionserver.cc:1342<o:p></o:p>     #7 SockHandler::ioReady (this= =3D0x80b2ac0, fd=3D33) at sessionserver.cc:1455<o:p></o:p>     #8 0x00ba2592 in runCoreDispat= cher (default_t=3D<value optimized out>, flags=3D-1) at fds.cc:889<o:= p></o:p>     #9 0x00ba2ff7 in DSEvntFds::ru= nDispatcher () at fds.cc:945<o:p></o:p>     #10 0x080559c6 in main () at s= essionserver.cc:1739<o:p></o:p> <o:p> </o:p> Here it is helpful to examine frames #0 and #1 in= detail in relation to the relevant source code /libraries/liblmdb/mdb.c.<o= :p></o:p> <o:p> </o:p> Frame #1<o:p></o:p> <o:p> </o:p> #1  0x0011c8ec in mdb_put (txn=3D0xdc13f008,= dbi=3D2, key=3D0xffe52e74, data=3D0xffe52e6c, flags=3D0) at mdb.c:8771<o:p= ></o:p>         mc =3D= {mc_next =3D 0x0, mc_backup =3D 0x0, mc_xcursor =3D 0x0, mc_txn =3D 0xdc13= f008, mc_dbi =3D 2, mc_db =3D 0xdc13f088, mc_dbx =3D 0xe1215038, mc_dbflag = =3D 0xdc1cf09a "\033", '\032' <repeats 51 times>, mc_snum = =3D 0, mc_top =3D 0, mc_flags =3D 0, mc_pg =3D {0x0, 0x80b2ac0, 0xffe52d68, 0x28afce, 0xdc13f008, 0x80ce9b0, 0x= ffe52dc8, 0x4c5ff4, 0x2b, 0x80b2ac0, 0xffe52d98, 0x49209a, 0x2b, 0x4c5ff4, = 0x2121cb, 0x21576c, 0x8297d34, 0x3497af, 0x21576c, 0x21316a, 0x8297d34, 0x8= 0ac7b0, 0x1e, 0x46eed6, 0x2b, 0x0, 0x80ce9b0, 0x4c5ff4, 0x80ac7b0, 0x8297d34, 0xffe52df8, 0x46fb96}, mc_ki = =3D {0, 2089, 51120, 2058, 30, 0, 16796, 17, 11760, 65509, 3, 0, 32040, 208= 9, 30, 0, 0, 0, 0, 0, 3, 0, 24564, 76, 51120, 2058, 10944, 2059, 11800, 655= 09, 64758, 70}} <--------<o:p></o:p>         &= nbsp;       <o:p></o:p>         mx =3D= {mx_cursor =3D {mc_next =3D 0x38, mc_backup =3D 0x3, mc_xcursor =3D 0x0, m= c_txn =3D 0x0, mc_dbi =3D 0, mc_db =3D 0x28, mc_dbx =3D 0x0, mc_dbflag =3D = 0x0, mc_snum =3D 3, mc_top =3D 0, mc_flags =3D 17, mc_pg =3D {0x3a93d8, 0x1= 0, 0x0, 0x18, 0x3a93a0, 0x0, 0x3a93d0, 0x289abe, 0x3a93a0, 0xffe52d3f, 0xffe52c68, 0x28afce, 0xffe= 52dbc, 0xffe52db4, 0x3, 0x4c5ff4, 0x11, 0x36c99b, 0x6e, 0x77, 0x7c, 0x5b, 0= x38, 0x289abe, 0x0, 0x0, 0x0, 0x28, 0x0, 0x0, 0x3, 0x10}, mc_ki =3D {37848,= 58, 51611, 54, 110, 0, 119, 0, 124, 0, 91, 0, 58, 0, 39614, 40, 0, 0, 0, 0, 0, 0, 160, 0, 0, 0, 2, 0, 18, 0, 1= 36, 0}}, mx_db =3D {md_pad =3D 3838936, md_flags =3D 51611, md_depth =3D 54= , md_branch_pages =3D 110, md_leaf_pages =3D 119, md_overflow_pages =3D 124= , md_entries =3D 91, md_root =3D 56}, mx_dbx =3D {md_name =3D {mv_size =3D 6, mv_data =3D 0x0}, md_cmp =3D 0, md_dcmp =3D 0, md_rel = =3D 0x40, md_relctx =3D 0x0}, mx_dbflag =3D 0 '\000'}<o:p></o:p>         rc =3D= 0 <o:p></o:p> <o:p> </o:p> And source code for mdb_put() is from mdb.c<o:p><= /o:p> <o:p> </o:p> int<o:p></o:p> mdb_put(MDB_txn *txn, MDB_dbi dbi,<o:p></o:p>     MDB_val *key, MDB_val *data, u= nsigned int flags) {<o:p></o:p>         &= nbsp;       MDB_cursor mc;<o:p></o:p>         &= nbsp;       MDB_xcursor mx;<o:p></o:p>         &= nbsp;       int rc;<o:p></o:p> <o:p> </o:p>         &= nbsp;       if (!key || !data || !TXN_DBI_EXI= ST(txn, dbi, DB_USRVALID))<o:p></o:p>         &= nbsp;           &nbs= p;           return EINVA= L;<o:p></o:p> <o:p> </o:p>         &= nbsp;       if (flags & ~(MDB_NOOVERWRITE= |MDB_NODUPDATA|MDB_RESERVE|MDB_APPEND|MDB_APPENDDUP))<o:p></o:p>         &= nbsp;           &nbs= p;           return EINVA= L;<o:p></o:p> <o:p> </o:p>         &= nbsp;       if (txn->mt_flags & (MDB_T= XN_RDONLY|MDB_TXN_BLOCKED))<o:p></o:p>         &= nbsp;           &nbs= p;           return (txn-= >mt_flags & MDB_TXN_RDONLY) ? EACCES : MDB_BAD_TXN;<o:p></o:p> <o:p> </o:p>         &= nbsp;       mdb_cursor_init(&mc, txn, dbi= , &mx); <-------- see source code below<o:p></o:p>         &= nbsp;       mc.mc_next =3D txn->mt_cursors= [dbi];<o:p></o:p>         &= nbsp;       txn->mt_cursors[dbi] =3D &= mc;<o:p></o:p>         &= nbsp;       rc =3D mdb_cursor_put(&mc, ke= y, data, flags); <-------- see Frame #0<o:p></o:p>         &= nbsp;       txn->mt_cursors[dbi] =3D mc.mc= _next;<o:p></o:p>         &= nbsp;       return rc;<o:p></o:p> }<o:p></o:p> <o:p> </o:p> <o:p> </o:p> Source Code For mdb_cursor_init() From mdb.c<o:p>= </o:p> /** Initialize a cursor for a given transaction a= nd database. */ static void mdb_cursor_init(MDB_cursor *mc, MDB_txn *txn, M= DB_dbi dbi, MDB_xcursor *mx) {<o:p></o:p>         &= nbsp;       mc->mc_next =3D NULL;<o:p></o:= p>         &= nbsp;       mc->mc_backup =3D NULL;<o:p></= o:p>         &= nbsp;       mc->mc_dbi =3D dbi;<o:p></o:p>=         &= nbsp;       mc->mc_txn =3D txn;<o:p></o:p>=         &= nbsp;       mc->mc_db =3D &txn->mt_= dbs[dbi];<o:p></o:p>         &= nbsp;       mc->mc_dbx =3D &txn->mt= _dbxs[dbi];<o:p></o:p>         &= nbsp;       mc->mc_dbflag =3D &txn-&gt= ;mt_dbflags[dbi];<o:p></o:p>         &= nbsp;       mc->mc_snum =3D 0; <-------= - <o:p></o:p>         &= nbsp;       mc->mc_top =3D 0; <--------=   <o:p></o:p>         &= nbsp;       mc->mc_pg[0] =3D 0; <------= -- <o:p></o:p>         &= nbsp;       mc->mc_ki[0] =3D 0;<o:p></o:p>=         &= nbsp;       mc->mc_flags =3D 0;<o:p></o:p>=         &= nbsp;       if (txn->mt_dbs[dbi].md_flags = & MDB_DUPSORT) {<o:p></o:p>         &= nbsp;           &nbs= p;           mdb_tassert(= txn, mx !=3D NULL);<o:p></o:p>         &= nbsp;           &nbs= p;           mc->mc_xc= ursor =3D mx;<o:p></o:p>         &= nbsp;           &nbs= p;           mdb_xcursor_= init0(mc);<o:p></o:p>         &= nbsp;       } else {<o:p></o:p>         &= nbsp;           &nbs= p;           mc->mc_xc= ursor =3D NULL;<o:p></o:p>         &= nbsp;       }<o:p></o:p>         &= nbsp;       if (*mc->mc_dbflag & DB_ST= ALE) { <-------- false *mc->mc_dbflag =3D 27, DB_STALE =3D 0x02 =             &nb= sp;    /**< Named-DB record is older than txnID */<o:p></= o:p>         &= nbsp;           &nbs= p;           mdb_page_sea= rch(mc, NULL, MDB_PS_ROOTONLY); <o:p></o:p>         &= nbsp;       }<o:p></o:p> }<o:p></o:p> <o:p> </o:p> From this we know that when mdb_cursor_put() is c= alled, the following values remain assigned:<o:p></o:p> <o:p> </o:p>         &= nbsp;     mc->mc_snum =3D 0; <--------<o:p></o:p>=         &= nbsp;       mc->mc_top =3D 0; <--------=   <o:p></o:p>         &= nbsp;       mc->mc_pg[0] =3D 0; <------= -- <o:p></o:p> <o:p> </o:p> and its noted these two values still have their i= nitial values at the time of the segmentation fault.<o:p></o:p> <o:p> </o:p> (gdb) fr 0<o:p></o:p> #0 0x0011b62e in mdb_cursor_put (mc=3D0xffe52d18,= key=3D0xffe52e74, data=3D0xffe52e6c, flags=3D0) at mdb.c:6688<o:p></o:p></= p> 6688 in mdb.c<o:p></o:p> (gdb) p mc->mc_top<o:p></o:p> $1 =3D 0<o:p></o:p> (gdb) p mc->mc_pg[mc->mc_top]<o:p></o:p></p= > $19 =3D (MDB_page *) 0x0<o:p></o:p> <o:p> </o:p> Let us consider how the path taken through mdb_cu= rsor_put() could account for these null variable values which result in a s= egmentation fault.<o:p></o:p> <o:p> </o:p> Frame #0<o:p></o:p> <o:p> </o:p> #0  0x0011b62e in mdb_cursor_put (mc=3D0xffe= 52d18, key=3D0xffe52e74, data=3D0xffe52e6c, flags=3D0) at mdb.c:6688 <--= ------ flags =3D 0<o:p></o:p>         env = =3D 0x80ce9b0<o:p></o:p>         leaf = =3D <value optimized out><o:p></o:p>         fp =3D= <value optimized out><o:p></o:p>         mp =3D= 0x38<o:p></o:p>         sub_ro= ot =3D 0x0 <o:p></o:p>         f= p_flags =3D <value optimized out><o:p></o:p>         xdata = =3D {mv_size =3D 3838880, mv_data =3D 0xdc1cf09a}<o:p></o:p>         rdata = =3D 0xffe52e6c<o:p></o:p>         dkey = =3D {mv_size =3D 0, mv_data =3D 0x3a93cc}<o:p></o:p>         olddat= a =3D {mv_size =3D 22, mv_data =3D 0x36c820}<o:p></o:p>         dummy = =3D {md_pad =3D 22, md_flags =3D 11128, md_depth =3D 65509, md_branch_pages= =3D 16, md_leaf_pages =3D 1507329, md_overflow_pages =3D 0, md_entries =3D= 3838928, md_root =3D 3833844}<o:p></o:p>         do_sub= =3D 0<o:p></o:p>         insert= _key =3D -30798 <-------- #define MDB_NOTFOUND    &n= bsp;     (-30798)<o:p></o:p>         insert= _data =3D -30798 <-------- #define MDB_NOTFOUND    &= nbsp;   (-30798)<o:p></o:p>         mcount= =3D 0<o:p></o:p>         dcount= =3D 0<o:p></o:p>         nospil= l =3D 0 <--------<o:p></o:p>         nsize = =3D <value optimized out><o:p></o:p>         rc =3D= 0 <o:p></o:p>         r= c2 =3D <value optimized out><o:p></o:p>         nflags= =3D 0 <--------<o:p></o:p>         __func= __ =3D "mdb_cursor_put"       =    <o:p></o:p> <o:p> </o:p> We know that mc->mc_top and mc->mc_pg[0] we= re initialized to 0 in mdb_cursor_init(). As my annotations of the source c= ode below suggest, careful analysis reveals that the paths actually taken t= hrough mdb_cursor_put() do not modify these assignments.<o:p></o:p> <o:p> </o:p> Excerpted Source Code For mdb_cursor_put() From m= db.c<o:p></o:p> <o:p> </o:p> int<o:p></o:p> mdb_cursor_put(MDB_cursor *mc, MDB_val *key, MDB_= val *data,<o:p></o:p>     unsigned int flags)<o:p></o:p>= {<o:p></o:p>         &= nbsp;       MDB_env    &n= bsp;           &nbsp= ;           *env;<o:p></o= :p>         &= nbsp;       MDB_node    &= nbsp;    *leaf =3D NULL;<o:p></o:p>         &= nbsp;       MDB_page    &= nbsp;    *fp, *mp, *sub_root =3D NULL;<o:p></o:p>         &= nbsp;       uint16_t    &= nbsp;          fp_flags;<o:p><= /o:p>         &= nbsp;       MDB_val    &n= bsp;           &nbsp= ;             x= data, *rdata, dkey, olddata;<o:p></o:p>         &= nbsp;       MDB_db dummy;<o:p></o:p>         &= nbsp;       int do_sub =3D 0, insert_key, ins= ert_data;<o:p></o:p>         &= nbsp;       unsigned int mcount =3D 0, dcount= =3D 0, nospill;<o:p></o:p>         &= nbsp;       size_t nsize;<o:p></o:p>         &= nbsp;       int rc, rc2;<o:p></o:p>         &= nbsp;       unsigned int nflags;<o:p></o:p></= p>         &= nbsp;       DKBUF;<o:p></o:p> <o:p> </o:p>         &= nbsp;       if (mc =3D=3D NULL || key =3D=3D = NULL)<o:p></o:p>         &= nbsp;           &nbs= p;           return EINVA= L;<o:p></o:p> <o:p> </o:p>         &= nbsp;       env =3D mc->mc_txn->mt_env;= <o:p></o:p> <o:p> </o:p>         &= nbsp;       /* Check this first so counter wi= ll always be zero on any<o:p></o:p>         &= nbsp;       * early failures.<o:p></o:p>         &= nbsp;       */<o:p></o:p>         &= nbsp;       if (flags & MDB_MULTIPLE) {<o= :p></o:p>         &= nbsp;           &nbs= p;           dcount =3D d= ata[1].mv_size;<o:p></o:p>         &= nbsp;           &nbs= p;           data[1].mv_s= ize =3D 0;<o:p></o:p>         &= nbsp;           &nbs= p;           if (!F_ISSET= (mc->mc_db->md_flags, MDB_DUPFIXED))<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  return MDB_INCOMPATIBLE;<o:p></o:p>         &= nbsp;       }<o:p></o:p> <o:p> </o:p>         &= nbsp;       nospill =3D flags & MDB_NOSPI= LL;<o:p></o:p>         &= nbsp;       flags &=3D ~MDB_NOSPILL;<o:p>= </o:p> <o:p> </o:p>         &= nbsp;       if (mc->mc_txn->mt_flags &a= mp; (MDB_TXN_RDONLY|MDB_TXN_BLOCKED))<o:p></o:p>         &= nbsp;           &nbs= p;           return (mc-&= gt;mc_txn->mt_flags & MDB_TXN_RDONLY) ? EACCES : MDB_BAD_TXN;<o:p></= o:p> <o:p> </o:p>         &= nbsp;       if (key->mv_size-1 >=3D ENV= _MAXKEY(env))<o:p></o:p>         &= nbsp;           &nbs= p;           return MDB_B= AD_VALSIZE;<o:p></o:p> <o:p> </o:p> #if SIZE_MAX > MAXDATASIZE<o:p></o:p>         &= nbsp;       if (data->mv_size > ((mc-&g= t;mc_db->md_flags & MDB_DUPSORT) ? ENV_MAXKEY(env) : MAXDATASIZE))<o= :p></o:p>         &= nbsp;           &nbs= p;           return MDB_B= AD_VALSIZE;<o:p></o:p> #else<o:p></o:p>         &= nbsp;       if ((mc->mc_db->md_flags &a= mp; MDB_DUPSORT) && data->mv_size > ENV_MAXKEY(env))<o:p></o:= p>         &= nbsp;           &nbs= p;           return MDB_B= AD_VALSIZE;<o:p></o:p> #endif<o:p></o:p> <o:p> </o:p>         &= nbsp;       DPRINTF(("=3D=3D> put db = %d key [%s], size %"Z"u, data size %"Z"u",<o:p></o= :p>         &= nbsp;           &nbs= p;           DDBI(mc), DK= EY(key), key ? key->mv_size : 0, data->mv_size));<o:p></o:p> <o:p> </o:p>         &= nbsp;       dkey.mv_size =3D 0;<o:p></o:p></p= > <o:p> </o:p>         &= nbsp;       if (flags =3D=3D MDB_CURRENT) { &= lt;-------- false flags =3D 0<o:p></o:p>         &= nbsp;           &nbs= p;           if (!(mc-&gt= ;mc_flags & C_INITIALIZED))<o:p></o:p>         &= nbsp;           &nbs= p;             =             &nb= sp; return EINVAL;<o:p></o:p>         &= nbsp;           &nbs= p;           rc =3D MDB_S= UCCESS;<o:p></o:p>         &= nbsp;       } else if (mc->mc_db->md_ro= ot =3D=3D P_INVALID) { <-------- false, mc->mc_db->md_root =3D 68<= o:p></o:p>         &= nbsp;           &nbs= p;           /* new datab= ase, cursor has nothing to point to */<o:p></o:p>         &= nbsp;           &nbs= p;           mc->mc_sn= um =3D 0;<o:p></o:p>         &= nbsp;           &nbs= p;           mc->mc_to= p =3D 0;<o:p></o:p>         &= nbsp;           &nbs= p;           mc->mc_fl= ags &=3D ~C_INITIALIZED;<o:p></o:p>         &= nbsp;           &nbs= p;           rc =3D MDB_N= O_ROOT;  <o:p></o:p>         &= nbsp;       } else { <-------- executed<o:= p></o:p>         &= nbsp;           &nbs= p;           int exact = =3D 0;<o:p></o:p>         &= nbsp;           &nbs= p;           MDB_val d2;<= o:p></o:p>         &= nbsp;           &nbs= p;           if (flags &a= mp; MDB_APPEND) { <-------- false flags =3D 0<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  MDB_val k2;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  rc =3D mdb_cursor_last(mc, &k2, &d2);<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  if (rc =3D=3D 0) {<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     rc =3D mc->mc_dbx->md_cmp(key, &k2)= ;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     if (rc > 0) {<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         rc =3D MDB_NOTFOUND;<o:p= ></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         mc->mc_ki[mc->mc_t= op]++;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     } else {<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         /* new key is <=3D la= st key */<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         rc =3D MDB_KEYEXIST;<o:p= ></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     }<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  }<o:p></o:p>         &= nbsp;           &nbs= p;           } else {<= -------- executed<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  rc =3D mdb_cursor_set(mc, key, &d2, MDB_SET, &exact); <= -------- returns -30798 which is MDB_NOTFOUND<o:p></o:p>         &= nbsp;           &nbs= p;           }<-------= - executed<o:p></o:p>         &= nbsp;           &nbs= p;           if ((flags &= amp; MDB_NOOVERWRITE) && rc =3D=3D 0) { <-------- false flags = =3D 0<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  DPRINTF(("duplicate key [%s]", DKEY(key)));<o:p></o:p></= p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  *data =3D d2;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  return MDB_KEYEXIST;<o:p></o:p>         &= nbsp;           &nbs= p;           }<o:p></o:p>=         &= nbsp;           &nbs= p;           if (rc &= & rc !=3D MDB_NOTFOUND) <-------- false rc =3D MDB_NOTFOUND<o:p></o:= p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  return rc;<o:p></o:p>         &= nbsp;       }<-------- executed<o:p></o:p>= <o:p> </o:p>         &= nbsp;       if (mc->mc_flags & C_DEL)<= o:p></o:p>         &= nbsp;           &nbs= p;           mc->mc_fl= ags ^=3D C_DEL;<o:p></o:p> <o:p> </o:p>         &= nbsp;       /* Cursor is positioned, check fo= r room in the dirty list */<o:p></o:p>         &= nbsp;       if (!nospill) { <-------- true= <o:p></o:p>         &= nbsp;           &nbs= p;           if (flags &a= mp; MDB_MULTIPLE) {<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  rdata =3D &xdata;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  xdata.mv_size =3D data->mv_size * dcount;<o:p></o:p>         &= nbsp;           &nbs= p;           } else {<o:p= ></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  rdata =3D data; <-------- executed<o:p></o:p>         &= nbsp;           &nbs= p;           }<o:p></o:p>=         &= nbsp;           &nbs= p;           if ((rc2 =3D= mdb_page_spill(mc, key, rdata))) <-------- returned 0 or would have ret= urned<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  return rc2;<o:p></o:p>         &= nbsp;       }<-------- executed<o:p></o:p>= <o:p> </o:p>         &= nbsp;       if (rc =3D=3D MDB_NO_ROOT) { <= -------- false <o:p> </o:p>         &= nbsp;           &nbs= p;           MDB_page *np= ;<o:p></o:p>         &= nbsp;           &nbs= p;           /* new datab= ase, write a root leaf page */<o:p></o:p>         &= nbsp;           &nbs= p;           DPUTS("= allocating new root leaf page");<o:p></o:p>         &= nbsp;           &nbs= p;           if ((rc2 =3D= mdb_page_new(mc, P_LEAF, 1, &np))) {<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  return rc2;<o:p></o:p>         &= nbsp;           &nbs= p;           }<o:p></o:p>=         &= nbsp;           &nbs= p;           mdb_cursor_p= ush(mc, np);<o:p></o:p>         &= nbsp;           &nbs= p;           mc->mc_db= ->md_root =3D np->mp_pgno;<o:p></o:p>         &= nbsp;           &nbs= p;           mc->mc_db= ->md_depth++;<o:p></o:p>         &= nbsp;           &nbs= p;           *mc->mc_d= bflag |=3D DB_DIRTY;<o:p></o:p>         &= nbsp;           &nbs= p;           if ((mc->= mc_db->md_flags & (MDB_DUPSORT|MDB_DUPFIXED))<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  =3D=3D MDB_DUPFIXED)<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  np->mp_flags |=3D P_LEAF2;<o:p></o:p>         &= nbsp;           &nbs= p;           mc->mc_fl= ags |=3D C_INITIALIZED;<o:p></o:p>         &= nbsp;       } else {<o:p></o:p>         &= nbsp;           &nbs= p;           /* make sure= all cursor pages are writable */<o:p></o:p>         &= nbsp;           &nbs= p;           rc2 =3D mdb_= cursor_touch(mc); <-------- is this called and returns 0 because it does= not return<o:p></o:p>         &= nbsp;           &nbs= p;           if (rc2)<o:p= ></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  return rc2;<o:p></o:p>         &= nbsp;       } <-------- executed<o:p></o:p= > <o:p> </o:p>         &= nbsp;       insert_key =3D insert_data =3D rc= ; <-------- rc =3D -30798 or MDB_NOTFOUND!<o:p></o:p>         &= nbsp;       if (insert_key) { <-------- tr= ue<o:p></o:p>         &= nbsp;           &nbs= p;           /* The key d= oes not exist */<o:p></o:p>         &= nbsp;           &nbs= p;           DPRINTF((&qu= ot;inserting key at index %i", mc->mc_ki[mc->mc_top]));<o:p></o:= p>         &= nbsp;           &nbs= p;           if ((mc->= mc_db->md_flags & MDB_DUPSORT) &&<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  LEAFSIZE(key, data) > env->me_nodemax) <-------- false si= nce mc->mc_db->md_flags =3D 0<o:p></o:p>         &= nbsp;           &nbs= p;           {<o:p></o:p>=         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  /* Too big for a node, insert in sub-DB.  Set up an empty<o:p= ></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  * "old sub-page" for prep_subDB to expand to a full page= .<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  */<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  fp_flags =3D P_LEAF|P_DIRTY;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  fp =3D env->me_pbuf;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  fp->mp_pad =3D data->mv_size; /* used if MDB_DUPFIXED */<o:p= ></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  fp->mp_lower =3D fp->mp_upper =3D (PAGEHDRSZ-PAGEBASE);<o:p>= </o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  olddata.mv_size =3D PAGEHDRSZ;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  goto prep_subDB;<o:p></o:p>         &= nbsp;           &nbs= p;           } <------= -- this was not executed, and the very long external else contingency below= should be skipped!<o:p></o:p>         &= nbsp;       } else {<o:p></o:p>         &= nbsp;           &nbs= p;           /* there's o= nly a key anyway, so this is a no-op */<o:p></o:p>         &= nbsp;           &nbs= p;           if (IS_LEAF2= (mc->mc_pg[mc->mc_top])) {<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  char *ptr;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  unsigned int ksize =3D mc->mc_db->md_pad;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  if (key->mv_size !=3D ksize)<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     return MDB_BAD_VALSIZE;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  ptr =3D LEAF2KEY(mc->mc_pg[mc->mc_top], mc->mc_ki[mc->= mc_top], ksize);<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  memcpy(ptr, key->mv_data, ksize);<o:p></o:p> fix_parent:<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  /* if overwriting slot 0 of leaf, need to<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  * update branch key if there is a parent page<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  */<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  if (mc->mc_top && !mc->mc_ki[mc->mc_top]) {<o:p><= /o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     unsigned short dtop =3D 1;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     mc->mc_top--;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     /* slot 0 is always an empty key, find real s= lot */<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     while (mc->mc_top && !mc->mc_ki= [mc->mc_top]) {<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         mc->mc_top--;<o:p></o= :p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         dtop++;<o:p></o:= p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     }<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     if (mc->mc_ki[mc->mc_top])<o:p></o:p></= p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         rc2 =3D mdb_update_key(m= c, key);<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     else<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         rc2 =3D MDB_SUCCESS;<o:p= ></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     mc->mc_top +=3D dtop;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     if (rc2)<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         return rc2;<o:p></o:p></= p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  }<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  return MDB_SUCCESS;<o:p></o:p>         &= nbsp;           &nbs= p;           }<o:p></o:p>= <o:p> </o:p> more:<o:p></o:p>         &= nbsp;           &nbs= p;           leaf =3D NOD= EPTR(mc->mc_pg[mc->mc_top], mc->mc_ki[mc->mc_top]);<o:p></o:p><= /p>         &= nbsp;           &nbs= p;           olddata.mv_s= ize =3D NODEDSZ(leaf);<o:p></o:p>         &= nbsp;           &nbs= p;           olddata.mv_d= ata =3D NODEDATA(leaf);<o:p></o:p> <o:p> </o:p>         &= nbsp;           &nbs= p;           /* DB has du= ps? */<o:p></o:p>         &= nbsp;           &nbs= p;           if (F_ISSET(= mc->mc_db->md_flags, MDB_DUPSORT)) { <o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  /* Prepare (sub-)page/sub-DB to accept the new item,<o:p></o:p></p= >         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  * if needed.  fp: old sub-page or a header faking<o:p></o:p><= /p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  * it.  mp: new (sub-)page.  offset: growth in page<o:p><= /o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  * size.  xdata: node data with new page or DB.<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  */<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  unsigned         &nbs= p;   i, offset =3D 0;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  mp =3D fp =3D xdata.mv_data =3D env->me_pbuf;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  mp->mp_pgno =3D mc->mc_pg[mc->mc_top]->mp_pgno;<o:p></= o:p> <o:p> </o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  /* Was a single item before, must convert now */<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  if (!F_ISSET(leaf->mn_flags, F_DUPDATA)) {<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     MDB_cmp_func *dcmp;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     /* Just overwrite the current item */<o:p></o= :p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     if (flags =3D=3D MDB_CURRENT)<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         goto current;<o:p></o:p>=         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     dcmp =3D mc->mc_dbx->md_dcmp;<o:p></o:p= > #if UINT_MAX < SIZE_MAX<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     if (dcmp =3D=3D mdb_cmp_int && olddat= a.mv_size =3D=3D sizeof(size_t))<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         dcmp =3D mdb_cmp_clong;<= o:p></o:p> #endif<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     /* does data match? */<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     if (!dcmp(data, &olddata)) {<o:p></o:p></= p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         if (flags & (MDB_NOD= UPDATA|MDB_APPENDDUP))<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;            ret= urn MDB_KEYEXIST;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         /* overwrite it */<o:p><= /o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         goto current;<o:p></o:p>=         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     }<o:p></o:p> <o:p> </o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     /* Back up original data item */<o:p></o:p></= p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     dkey.mv_size =3D olddata.mv_size;<o:p></o:p><= /p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     dkey.mv_data =3D memcpy(fp+1, olddata.mv_= data, olddata.mv_size);<o:p></o:p> <o:p> </o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     /* Make sub-page header for the dup items, wi= th dummy body */<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     fp->mp_flags =3D P_LEAF|P_DIRTY|P_SUBP;<o:= p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     fp->mp_lower =3D (PAGEHDRSZ-PAGEBASE);<o:p= ></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     xdata.mv_size =3D PAGEHDRSZ + dkey.mv_siz= e + data->mv_size;<o:p></o:p>         &= nbsp;           &nbs= p;             =             &nb= sp;            =      if (mc->mc_db->md_flags & MDB_DUPFIXED) = {<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         fp->mp_flags |=3D P_L= EAF2;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         fp->mp_pad =3D data-&= gt;mv_size;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         xdata.mv_size +=3D 2= * data->mv_size;      /* leave space for 2 mor= e */<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     } else {<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         xdata.mv_size +=3D 2= * (sizeof(indx_t) + NODESIZE) +<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;            (dk= ey.mv_size & 1) + (data->mv_size & 1);<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     }<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     fp->mp_upper =3D xdata.mv_size - PAGEBASE;= <o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     olddata.mv_size =3D xdata.mv_size; /* pretend= olddata is fp */<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  } else if (leaf->mn_flags & F_SUBDATA) {<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     /* Data is on sub-DB, just store it */<o:p></= o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     flags |=3D F_DUPDATA|F_SUBDATA;<o:p></o:p></p= >         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     goto put_sub;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  } else {<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     /* Data is on sub-page */<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     fp =3D olddata.mv_data;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     switch (flags) {<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     default:<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         if (!(mc->mc_db->m= d_flags & MDB_DUPFIXED)) {<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;            off= set =3D EVEN(NODESIZE + sizeof(indx_t) +<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;   data->mv_size);<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;            bre= ak;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         }<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         offset =3D fp->mp_pad= ;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         if (SIZELEFT(fp) < of= fset) {<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;            off= set *=3D 4; /* space for 4 more */<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;            bre= ak;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         }<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         /* FALLTHRU: Big enough = MDB_DUPFIXED sub-page */<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     case MDB_CURRENT:<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         fp->mp_flags |=3D P_D= IRTY;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         COPY_PGNO(fp->mp_pgno= , mp->mp_pgno);<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         mc->mc_xcursor->mx= _cursor.mc_pg[0] =3D fp;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         flags |=3D F_DUPDATA;<o:= p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         goto put_sub;<o:p></o:p>=         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     }<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     xdata.mv_size =3D olddata.mv_size + offse= t;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  }<o:p></o:p> <o:p> </o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  fp_flags =3D fp->mp_flags;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  if (NODESIZE + NODEKSZ(leaf) + xdata.mv_size > env->= me_nodemax) {<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         /* Too big for a sub-pag= e, convert to sub-DB */<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         fp_flags &=3D ~P_SUB= P;<o:p></o:p> prep_subDB:<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         if (mc->mc_db->md_= flags & MDB_DUPFIXED) {<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;            fp_= flags |=3D P_LEAF2;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;            dum= my.md_pad =3D fp->mp_pad;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;            dum= my.md_flags =3D MDB_DUPFIXED;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;            if = (mc->mc_db->md_flags & MDB_INTEGERDUP)<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;   dummy.md_flags |=3D MDB_INTEGERKEY;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         } else {<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;            dum= my.md_pad =3D 0;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;            dum= my.md_flags =3D 0;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         }<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         dummy.md_depth =3D 1;<o:= p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         dummy.md_branch_pages = =3D 0;<o:p></o:p>         &= nbsp;           &nbs= p;             =             &nb= sp;            =             &nb= sp;        dummy.md_leaf_pages =3D 1;<o:= p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         dummy.md_overflow_pages = =3D 0;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         dummy.md_entries =3D NUM= KEYS(fp);<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         xdata.mv_size =3D sizeof= (MDB_db);<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         xdata.mv_data =3D &d= ummy;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         if ((rc =3D mdb_page_all= oc(mc, 1, &mp)))<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;            ret= urn rc;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         offset =3D env->me_ps= ize - olddata.mv_size;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         flags |=3D F_DUPDATA|F_S= UBDATA;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         dummy.md_root =3D mp-&gt= ;mp_pgno;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         sub_root =3D mp;<o:p></o= :p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  }<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  if (mp !=3D fp) {<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     mp->mp_flags =3D fp_flags | P_DIRTY;<o:p><= /o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     mp->mp_pad   =3D fp->mp_pad;<= o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     mp->mp_lower =3D fp->mp_lower;<o:p></o:= p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     mp->mp_upper =3D fp->mp_upper + off= set;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     if (fp_flags & P_LEAF2) {<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         memcpy(METADATA(mp), MET= ADATA(fp), NUMKEYS(fp) * fp->mp_pad);<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     } else {<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         memcpy((char *)mp + = mp->mp_upper + PAGEBASE, (char *)fp + fp->mp_upper + PAGE= BASE,<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;            old= data.mv_size - fp->mp_upper - PAGEBASE);<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         for (i=3D0; i<NUMKEYS= (fp); i++)<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;            mp-= >mp_ptrs[i] =3D fp->mp_ptrs[i] + offset;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     }<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  }<o:p></o:p> <o:p> </o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  rdata =3D &xdata;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  flags |=3D F_DUPDATA;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  do_sub =3D 1;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  if (!insert_key)<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     mdb_node_del(mc, 0);<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  goto new_sub;<o:p></o:p>         &= nbsp;           &nbs= p;           }<o:p></o:p>= current:<o:p></o:p>         &= nbsp;           &nbs= p;           /* LMDB pass= es F_SUBDATA in 'flags' to write a DB record */<o:p></o:p>         &= nbsp;           &nbs= p;           if ((leaf-&g= t;mn_flags ^ flags) & F_SUBDATA)<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  return MDB_INCOMPATIBLE;<o:p></o:p>         &= nbsp;           &nbs= p;           /* overflow = page overwrites need special handling */<o:p></o:p>         &= nbsp;           &nbs= p;           if (F_ISSET(= leaf->mn_flags, F_BIGDATA)) { <-------- is this taken?<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  MDB_page *omp;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  pgno_t pg;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  int level, ovpages, dpages =3D OVPAGES(data->mv_size, env->m= e_psize);<o:p></o:p> <o:p> </o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  memcpy(&pg, olddata.mv_data, sizeof(pg));<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  if ((rc2 =3D mdb_page_get(mc->mc_txn, pg, &omp, &level)= ) !=3D 0)<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     return rc2;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  ovpages =3D omp->mp_pages;<o:p></o:p> <o:p> </o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  /* Is the ov page large enough? */<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  if (ovpages >=3D dpages) {<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;    if (!(omp->mp_flags & P_DIRTY) &&<o:p></o:p>=         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;       (level || (env->me_flags & MDB_= WRITEMAP)))<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;    {<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     rc =3D mdb_page_unspill(mc->mc_txn, omp, &= amp;omp); <-------- is this taken?<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     if (rc)<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         return rc;<o:p></o:p></p= >         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     level =3D 0;     &nb= sp;            =             /* dirty= in this txn or clean */<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;    }<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;    /* Is it dirty? */<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;    if (omp->mp_flags & P_DIRTY) {<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     /* yes, overwrite it. Note in this case we do= n't<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     * bother to try shrinking the page if the new= data<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     * is smaller than the overflow threshold.<o:p= ></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     */<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     if (level > 1) {<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         /* It is writable only i= n a parent txn */<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         size_t sz =3D (size_t) e= nv->me_psize * ovpages, off;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         MDB_page *np =3D mdb_pag= e_malloc(mc->mc_txn, ovpages);<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         MDB_ID2 id2;<o:p></o:p><= /p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         if (!np)<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;            ret= urn ENOMEM;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         id2.mid =3D pg;<o:p></o:= p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         id2.mptr =3D np;<o:p></o= :p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         /* Note - this page is a= lready counted in parent's dirty_room */<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         rc2 =3D mdb_mid2l_insert= (mc->mc_txn->mt_u.dirty_list, &id2);<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         mdb_cassert(mc, rc2 =3D= =3D 0);<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         if (!(flags & MDB_RE= SERVE)) {<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;            /* = Copy end of page, adjusting alignment so<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;            * c= ompiler may copy words instead of bytes.<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;            */<= o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;            off= =3D (PAGEHDRSZ + data->mv_size) & -sizeof(size_t);<o:p></o:p></= p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;            mem= cpy((size_t *)((char *)np + off),<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;   (size_t *)((char *)omp + off), sz - off);<o:p></o:p><= /p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;            sz = =3D PAGEHDRSZ;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         }<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         memcpy(np, omp, sz); /* = Copy beginning of page */<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         omp =3D np;<o:p></o:p></= p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     }<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     SETDSZ(leaf, data->mv_size);<o:p></o:p></p= >         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     if (F_ISSET(flags, MDB_RESERVE))<o:p></o:p></= p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         data->mv_data =3D MET= ADATA(omp);<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     else<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;         memcpy(METADATA(omp), da= ta->mv_data, data->mv_size);<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     return MDB_SUCCESS;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;    }<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  }<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  if ((rc2 =3D mdb_ovpage_free(mc, omp)) !=3D MDB_SUCCESS)<o:p></o:p= >         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     return rc2;<o:p></o:p>         &= nbsp;           &nbs= p;           } else if (d= ata->mv_size =3D=3D olddata.mv_size) {<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  /* same size, just replace it. Note that we could<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  * also reuse this node if the new data is smaller,<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  * but instead we opt to shrink the node in that case.<o:p></o:p></= p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  */<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  if (F_ISSET(flags, MDB_RESERVE))<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     data->mv_data =3D olddata.mv_data;<o:p></o= :p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  else if (!(mc->mc_flags & C_SUB))<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     memcpy(olddata.mv_data, data->mv_data, dat= a->mv_size);<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  else {<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     memcpy(NODEKEY(leaf), key->mv_data, key-&g= t;mv_size);<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;     goto fix_parent;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  }<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  return MDB_SUCCESS;<o:p></o:p>         &= nbsp;           &nbs= p;           }<o:p></o:p>=         &= nbsp;           &nbs= p;           mdb_node_del= (mc, 0);<o:p></o:p>         &= nbsp;       } <-------- This entire else c= ontingency was likely skipped!!<o:p></o:p> <o:p> </o:p>         &= nbsp;       rdata =3D data;<o:p></o:p> <o:p> </o:p> new_sub:<o:p></o:p>         &= nbsp;       nflags =3D flags & NODE_ADD_F= LAGS;<o:p></o:p>         &= nbsp;       nsize =3D IS_LEAF2(mc->mc_pg[m= c->mc_top]) ? key->mv_size : mdb_leaf_size(env, key, rdata); <----= ---- SEGV<o:p></o:p> ...       &nbs= p;    <o:p></o:p> <o:p> </o:p> This may be an identifiable bug in mdb_cursor_put= () as It does not safely handle the case when the noted call to mdb_cursor_= set() returns MDB_NOTFOUND (-30798)!<o:p></o:p> <o:p> </o:p> It explicitly fails to preemptively exit by retur= ning an error in this case.<o:p></o:p> <o:p> </o:p> The value of -30798 is in turn assigned to insert= _key, which is therefore non-null in the condition of the subsequent if sta= tement.<o:p></o:p> <o:p> </o:p>         &= nbsp;       insert_key =3D insert_data =3D rc= ; <-------- insert_key =3D -30798 or MDB_NOTFOUND!<o:p></o:p>         &= nbsp;       if (insert_key) { <-------- tr= ue<o:p></o:p>         &= nbsp;           &nbs= p;           /* The key d= oes not exist */<o:p></o:p>         &= nbsp;           &nbs= p;           DPRINTF((&qu= ot;inserting key at index %i", mc->mc_ki[mc->mc_top]));<o:p></o:= p>         &= nbsp;           &nbs= p;           if ((mc->= mc_db->md_flags & MDB_DUPSORT) &&<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  LEAFSIZE(key, data) > env->me_nodemax) <-------- false si= nce mc->mc_db->md_flags =3D 0<o:p></o:p>         &= nbsp;           &nbs= p;           {<o:p></o:p>=         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  /* Too big for a node, insert in sub-DB.  Set up an empty<o:p= ></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  * "old sub-page" for prep_subDB to expand to a full page= .<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  */<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  fp_flags =3D P_LEAF|P_DIRTY;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  fp =3D env->me_pbuf;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  fp->mp_pad =3D data->mv_size; /* used if MDB_DUPFIXED */<o:p= ></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  fp->mp_lower =3D fp->mp_upper =3D (PAGEHDRSZ-PAGEBASE);<o:p>= </o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  olddata.mv_size =3D PAGEHDRSZ;<o:p></o:p>         &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  goto prep_subDB;<o:p></o:p>         &= nbsp;           &nbs= p;           } <------= -- this was not executed, and the very long else contingency below is skipp= ed!<o:p></o:p>         &= nbsp;       } else {<o:p></o:p> Since mc->mc_db->md_flags =3D 0, the contin= gency in the embedded if statement fails.<o:p></o:p> <o:p> </o:p> The segmentation fault occurs due to the call of = IS_LEAF2 on the null pointer mc->mc_pg[0].<o:p></o:p> <o:p> </o:p>         &= nbsp;       rdata =3D data;<o:p></o:p> <o:p> </o:p> new_sub:<o:p></o:p>         &= nbsp;       nflags =3D flags & NODE_ADD_F= LAGS;<o:p></o:p>         &= nbsp;       nsize =3D IS_LEAF2(mc->mc_pg[m= c->mc_top]) ? key->mv_size : mdb_leaf_size(env, key, rdata); <----= ---- SEGV<o:p></o:p> <o:p> </o:p> Regards,<o:p></o:p> Robins.<o:p></o:p> </div> </body> </html> --_000_MN2PR06MB55017A7261FA27380778753BAFEA0MN2PR06MB5501namp_--

1 0

Re: (ITS#9037) observing crash in mdb_cursor_put()
by hyc＠symas.com 18 Jun '19

18 Jun '19

Robins George wrote: > Hello Howard, > we are not able to reproduce the issue consistently so test code is not available at this time, and I do believe this issue will be present in the latest code as well since Myk also reported the same.. Attaching analysis from our side, hope it helps. Thanks. Notes below... > > The call stack what we have seen is, > > #0 0x0011b62e in mdb_cursor_put (mc=0xffe52d18, key=0xffe52e74, data=0xffe52e6c, flags=0) at mdb.c:6688 > #1 0x0011c8ec in mdb_put (txn=0xdc13f008, dbi=2, key=0xffe52e74, data=0xffe52e6c, flags=0) at mdb.c:8771 > #2 0x00b792b8 in LMDB::LMDBContext::createSession (this=0x80ca418, sid=0x8297be4 "sidebf3708f0fd9fcb8e062529da47adf51402dfc4600000000+") at lmdbint.cc:460 > #3 0x08053564 in updateLMDB (request=..., forward=@0xffe5315f) at sessionserver.cc:1075 > #4 processRequestWithoutResponse (request=..., forward=@0xffe5315f) at sessionserver.cc:1160 > #5 0x08056a80 in processRequest (this=0x80b2ac0, fd=33) at sessionserver.cc:1189 > #6 readFromSock (this=0x80b2ac0, fd=33) at sessionserver.cc:1342 > #7 SockHandler::ioReady (this=0x80b2ac0, fd=33) at sessionserver.cc:1455 > #8 0x00ba2592 in runCoreDispatcher (default_t=<value optimized out>, flags=-1) at fds.cc:889 > #9 0x00ba2ff7 in DSEvntFds::runDispatcher () at fds.cc:945 > #10 0x080559c6 in main () at sessionserver.cc:1739 > > Here it is helpful to examine frames #0 and #1 in detail in relation to the relevant source code /libraries/liblmdb/mdb.c. > > Frame #1 > > #1 0x0011c8ec in mdb_put (txn=0xdc13f008, dbi=2, key=0xffe52e74, data=0xffe52e6c, flags=0) at mdb.c:8771 > mc = {mc_next = 0x0, mc_backup = 0x0, mc_xcursor = 0x0, mc_txn = 0xdc13f008, mc_dbi = 2, mc_db = 0xdc13f088, mc_dbx = 0xe1215038, mc_dbflag = 0xdc1cf09a "\033", '\032' <repeats 51 times>, mc_snum = 0, mc_top = 0, mc_flags = 0, mc_pg = {0x0, 0x80b2ac0, 0xffe52d68, 0x28afce, 0xdc13f008, 0x80ce9b0, 0xffe52dc8, 0x4c5ff4, 0x2b, 0x80b2ac0, 0xffe52d98, 0x49209a, 0x2b, 0x4c5ff4, 0x2121cb, 0x21576c, 0x8297d34, 0x3497af, 0x21576c, 0x21316a, 0x8297d34, 0x80ac7b0, 0x1e, 0x46eed6, 0x2b, 0x0, 0x80ce9b0, 0x4c5ff4, 0x80ac7b0, 0x8297d34, 0xffe52df8, 0x46fb96}, mc_ki = {0, 2089, 51120, 2058, 30, 0, 16796, 17, 11760, 65509, 3, 0, 32040, 2089, 30, 0, 0, 0, 0, 0, 3, 0, 24564, 76, 51120, 2058, 10944, 2059, 11800, 65509, 64758, 70}} <-------- > > mx = {mx_cursor = {mc_next = 0x38, mc_backup = 0x3, mc_xcursor = 0x0, mc_txn = 0x0, mc_dbi = 0, mc_db = 0x28, mc_dbx = 0x0, mc_dbflag = 0x0, mc_snum = 3, mc_top = 0, mc_flags = 17, mc_pg = {0x3a93d8, 0x10, 0x0, 0x18, 0x3a93a0, 0x0, 0x3a93d0, 0x289abe, 0x3a93a0, 0xffe52d3f, 0xffe52c68, 0x28afce, 0xffe52dbc, 0xffe52db4, 0x3, 0x4c5ff4, 0x11, 0x36c99b, 0x6e, 0x77, 0x7c, 0x5b, 0x38, 0x289abe, 0x0, 0x0, 0x0, 0x28, 0x0, 0x0, 0x3, 0x10}, mc_ki = {37848, 58, 51611, 54, 110, 0, 119, 0, 124, 0, 91, 0, 58, 0, 39614, 40, 0, 0, 0, 0, 0, 0, 160, 0, 0, 0, 2, 0, 18, 0, 136, 0}}, mx_db = {md_pad = 3838936, md_flags = 51611, md_depth = 54, md_branch_pages = 110, md_leaf_pages = 119, md_overflow_pages = 124, md_entries = 91, md_root = 56}, mx_dbx = {md_name = {mv_size = 6, mv_data = 0x0}, md_cmp = 0, md_dcmp = 0, md_rel = 0x40, md_relctx = 0x0}, mx_dbflag = 0 '\000'} > rc = 0 > > And source code for mdb_put() is from mdb.c > > int > mdb_put(MDB_txn *txn, MDB_dbi dbi, > MDB_val *key, MDB_val *data, unsigned int flags) > { > MDB_cursor mc; > MDB_xcursor mx; > int rc; > > if (!key || !data || !TXN_DBI_EXIST(txn, dbi, DB_USRVALID)) > return EINVAL; > > if (flags & ~(MDB_NOOVERWRITE|MDB_NODUPDATA|MDB_RESERVE|MDB_APPEND|MDB_APPENDDUP)) > return EINVAL; > > if (txn->mt_flags & (MDB_TXN_RDONLY|MDB_TXN_BLOCKED)) > return (txn->mt_flags & MDB_TXN_RDONLY) ? EACCES : MDB_BAD_TXN; > > mdb_cursor_init(&mc, txn, dbi, &mx); <-------- see source code below > mc.mc_next = txn->mt_cursors[dbi]; > txn->mt_cursors[dbi] = &mc; > rc = mdb_cursor_put(&mc, key, data, flags); <-------- see Frame #0 > txn->mt_cursors[dbi] = mc.mc_next; > return rc; > } > > > Source Code For mdb_cursor_init() From mdb.c > /** Initialize a cursor for a given transaction and database. */ > static void > mdb_cursor_init(MDB_cursor *mc, MDB_txn *txn, MDB_dbi dbi, MDB_xcursor *mx) > { > mc->mc_next = NULL; > mc->mc_backup = NULL; > mc->mc_dbi = dbi; > mc->mc_txn = txn; > mc->mc_db = &txn->mt_dbs[dbi]; > mc->mc_dbx = &txn->mt_dbxs[dbi]; > mc->mc_dbflag = &txn->mt_dbflags[dbi]; > mc->mc_snum = 0; <-------- > mc->mc_top = 0; <-------- > mc->mc_pg[0] = 0; <-------- > mc->mc_ki[0] = 0; > mc->mc_flags = 0; > if (txn->mt_dbs[dbi].md_flags & MDB_DUPSORT) { > mdb_tassert(txn, mx != NULL); > mc->mc_xcursor = mx; > mdb_xcursor_init0(mc); > } else { > mc->mc_xcursor = NULL; > } > if (*mc->mc_dbflag & DB_STALE) { <-------- false *mc->mc_dbflag = 27, DB_STALE = 0x02 /**< Named-DB record is older than txnID */ 27 = 0x1b, so DB_STALE is true. mdb_page_search ought to have been invoked to set the root of the cursor. > mdb_page_search(mc, NULL, MDB_PS_ROOTONLY); > } > } > > From this we know that when mdb_cursor_put() is called, the following values remain assigned: > > mc->mc_snum = 0; <-------- > mc->mc_top = 0; <-------- > mc->mc_pg[0] = 0; <-------- > > and its noted these two values still have their initial values at the time of the segmentation fault. There is no issue with mdb_cursor_put then. The question is why didn't mdb_page_search find the DBI's root node? -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7996) Tighten race in ldap_int_initialize
by Armin.Tueting＠tueting-online.com 18 Jun '19

18 Jun '19

Am Montag, den 17.06.2019, 15:40 -0700 schrieb Quanah Gibson-Mount: > --On Monday, June 17, 2019 7:16 PM +0200 Armin T=C3=BCting=20 > <Armin.Tueting(a)tueting-online.com> wrote: >=20 > > > I.e., it started and then got as far as reading your ldap.conf file. > > > What is the contents of ldap.conf? > > Attached 'ldap.conf'. Nothing unusual... > >=20 > > > Have you run the test suite (make test)? Does it pass? fail? > > Attached 'make_test.txt'. As far as I can see - it has been passed. >=20 > Ok, so make test passes without issue, so it would appear there's somethi= ng=20 > specific with your configuration that is triggering the problem. Would y= ou=20 > be able to provide your slapd configuration (minus any passwords and the= =20 > like)? I'll privately send it to you in a seperate email. >=20 > Additionally, if you could get a full gdb backtrace of the hung slapd=20 > process that would be useful as well. I.e.: >=20 > start up slapd > gdb /path/to/slapd <pid #> (gdb) thr apply all bt full Thread 1 (Thread 0x7ffa4ad71880 (LWP 9207)): #0 0x00007ffa4918c4ed in __lll_lock_wait () from /lib64/libpthread.so.0 No symbol table info available. #1 0x00007ffa49187dcb in _L_lock_883 () from /lib64/libpthread.so.0 No symbol table info available. #2 0x00007ffa49187c98 in pthread_mutex_lock () from /lib64/libpthread.so.0 No symbol table info available. #3 0x00007ffa4a92a135 in ldap_pvt_thread_mutex_lock (mutex=3D<optimized out>) at thr_posix.c:296 No locals. #4 0x00007ffa4a944e6e in ldap_set_option (ld=3Dld@entry=3D0x0,=20 option=3Doption@entry=3D20486, invalue=3Dinvalue@entry=3D0x7ffdceb7aeb4) at options.c:465 lo =3D 0x7176c0 <ldap_int_global_options> dbglvl =3D 0x0 rc =3D -1 __PRETTY_FUNCTION__ =3D "ldap_set_option" #5 0x00007ffa4a943ed3 in openldap_ldap_init_w_conf (file=3Dfile@entry=3D0x 7ffa4a95a750 "/opt/openldap/etc/openldap/ldap.conf",=20 userconf=3Duserconf@entry=3D0) at init.c:270 p =3D <optimized out> linebuf =3D "URI\000ldap://localhost/\000\000tions\n\000ware/man.cgi?query=3Dldap.con f\n\000ldapusingtlswithsshapasswords\n\000\000\377\377\000\000\177\003" , '\000' <repeats 22 times>, "\200\037\000\000\377\377", '\000' <repeats 152 times>... fp =3D 0x8177b0 i =3D <optimized out> opt =3D 0x7ffdceb7aeb4 "ldap://localhost/" start =3D 0x7ffdceb7aeb4 "ldap://localhost/" end =3D <optimized out> gopts =3D 0x7ffa4ab67120 <ldap_int_global_options> #6 0x00007ffa4a944318 in openldap_ldap_init_w_sysconf (file=3D0x7ffa4a95a750 "/opt/openldap/etc/openldap/ldap.conf") at init.c:316 No locals. #7 ldap_int_initialize (gopts=3Dgopts@entry=3D0x7176c0 <ldap_int_global_options>, dbglvl=3Ddbglvl@entry=3D0x719050 <slap_debug>) at init.c:684 No locals. #8 0x00007ffa4a944e38 in ldap_set_option (ld=3D0x0, option=3D20481, invalue=3D0x719050 <slap_debug>) at options.c:450 lo =3D <optimized out> dbglvl =3D 0x719050 <slap_debug> rc =3D -1 __PRETTY_FUNCTION__ =3D "ldap_set_option" #9 0x0000000000413cd6 in main () No symbol table info available. > at the gdb prompt: >=20 > thr apply all bt full >=20 > Thanks! >=20 > --Quanah >=20 >=20 >=20 > -- >=20 > Quanah Gibson-Mount > Product Architect > Symas Corporation > Packaged, certified, and supported LDAP solutions powered by OpenLDAP: > <http://www.symas.com> >=20

1 0

Re: (ITS#7996) Tighten race in ldap_int_initialize
by Armin.Tueting＠tueting-online.com 18 Jun '19

18 Jun '19

1 0

RE: (ITS#9037) observing crash in mdb_cursor_put()
by grobins＠pulsesecure.net 17 Jun '19

17 Jun '19

SGVsbG8gSG93YXJkLA0Kd2UgYXJlIG5vdCBhYmxlIHRvIHJlcHJvZHVjZSB0aGUgaXNzdWUgY29u c2lzdGVudGx5IHNvIHRlc3QgY29kZSBpcyBub3QgYXZhaWxhYmxlIGF0IHRoaXMgdGltZSwgYW5k IEkgZG8gYmVsaWV2ZSB0aGlzIGlzc3VlIHdpbGwgYmUgcHJlc2VudCBpbiB0aGUgbGF0ZXN0IGNv ZGUgYXMgd2VsbCBzaW5jZSBNeWsgYWxzbyByZXBvcnRlZCB0aGUgc2FtZS4uICBBdHRhY2hpbmcg YW5hbHlzaXMgZnJvbSBvdXIgc2lkZSwgaG9wZSBpdCBoZWxwcy4NCg0KVGhlIGNhbGwgc3RhY2sg d2hhdCB3ZSBoYXZlIHNlZW4gaXMsDQoNCiAgICAjMCAweDAwMTFiNjJlIGluIG1kYl9jdXJzb3Jf cHV0IChtYz0weGZmZTUyZDE4LCBrZXk9MHhmZmU1MmU3NCwgZGF0YT0weGZmZTUyZTZjLCBmbGFn cz0wKSBhdCBtZGIuYzo2Njg4DQogICAgIzEgMHgwMDExYzhlYyBpbiBtZGJfcHV0ICh0eG49MHhk YzEzZjAwOCwgZGJpPTIsIGtleT0weGZmZTUyZTc0LCBkYXRhPTB4ZmZlNTJlNmMsIGZsYWdzPTAp IGF0IG1kYi5jOjg3NzENCiAgICAjMiAweDAwYjc5MmI4IGluIExNREI6OkxNREJDb250ZXh0Ojpj cmVhdGVTZXNzaW9uICh0aGlzPTB4ODBjYTQxOCwgc2lkPTB4ODI5N2JlNCAic2lkZWJmMzcwOGYw ZmQ5ZmNiOGUwNjI1MjlkYTQ3YWRmNTE0MDJkZmM0NjAwMDAwMDAwKyIpIGF0IGxtZGJpbnQuY2M6 NDYwDQogICAgIzMgMHgwODA1MzU2NCBpbiB1cGRhdGVMTURCIChyZXF1ZXN0PS4uLiwgZm9yd2Fy ZD1AMHhmZmU1MzE1ZikgYXQgc2Vzc2lvbnNlcnZlci5jYzoxMDc1DQogICAgIzQgcHJvY2Vzc1Jl cXVlc3RXaXRob3V0UmVzcG9uc2UgKHJlcXVlc3Q9Li4uLCBmb3J3YXJkPUAweGZmZTUzMTVmKSBh dCBzZXNzaW9uc2VydmVyLmNjOjExNjANCiAgICAjNSAweDA4MDU2YTgwIGluIHByb2Nlc3NSZXF1 ZXN0ICh0aGlzPTB4ODBiMmFjMCwgZmQ9MzMpIGF0IHNlc3Npb25zZXJ2ZXIuY2M6MTE4OQ0KICAg ICM2IHJlYWRGcm9tU29jayAodGhpcz0weDgwYjJhYzAsIGZkPTMzKSBhdCBzZXNzaW9uc2VydmVy LmNjOjEzNDINCiAgICAjNyBTb2NrSGFuZGxlcjo6aW9SZWFkeSAodGhpcz0weDgwYjJhYzAsIGZk PTMzKSBhdCBzZXNzaW9uc2VydmVyLmNjOjE0NTUNCiAgICAjOCAweDAwYmEyNTkyIGluIHJ1bkNv cmVEaXNwYXRjaGVyIChkZWZhdWx0X3Q9PHZhbHVlIG9wdGltaXplZCBvdXQ+LCBmbGFncz0tMSkg YXQgZmRzLmNjOjg4OQ0KICAgICM5IDB4MDBiYTJmZjcgaW4gRFNFdm50RmRzOjpydW5EaXNwYXRj aGVyICgpIGF0IGZkcy5jYzo5NDUNCiAgICAjMTAgMHgwODA1NTljNiBpbiBtYWluICgpIGF0IHNl c3Npb25zZXJ2ZXIuY2M6MTczOQ0KDQpIZXJlIGl0IGlzIGhlbHBmdWwgdG8gZXhhbWluZSBmcmFt ZXMgIzAgYW5kICMxIGluIGRldGFpbCBpbiByZWxhdGlvbiB0byB0aGUgcmVsZXZhbnQgc291cmNl IGNvZGUgL2xpYnJhcmllcy9saWJsbWRiL21kYi5jLg0KDQpGcmFtZSAjMQ0KDQojMSAgMHgwMDEx YzhlYyBpbiBtZGJfcHV0ICh0eG49MHhkYzEzZjAwOCwgZGJpPTIsIGtleT0weGZmZTUyZTc0LCBk YXRhPTB4ZmZlNTJlNmMsIGZsYWdzPTApIGF0IG1kYi5jOjg3NzENCiAgICAgICAgbWMgPSB7bWNf bmV4dCA9IDB4MCwgbWNfYmFja3VwID0gMHgwLCBtY194Y3Vyc29yID0gMHgwLCBtY190eG4gPSAw eGRjMTNmMDA4LCBtY19kYmkgPSAyLCBtY19kYiA9IDB4ZGMxM2YwODgsIG1jX2RieCA9IDB4ZTEy MTUwMzgsIG1jX2RiZmxhZyA9IDB4ZGMxY2YwOWEgIlwwMzMiLCAnXDAzMicgPHJlcGVhdHMgNTEg dGltZXM+LCBtY19zbnVtID0gMCwgbWNfdG9wID0gMCwgbWNfZmxhZ3MgPSAwLCBtY19wZyA9IHsw eDAsIDB4ODBiMmFjMCwgMHhmZmU1MmQ2OCwgMHgyOGFmY2UsIDB4ZGMxM2YwMDgsIDB4ODBjZTli MCwgMHhmZmU1MmRjOCwgMHg0YzVmZjQsIDB4MmIsIDB4ODBiMmFjMCwgMHhmZmU1MmQ5OCwgMHg0 OTIwOWEsIDB4MmIsIDB4NGM1ZmY0LCAweDIxMjFjYiwgMHgyMTU3NmMsIDB4ODI5N2QzNCwgMHgz NDk3YWYsIDB4MjE1NzZjLCAweDIxMzE2YSwgMHg4Mjk3ZDM0LCAweDgwYWM3YjAsIDB4MWUsIDB4 NDZlZWQ2LCAweDJiLCAweDAsIDB4ODBjZTliMCwgMHg0YzVmZjQsIDB4ODBhYzdiMCwgMHg4Mjk3 ZDM0LCAweGZmZTUyZGY4LCAweDQ2ZmI5Nn0sIG1jX2tpID0gezAsIDIwODksIDUxMTIwLCAyMDU4 LCAzMCwgMCwgMTY3OTYsIDE3LCAxMTc2MCwgNjU1MDksIDMsIDAsIDMyMDQwLCAyMDg5LCAzMCwg MCwgMCwgMCwgMCwgMCwgMywgMCwgMjQ1NjQsIDc2LCA1MTEyMCwgMjA1OCwgMTA5NDQsIDIwNTks IDExODAwLCA2NTUwOSwgNjQ3NTgsIDcwfX0gPC0tLS0tLS0tDQoJDQogICAgICAgIG14ID0ge214 X2N1cnNvciA9IHttY19uZXh0ID0gMHgzOCwgbWNfYmFja3VwID0gMHgzLCBtY194Y3Vyc29yID0g MHgwLCBtY190eG4gPSAweDAsIG1jX2RiaSA9IDAsIG1jX2RiID0gMHgyOCwgbWNfZGJ4ID0gMHgw LCBtY19kYmZsYWcgPSAweDAsIG1jX3NudW0gPSAzLCBtY190b3AgPSAwLCBtY19mbGFncyA9IDE3 LCBtY19wZyA9IHsweDNhOTNkOCwgMHgxMCwgMHgwLCAweDE4LCAweDNhOTNhMCwgMHgwLCAweDNh OTNkMCwgMHgyODlhYmUsIDB4M2E5M2EwLCAweGZmZTUyZDNmLCAweGZmZTUyYzY4LCAweDI4YWZj ZSwgMHhmZmU1MmRiYywgMHhmZmU1MmRiNCwgMHgzLCAweDRjNWZmNCwgMHgxMSwgMHgzNmM5OWIs IDB4NmUsIDB4NzcsIDB4N2MsIDB4NWIsIDB4MzgsIDB4Mjg5YWJlLCAweDAsIDB4MCwgMHgwLCAw eDI4LCAweDAsIDB4MCwgMHgzLCAweDEwfSwgbWNfa2kgPSB7Mzc4NDgsIDU4LCA1MTYxMSwgNTQs IDExMCwgMCwgMTE5LCAwLCAxMjQsIDAsIDkxLCAwLCA1OCwgMCwgMzk2MTQsIDQwLCAwLCAwLCAw LCAwLCAwLCAwLCAxNjAsIDAsIDAsIDAsIDIsIDAsIDE4LCAwLCAxMzYsIDB9fSwgbXhfZGIgPSB7 bWRfcGFkID0gMzgzODkzNiwgbWRfZmxhZ3MgPSA1MTYxMSwgbWRfZGVwdGggPSA1NCwgbWRfYnJh bmNoX3BhZ2VzID0gMTEwLCBtZF9sZWFmX3BhZ2VzID0gMTE5LCBtZF9vdmVyZmxvd19wYWdlcyA9 IDEyNCwgbWRfZW50cmllcyA9IDkxLCBtZF9yb290ID0gNTZ9LCBteF9kYnggPSB7bWRfbmFtZSA9 IHttdl9zaXplID0gNiwgbXZfZGF0YSA9IDB4MH0sIG1kX2NtcCA9IDAsIG1kX2RjbXAgPSAwLCBt ZF9yZWwgPSAweDQwLCBtZF9yZWxjdHggPSAweDB9LCBteF9kYmZsYWcgPSAwICdcMDAwJ30NCiAg ICAgICAgcmMgPSAwIA0KDQpBbmQgc291cmNlIGNvZGUgZm9yIG1kYl9wdXQoKSBpcyBmcm9tIG1k Yi5jDQoNCmludA0KbWRiX3B1dChNREJfdHhuICp0eG4sIE1EQl9kYmkgZGJpLA0KICAgIE1EQl92 YWwgKmtleSwgTURCX3ZhbCAqZGF0YSwgdW5zaWduZWQgaW50IGZsYWdzKQ0Kew0KCU1EQl9jdXJz b3IgbWM7DQoJTURCX3hjdXJzb3IgbXg7DQoJaW50IHJjOw0KDQoJaWYgKCFrZXkgfHwgIWRhdGEg fHwgIVRYTl9EQklfRVhJU1QodHhuLCBkYmksIERCX1VTUlZBTElEKSkNCgkJcmV0dXJuIEVJTlZB TDsNCg0KCWlmIChmbGFncyAmIH4oTURCX05PT1ZFUldSSVRFfE1EQl9OT0RVUERBVEF8TURCX1JF U0VSVkV8TURCX0FQUEVORHxNREJfQVBQRU5ERFVQKSkNCgkJcmV0dXJuIEVJTlZBTDsNCg0KCWlm ICh0eG4tPm10X2ZsYWdzICYgKE1EQl9UWE5fUkRPTkxZfE1EQl9UWE5fQkxPQ0tFRCkpDQoJCXJl dHVybiAodHhuLT5tdF9mbGFncyAmIE1EQl9UWE5fUkRPTkxZKSA/IEVBQ0NFUyA6IE1EQl9CQURf VFhOOw0KDQoJbWRiX2N1cnNvcl9pbml0KCZtYywgdHhuLCBkYmksICZteCk7IDwtLS0tLS0tLSBz ZWUgc291cmNlIGNvZGUgYmVsb3cNCgltYy5tY19uZXh0ID0gdHhuLT5tdF9jdXJzb3JzW2RiaV07 DQoJdHhuLT5tdF9jdXJzb3JzW2RiaV0gPSAmbWM7DQoJcmMgPSBtZGJfY3Vyc29yX3B1dCgmbWMs IGtleSwgZGF0YSwgZmxhZ3MpOyA8LS0tLS0tLS0gc2VlIEZyYW1lICMwDQoJdHhuLT5tdF9jdXJz b3JzW2RiaV0gPSBtYy5tY19uZXh0Ow0KCXJldHVybiByYzsNCn0NCg0KDQpTb3VyY2UgQ29kZSBG b3IgbWRiX2N1cnNvcl9pbml0KCkgRnJvbSBtZGIuYw0KLyoqIEluaXRpYWxpemUgYSBjdXJzb3Ig Zm9yIGEgZ2l2ZW4gdHJhbnNhY3Rpb24gYW5kIGRhdGFiYXNlLiAqLw0Kc3RhdGljIHZvaWQNCm1k Yl9jdXJzb3JfaW5pdChNREJfY3Vyc29yICptYywgTURCX3R4biAqdHhuLCBNREJfZGJpIGRiaSwg TURCX3hjdXJzb3IgKm14KQ0Kew0KCW1jLT5tY19uZXh0ID0gTlVMTDsNCgltYy0+bWNfYmFja3Vw ID0gTlVMTDsNCgltYy0+bWNfZGJpID0gZGJpOw0KCW1jLT5tY190eG4gPSB0eG47DQoJbWMtPm1j X2RiID0gJnR4bi0+bXRfZGJzW2RiaV07DQoJbWMtPm1jX2RieCA9ICZ0eG4tPm10X2RieHNbZGJp XTsNCgltYy0+bWNfZGJmbGFnID0gJnR4bi0+bXRfZGJmbGFnc1tkYmldOw0KCW1jLT5tY19zbnVt ID0gMDsgPC0tLS0tLS0tIA0KCW1jLT5tY190b3AgPSAwOyA8LS0tLS0tLS0gIA0KCW1jLT5tY19w Z1swXSA9IDA7IDwtLS0tLS0tLSANCgltYy0+bWNfa2lbMF0gPSAwOw0KCW1jLT5tY19mbGFncyA9 IDA7DQoJaWYgKHR4bi0+bXRfZGJzW2RiaV0ubWRfZmxhZ3MgJiBNREJfRFVQU09SVCkgew0KCQlt ZGJfdGFzc2VydCh0eG4sIG14ICE9IE5VTEwpOw0KCQltYy0+bWNfeGN1cnNvciA9IG14Ow0KCQlt ZGJfeGN1cnNvcl9pbml0MChtYyk7DQoJfSBlbHNlIHsNCgkJbWMtPm1jX3hjdXJzb3IgPSBOVUxM Ow0KCX0NCglpZiAoKm1jLT5tY19kYmZsYWcgJiBEQl9TVEFMRSkgeyA8LS0tLS0tLS0gZmFsc2Ug Km1jLT5tY19kYmZsYWcgPSAyNywgREJfU1RBTEUgPSAweDAyCQkvKio8IE5hbWVkLURCIHJlY29y ZCBpcyBvbGRlciB0aGFuIHR4bklEICovDQoJCW1kYl9wYWdlX3NlYXJjaChtYywgTlVMTCwgTURC X1BTX1JPT1RPTkxZKTsgDQoJfQ0KfQ0KDQpGcm9tIHRoaXMgd2Uga25vdyB0aGF0IHdoZW4gbWRi X2N1cnNvcl9wdXQoKSBpcyBjYWxsZWQsIHRoZSBmb2xsb3dpbmcgdmFsdWVzIHJlbWFpbiBhc3Np Z25lZDoNCg0KICAgICAgICAgICAgICBtYy0+bWNfc251bSA9IDA7IDwtLS0tLS0tLQ0KCW1jLT5t Y190b3AgPSAwOyA8LS0tLS0tLS0gIA0KCW1jLT5tY19wZ1swXSA9IDA7IDwtLS0tLS0tLSANCg0K YW5kIGl0cyBub3RlZCB0aGVzZSB0d28gdmFsdWVzIHN0aWxsIGhhdmUgdGhlaXIgaW5pdGlhbCB2 YWx1ZXMgYXQgdGhlIHRpbWUgb2YgdGhlIHNlZ21lbnRhdGlvbiBmYXVsdC4NCg0KKGdkYikgZnIg MA0KIzAgMHgwMDExYjYyZSBpbiBtZGJfY3Vyc29yX3B1dCAobWM9MHhmZmU1MmQxOCwga2V5PTB4 ZmZlNTJlNzQsIGRhdGE9MHhmZmU1MmU2YywgZmxhZ3M9MCkgYXQgbWRiLmM6NjY4OA0KNjY4OCBp biBtZGIuYw0KKGdkYikgcCBtYy0+bWNfdG9wDQokMSA9IDANCihnZGIpIHAgbWMtPm1jX3BnW21j LT5tY190b3BdDQokMTkgPSAoTURCX3BhZ2UgKikgMHgwDQoNCkxldCB1cyBjb25zaWRlciBob3cg dGhlIHBhdGggdGFrZW4gdGhyb3VnaCBtZGJfY3Vyc29yX3B1dCgpIGNvdWxkIGFjY291bnQgZm9y IHRoZXNlIG51bGwgdmFyaWFibGUgdmFsdWVzIHdoaWNoIHJlc3VsdCBpbiBhIHNlZ21lbnRhdGlv biBmYXVsdC4NCg0KRnJhbWUgIzANCg0KIzAgIDB4MDAxMWI2MmUgaW4gbWRiX2N1cnNvcl9wdXQg KG1jPTB4ZmZlNTJkMTgsIGtleT0weGZmZTUyZTc0LCBkYXRhPTB4ZmZlNTJlNmMsIGZsYWdzPTAp IGF0IG1kYi5jOjY2ODggPC0tLS0tLS0tIGZsYWdzID0gMA0KICAgICAgICBlbnYgPSAweDgwY2U5 YjANCiAgICAgICAgbGVhZiA9IDx2YWx1ZSBvcHRpbWl6ZWQgb3V0Pg0KICAgICAgICBmcCA9IDx2 YWx1ZSBvcHRpbWl6ZWQgb3V0Pg0KICAgICAgICBtcCA9IDB4MzgNCiAgICAgICAgc3ViX3Jvb3Qg PSAweDAgDQogICAgICAgIGZwX2ZsYWdzID0gPHZhbHVlIG9wdGltaXplZCBvdXQ+DQogICAgICAg IHhkYXRhID0ge212X3NpemUgPSAzODM4ODgwLCBtdl9kYXRhID0gMHhkYzFjZjA5YX0NCiAgICAg ICAgcmRhdGEgPSAweGZmZTUyZTZjDQogICAgICAgIGRrZXkgPSB7bXZfc2l6ZSA9IDAsIG12X2Rh dGEgPSAweDNhOTNjY30NCiAgICAgICAgb2xkZGF0YSA9IHttdl9zaXplID0gMjIsIG12X2RhdGEg PSAweDM2YzgyMH0NCiAgICAgICAgZHVtbXkgPSB7bWRfcGFkID0gMjIsIG1kX2ZsYWdzID0gMTEx MjgsIG1kX2RlcHRoID0gNjU1MDksIG1kX2JyYW5jaF9wYWdlcyA9IDE2LCBtZF9sZWFmX3BhZ2Vz ID0gMTUwNzMyOSwgbWRfb3ZlcmZsb3dfcGFnZXMgPSAwLCBtZF9lbnRyaWVzID0gMzgzODkyOCwg bWRfcm9vdCA9IDM4MzM4NDR9DQogICAgICAgIGRvX3N1YiA9IDANCiAgICAgICAgaW5zZXJ0X2tl eSA9IC0zMDc5OCA8LS0tLS0tLS0gI2RlZmluZSBNREJfTk9URk9VTkQJKC0zMDc5OCkNCiAgICAg ICAgaW5zZXJ0X2RhdGEgPSAtMzA3OTggPC0tLS0tLS0tICNkZWZpbmUgTURCX05PVEZPVU5ECSgt MzA3OTgpDQogICAgICAgIG1jb3VudCA9IDANCiAgICAgICAgZGNvdW50ID0gMA0KICAgICAgICBu b3NwaWxsID0gMCA8LS0tLS0tLS0NCiAgICAgICAgbnNpemUgPSA8dmFsdWUgb3B0aW1pemVkIG91 dD4NCiAgICAgICAgcmMgPSAwIA0KICAgICAgICByYzIgPSA8dmFsdWUgb3B0aW1pemVkIG91dD4N CiAgICAgICAgbmZsYWdzID0gMCA8LS0tLS0tLS0NCiAgICAgICAgX19mdW5jX18gPSAibWRiX2N1 cnNvcl9wdXQiCQ0KDQpXZSBrbm93IHRoYXQgbWMtPm1jX3RvcCBhbmQgbWMtPm1jX3BnWzBdIHdl cmUgaW5pdGlhbGl6ZWQgdG8gMCBpbiBtZGJfY3Vyc29yX2luaXQoKS4gQXMgbXkgYW5ub3RhdGlv bnMgb2YgdGhlIHNvdXJjZSBjb2RlIGJlbG93IHN1Z2dlc3QsIGNhcmVmdWwgYW5hbHlzaXMgcmV2 ZWFscyB0aGF0IHRoZSBwYXRocyBhY3R1YWxseSB0YWtlbiB0aHJvdWdoIG1kYl9jdXJzb3JfcHV0 KCkgZG8gbm90IG1vZGlmeSB0aGVzZSBhc3NpZ25tZW50cy4NCg0KRXhjZXJwdGVkIFNvdXJjZSBD b2RlIEZvciBtZGJfY3Vyc29yX3B1dCgpIEZyb20gbWRiLmMNCg0KaW50DQptZGJfY3Vyc29yX3B1 dChNREJfY3Vyc29yICptYywgTURCX3ZhbCAqa2V5LCBNREJfdmFsICpkYXRhLA0KICAgIHVuc2ln bmVkIGludCBmbGFncykNCnsNCglNREJfZW52CQkqZW52Ow0KCU1EQl9ub2RlCSpsZWFmID0gTlVM TDsNCglNREJfcGFnZQkqZnAsICptcCwgKnN1Yl9yb290ID0gTlVMTDsNCgl1aW50MTZfdAlmcF9m bGFnczsNCglNREJfdmFsCQl4ZGF0YSwgKnJkYXRhLCBka2V5LCBvbGRkYXRhOw0KCU1EQl9kYiBk dW1teTsNCglpbnQgZG9fc3ViID0gMCwgaW5zZXJ0X2tleSwgaW5zZXJ0X2RhdGE7DQoJdW5zaWdu ZWQgaW50IG1jb3VudCA9IDAsIGRjb3VudCA9IDAsIG5vc3BpbGw7DQoJc2l6ZV90IG5zaXplOw0K CWludCByYywgcmMyOw0KCXVuc2lnbmVkIGludCBuZmxhZ3M7DQoJREtCVUY7DQoNCglpZiAobWMg PT0gTlVMTCB8fCBrZXkgPT0gTlVMTCkNCgkJcmV0dXJuIEVJTlZBTDsNCg0KCWVudiA9IG1jLT5t Y190eG4tPm10X2VudjsNCg0KCS8qIENoZWNrIHRoaXMgZmlyc3Qgc28gY291bnRlciB3aWxsIGFs d2F5cyBiZSB6ZXJvIG9uIGFueQ0KCSAqIGVhcmx5IGZhaWx1cmVzLg0KCSAqLw0KCWlmIChmbGFn cyAmIE1EQl9NVUxUSVBMRSkgew0KCQlkY291bnQgPSBkYXRhWzFdLm12X3NpemU7DQoJCWRhdGFb MV0ubXZfc2l6ZSA9IDA7DQoJCWlmICghRl9JU1NFVChtYy0+bWNfZGItPm1kX2ZsYWdzLCBNREJf RFVQRklYRUQpKQ0KCQkJcmV0dXJuIE1EQl9JTkNPTVBBVElCTEU7DQoJfQ0KDQoJbm9zcGlsbCA9 IGZsYWdzICYgTURCX05PU1BJTEw7DQoJZmxhZ3MgJj0gfk1EQl9OT1NQSUxMOw0KDQoJaWYgKG1j LT5tY190eG4tPm10X2ZsYWdzICYgKE1EQl9UWE5fUkRPTkxZfE1EQl9UWE5fQkxPQ0tFRCkpDQoJ CXJldHVybiAobWMtPm1jX3R4bi0+bXRfZmxhZ3MgJiBNREJfVFhOX1JET05MWSkgPyBFQUNDRVMg OiBNREJfQkFEX1RYTjsNCg0KCWlmIChrZXktPm12X3NpemUtMSA+PSBFTlZfTUFYS0VZKGVudikp DQoJCXJldHVybiBNREJfQkFEX1ZBTFNJWkU7DQoNCiNpZiBTSVpFX01BWCA+IE1BWERBVEFTSVpF DQoJaWYgKGRhdGEtPm12X3NpemUgPiAoKG1jLT5tY19kYi0+bWRfZmxhZ3MgJiBNREJfRFVQU09S VCkgPyBFTlZfTUFYS0VZKGVudikgOiBNQVhEQVRBU0laRSkpDQoJCXJldHVybiBNREJfQkFEX1ZB TFNJWkU7DQojZWxzZQ0KCWlmICgobWMtPm1jX2RiLT5tZF9mbGFncyAmIE1EQl9EVVBTT1JUKSAm JiBkYXRhLT5tdl9zaXplID4gRU5WX01BWEtFWShlbnYpKQ0KCQlyZXR1cm4gTURCX0JBRF9WQUxT SVpFOw0KI2VuZGlmDQoNCglEUFJJTlRGKCgiPT0+IHB1dCBkYiAlZCBrZXkgWyVzXSwgc2l6ZSAl IloidSwgZGF0YSBzaXplICUiWiJ1IiwNCgkJRERCSShtYyksIERLRVkoa2V5KSwga2V5ID8ga2V5 LT5tdl9zaXplIDogMCwgZGF0YS0+bXZfc2l6ZSkpOw0KDQoJZGtleS5tdl9zaXplID0gMDsNCg0K CWlmIChmbGFncyA9PSBNREJfQ1VSUkVOVCkgeyA8LS0tLS0tLS0gZmFsc2UgZmxhZ3MgPSAwDQoJ CWlmICghKG1jLT5tY19mbGFncyAmIENfSU5JVElBTElaRUQpKQ0KCQkJcmV0dXJuIEVJTlZBTDsN CgkJcmMgPSBNREJfU1VDQ0VTUzsNCgl9IGVsc2UgaWYgKG1jLT5tY19kYi0+bWRfcm9vdCA9PSBQ X0lOVkFMSUQpIHsgPC0tLS0tLS0tIGZhbHNlLCBtYy0+bWNfZGItPm1kX3Jvb3QgPSA2OA0KCQkv KiBuZXcgZGF0YWJhc2UsIGN1cnNvciBoYXMgbm90aGluZyB0byBwb2ludCB0byAqLw0KCQltYy0+ bWNfc251bSA9IDA7DQoJCW1jLT5tY190b3AgPSAwOw0KCQltYy0+bWNfZmxhZ3MgJj0gfkNfSU5J VElBTElaRUQ7DQoJCXJjID0gTURCX05PX1JPT1Q7ICANCgl9IGVsc2UgeyA8LS0tLS0tLS0gZXhl Y3V0ZWQNCgkJaW50IGV4YWN0ID0gMDsNCgkJTURCX3ZhbCBkMjsNCgkJaWYgKGZsYWdzICYgTURC X0FQUEVORCkgeyA8LS0tLS0tLS0gZmFsc2UgZmxhZ3MgPSAwDQoJCQlNREJfdmFsIGsyOw0KCQkJ cmMgPSBtZGJfY3Vyc29yX2xhc3QobWMsICZrMiwgJmQyKTsNCgkJCWlmIChyYyA9PSAwKSB7DQoJ CQkJcmMgPSBtYy0+bWNfZGJ4LT5tZF9jbXAoa2V5LCAmazIpOw0KCQkJCWlmIChyYyA+IDApIHsN CgkJCQkJcmMgPSBNREJfTk9URk9VTkQ7DQoJCQkJCW1jLT5tY19raVttYy0+bWNfdG9wXSsrOw0K CQkJCX0gZWxzZSB7DQoJCQkJCS8qIG5ldyBrZXkgaXMgPD0gbGFzdCBrZXkgKi8NCgkJCQkJcmMg PSBNREJfS0VZRVhJU1Q7DQoJCQkJfQ0KCQkJfQ0KCQl9IGVsc2UgezwtLS0tLS0tLSBleGVjdXRl ZA0KCQkJcmMgPSBtZGJfY3Vyc29yX3NldChtYywga2V5LCAmZDIsIE1EQl9TRVQsICZleGFjdCk7 IDwtLS0tLS0tLSByZXR1cm5zIC0zMDc5OCB3aGljaCBpcyBNREJfTk9URk9VTkQNCgkJfTwtLS0t LS0tLSBleGVjdXRlZA0KCQlpZiAoKGZsYWdzICYgTURCX05PT1ZFUldSSVRFKSAmJiByYyA9PSAw KSB7IDwtLS0tLS0tLSBmYWxzZSBmbGFncyA9IDANCgkJCURQUklOVEYoKCJkdXBsaWNhdGUga2V5 IFslc10iLCBES0VZKGtleSkpKTsNCgkJCSpkYXRhID0gZDI7DQoJCQlyZXR1cm4gTURCX0tFWUVY SVNUOw0KCQl9DQoJCWlmIChyYyAmJiByYyAhPSBNREJfTk9URk9VTkQpIDwtLS0tLS0tLSBmYWxz ZSByYyA9IE1EQl9OT1RGT1VORA0KCQkJcmV0dXJuIHJjOw0KCX08LS0tLS0tLS0gZXhlY3V0ZWQN Cg0KCWlmIChtYy0+bWNfZmxhZ3MgJiBDX0RFTCkNCgkJbWMtPm1jX2ZsYWdzIF49IENfREVMOw0K DQoJLyogQ3Vyc29yIGlzIHBvc2l0aW9uZWQsIGNoZWNrIGZvciByb29tIGluIHRoZSBkaXJ0eSBs aXN0ICovDQoJaWYgKCFub3NwaWxsKSB7IDwtLS0tLS0tLSB0cnVlDQoJCWlmIChmbGFncyAmIE1E Ql9NVUxUSVBMRSkgew0KCQkJcmRhdGEgPSAmeGRhdGE7DQoJCQl4ZGF0YS5tdl9zaXplID0gZGF0 YS0+bXZfc2l6ZSAqIGRjb3VudDsNCgkJfSBlbHNlIHsNCgkJCXJkYXRhID0gZGF0YTsgPC0tLS0t LS0tIGV4ZWN1dGVkDQoJCX0NCgkJaWYgKChyYzIgPSBtZGJfcGFnZV9zcGlsbChtYywga2V5LCBy ZGF0YSkpKSA8LS0tLS0tLS0gcmV0dXJuZWQgMCBvciB3b3VsZCBoYXZlIHJldHVybmVkDQoJCQly ZXR1cm4gcmMyOw0KCX08LS0tLS0tLS0gZXhlY3V0ZWQNCg0KCWlmIChyYyA9PSBNREJfTk9fUk9P VCkgeyA8LS0tLS0tLS0gZmFsc2UgDQoJCU1EQl9wYWdlICpucDsNCgkJLyogbmV3IGRhdGFiYXNl LCB3cml0ZSBhIHJvb3QgbGVhZiBwYWdlICovDQoJCURQVVRTKCJhbGxvY2F0aW5nIG5ldyByb290 IGxlYWYgcGFnZSIpOw0KCQlpZiAoKHJjMiA9IG1kYl9wYWdlX25ldyhtYywgUF9MRUFGLCAxLCAm bnApKSkgew0KCQkJcmV0dXJuIHJjMjsNCgkJfQ0KCQltZGJfY3Vyc29yX3B1c2gobWMsIG5wKTsN CgkJbWMtPm1jX2RiLT5tZF9yb290ID0gbnAtPm1wX3Bnbm87DQoJCW1jLT5tY19kYi0+bWRfZGVw dGgrKzsNCgkJKm1jLT5tY19kYmZsYWcgfD0gREJfRElSVFk7DQoJCWlmICgobWMtPm1jX2RiLT5t ZF9mbGFncyAmIChNREJfRFVQU09SVHxNREJfRFVQRklYRUQpKQ0KCQkJPT0gTURCX0RVUEZJWEVE KQ0KCQkJbnAtPm1wX2ZsYWdzIHw9IFBfTEVBRjI7DQoJCW1jLT5tY19mbGFncyB8PSBDX0lOSVRJ QUxJWkVEOw0KCX0gZWxzZSB7DQoJCS8qIG1ha2Ugc3VyZSBhbGwgY3Vyc29yIHBhZ2VzIGFyZSB3 cml0YWJsZSAqLw0KCQlyYzIgPSBtZGJfY3Vyc29yX3RvdWNoKG1jKTsgPC0tLS0tLS0tIGlzIHRo aXMgY2FsbGVkIGFuZCByZXR1cm5zIDAgYmVjYXVzZSBpdCBkb2VzIG5vdCByZXR1cm4NCgkJaWYg KHJjMikNCgkJCXJldHVybiByYzI7DQoJfSA8LS0tLS0tLS0gZXhlY3V0ZWQNCg0KCWluc2VydF9r ZXkgPSBpbnNlcnRfZGF0YSA9IHJjOyA8LS0tLS0tLS0gcmMgPSAtMzA3OTggb3IgTURCX05PVEZP VU5EIQ0KCWlmIChpbnNlcnRfa2V5KSB7IDwtLS0tLS0tLSB0cnVlDQoJCS8qIFRoZSBrZXkgZG9l cyBub3QgZXhpc3QgKi8NCgkJRFBSSU5URigoImluc2VydGluZyBrZXkgYXQgaW5kZXggJWkiLCBt Yy0+bWNfa2lbbWMtPm1jX3RvcF0pKTsNCgkJaWYgKChtYy0+bWNfZGItPm1kX2ZsYWdzICYgTURC X0RVUFNPUlQpICYmDQoJCQlMRUFGU0laRShrZXksIGRhdGEpID4gZW52LT5tZV9ub2RlbWF4KSA8 LS0tLS0tLS0gZmFsc2Ugc2luY2UgbWMtPm1jX2RiLT5tZF9mbGFncyA9IDANCgkJew0KCQkJLyog VG9vIGJpZyBmb3IgYSBub2RlLCBpbnNlcnQgaW4gc3ViLURCLiAgU2V0IHVwIGFuIGVtcHR5DQoJ CQkgKiAib2xkIHN1Yi1wYWdlIiBmb3IgcHJlcF9zdWJEQiB0byBleHBhbmQgdG8gYSBmdWxsIHBh Z2UuDQoJCQkgKi8NCgkJCWZwX2ZsYWdzID0gUF9MRUFGfFBfRElSVFk7DQoJCQlmcCA9IGVudi0+ bWVfcGJ1ZjsNCgkJCWZwLT5tcF9wYWQgPSBkYXRhLT5tdl9zaXplOyAvKiB1c2VkIGlmIE1EQl9E VVBGSVhFRCAqLw0KCQkJZnAtPm1wX2xvd2VyID0gZnAtPm1wX3VwcGVyID0gKFBBR0VIRFJTWi1Q QUdFQkFTRSk7DQoJCQlvbGRkYXRhLm12X3NpemUgPSBQQUdFSERSU1o7DQoJCQlnb3RvIHByZXBf c3ViREI7DQoJCX0gPC0tLS0tLS0tIHRoaXMgd2FzIG5vdCBleGVjdXRlZCwgYW5kIHRoZSB2ZXJ5 IGxvbmcgZXh0ZXJuYWwgZWxzZSBjb250aW5nZW5jeSBiZWxvdyBzaG91bGQgYmUgc2tpcHBlZCEN Cgl9IGVsc2Ugew0KCQkvKiB0aGVyZSdzIG9ubHkgYSBrZXkgYW55d2F5LCBzbyB0aGlzIGlzIGEg bm8tb3AgKi8NCgkJaWYgKElTX0xFQUYyKG1jLT5tY19wZ1ttYy0+bWNfdG9wXSkpIHsNCgkJCWNo YXIgKnB0cjsNCgkJCXVuc2lnbmVkIGludCBrc2l6ZSA9IG1jLT5tY19kYi0+bWRfcGFkOw0KCQkJ aWYgKGtleS0+bXZfc2l6ZSAhPSBrc2l6ZSkNCgkJCQlyZXR1cm4gTURCX0JBRF9WQUxTSVpFOw0K CQkJcHRyID0gTEVBRjJLRVkobWMtPm1jX3BnW21jLT5tY190b3BdLCBtYy0+bWNfa2lbbWMtPm1j X3RvcF0sIGtzaXplKTsNCgkJCW1lbWNweShwdHIsIGtleS0+bXZfZGF0YSwga3NpemUpOw0KZml4 X3BhcmVudDoNCgkJCS8qIGlmIG92ZXJ3cml0aW5nIHNsb3QgMCBvZiBsZWFmLCBuZWVkIHRvDQoJ CQkgKiB1cGRhdGUgYnJhbmNoIGtleSBpZiB0aGVyZSBpcyBhIHBhcmVudCBwYWdlDQoJCQkgKi8N CgkJCWlmIChtYy0+bWNfdG9wICYmICFtYy0+bWNfa2lbbWMtPm1jX3RvcF0pIHsNCgkJCQl1bnNp Z25lZCBzaG9ydCBkdG9wID0gMTsNCgkJCQltYy0+bWNfdG9wLS07DQoJCQkJLyogc2xvdCAwIGlz IGFsd2F5cyBhbiBlbXB0eSBrZXksIGZpbmQgcmVhbCBzbG90ICovDQoJCQkJd2hpbGUgKG1jLT5t Y190b3AgJiYgIW1jLT5tY19raVttYy0+bWNfdG9wXSkgew0KCQkJCQltYy0+bWNfdG9wLS07DQoJ CQkJCWR0b3ArKzsNCgkJCQl9DQoJCQkJaWYgKG1jLT5tY19raVttYy0+bWNfdG9wXSkNCgkJCQkJ cmMyID0gbWRiX3VwZGF0ZV9rZXkobWMsIGtleSk7DQoJCQkJZWxzZQ0KCQkJCQlyYzIgPSBNREJf U1VDQ0VTUzsNCgkJCQltYy0+bWNfdG9wICs9IGR0b3A7DQoJCQkJaWYgKHJjMikNCgkJCQkJcmV0 dXJuIHJjMjsNCgkJCX0NCgkJCXJldHVybiBNREJfU1VDQ0VTUzsNCgkJfQ0KDQptb3JlOg0KCQls ZWFmID0gTk9ERVBUUihtYy0+bWNfcGdbbWMtPm1jX3RvcF0sIG1jLT5tY19raVttYy0+bWNfdG9w XSk7DQoJCW9sZGRhdGEubXZfc2l6ZSA9IE5PREVEU1oobGVhZik7DQoJCW9sZGRhdGEubXZfZGF0 YSA9IE5PREVEQVRBKGxlYWYpOw0KDQoJCS8qIERCIGhhcyBkdXBzPyAqLw0KCQlpZiAoRl9JU1NF VChtYy0+bWNfZGItPm1kX2ZsYWdzLCBNREJfRFVQU09SVCkpIHsgDQoJCQkvKiBQcmVwYXJlIChz dWItKXBhZ2Uvc3ViLURCIHRvIGFjY2VwdCB0aGUgbmV3IGl0ZW0sDQoJCQkgKiBpZiBuZWVkZWQu ICBmcDogb2xkIHN1Yi1wYWdlIG9yIGEgaGVhZGVyIGZha2luZw0KCQkJICogaXQuICBtcDogbmV3 IChzdWItKXBhZ2UuICBvZmZzZXQ6IGdyb3d0aCBpbiBwYWdlDQoJCQkgKiBzaXplLiAgeGRhdGE6 IG5vZGUgZGF0YSB3aXRoIG5ldyBwYWdlIG9yIERCLg0KCQkJICovDQoJCQl1bnNpZ25lZAlpLCBv ZmZzZXQgPSAwOw0KCQkJbXAgPSBmcCA9IHhkYXRhLm12X2RhdGEgPSBlbnYtPm1lX3BidWY7DQoJ CQltcC0+bXBfcGdubyA9IG1jLT5tY19wZ1ttYy0+bWNfdG9wXS0+bXBfcGdubzsNCg0KCQkJLyog V2FzIGEgc2luZ2xlIGl0ZW0gYmVmb3JlLCBtdXN0IGNvbnZlcnQgbm93ICovDQoJCQlpZiAoIUZf SVNTRVQobGVhZi0+bW5fZmxhZ3MsIEZfRFVQREFUQSkpIHsNCgkJCQlNREJfY21wX2Z1bmMgKmRj bXA7DQoJCQkJLyogSnVzdCBvdmVyd3JpdGUgdGhlIGN1cnJlbnQgaXRlbSAqLw0KCQkJCWlmIChm bGFncyA9PSBNREJfQ1VSUkVOVCkNCgkJCQkJZ290byBjdXJyZW50Ow0KCQkJCWRjbXAgPSBtYy0+ bWNfZGJ4LT5tZF9kY21wOw0KI2lmIFVJTlRfTUFYIDwgU0laRV9NQVgNCgkJCQlpZiAoZGNtcCA9 PSBtZGJfY21wX2ludCAmJiBvbGRkYXRhLm12X3NpemUgPT0gc2l6ZW9mKHNpemVfdCkpDQoJCQkJ CWRjbXAgPSBtZGJfY21wX2Nsb25nOw0KI2VuZGlmDQoJCQkJLyogZG9lcyBkYXRhIG1hdGNoPyAq Lw0KCQkJCWlmICghZGNtcChkYXRhLCAmb2xkZGF0YSkpIHsNCgkJCQkJaWYgKGZsYWdzICYgKE1E Ql9OT0RVUERBVEF8TURCX0FQUEVORERVUCkpDQoJCQkJCQlyZXR1cm4gTURCX0tFWUVYSVNUOw0K CQkJCQkvKiBvdmVyd3JpdGUgaXQgKi8NCgkJCQkJZ290byBjdXJyZW50Ow0KCQkJCX0NCg0KCQkJ CS8qIEJhY2sgdXAgb3JpZ2luYWwgZGF0YSBpdGVtICovDQoJCQkJZGtleS5tdl9zaXplID0gb2xk ZGF0YS5tdl9zaXplOw0KCQkJCWRrZXkubXZfZGF0YSA9IG1lbWNweShmcCsxLCBvbGRkYXRhLm12 X2RhdGEsIG9sZGRhdGEubXZfc2l6ZSk7DQoNCgkJCQkvKiBNYWtlIHN1Yi1wYWdlIGhlYWRlciBm b3IgdGhlIGR1cCBpdGVtcywgd2l0aCBkdW1teSBib2R5ICovDQoJCQkJZnAtPm1wX2ZsYWdzID0g UF9MRUFGfFBfRElSVFl8UF9TVUJQOw0KCQkJCWZwLT5tcF9sb3dlciA9IChQQUdFSERSU1otUEFH RUJBU0UpOw0KCQkJCXhkYXRhLm12X3NpemUgPSBQQUdFSERSU1ogKyBka2V5Lm12X3NpemUgKyBk YXRhLT5tdl9zaXplOw0KCQkJCWlmIChtYy0+bWNfZGItPm1kX2ZsYWdzICYgTURCX0RVUEZJWEVE KSB7DQoJCQkJCWZwLT5tcF9mbGFncyB8PSBQX0xFQUYyOw0KCQkJCQlmcC0+bXBfcGFkID0gZGF0 YS0+bXZfc2l6ZTsNCgkJCQkJeGRhdGEubXZfc2l6ZSArPSAyICogZGF0YS0+bXZfc2l6ZTsJLyog bGVhdmUgc3BhY2UgZm9yIDIgbW9yZSAqLw0KCQkJCX0gZWxzZSB7DQoJCQkJCXhkYXRhLm12X3Np emUgKz0gMiAqIChzaXplb2YoaW5keF90KSArIE5PREVTSVpFKSArDQoJCQkJCQkoZGtleS5tdl9z aXplICYgMSkgKyAoZGF0YS0+bXZfc2l6ZSAmIDEpOw0KCQkJCX0NCgkJCQlmcC0+bXBfdXBwZXIg PSB4ZGF0YS5tdl9zaXplIC0gUEFHRUJBU0U7DQoJCQkJb2xkZGF0YS5tdl9zaXplID0geGRhdGEu bXZfc2l6ZTsgLyogcHJldGVuZCBvbGRkYXRhIGlzIGZwICovDQoJCQl9IGVsc2UgaWYgKGxlYWYt Pm1uX2ZsYWdzICYgRl9TVUJEQVRBKSB7DQoJCQkJLyogRGF0YSBpcyBvbiBzdWItREIsIGp1c3Qg c3RvcmUgaXQgKi8NCgkJCQlmbGFncyB8PSBGX0RVUERBVEF8Rl9TVUJEQVRBOw0KCQkJCWdvdG8g cHV0X3N1YjsNCgkJCX0gZWxzZSB7DQoJCQkJLyogRGF0YSBpcyBvbiBzdWItcGFnZSAqLw0KCQkJ CWZwID0gb2xkZGF0YS5tdl9kYXRhOw0KCQkJCXN3aXRjaCAoZmxhZ3MpIHsNCgkJCQlkZWZhdWx0 Og0KCQkJCQlpZiAoIShtYy0+bWNfZGItPm1kX2ZsYWdzICYgTURCX0RVUEZJWEVEKSkgew0KCQkJ CQkJb2Zmc2V0ID0gRVZFTihOT0RFU0laRSArIHNpemVvZihpbmR4X3QpICsNCgkJCQkJCQlkYXRh LT5tdl9zaXplKTsNCgkJCQkJCWJyZWFrOw0KCQkJCQl9DQoJCQkJCW9mZnNldCA9IGZwLT5tcF9w YWQ7DQoJCQkJCWlmIChTSVpFTEVGVChmcCkgPCBvZmZzZXQpIHsNCgkJCQkJCW9mZnNldCAqPSA0 OyAvKiBzcGFjZSBmb3IgNCBtb3JlICovDQoJCQkJCQlicmVhazsNCgkJCQkJfQ0KCQkJCQkvKiBG QUxMVEhSVTogQmlnIGVub3VnaCBNREJfRFVQRklYRUQgc3ViLXBhZ2UgKi8NCgkJCQljYXNlIE1E Ql9DVVJSRU5UOg0KCQkJCQlmcC0+bXBfZmxhZ3MgfD0gUF9ESVJUWTsNCgkJCQkJQ09QWV9QR05P KGZwLT5tcF9wZ25vLCBtcC0+bXBfcGdubyk7DQoJCQkJCW1jLT5tY194Y3Vyc29yLT5teF9jdXJz b3IubWNfcGdbMF0gPSBmcDsNCgkJCQkJZmxhZ3MgfD0gRl9EVVBEQVRBOw0KCQkJCQlnb3RvIHB1 dF9zdWI7DQoJCQkJfQ0KCQkJCXhkYXRhLm12X3NpemUgPSBvbGRkYXRhLm12X3NpemUgKyBvZmZz ZXQ7DQoJCQl9DQoNCgkJCWZwX2ZsYWdzID0gZnAtPm1wX2ZsYWdzOw0KCQkJaWYgKE5PREVTSVpF ICsgTk9ERUtTWihsZWFmKSArIHhkYXRhLm12X3NpemUgPiBlbnYtPm1lX25vZGVtYXgpIHsNCgkJ CQkJLyogVG9vIGJpZyBmb3IgYSBzdWItcGFnZSwgY29udmVydCB0byBzdWItREIgKi8NCgkJCQkJ ZnBfZmxhZ3MgJj0gflBfU1VCUDsNCnByZXBfc3ViREI6DQoJCQkJCWlmIChtYy0+bWNfZGItPm1k X2ZsYWdzICYgTURCX0RVUEZJWEVEKSB7DQoJCQkJCQlmcF9mbGFncyB8PSBQX0xFQUYyOw0KCQkJ CQkJZHVtbXkubWRfcGFkID0gZnAtPm1wX3BhZDsNCgkJCQkJCWR1bW15Lm1kX2ZsYWdzID0gTURC X0RVUEZJWEVEOw0KCQkJCQkJaWYgKG1jLT5tY19kYi0+bWRfZmxhZ3MgJiBNREJfSU5URUdFUkRV UCkNCgkJCQkJCQlkdW1teS5tZF9mbGFncyB8PSBNREJfSU5URUdFUktFWTsNCgkJCQkJfSBlbHNl IHsNCgkJCQkJCWR1bW15Lm1kX3BhZCA9IDA7DQoJCQkJCQlkdW1teS5tZF9mbGFncyA9IDA7DQoJ CQkJCX0NCgkJCQkJZHVtbXkubWRfZGVwdGggPSAxOw0KCQkJCQlkdW1teS5tZF9icmFuY2hfcGFn ZXMgPSAwOw0KCQkJCQlkdW1teS5tZF9sZWFmX3BhZ2VzID0gMTsNCgkJCQkJZHVtbXkubWRfb3Zl cmZsb3dfcGFnZXMgPSAwOw0KCQkJCQlkdW1teS5tZF9lbnRyaWVzID0gTlVNS0VZUyhmcCk7DQoJ CQkJCXhkYXRhLm12X3NpemUgPSBzaXplb2YoTURCX2RiKTsNCgkJCQkJeGRhdGEubXZfZGF0YSA9 ICZkdW1teTsNCgkJCQkJaWYgKChyYyA9IG1kYl9wYWdlX2FsbG9jKG1jLCAxLCAmbXApKSkNCgkJ CQkJCXJldHVybiByYzsNCgkJCQkJb2Zmc2V0ID0gZW52LT5tZV9wc2l6ZSAtIG9sZGRhdGEubXZf c2l6ZTsNCgkJCQkJZmxhZ3MgfD0gRl9EVVBEQVRBfEZfU1VCREFUQTsNCgkJCQkJZHVtbXkubWRf cm9vdCA9IG1wLT5tcF9wZ25vOw0KCQkJCQlzdWJfcm9vdCA9IG1wOw0KCQkJfQ0KCQkJaWYgKG1w ICE9IGZwKSB7DQoJCQkJbXAtPm1wX2ZsYWdzID0gZnBfZmxhZ3MgfCBQX0RJUlRZOw0KCQkJCW1w LT5tcF9wYWQgICA9IGZwLT5tcF9wYWQ7DQoJCQkJbXAtPm1wX2xvd2VyID0gZnAtPm1wX2xvd2Vy Ow0KCQkJCW1wLT5tcF91cHBlciA9IGZwLT5tcF91cHBlciArIG9mZnNldDsNCgkJCQlpZiAoZnBf ZmxhZ3MgJiBQX0xFQUYyKSB7DQoJCQkJCW1lbWNweShNRVRBREFUQShtcCksIE1FVEFEQVRBKGZw KSwgTlVNS0VZUyhmcCkgKiBmcC0+bXBfcGFkKTsNCgkJCQl9IGVsc2Ugew0KCQkJCQltZW1jcHko KGNoYXIgKiltcCArIG1wLT5tcF91cHBlciArIFBBR0VCQVNFLCAoY2hhciAqKWZwICsgZnAtPm1w X3VwcGVyICsgUEFHRUJBU0UsDQoJCQkJCQlvbGRkYXRhLm12X3NpemUgLSBmcC0+bXBfdXBwZXIg LSBQQUdFQkFTRSk7DQoJCQkJCWZvciAoaT0wOyBpPE5VTUtFWVMoZnApOyBpKyspDQoJCQkJCQlt cC0+bXBfcHRyc1tpXSA9IGZwLT5tcF9wdHJzW2ldICsgb2Zmc2V0Ow0KCQkJCX0NCgkJCX0NCg0K CQkJcmRhdGEgPSAmeGRhdGE7DQoJCQlmbGFncyB8PSBGX0RVUERBVEE7DQoJCQlkb19zdWIgPSAx Ow0KCQkJaWYgKCFpbnNlcnRfa2V5KQ0KCQkJCW1kYl9ub2RlX2RlbChtYywgMCk7DQoJCQlnb3Rv IG5ld19zdWI7DQoJCX0NCmN1cnJlbnQ6DQoJCS8qIExNREIgcGFzc2VzIEZfU1VCREFUQSBpbiAn ZmxhZ3MnIHRvIHdyaXRlIGEgREIgcmVjb3JkICovDQoJCWlmICgobGVhZi0+bW5fZmxhZ3MgXiBm bGFncykgJiBGX1NVQkRBVEEpDQoJCQlyZXR1cm4gTURCX0lOQ09NUEFUSUJMRTsNCgkJLyogb3Zl cmZsb3cgcGFnZSBvdmVyd3JpdGVzIG5lZWQgc3BlY2lhbCBoYW5kbGluZyAqLw0KCQlpZiAoRl9J U1NFVChsZWFmLT5tbl9mbGFncywgRl9CSUdEQVRBKSkgeyA8LS0tLS0tLS0gaXMgdGhpcyB0YWtl bj8NCgkJCU1EQl9wYWdlICpvbXA7DQoJCQlwZ25vX3QgcGc7DQoJCQlpbnQgbGV2ZWwsIG92cGFn ZXMsIGRwYWdlcyA9IE9WUEFHRVMoZGF0YS0+bXZfc2l6ZSwgZW52LT5tZV9wc2l6ZSk7DQoNCgkJ CW1lbWNweSgmcGcsIG9sZGRhdGEubXZfZGF0YSwgc2l6ZW9mKHBnKSk7DQoJCQlpZiAoKHJjMiA9 IG1kYl9wYWdlX2dldChtYy0+bWNfdHhuLCBwZywgJm9tcCwgJmxldmVsKSkgIT0gMCkNCgkJCQly ZXR1cm4gcmMyOw0KCQkJb3ZwYWdlcyA9IG9tcC0+bXBfcGFnZXM7DQoNCgkJCS8qIElzIHRoZSBv diBwYWdlIGxhcmdlIGVub3VnaD8gKi8NCgkJCWlmIChvdnBhZ2VzID49IGRwYWdlcykgew0KCQkJ ICBpZiAoIShvbXAtPm1wX2ZsYWdzICYgUF9ESVJUWSkgJiYNCgkJCQkgIChsZXZlbCB8fCAoZW52 LT5tZV9mbGFncyAmIE1EQl9XUklURU1BUCkpKQ0KCQkJICB7DQoJCQkJcmMgPSBtZGJfcGFnZV91 bnNwaWxsKG1jLT5tY190eG4sIG9tcCwgJm9tcCk7IDwtLS0tLS0tLSBpcyB0aGlzIHRha2VuPw0K CQkJCWlmIChyYykNCgkJCQkJcmV0dXJuIHJjOw0KCQkJCWxldmVsID0gMDsJCS8qIGRpcnR5IGlu IHRoaXMgdHhuIG9yIGNsZWFuICovDQoJCQkgIH0NCgkJCSAgLyogSXMgaXQgZGlydHk/ICovDQoJ CQkgIGlmIChvbXAtPm1wX2ZsYWdzICYgUF9ESVJUWSkgew0KCQkJCS8qIHllcywgb3ZlcndyaXRl IGl0LiBOb3RlIGluIHRoaXMgY2FzZSB3ZSBkb24ndA0KCQkJCSAqIGJvdGhlciB0byB0cnkgc2hy aW5raW5nIHRoZSBwYWdlIGlmIHRoZSBuZXcgZGF0YQ0KCQkJCSAqIGlzIHNtYWxsZXIgdGhhbiB0 aGUgb3ZlcmZsb3cgdGhyZXNob2xkLg0KCQkJCSAqLw0KCQkJCWlmIChsZXZlbCA+IDEpIHsNCgkJ CQkJLyogSXQgaXMgd3JpdGFibGUgb25seSBpbiBhIHBhcmVudCB0eG4gKi8NCgkJCQkJc2l6ZV90 IHN6ID0gKHNpemVfdCkgZW52LT5tZV9wc2l6ZSAqIG92cGFnZXMsIG9mZjsNCgkJCQkJTURCX3Bh Z2UgKm5wID0gbWRiX3BhZ2VfbWFsbG9jKG1jLT5tY190eG4sIG92cGFnZXMpOw0KCQkJCQlNREJf SUQyIGlkMjsNCgkJCQkJaWYgKCFucCkNCgkJCQkJCXJldHVybiBFTk9NRU07DQoJCQkJCWlkMi5t aWQgPSBwZzsNCgkJCQkJaWQyLm1wdHIgPSBucDsNCgkJCQkJLyogTm90ZSAtIHRoaXMgcGFnZSBp cyBhbHJlYWR5IGNvdW50ZWQgaW4gcGFyZW50J3MgZGlydHlfcm9vbSAqLw0KCQkJCQlyYzIgPSBt ZGJfbWlkMmxfaW5zZXJ0KG1jLT5tY190eG4tPm10X3UuZGlydHlfbGlzdCwgJmlkMik7DQoJCQkJ CW1kYl9jYXNzZXJ0KG1jLCByYzIgPT0gMCk7DQoJCQkJCWlmICghKGZsYWdzICYgTURCX1JFU0VS VkUpKSB7DQoJCQkJCQkvKiBDb3B5IGVuZCBvZiBwYWdlLCBhZGp1c3RpbmcgYWxpZ25tZW50IHNv DQoJCQkJCQkgKiBjb21waWxlciBtYXkgY29weSB3b3JkcyBpbnN0ZWFkIG9mIGJ5dGVzLg0KCQkJ CQkJICovDQoJCQkJCQlvZmYgPSAoUEFHRUhEUlNaICsgZGF0YS0+bXZfc2l6ZSkgJiAtc2l6ZW9m KHNpemVfdCk7DQoJCQkJCQltZW1jcHkoKHNpemVfdCAqKSgoY2hhciAqKW5wICsgb2ZmKSwNCgkJ CQkJCQkoc2l6ZV90ICopKChjaGFyICopb21wICsgb2ZmKSwgc3ogLSBvZmYpOw0KCQkJCQkJc3og PSBQQUdFSERSU1o7DQoJCQkJCX0NCgkJCQkJbWVtY3B5KG5wLCBvbXAsIHN6KTsgLyogQ29weSBi ZWdpbm5pbmcgb2YgcGFnZSAqLw0KCQkJCQlvbXAgPSBucDsNCgkJCQl9DQoJCQkJU0VURFNaKGxl YWYsIGRhdGEtPm12X3NpemUpOw0KCQkJCWlmIChGX0lTU0VUKGZsYWdzLCBNREJfUkVTRVJWRSkp DQoJCQkJCWRhdGEtPm12X2RhdGEgPSBNRVRBREFUQShvbXApOw0KCQkJCWVsc2UNCgkJCQkJbWVt Y3B5KE1FVEFEQVRBKG9tcCksIGRhdGEtPm12X2RhdGEsIGRhdGEtPm12X3NpemUpOw0KCQkJCXJl dHVybiBNREJfU1VDQ0VTUzsNCgkJCSAgfQ0KCQkJfQ0KCQkJaWYgKChyYzIgPSBtZGJfb3ZwYWdl X2ZyZWUobWMsIG9tcCkpICE9IE1EQl9TVUNDRVNTKQ0KCQkJCXJldHVybiByYzI7DQoJCX0gZWxz ZSBpZiAoZGF0YS0+bXZfc2l6ZSA9PSBvbGRkYXRhLm12X3NpemUpIHsNCgkJCS8qIHNhbWUgc2l6 ZSwganVzdCByZXBsYWNlIGl0LiBOb3RlIHRoYXQgd2UgY291bGQNCgkJCSAqIGFsc28gcmV1c2Ug dGhpcyBub2RlIGlmIHRoZSBuZXcgZGF0YSBpcyBzbWFsbGVyLA0KCQkJICogYnV0IGluc3RlYWQg d2Ugb3B0IHRvIHNocmluayB0aGUgbm9kZSBpbiB0aGF0IGNhc2UuDQoJCQkgKi8NCgkJCWlmIChG X0lTU0VUKGZsYWdzLCBNREJfUkVTRVJWRSkpDQoJCQkJZGF0YS0+bXZfZGF0YSA9IG9sZGRhdGEu bXZfZGF0YTsNCgkJCWVsc2UgaWYgKCEobWMtPm1jX2ZsYWdzICYgQ19TVUIpKQ0KCQkJCW1lbWNw eShvbGRkYXRhLm12X2RhdGEsIGRhdGEtPm12X2RhdGEsIGRhdGEtPm12X3NpemUpOw0KCQkJZWxz ZSB7DQoJCQkJbWVtY3B5KE5PREVLRVkobGVhZiksIGtleS0+bXZfZGF0YSwga2V5LT5tdl9zaXpl KTsNCgkJCQlnb3RvIGZpeF9wYXJlbnQ7DQoJCQl9DQoJCQlyZXR1cm4gTURCX1NVQ0NFU1M7DQoJ CX0NCgkJbWRiX25vZGVfZGVsKG1jLCAwKTsNCgl9IDwtLS0tLS0tLSBUaGlzIGVudGlyZSBlbHNl IGNvbnRpbmdlbmN5IHdhcyBsaWtlbHkgc2tpcHBlZCEhDQoNCglyZGF0YSA9IGRhdGE7DQoNCm5l d19zdWI6DQoJbmZsYWdzID0gZmxhZ3MgJiBOT0RFX0FERF9GTEFHUzsNCgluc2l6ZSA9IElTX0xF QUYyKG1jLT5tY19wZ1ttYy0+bWNfdG9wXSkgPyBrZXktPm12X3NpemUgOiBtZGJfbGVhZl9zaXpl KGVudiwga2V5LCByZGF0YSk7IDwtLS0tLS0tLSBTRUdWDQouLi4JDQoNClRoaXMgbWF5IGJlIGFu IGlkZW50aWZpYWJsZSBidWcgaW4gbWRiX2N1cnNvcl9wdXQoKSBhcyBJdCBkb2VzIG5vdCBzYWZl bHkgaGFuZGxlIHRoZSBjYXNlIHdoZW4gdGhlIG5vdGVkIGNhbGwgdG8gbWRiX2N1cnNvcl9zZXQo KSByZXR1cm5zIE1EQl9OT1RGT1VORCAoLTMwNzk4KSENCg0KSXQgZXhwbGljaXRseSBmYWlscyB0 byBwcmVlbXB0aXZlbHkgZXhpdCBieSByZXR1cm5pbmcgYW4gZXJyb3IgaW4gdGhpcyBjYXNlLg0K DQpUaGUgdmFsdWUgb2YgLTMwNzk4IGlzIGluIHR1cm4gYXNzaWduZWQgdG8gaW5zZXJ0X2tleSwg d2hpY2ggaXMgdGhlcmVmb3JlIG5vbi1udWxsIGluIHRoZSBjb25kaXRpb24gb2YgdGhlIHN1YnNl cXVlbnQgaWYgc3RhdGVtZW50Lg0KDQoJaW5zZXJ0X2tleSA9IGluc2VydF9kYXRhID0gcmM7IDwt LS0tLS0tLSBpbnNlcnRfa2V5ID0gLTMwNzk4IG9yIE1EQl9OT1RGT1VORCENCglpZiAoaW5zZXJ0 X2tleSkgeyA8LS0tLS0tLS0gdHJ1ZQ0KCQkvKiBUaGUga2V5IGRvZXMgbm90IGV4aXN0ICovDQoJ CURQUklOVEYoKCJpbnNlcnRpbmcga2V5IGF0IGluZGV4ICVpIiwgbWMtPm1jX2tpW21jLT5tY190 b3BdKSk7DQoJCWlmICgobWMtPm1jX2RiLT5tZF9mbGFncyAmIE1EQl9EVVBTT1JUKSAmJg0KCQkJ TEVBRlNJWkUoa2V5LCBkYXRhKSA+IGVudi0+bWVfbm9kZW1heCkgPC0tLS0tLS0tIGZhbHNlIHNp bmNlIG1jLT5tY19kYi0+bWRfZmxhZ3MgPSAwDQoJCXsNCgkJCS8qIFRvbyBiaWcgZm9yIGEgbm9k ZSwgaW5zZXJ0IGluIHN1Yi1EQi4gIFNldCB1cCBhbiBlbXB0eQ0KCQkJICogIm9sZCBzdWItcGFn ZSIgZm9yIHByZXBfc3ViREIgdG8gZXhwYW5kIHRvIGEgZnVsbCBwYWdlLg0KCQkJICovDQoJCQlm cF9mbGFncyA9IFBfTEVBRnxQX0RJUlRZOw0KCQkJZnAgPSBlbnYtPm1lX3BidWY7DQoJCQlmcC0+ bXBfcGFkID0gZGF0YS0+bXZfc2l6ZTsgLyogdXNlZCBpZiBNREJfRFVQRklYRUQgKi8NCgkJCWZw LT5tcF9sb3dlciA9IGZwLT5tcF91cHBlciA9IChQQUdFSERSU1otUEFHRUJBU0UpOw0KCQkJb2xk ZGF0YS5tdl9zaXplID0gUEFHRUhEUlNaOw0KCQkJZ290byBwcmVwX3N1YkRCOw0KCQl9IDwtLS0t LS0tLSB0aGlzIHdhcyBub3QgZXhlY3V0ZWQsIGFuZCB0aGUgdmVyeSBsb25nIGVsc2UgY29udGlu Z2VuY3kgYmVsb3cgaXMgc2tpcHBlZCENCgl9IGVsc2Ugew0KU2luY2UgbWMtPm1jX2RiLT5tZF9m bGFncyA9IDAsIHRoZSBjb250aW5nZW5jeSBpbiB0aGUgZW1iZWRkZWQgaWYgc3RhdGVtZW50IGZh aWxzLg0KDQpUaGUgc2VnbWVudGF0aW9uIGZhdWx0IG9jY3VycyBkdWUgdG8gdGhlIGNhbGwgb2Yg SVNfTEVBRjIgb24gdGhlIG51bGwgcG9pbnRlciBtYy0+bWNfcGdbMF0uDQoNCglyZGF0YSA9IGRh dGE7DQoNCm5ld19zdWI6DQoJbmZsYWdzID0gZmxhZ3MgJiBOT0RFX0FERF9GTEFHUzsNCgluc2l6 ZSA9IElTX0xFQUYyKG1jLT5tY19wZ1ttYy0+bWNfdG9wXSkgPyBrZXktPm12X3NpemUgOiBtZGJf bGVhZl9zaXplKGVudiwga2V5LCByZGF0YSk7IDwtLS0tLS0tLSBTRUdWDQoNCg0KUmVnYXJkcywN ClJvYmlucy4NCg0KLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCkZyb206IEhvd2FyZCBDaHUg PGh5Y0BzeW1hcy5jb20+IA0KU2VudDogTW9uZGF5LCBKdW5lIDE3LCAyMDE5IDU6NTUgUE0NClRv OiBSb2JpbnMgR2VvcmdlIDxncm9iaW5zQHB1bHNlc2VjdXJlLm5ldD47IG9wZW5sZGFwLWl0c0BP cGVuTERBUC5vcmcNClN1YmplY3Q6IFJlOiAoSVRTIzkwMzcpIG9ic2VydmluZyBjcmFzaCBpbiBt ZGJfY3Vyc29yX3B1dCgpDQoNCkNBVVRJT046IFRoaXMgZW1haWwgb3JpZ2luYXRlZCBmcm9tIG91 dHNpZGUgb2YgUHVsc2VTZWN1cmUuIERvIG5vdCBjbGljayBsaW5rcyBvciBvcGVuIGF0dGFjaG1l bnRzIHVubGVzcyB5b3UgcmVjb2duaXplIHRoZSBzZW5kZXIgYW5kIGtub3cgdGhlIGNvbnRlbnQg aXMgc2FmZS4NCg0KDQpncm9iaW5zQHB1bHNlc2VjdXJlLm5ldCB3cm90ZToNCj4gRnVsbF9OYW1l OiBSb2JpbnMgR2VvcmdlDQo+IFZlcnNpb246IDAuOS4xOA0KPiBPUzogQ2VudG9zDQo+IFVSTDog ZnRwOi8vZnRwLm9wZW5sZGFwLm9yZy9pbmNvbWluZy8NCj4gU3VibWlzc2lvbiBmcm9tOiAoTlVM TCkgKDEyMS4yNDQuMTU0LjEzNCkNCj4NCj4NCj4gSSBhbSBzZWVpbmcgTE1EQiBjcmFzaCwgcGxl YXNlIGZpbmQgdGhlIHN0YWNrIHRyYWNlLg0KPg0KPiAjMCAgMHgwMDllMzYyZSBpbiBtZGJfY3Vy c29yX3B1dCAobWM9MHhmZmQ1OWJiOCwga2V5PTB4ZmZkNTlkMTQsIA0KPiBkYXRhPTB4ZmZkNTlk MGMsIGZsYWdzPTApIGF0IG1kYi5jOjY2ODgNCj4gIzEgIDB4MDA5ZTQ4ZWMgaW4gbWRiX3B1dCAo dHhuPTB4ZDc0ZDMwMDgsIGRiaT0yLCBrZXk9MHhmZmQ1OWQxNCwgDQo+IGRhdGE9MHhmZmQ1OWQw YywgZmxhZ3M9MCkgYXQgbWRiLmM6ODc3MQ0KPiAjMiAgMHgwMGY1Y2VmOCBpbiBMTURCOjpMTURC Q29udGV4dDo6Y3JlYXRlU2Vzc2lvbiAodGhpcz0weDgwY2M2MTAsIA0KPiBzaWQ9MHg4MGNkMGZj ICJzaWQxNDIyNDE5ZTVmZDZjZDIyNGYyNzhkNzQyOTNjNTBmNWVmOTY1OTM3MDAwMDAwMDArIikg DQo+IGF0DQo+IGxtZGJpbnQuY2M6NDYwDQo+ICMzICAweDA4MDUzOThlIGluIHVwZGF0ZUxNREIg KHJlcXVlc3Q9Li4uLCBmb3J3YXJkPUAweGZmZDU5ZmZmKSBhdA0KPiBzZXNzaW9uc2VydmVyLmNj OjExNTQNCj4gIzQgIHByb2Nlc3NSZXF1ZXN0V2l0aG91dFJlc3BvbnNlIChyZXF1ZXN0PS4uLiwg Zm9yd2FyZD1AMHhmZmQ1OWZmZikgDQo+IGF0DQo+IHNlc3Npb25zZXJ2ZXIuY2M6MTI0MA0KPiAj NSAgMHgwODA1NjkzNiBpbiBaU3ViSGFuZGxlcjo6aW9SZWFkeSAodGhpcz0weGZmZDVhYTFjLCBm ZD0yMCkgYXQNCj4gc2Vzc2lvbnNlcnZlci5jYzoxNTAxDQo+ICM2ICAweDAwZjg3MGQyIGluIHJ1 bkNvcmVEaXNwYXRjaGVyIChkZWZhdWx0X3Q9PHZhbHVlIG9wdGltaXplZCBvdXQ+LCANCj4gZmxh Z3M9LTEpIGF0IGZkcy5jYzo4ODkNCj4gIzcgIDB4MDBmODdiMzcgaW4gRFNFdm50RmRzOjpydW5E aXNwYXRjaGVyICgpIGF0IGZkcy5jYzo5NDUNCj4gIzggIDB4MDgwNTYyNDggaW4gbWFpbiAoKSBh dCBzZXNzaW9uc2VydmVyLmNjOjE4MjANCj4NCj4gQW55Ym9keSBoYXMgc2VlbiBzaW1pbGFyIGlz c3VlIGJlZm9yZT8NCg0KRG9lc24ndCBzb3VuZCBmYW1pbGlhci4gQnV0IDAuOS4xOCBpcyBxdWl0 ZSBvbGQuIElmIHlvdSBjYW4gcmVwcm9kdWNlIHRoaXMgaXNzdWUgaW4gMC45LjIzIHRoZW4gd2Un bGwgdGFrZSBhIGxvb2suIFRlc3QgY29kZSB0byByZXByb2R1Y2UgdGhlIHByb2JsZW0gd291bGQg YWxzbyBiZSBuZWVkZWQuDQoNCi0tDQogIC0tIEhvd2FyZCBDaHUNCiAgQ1RPLCBTeW1hcyBDb3Jw LiAgICAgICAgICAgaHR0cHM6Ly9uYW0wNC5zYWZlbGlua3MucHJvdGVjdGlvbi5vdXRsb29rLmNv bS8/dXJsPWh0dHAlM0ElMkYlMkZ3d3cuc3ltYXMuY29tJmFtcDtkYXRhPTAyJTdDMDElN0MlN0Mw MDNiOTA3ZDJkZGU0NDAyOGFkYzA4ZDZmMzFlZDljNCU3QzMyOTBhOTE3OWRkNjQzZGI4NDNiYTNl Mzc2ZjlmOTZjJTdDMCU3QzAlN0M2MzY5NjM3MTExNjkyNTE5NDkmYW1wO3NkYXRhPTJ1R1JvTUdW ZGhDbmlVRGhYZnpQZ0FtaVYxa3Q2NmRXN3U4ckRtR3Q2YVklM0QmYW1wO3Jlc2VydmVkPTANCiAg RGlyZWN0b3IsIEhpZ2hsYW5kIFN1biAgICAgaHR0cHM6Ly9uYW0wNC5zYWZlbGlua3MucHJvdGVj dGlvbi5vdXRsb29rLmNvbS8/dXJsPWh0dHAlM0ElMkYlMkZoaWdobGFuZHN1bi5jb20lMkZoeWMl MkYmYW1wO2RhdGE9MDIlN0MwMSU3QyU3QzAwM2I5MDdkMmRkZTQ0MDI4YWRjMDhkNmYzMWVkOWM0 JTdDMzI5MGE5MTc5ZGQ2NDNkYjg0M2JhM2UzNzZmOWY5NmMlN0MwJTdDMCU3QzYzNjk2MzcxMTE2 OTI1MTk0OSZhbXA7c2RhdGE9MyUyRkxSUE0lMkJIdElhUm9LYU16JTJGMjclMkZvNFpUOFhoVGNG SlR2eXBCc3N4MHI4JTNEJmFtcDtyZXNlcnZlZD0wDQogIENoaWVmIEFyY2hpdGVjdCwgT3BlbkxE QVAgIGh0dHBzOi8vbmFtMDQuc2FmZWxpbmtzLnByb3RlY3Rpb24ub3V0bG9vay5jb20vP3VybD1o dHRwJTNBJTJGJTJGd3d3Lm9wZW5sZGFwLm9yZyUyRnByb2plY3QlMkYmYW1wO2RhdGE9MDIlN0Mw MSU3QyU3QzAwM2I5MDdkMmRkZTQ0MDI4YWRjMDhkNmYzMWVkOWM0JTdDMzI5MGE5MTc5ZGQ2NDNk Yjg0M2JhM2UzNzZmOWY5NmMlN0MwJTdDMCU3QzYzNjk2MzcxMTE2OTI1MTk0OSZhbXA7c2RhdGE9 N2lUWE1nMk5TZVhHJTJGS3A5cWFIUVBPQyUyRktiakNWJTJCSzNKcyUyRmVuaEZwTEdZJTNEJmFt cDtyZXNlcnZlZD0wDQo=

1 0

Re: (ITS#9037) observing crash in mdb_cursor_put()
by myk＠mykzilla.org 17 Jun '19

17 Jun '19

hyc(a)symas.com wrote on 2019-06-17 05:25: > grobins(a)pulsesecure.net wrote: >> I am seeing LMDB crash, please find the stack trace. >> >> #0 0x009e362e in mdb_cursor_put (mc=0xffd59bb8, key=0xffd59d14, >> data=0xffd59d0c, flags=0) at mdb.c:6688 >> #1 0x009e48ec in mdb_put (txn=0xd74d3008, dbi=2, key=0xffd59d14, >> data=0xffd59d0c, flags=0) at mdb.c:8771 >> … >> >> Anybody has seen similar issue before? > Doesn't sound familiar. But 0.9.18 is quite old. If you can reproduce this > issue in 0.9.23 then we'll take a look. Test code to reproduce the problem > would also be needed. I'm seeing this in Firefox, which uses LMDB 0.9.23 (with minor changes). Line 6688 in 0.9.18 occurs at line 6938 in 0.9.23 (line 6937 in Firefox since we landed ITS#9030), and that's where we see crashes. I haven't reported it here yet because I haven't been able to confirm that it's a bug in LMDB as opposed to my own code. In fact I haven't been able to reproduce it at all, I've only seen it in crash reports submitted by Firefox installations (almost exclusively on Windows). So I don't have test code to reproduce the problem. Nevertheless, FWIW, here's the Firefox bug that tracks the issue: https://bugzilla.mozilla.org/show_bug.cgi?id=1538541. And here are its crash reports: https://crash-stats.mozilla.org/signature/?signature=mdb_cursor_put (only the last seven days of reports shown by default, but this has been happening since we started using LMDB in Firefox nightly builds a couple of months ago). I've examined some of the dumps, and mc->mc_top is 0 when the crash occurs, while mc->mc_pg[0] is a NULL pointer. So presumably the crash occurs because IS_LEAF2 tries to dereference mc->mc_pg[mc->mc_top]. Further investigation shows that insert_data and insert_key are both MDB_NOTFOUND, and flags is 0, so it isn't MDB_CURRENT, nor does it contain MDB_APPEND. If I understand the code in mdb_cursor_put correctly, this means that mdb_cursor_set was called on line 6614. And mdb_cursor_set is in the stack of another crash I've been investigating in mdb_page_search_root (https://bugzilla.mozilla.org/show_bug.cgi?id=1550174, https://crash-stats.mozilla.org/signature/?signature=mdb_page_search_root), which happens on all of Firefox's primary platforms (Windows, macOS, Linux). But I haven't been able to reproduce that one either, on any of those platforms, so I have no idea if they're related (nor if mdb_cursor_set is even implicated in this crash). And I still can't say that either is an LMDB bug. -myk

1 0

Re: (ITS#7996) Tighten race in ldap_int_initialize
by quanah＠symas.com 17 Jun '19

17 Jun '19

--On Monday, June 17, 2019 7:16 PM +0200 Armin T=C3=BCting=20 <Armin.Tueting(a)tueting-online.com> wrote: >> I.e., it started and then got as far as reading your ldap.conf file. >> What is the contents of ldap.conf? > Attached 'ldap.conf'. Nothing unusual... > >> Have you run the test suite (make test)? Does it pass? fail? > Attached 'make_test.txt'. As far as I can see - it has been passed. Ok, so make test passes without issue, so it would appear there's something = specific with your configuration that is triggering the problem. Would you = be able to provide your slapd configuration (minus any passwords and the=20 like)? Additionally, if you could get a full gdb backtrace of the hung slapd=20 process that would be useful as well. I.e.: start up slapd gdb /path/to/slapd <pid #> at the gdb prompt: thr apply all bt full Thanks! --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

Re: (ITS#9017) Improving performance of commit sync in Windows
by kriszyp＠gmail.com 17 Jun '19

17 Jun '19

--000000000000db50ce058b89ac2f Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Is this still being considered/reviewed? Let me know if there are any other changes you would like me to make. This patch has continued to yield significant and reliable performance improvements for us, and seems like it would be nice for this to be available for other Windows users. On Fri, May 3, 2019 at 3:52 PM Kris Zyp <kriszyp(a)gmail.com> wrote: > For the sake of putting this in the email thread (other code discussion i= n > GitHub), here is the latest squashed commit of the proposed patch (with t= he > on-demand, retained overlapped array to reduce re-malloc and opening even= t > handles): > https://github.com/kriszyp/node-lmdb/commit/726a9156662c703bf3d453aab75ee= 222072b990f > > > > Thanks, > Kris > > > > *From: *Kris Zyp <kriszyp(a)gmail.com> > *Sent: *April 30, 2019 12:43 PM > *To: *Howard Chu <hyc(a)symas.com>; openldap-its(a)OpenLDAP.org > *Subject: *RE: (ITS#9017) Improving performance of commit sync in Windows > > > > > What is the point of using writemap mode if you still need to use > WriteFile > > > on every individual page? > > > > As I understood from the documentation, and have observed, using writemap > mode is faster (and uses less temporary memory) because it doesn=E2=80=99= t require > mallocs to allocate pages (docs: =E2=80=9CThis is faster and uses fewer m= allocs=E2=80=9D). > To be clear though, LMDB is so incredibly fast and efficient, that in > sync-mode, it takes enormous transactions before the time spent allocatin= g > and creating the dirty pages with the updated b-tree is anywhere even > remotely close to the time it takes to wait for disk flushing, even with = an > SSD. But the more pertinent question is efficiency, and measuring CPU > cycles rather than time spent (efficiency is more important than just tim= e > spent). When I ran my tests this morning of 100 (sync) transactions with > 100 puts per transaction, times varied quite a bit, but it seemed like > running with writemap enabled typically averages about 500ms of CPU and > with writemap disabled it typically averages around 600ms. Not a huge > difference, but still definitely worthwhile, I think. > > > > Caveat emptor: Measuring LMDB performance with sync interactions on > Windows is one of the most frustratingly erratic things to measure. It is > sunny outside right now, times could be different when it starts raining > later, but, this is what I saw this morning... > > > > > What is the performance difference between your patch using writemap, > and just > > > not using writemap in the first place? > > > > Running 1000 sync transactions on 3GB db with a single put per > transaction, without writemap map, without the patch took about 60 second= s. > And it took about 1 second with the patch with writemap mode enabled! > (there is no significant difference in sync times with writemap enabled o= r > disabled with the patch.) So the difference was huge in my test. And not > only that, without the patch, the CPU usage was actually _*higher*_ > during that 60 seconds (close to 100% of a core) than during the executio= n > with the patch for one second (close to 50%). Anyway, there are certainl= y > tests I have run where the differences are not as large (doing small > commits on large dbs accentuates the differences), but the patch always > seems to win. It could also be that my particular configuration causes > bigger differences (on an SSD drive, and maybe a more fragmented file?). > > > > Anyway, I added error handling for the malloc, and fixed/changed the othe= r > things you suggested. Be happy to make any other changes you want. The > updated patch is here: > > > https://github.com/kriszyp/node-lmdb/commit/25366dea9453749cf6637f43ec17b= 9b62094acde > > > > > OVERLAPPED* ov =3D malloc((pagecount - keep) * sizeof(OVERLAPPED)); > > > Probably this ought to just be pre-allocated based on the maximum > number of dirty pages a txn allows. > > > > I wasn=E2=80=99t sure I understood this comment. Are you suggesting we ma= lloc(MDB_IDL_UM_MAX > * sizeof(OVERLAPPED)) for each environment, and retain it for the life of > the environment? I think that is 4MB, if my math is right, which seems li= ke > a lot of memory to keep allocated (we usually have a lot of open > environments). If the goal is to reduce the number of mallocs, how about = we > retain the OVERLAPPED array, and only free and re-malloc if the previous > allocation wasn=E2=80=99t large enough? Then there isn=E2=80=99t unnecess= ary allocation, > and we only malloc when there is a bigger transaction than any previous. = I > put this together in a separate commit, as I wasn=E2=80=99t sure if this = what you > wanted (can squash if you prefer): > https://github.com/kriszyp/node-lmdb/commit/2fe68fb5269c843e2e789746a17a4= b2adefaac40 > > > > Thank you for the review! > > > > Thanks, > Kris > > > > *From: *Howard Chu <hyc(a)symas.com> > *Sent: *April 30, 2019 7:12 AM > *To: *kriszyp(a)gmail.com; openldap-its(a)OpenLDAP.org > *Subject: *Re: (ITS#9017) Improving performance of commit sync in Windows > > > > kriszyp(a)gmail.com wrote: > > > Full_Name: Kristopher William Zyp > > > Version: LMDB 0.9.23 > > > OS: Windows > > > URL: > https://github.com/kriszyp/node-lmdb/commit/7ff525ae57684a163d32af74a0ab9= 332b7fc4ce9 > > > Submission from: (NULL) (71.199.6.148) > > > > > > > > > We have seen very poor performance on the sync of commits on large > databases in > > > Windows. On databases with 2GB of data, in writemap mode, the sync of > even small > > > commits is consistently well over 100ms (without writemap it is faster, > but > > > still slow). It is expected that a sync should take some time while > waiting for > > > disk confirmation of the writes, but more concerning is that these sync > > > operations (in writemap mode) are instead dominated by nearly 100% > system CPU > > > utilization, so operations that requires sub-millisecond b-tree update > > > operations are then dominated by very large amounts of system CPU cycle= s > during > > > the sync phase. > > > > > > I think that the fundamental problem is that FlushViewOfFile seems to b= e > an O(n) > > > operation where n is the size of the file (or map). I presume that > Windows is > > > scanning the entire map/file for dirty pages to flush, I'm guessing > because it > > > doesn't have an internal index of all the dirty pages for every > file/map-view in > > > the OS disk cache. Therefore, the turns into an extremely expensive, > CPU-bound > > > operation to find the dirty pages for (large file) and initiate their > writes, > > > which, of course, is contrary to the whole goal of a scalable database > system. > > > And FlushFileBuffers is also relatively slow as well. We have attempted > to batch > > > as many operations into single transaction as possible, but this is > still a very > > > large overhead. > > > > > > The Windows docs for FlushFileBuffers itself warns about the > inefficiencies of > > > this function ( > https://docs.microsoft.com/en-us/windows/desktop/api/fileapi/nf-fileapi-f= lushfilebuffers > ). > > > Which also points to the solution: it is much faster to write out the > dirty > > > pages with WriteFile through a sync file handle > (FILE_FLAG_WRITE_THROUGH). > > > > > > The associated patch > > > ( > https://github.com/kriszyp/node-lmdb/commit/7ff525ae57684a163d32af74a0ab9= 332b7fc4ce9 > ) > > > is my attempt at implementing this solution, for Windows. Fortunately, > with the > > > design of LMDB, this is relatively straightforward. LMDB already suppor= ts > > > writing out dirty pages with WriteFile calls. I added a write-through > handle for > > > sending these writes directly to disk. I then made that file-handle > > > overlapped/asynchronously, so all the writes for a commit could be > started in > > > overlap mode, and (at least theoretically) transfer in parallel to the > drive and > > > then used GetOverlappedResult to wait for the completion. So basically > > > mdb_page_flush becomes the sync. I extended the writing of dirty pages > through > > > WriteFile to writemap mode as well (for writing meta too), so that > WriteFile > > > with write-through can be used to flush the data without ever needing t= o > call > > > FlushViewOfFile or FlushFileBuffers. I also implemented support for wri= te > > > gathering in writemap mode where contiguous file positions infers > contiguous > > > memory (by tracking the starting position with wdp and writing > contiguous pages > > > in single operations). Sorting of the dirty list is maintained even in > writemap > > > mode for this purpose. > > > > What is the point of using writemap mode if you still need to use WriteFi= le > > on every individual page? > > > > > The performance benefits of this patch, in my testing, are considerable= . > Writing > > > out/syncing transactions is typically over 5x faster in writemap mode, > and 2x > > > faster in standard mode. And perhaps more importantly (especially in > environment > > > with many threads/processes), the efficiency benefits are even larger, > > > particularly in writemap mode, where there can be a 50-100x reduction i= n > the > > > system CPU usage by using this patch. This brings windows performance > with > > > sync'ed transactions in LMDB back into the range of "lightning" > performance :). > > > > What is the performance difference between your patch using writemap, and > just > > not using writemap in the first place? > > > > -- > > -- Howard Chu > > CTO, Symas Corp. http://www.symas.com > > Director, Highland Sun http://highlandsun.com/hyc/ > > Chief Architect, OpenLDAP http://www.openldap.org/project/ > > > > > --000000000000db50ce058b89ac2f Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr">Is this still being considered/reviewed? Let me know if th= ere are any other changes you would like me to make. This patch has continu= ed to yield significant and reliable performance improvements for us, and s= eems like it would be nice for this to be available for other Windows users= .</div> <div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr"= >On Fri, May 3, 2019 at 3:52 PM Kris Zyp <<a href=3D"mailto:kriszyp@gmai= l.com">kriszyp(a)gmail.com</a>> wrote: </div><blockquote class=3D"gmail= _quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204= ,204);padding-left:1ex"><div lang=3D"EN-CA"><div class=3D"gmail-m_524243755= 9147988140WordSection1">For the sake of putting this= in the email thread (other code discussion in GitHub), here is the latest = squashed commit of the proposed patch (with the on-demand, retained overlap= ped array to reduce re-malloc and opening event handles): <a href=3D"https:= //github.com/kriszyp/node-lmdb/commit/726a9156662c703bf3d453aab75ee222072b9= 90f" target=3D"_blank">https://github.com/kriszyp/node-lmdb/commit/726a9156= 662c703bf3d453aab75ee222072b990f</a>=C2= =A0Thanks, Kris=C2=A0<div style=3D"border-right:none;border-bottom:n= one;border-left:none;border-top:1pt solid rgb(225,225,225);padding:3pt 0cm = 0cm">From: = <a href=3D"mailto:kriszyp@gmail.com" target=3D"_blank">Kris Zyp</a> S= ent: April 30, 2019 12:43 PM To: <a href=3D"mailto:hyc@symas.= com" target=3D"_blank">Howard Chu</a>; <a href=3D"mailto:openldap-its@OpenL= DAP.org" target=3D"_blank">openldap-its(a)OpenLDAP.org</a> Subject: </b= >RE: (ITS#9017) Improving performance of commit sync in Windows</div><p= class=3D"MsoNormal">=C2=A0> Wh= at is the point of using writemap mode if you still need to use WriteFile<u= >> on every individual page?</u= >=C2=A0As I understood from the documentation, and have observed, using wri= temap mode is faster (and uses less temporary memory) because it doesn=E2= =80=99t require mallocs to allocate pages (docs: =E2=80=9CThis is faster an= d uses fewer mallocs=E2=80=9D). To be clear though, LMDB is so incredibly f= ast and efficient, that in sync-mode, it takes enormous transactions before= the time spent allocating and creating the dirty pages with the updated b-= tree is anywhere even remotely close to the time it takes to wait for disk = flushing, even with an SSD. But the more pertinent question is efficiency, = and measuring CPU cycles rather than time spent (efficiency is more importa= nt than just time spent). When I ran my tests this morning of 100 (sync) tr= ansactions with 100 puts per transaction, times varied quite a bit, but it = seemed like running with writemap enabled typically averages about 500ms of= CPU and with writemap disabled it typically averages around 600ms. Not a h= uge difference, but still definitely worthwhile, I think.= =C2=A0Cavea= t emptor: Measuring LMDB performance with sync interactions on Windows is o= ne of the most frustratingly erratic things to measure. It is sunny outside= right now, times could be different when it starts raining later, but, thi= s is what I saw this morning...= =C2=A0> What is the performance di= fference between your patch using writemap, and just> not using writemap in the first place?= =C2=A0R= unning 1000 sync transactions on 3GB db with a single put per transaction, = without writemap map, without the patch took about 60 seconds. And it took = about 1 second with the patch with writemap mode enabled! (there is no sign= ificant difference in sync times with writemap enabled or disabled with the= patch.) So the difference was huge in my test. And not only that, without = the patch, the CPU usage was actually _higher_ during that 60 second= s (close to 100% of a core) than during the execution with the patch for on= e second (close to 50%).=C2=A0 Anyway, there are certainly tests I have run= where the differences are not as large (doing small commits on large dbs a= ccentuates the differences), but the patch always seems to win. It could al= so be that my particular configuration causes bigger differences (on an SSD= drive, and maybe a more fragmented file?).=C2=A0Anyway, I added err= or handling for the malloc, and fixed/changed the other things you suggeste= d. Be happy to make any other changes you want. The updated patch is here:<= u><a href=3D"https://github.com/krisz= yp/node-lmdb/commit/25366dea9453749cf6637f43ec17b9b62094acde" target=3D"_bl= ank">https://github.com/kriszyp/node-lmdb/commit/25366dea9453749cf6637f43ec= 17b9b62094acde</a>=C2=A0= > OVERLAPPED* ov =3D malloc((pagecount - keep) * sizeof(OVERLAPPED));<= span style=3D"font-size:9pt;font-family:Consolas;color:rgb(36,41,46)">> = Probably this ought to= just be pre-allocated based on the maximum number of dirty pages a txn all= ows.=C2=A0<p= class=3D"MsoNormal">I wasn=E2=80=99= t sure I understood this comment. Are you suggesting we malloc(MDB_I= DL_UM_MAX * sizeof(OVERLAPPED)) for each environment, and retain it for the= life of the environment? I think that is 4MB, if my math is right, which s= eems like a lot of memory to keep allocated (we usually have a lot of open = environments). If the goal is to reduce the number of mallocs, how about we= retain the OVERLAPPED array, and only free and re-malloc if the previous a= llocation wasn=E2=80=99t large enough? Then there isn=E2=80=99t unnecessary= allocation, and we only malloc when there is a bigger transaction than any= previous. I put this together in a separate commit, as I wasn=E2=80=99t su= re if this what you wanted (can squash if you prefer): <a href=3D"https://g= ithub.com/kriszyp/node-lmdb/commit/2fe68fb5269c843e2e789746a17a4b2adefaac40= " target=3D"_blank">https://github.com/kriszyp/node-lmdb/commit/2fe68fb5269= c843e2e789746a17a4b2adefaac40</a><= u>=C2=A0Thank you for the review! <sp= an style=3D"font-size:10.5pt;font-family:"Segoe UI",sans-serif;co= lor:rgb(36,41,46);background:white">=C2=A0Thanks, Kris<= /u>=C2=A0<div style=3D"= border-right:none;border-bottom:none;border-left:none;border-top:1pt solid = rgb(225,225,225);padding:3pt 0cm 0cm">From: <= a href=3D"mailto:hyc@symas.com" target=3D"_blank">Howard Chu</a> Sent= : April 30, 2019 7:12 AM To: <a href=3D"mailto:kriszyp@gmail.= com" target=3D"_blank">kriszyp(a)gmail.com</a>; <a href=3D"mailto:openldap-it= s(a)OpenLDAP.org" target=3D"_blank">openldap-its(a)OpenLDAP.org</a> Subje= ct: Re: (ITS#9017) Improving performance of commit sync in Windows</= u></div>=C2=A0<a href=3D"mailto:kriszyp@gmail.com" target=3D"_blank">krisz= yp(a)gmail.com</a> wrote:> Full_N= ame: Kristopher William Zyp> Ve= rsion: LMDB 0.9.23> OS: Windows= > URL: <a href=3D"https://githu= b.com/kriszyp/node-lmdb/commit/7ff525ae57684a163d32af74a0ab9332b7fc4ce9" ta= rget=3D"_blank">https://github.com/kriszyp/node-lmdb/commit/7ff525ae57684a1= 63d32af74a0ab9332b7fc4ce9</a>> = Submission from: (NULL) (71.199.6.148)> > <p= class=3D"MsoNormal">> We have seen very poor performance on the sync of= commits on large databases in>= Windows. On databases with 2GB of data, in writemap mode, the sync of even= small> commits is consistently= well over 100ms (without writemap it is faster, but> still slow). It is expected that a sync should take = some time while waiting for> di= sk confirmation of the writes, but more concerning is that these sync</u= >> operations (in writemap mode) are i= nstead dominated by nearly 100% system CPU> utilization, so operations that requires sub-millisecond b-tre= e update> operations are then d= ominated by very large amounts of system CPU cycles during</p= >> the sync phase.> > I think that t= he fundamental problem is that FlushViewOfFile seems to be an O(n)<u= >> operation where n is the size of the f= ile (or map). I presume that Windows is> scanning the entire map/file for dirty pages to flush, I'm gu= essing because it> doesn't = have an internal index of all the dirty pages for every file/map-view in= > the OS disk cache. Therefore, th= e turns into an extremely expensive, CPU-bound> operation to find the dirty pages for (large file) and ini= tiate their writes,> which, of = course, is contrary to the whole goal of a scalable database system.= > And FlushFileBuffers is also relativ= ely slow as well. We have attempted to batch> as many operations into single transaction as possible, but = this is still a very> large ove= rhead.> > The Windows docs for FlushFileBuffers itself warns a= bout the inefficiencies of> thi= s function (<a href=3D"https://docs.microsoft.com/en-us/windows/desktop/api= /fileapi/nf-fileapi-flushfilebuffers" target=3D"_blank">https://docs.micros= oft.com/en-us/windows/desktop/api/fileapi/nf-fileapi-flushfilebuffers</a>).= > Which also points to the solu= tion: it is much faster to write out the dirty> pages with WriteFile through a sync file handle (FILE_FLAG= _WRITE_THROUGH).> </u= >> The associated patch> (<a href=3D"https://github.com/kriszyp/node-lmdb/co= mmit/7ff525ae57684a163d32af74a0ab9332b7fc4ce9" target=3D"_blank">https://gi= thub.com/kriszyp/node-lmdb/commit/7ff525ae57684a163d32af74a0ab9332b7fc4ce9<= /a>)> is my attempt at implemen= ting this solution, for Windows. Fortunately, with the> design of LMDB, this is relatively straightforward= . LMDB already supports> writin= g out dirty pages with WriteFile calls. I added a write-through handle for<= u>> sending these writes directly = to disk. I then made that file-handle> overlapped/asynchronously, so all the writes for a commit could be = started in> overlap mode, and (= at least theoretically) transfer in parallel to the drive and= > then used GetOverlappedResult to wait for t= he completion. So basically> md= b_page_flush becomes the sync. I extended the writing of dirty pages throug= h> WriteFile to writemap mode a= s well (for writing meta too), so that WriteFile> with write-through can be used to flush the data withou= t ever needing to call> FlushVi= ewOfFile or FlushFileBuffers. I also implemented support for write<u= >> gathering in writemap mode where conti= guous file positions infers contiguous> memory (by tracking the starting position with wdp and writing con= tiguous pages> in single operat= ions). Sorting of the dirty list is maintained even in writemap</= u>> mode for this purpose.<= p class=3D"MsoNormal">=C2=A0What i= s the point of using writemap mode if you still need to use WriteFile</u= >on every individual page?<= /p>=C2=A0&g= t; The performance benefits of this patch, in my testing, are considerable.= Writing> out/syncing transacti= ons is typically over 5x faster in writemap mode, and 2x<= p class=3D"MsoNormal">> faster in standard mode. And perhaps more import= antly (especially in environment&g= t; with many threads/processes), the efficiency benefits are even larger,<u= >> particularly in writemap mode, = where there can be a 50-100x reduction in the> system CPU usage by using this patch. This brings windows p= erformance with> sync'ed tr= ansactions in LMDB back into the range of "lightning" performance= :).=C2=A0What is the performance difference between your patch usin= g writemap, and justnot using writ= emap in the first place?=C2= =A0-- =C2=A0=C2=A0-- Howard Chu=C2= =A0 CTO, Symas Corp.=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 <a href=3D"http://www.symas.com" target=3D"_blank">http://www.symas.= com</a>=C2=A0 Director, Highland S= un=C2=A0=C2=A0=C2=A0=C2=A0 <a href=3D"http://highlandsun.com/hyc/" target= =3D"_blank">http://highlandsun.com/hyc/</a>=C2=A0 Chief Architect, OpenLDAP=C2=A0 <a href=3D"http://www.openl= dap.org/project/" target=3D"_blank">http://www.openldap.org/project/</a>= =C2=A0=C2=A0</div></div></blockquote></div> --000000000000db50ce058b89ac2f--

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs