openldap-bugs November 2013

openldap-bugs@openldap.org

18 participants
55 discussions

Re: (ITS#7756) LMDB assertion failed in mdb_page_search_root
by hyc＠symas.com 29 Nov '13

29 Nov '13

xramtsov(a)gmail.com wrote: > Full_Name: Evgeny Khramtsov > Version: LMDB 0.9.10 > OS: Debian Linux 3.10-3-amd64 SMP > URL: http://kuku.jabber.ru/~xram/lmdb_bug.tar.gz > Submission from: (NULL) (198.211.126.52) > > > The way to reproduce the crash: > 1) Write lots of key/vals such as "0", "1", "2", ..., "5000000" (all are "", but > this is irrelevant). > 2) Delete them in the same order. > 3) Get: mdb.c:4682: mdb_page_search_root: Assertion `(((mp)->mp_pb.pb.pb_lower - > ((unsigned) __builtin_offsetof (MDB_page, mp_ptrs))) >> 1) > 1' failed. > > On the url there is an example program reproducing the bug. > Unpack the tarball: > $ tar -xzf lmdb_bug.tar.gz > $ cd lmdb_bug > $ make > $ ./bug /path/to/some/non_existent/dir Thanks for the report. This is now fixed in git mdb.master. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#7756) LMDB assertion failed in mdb_page_search_root
by xramtsov＠gmail.com 28 Nov '13

28 Nov '13

Full_Name: Evgeny Khramtsov Version: LMDB 0.9.10 OS: Debian Linux 3.10-3-amd64 SMP URL: http://kuku.jabber.ru/~xram/lmdb_bug.tar.gz Submission from: (NULL) (198.211.126.52) The way to reproduce the crash: 1) Write lots of key/vals such as "0", "1", "2", ..., "5000000" (all are "", but this is irrelevant). 2) Delete them in the same order. 3) Get: mdb.c:4682: mdb_page_search_root: Assertion `(((mp)->mp_pb.pb.pb_lower - ((unsigned) __builtin_offsetof (MDB_page, mp_ptrs))) >> 1) > 1' failed. On the url there is an example program reproducing the bug. Unpack the tarball: $ tar -xzf lmdb_bug.tar.gz $ cd lmdb_bug $ make $ ./bug /path/to/some/non_existent/dir

1 0

(ITS#7755) not replicated nodes after recovery
by dmitrii.fonariuk＠gmail.com 27 Nov '13

27 Nov '13

Full_Name: Dmitrii Fonariuk Version: 2.4.38 OS: RHEL6.x86_64 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (91.210.4.1) hi. I have two servers in Multi-Master mode (type=refreshAndPersist) 1. I add's one DN (DN=1) on first node, after replication i see DN=1 on both nodes. 2. I've done backup via mdb_copy on 1st node. 3. After that i add's one DN on 2nd node (DN=2) and one DN on 1st node (DN=3). 4. I see DN=1, DN=2, DN=3 on both nodes and the same contextCSN. 5. I stop the slapd on 1st node and do recovery from last backup. 6. Start slapd. 7. Now I see DN=1, DN=2, DN=3 on 2nd node and DN=1, DN=2 on 1nd node and the same contextCSN. Why DN=3, added after backup on 1st node doesn't replicated from 2nd node? if i on 6 step start slapd with -c rid=001 - all OK. (performed long time if a lot of data) if i on 6 step before start slapd change his serverID on different value - all OK. (not suitable) I can conclude that the back is not replicated that DN, entryCSN value that matches the value serverID. This is a bug or a feature?

1 0

Re: (ITS#7754) Unaligned MDB_DUPSORT sub-pages
by h.b.furuseth＠usit.uio.no 26 Nov '13

26 Nov '13

I wrote: > (...) > A fix with a format change would be to put the padding byte in front > of the sub-page instead of at the end. Ignore next paragraph: > One fix with a format change could be that the data item gets uneven > size too, with a padding byte in front of the sub-page. Because that's repeated as the 2nd alternative after this one: > The most general way would be to do that in all nodes with uneven- > sized key and even-sized data: -- Hallvard

1 0

(ITS#7754) Unaligned MDB_DUPSORT sub-pages
by h.b.furuseth＠usit.uio.no 26 Nov '13

26 Nov '13

Full_Name: Hallvard B Furuseth Version: LMDB_0.9.10 OS: Linux x86_64 URL: Submission from: (NULL) (81.191.45.35) Submitted by: hallvard Uneven-sized keys in MDB_DUPSORT databases get unaligned sub-pages, and thus everything inside their sub-pages gets unaligned too. A fix preserving the current format when !MISALIGNED_OK would be more memcpying, e.g. in COPY_PGNO(). dcmp functions would do byte by byte comparison for such sub-pages. MDB_page.mp_flags could be split two uint8_t fields: mp_flags, mp_zero (zero for now anyway). Renumber P_KEEP to 0x80 or P_META. A fix with a format change would be to put the padding byte in front of the sub-page instead of at the end. One fix with a format change could be that the data item gets uneven size too, with a padding byte in front of the sub-page. The most general way would be to do that in all nodes with uneven- sized key and even-sized data: #define NODEDATA(node) ((void *) ((char *)(node)->mn_data + \ (node)->mn_ksize + ((node)->mn_ksize & ~(node)->mn_lo & 1))) Or do it only with sub-pages. Make the data item uneven-sized so code which is not interested in whether this is a sub-page, need not know about the hack. Access the sub-page specially: #define NODEPAGE(node) ((MDB_page *) \ ((char *)(node)->mn_data + (((node)->mn_ksize + 1) & -2))) That could be coded so the old format is still readable. Then a utility can walk all databases and adjust sub-pages so they have padding in front instead of at the end, so no dump/reload will be needed: /** Address of the possibly 2-byte aligned sub-page in a node. * If the data item has uneven size, then the first byte is * padding to re-align the sub-page after an uneven-sized key. */ #define NODEPAGE(node) ((MDB_page *) \ ((char *)(node)->mn_data + (node)->mn_ksize + ((node)->mn_lo & 1)))

1 0

(ITS#7753) "make" should not be hardcoded
by dam＠opencsw.org 26 Nov '13

26 Nov '13

Full_Name: Dagobert Michelsen Version: 2.4.38 OS: Solaris URL: http://sourceforge.net/apps/trac/gar/browser/csw/mgar/pkg/openldap/trunk/fi… Submission from: (NULL) (86.103.212.188) In Makefile.in the tests are invoked with "make test" which breaks if make has a different name (e.g. gmake) and should better be invoked with "$(MAKE) test". This tiny patch also fixes the issue: http://sourceforge.net/apps/trac/gar/browser/csw/mgar/pkg/openldap/trunk/fi…

1 0

Re: (ITS#7589) MDB nodesize issues
by h.b.furuseth＠usit.uio.no 25 Nov '13

25 Nov '13

Howard Chu writes: > Explicitly documented: > > /** Header for a single key/data pair within a page. > * We guarantee 2-byte alignment for nodes. > */ > typedef struct MDB_node { ...er, that's false for LEAF2 nodes since we don't reject uneven-sized DUPFIXED keys. But that part seems to work fine, it's the non-LEAF2 which breaks. -- Hallvard

1 0

Re: (ITS#7589) MDB nodesize issues
by h.b.furuseth＠usit.uio.no 25 Nov '13

25 Nov '13

Howard Chu writes: > Hallvard Breien Furuseth wrote: > "Non-LEAF2 nodes are >> even-sized" feels like a deeply embedded assumption, but I haven't >> really looked. > > Explicitly documented: > > /** Header for a single key/data pair within a page. > * We guarantee 2-byte alignment for nodes. > */ > typedef struct MDB_node { > > Obviously you cannot guarantee 2-byte alignment unless nodes are always even > sized. Indeed. But I was wondering how hard to try to rescue databases where that doc is false. -- Hallvard

1 0

Re: (ITS#7589) MDB nodesize issues
by hyc＠symas.com 25 Nov '13

25 Nov '13

Hallvard Breien Furuseth wrote: "Non-LEAF2 nodes are > even-sized" feels like a deeply embedded assumption, but I haven't > really looked. Explicitly documented: /** Header for a single key/data pair within a page. * We guarantee 2-byte alignment for nodes. */ typedef struct MDB_node { Obviously you cannot guarantee 2-byte alignment unless nodes are always even sized. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7589) MDB nodesize issues
by h.b.furuseth＠usit.uio.no 25 Nov '13

25 Nov '13

Howard Chu writes: > Fixed now in mdb.master. (...) The me_nodemax issue remains though. Should be fixed in my branch "mdb/its7589". However, another issue: mdb_node_shrink() can give the node containing the sub-page uneven size, if MDB_DUPFIXED keys have uneven size: mdb_dbi_open(txn, "db", MDB_DUPSORT|MDB_DUPFIXED, &dbi); mdb_put(txn, dbi, &(MDB_val){4, "key"}, &(MDB_val){1, "A"}, 0); mdb_put(txn, dbi, &(MDB_val){4, "key"}, &(MDB_val){1, "B"}, 0); mdb_del(txn, dbi, &(MDB_val){4, "key"}, &(MDB_val){1, "B"}); Fix: Do nothing if the node would be uneven-sized. Similar to how mdb_cursor_put() adds room for 4N nodes at a time, with MDB_DUPFIXED. Dunno if the cursor tracking code in mdb_cursor_del() must still run. I don't know what to do about databases that already contain such uneven-sized nodes. Maybe support reading them (if that needs any changes), but try to assert when writing. "Non-LEAF2 nodes are even-sized" feels like a deeply embedded assumption, but I haven't really looked. -- Hallvard

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs November 2013