October 2013 - openldap-bugs

Re: (ITS#7710) contextCSN values not updated by internal non-replicated ops

by hyc＠symas.com

Howard Chu wrote: > Michael Ströder wrote: >> On Wed, 09 Oct 2013 05:01:40 -0700 Howard Chu <hyc(a)symas.com> wrote >> >>> michael(a)stroeder.com wrote: >>>> I've updated the test data with LDIF files and a README describing the LDAP >>>> operations to perform: >>>> >>>> http://www.stroeder.com/temp/openldap-testbed-its7710.tar.bz2 >>>> >>>> I consider this to be a serious issue which could be the cause for other >>>> replication issues including dead-locks we're experiencing. (Maybe there >>>> are similar issues in slapo-refint since dead-locks in our deployment >>>> arised when moving entries into different subtree.) >>> >>> Thanks, fixed now in master. Looks like refint may have run into the same >>> issue, also patched in master. >> >> contextCSN values are correct now. >> >> But now I see this during initial refresh phase of second server: >> >> send_ldap_result: err=20 matched="" text="modify/add: memberOf: value #0 >> already exists" >> >> Is that expected? > > Not seeing that with your testcase. > Ah, I see it now. Yes, it's normal; memberOf on the provider already added the relevant values. The consumer receives a group entry and performs the same set of memberof updates, which are redundant at that point. It's harmless. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

9 years, 11 months

1
0
0 / 0

Re: (ITS#7710) contextCSN values not updated by internal non-replicated ops

by hyc＠symas.com

Michael Ströder wrote: > On Wed, 09 Oct 2013 05:01:40 -0700 Howard Chu <hyc(a)symas.com> wrote > >> michael(a)stroeder.com wrote: >>> I've updated the test data with LDIF files and a README describing the LDAP >>> operations to perform: >>> >>> http://www.stroeder.com/temp/openldap-testbed-its7710.tar.bz2 >>> >>> I consider this to be a serious issue which could be the cause for other >>> replication issues including dead-locks we're experiencing. (Maybe there >>> are similar issues in slapo-refint since dead-locks in our deployment >>> arised when moving entries into different subtree.) >> >> Thanks, fixed now in master. Looks like refint may have run into the same >> issue, also patched in master. > > contextCSN values are correct now. > > But now I see this during initial refresh phase of second server: > > send_ldap_result: err=20 matched="" text="modify/add: memberOf: value #0 > already exists" > > Is that expected? Not seeing that with your testcase. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

9 years, 11 months

1
0
0 / 0

(ITS#7717) Sudden memory increase leading to Master LDAP crash

by marco.pizzoli＠gmail.com

--089e0115ff765de02a04e84f36b1 Content-Type: text/plain; charset=ISO-8859-1 I think I faced this same problem last week during a massive load of users (300K +) using a single ldapadd execution. My db "exploded" (MDB_MAP_FULL) after about 120K users populated. The main issue was related to the size of the db than to the actual memory allocated. It grew up to over 12GB before reaching the limit of my file system. But with memory-mapped files topics are close, I suppose. Curious enough, populating the same db from a full syncrepl led the db to grew to only (more or less) 500MB. As additional information, this server was part of a 5-node multi-master. And after the ldapadd the db -size was very different on each of the servers being part of the cluster. I cannot share any conf, I'm sorry. Marco --089e0115ff765de02a04e84f36b1 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div><div><div>I think I faced this same problem last week= during a massive load of users (300K +) using a single ldapadd execution.<= br>My db "exploded" (MDB_MAP_FULL) after about 120K users populat= ed.<br> The main issue was related to the size of the db than to the actual memory = allocated. It grew up to over 12GB before reaching the limit of my file sys= tem. But with memory-mapped files topics are close, I suppose.<br><br> </div> Curious enough, populating the same db from a full syncrepl led the db to g= rew to only (more or less) 500MB.<br><br>As additional information, this se= rver was part of a 5-node multi-master. And after the ldapadd the db -size = was very different on each of the servers being part of the cluster.<br> <br></div>I cannot share any conf, I'm sorry.<br></div>Marco<br></div> --089e0115ff765de02a04e84f36b1--

9 years, 11 months

1
0
0 / 0

Re: (ITS#7710) contextCSN values not updated by internal non-replicated ops

by michael＠stroeder.com

On Wed, 09 Oct 2013 05:01:40 -0700 Howard Chu <hyc(a)symas.com> wrote > michael(a)stroeder.com wrote: > > I've updated the test data with LDIF files and a README describing the LDAP > > operations to perform: > > > > http://www.stroeder.com/temp/openldap-testbed-its7710.tar.bz2 > > > > I consider this to be a serious issue which could be the cause for other > > replication issues including dead-locks we're experiencing. (Maybe there > > are similar issues in slapo-refint since dead-locks in our deployment > > arised when moving entries into different subtree.) > > Thanks, fixed now in master. Looks like refint may have run into the same > issue, also patched in master. contextCSN values are correct now. But now I see this during initial refresh phase of second server: send_ldap_result: err=20 matched="" text="modify/add: memberOf: value #0 already exists" Is that expected? Ciao, Michael. 52555576 syncrepl_entry: rid=002 LDAP_RES_SEARCH_ENTRY(LDAP_SYNC_ADD) 52555576 syncrepl_entry: rid=002 inserted UUID b5736b9e-c52f-1032-99c4-31a68f008587 52555576 mdb_idl_fetch_key: [3bec70f1] 52555576 send_ldap_result: err=0 matched="" text="" 52555576 syncrepl_entry: rid=002 be_search (0) 52555576 syncrepl_entry: rid=002 cn=replicas,ou=groups,dc=example,dc=com 52555576 ==> mdb_add: cn=replicas,ou=groups,dc=example,dc=com 52555576 mdb_idl_insert_keys: b [f4f2ead9] 52555576 mdb_idl_insert_keys: b [8f2b53e8] 52555576 mdb_idl_insert_keys: b [cb13035d] 52555576 mdb_idl_insert_keys: b [c14a3e76] 52555576 mdb_idl_insert_keys: b [3bec70f1] 52555576 mdb_idl_insert_keys: b 52555576 send_ldap_result: err=0 matched="" text="" 52555576 mdb_modify: cn=slapd-1,ou=sys,dc=example,dc=com 52555576 mdb_modify_internal: add memberOf 52555576 dnMatch 0 "cn=replicas,ou=groups,dc=example,dc=com" "cn=replicas,ou=groups,dc=example,dc=com" 52555576 mdb_modify_internal: 20 modify/add: memberOf: value #0 already exists 52555576 send_ldap_result: err=20 matched="" text="modify/add: memberOf: value #0 already exists" 52555576 conn=-1 op=0: memberof_value_modify DN="cn=slapd-1,ou=sys,dc=example,dc=com" add memberOf="cn=replicas,ou=groups,dc=example,dc=com" failed err=20 52555576 mdb_modify: cn=slapd-2,ou=sys,dc=example,dc=com 52555576 mdb_modify_internal: add memberOf 52555576 dnMatch 0 "cn=replicas,ou=groups,dc=example,dc=com" "cn=replicas,ou=groups,dc=example,dc=com" 52555576 mdb_modify_internal: 20 modify/add: memberOf: value #0 already exists 52555576 send_ldap_result: err=20 matched="" text="modify/add: memberOf: value #0 already exists" 52555576 conn=-1 op=0: memberof_value_modify DN="cn=slapd-2,ou=sys,dc=example,dc=com" add memberOf="cn=replicas,ou=groups,dc=example,dc=com" failed err=20 52555576 syncrepl_entry: rid=002 be_add cn=replicas,ou=groups,dc=example,dc=com (0) 52555576 do_syncrep2: rid=002 LDAP_RES_INTERMEDIATE - REFRESH_DELETE 52555576 do_syncrep2: rid=002 cookie=rid=002,sid=001,csn=20131009130900.890770Z#000000#001#000000 52555576 slap_queue_csn: queing 0x7fd79c10c200 20131009130900.890770Z#000000#001#000000 52555576 base_candidates: base: "dc=example,dc=com" (0x00000001) 52555576 send_ldap_result: err=0 matched="" text="" 52555576 mdb_modify: dc=example,dc=com 52555576 mdb_modify_internal: replace contextCSN 52555576 send_ldap_result: err=0 matched="" text="" 52555576 slap_graduate_commit_csn: removing 0x7fd79c10c170 20131009130900.890770Z#000000#001#000000

9 years, 11 months

1
0
0 / 0

Re: (ITS#7710) contextCSN values not updated by internal non-replicated ops

by hyc＠symas.com

michael(a)stroeder.com wrote: > I've updated the test data with LDIF files and a README describing the LDAP > operations to perform: > > http://www.stroeder.com/temp/openldap-testbed-its7710.tar.bz2 > > I consider this to be a serious issue which could be the cause for other > replication issues including dead-locks we're experiencing. (Maybe there are > similar issues in slapo-refint since dead-locks in our deployment arised when > moving entries into different subtree.) Thanks, fixed now in master. Looks like refint may have run into the same issue, also patched in master. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

9 years, 11 months

1
0
0 / 0

Re: (ITS#7722) Assertion failure on MDB_NEXT of existing cursor following delete

by hyc＠symas.com

dw(a)botanicus.net wrote: > Full_Name: David Wilson > Version: bf0e97aa0e9 @ git://gitorious.org/mdb/mdb.git > OS: Linux > URL: https://gist.github.com/dw/ba100e59b42ed5fccb67/raw/f25b08334e6956d9558bb... > Submission from: (NULL) (178.238.153.20) > > > The attached cursor-del-break.c exhibits an assertion failure after successfully > deleting 1246 256-byte records from a set of 2048. This hammers the case where > one cursor is used to iterate, while a temporary cursor is used to delete the > record currently pointed to by the first cursor. > > cursor-del-break: src/py-lmdb/lib/mdb.c:5025: mdb_cursor_next: Assertion > `((((mp)->mp_flags) & (0x02)) == (0x02))' failed. > > > Thanks for the report, fixed now in mdb.master. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

9 years, 11 months

1
0
0 / 0

(ITS#7722) Assertion failure on MDB_NEXT of existing cursor following delete

by dw＠botanicus.net

Full_Name: David Wilson Version: bf0e97aa0e9 @ git://gitorious.org/mdb/mdb.git OS: Linux URL: https://gist.github.com/dw/ba100e59b42ed5fccb67/raw/f25b08334e6956d9558bb... Submission from: (NULL) (178.238.153.20) The attached cursor-del-break.c exhibits an assertion failure after successfully deleting 1246 256-byte records from a set of 2048. This hammers the case where one cursor is used to iterate, while a temporary cursor is used to delete the record currently pointed to by the first cursor. cursor-del-break: src/py-lmdb/lib/mdb.c:5025: mdb_cursor_next: Assertion `((((mp)->mp_flags) & (0x02)) == (0x02))' failed.

9 years, 11 months

1
0
0 / 0

(ITS#7721) slapo-lastbind patch to allow authTimestamp updates to be forwarded via updateref

by ck＠cksoft.de

Full_Name: Christian Kratzer Version: HEAD OS: CentOS 6.4 URL: http://www.cksoft.de/paste/34e81be96925e5ea28daea7da2b84b09049f7f3d/0001-... Submission from: (NULL) (2003:41:c010:8001:8de8:2245:8dd7:5c5e) I, Christian Kratzer, hereby place the following modifications to OpenLDAP Software (and only these modifications) into the public domain. Hence, these modifications may be freely used and/or redistributed for any purpose with or without attribution and/or other notice. I have adapted code from slapo-ppolicy to slapo-lastbind from contrib to allow forwarding of updates to authTimestamp via updateref and chaining. The patch introduces a new lastbind_forward_updates (olcLastBindForwardUpdates ) boolean parameter to slapo-lastbind that has the same semantics as ppolicy_forward_updates (olcPPolicyForwardUpdates) in slapo-ppolicy. I have a 2 master, 2 slave syncrepl test setup which chaining from the slaves to the masters ( to one of them to be precise ). - Bind on master1 results in authTimestamp getting replicated to master2 and both slaves. - Bind on slave1 with olcLastBindForwardUpdates=TRUE results in authTimestamp getting forwarded to the master with updateRef and chaining and replicated from there. Please review especially the one from ppolicy I did not merge marked with TODO. Greetings Christian

9 years, 11 months

1
0
0 / 0

Re: (ITS#7720) 2.4.36 build failure on HP-UX

by hyc＠symas.com

bugs-openldap(a)vendor.thewrittenword.com wrote: > Full_Name: Albert Chin > Version: 2.4.36 > OS: HP-UX 11.23/PA > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (69.67.212.94) Thanks for the report. Interesting, this should also have been flagged as illegal in gcc but wasn't. Fixed now in master. > While trying to build 2.4.36 on HP-UX 11.23/PA with the HP C compiler: > cc -z +O2 +Olit=all +Ofltacc +Oentrysched +Odataprefetch +Onolimit > -I../../../include -I../../../include -I.. -I./.. -I./../../../libraries/liblmdb > -I/opt/TWWfsw/tcpwrap/include -I/opt/TWWfsw/libsasl21/include > -I/opt/TWWfsw/libopenssl10/include -I/opt/TWWfsw/libdb47/include -c tools.c -o > tools.o > cc: "tools.c", line 1384: error 1539: Cannot do arithmetic with pointers to > objects of unknown size. > cc: "tools.c", line 1410: error 1539: Cannot do arithmetic with pointers to > objects of unknown size. > gmake[3]: *** [tools.lo] Error 1 > gmake[3]: Leaving directory `/opt/build/openldap-2.4.36/servers/slapd/back-mdb' > > ... > > cc -z +O2 +Olit=all +Ofltacc +Oentrysched +Odataprefetch +Onolimit > -I../../../include -I../../../include -I.. -I./.. -I./../../../libraries/liblmdb > -I/opt/TWWfsw/tcpwrap/include -I/opt/TWWfsw/libsasl21/include > -I/opt/TWWfsw/libopenssl10/include -I/opt/TWWfsw/libdb47/include -c dn2id.c -o > dn2id.o > cc: "dn2id.c", line 168: error 1539: Cannot do arithmetic with pointers to > objects of unknown size. > cc: "dn2id.c", line 181: error 1539: Cannot do arithmetic with pointers to > objects of unknown size. > cc: "dn2id.c", line 249: error 1539: Cannot do arithmetic with pointers to > objects of unknown size. > cc: "dn2id.c", line 262: error 1539: Cannot do arithmetic with pointers to > objects of unknown size. > cc: "dn2id.c", line 402: error 1539: Cannot do arithmetic with pointers to > objects of unknown size. > cc: "dn2id.c", line 798: error 1539: Cannot do arithmetic with pointers to > objects of unknown size. > gmake[3]: *** [dn2id.lo] Error 1 > gmake[3]: Leaving directory `/opt/build/openldap-2.4.36/servers/slapd/back-mdb' > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

9 years, 11 months

1
0
0 / 0

(ITS#7720) 2.4.36 build failure on HP-UX

by bugs-openldap＠vendor.thewrittenword.com

Full_Name: Albert Chin Version: 2.4.36 OS: HP-UX 11.23/PA URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (69.67.212.94) While trying to build 2.4.36 on HP-UX 11.23/PA with the HP C compiler: cc -z +O2 +Olit=all +Ofltacc +Oentrysched +Odataprefetch +Onolimit -I../../../include -I../../../include -I.. -I./.. -I./../../../libraries/liblmdb -I/opt/TWWfsw/tcpwrap/include -I/opt/TWWfsw/libsasl21/include -I/opt/TWWfsw/libopenssl10/include -I/opt/TWWfsw/libdb47/include -c tools.c -o tools.o cc: "tools.c", line 1384: error 1539: Cannot do arithmetic with pointers to objects of unknown size. cc: "tools.c", line 1410: error 1539: Cannot do arithmetic with pointers to objects of unknown size. gmake[3]: *** [tools.lo] Error 1 gmake[3]: Leaving directory `/opt/build/openldap-2.4.36/servers/slapd/back-mdb' ... cc -z +O2 +Olit=all +Ofltacc +Oentrysched +Odataprefetch +Onolimit -I../../../include -I../../../include -I.. -I./.. -I./../../../libraries/liblmdb -I/opt/TWWfsw/tcpwrap/include -I/opt/TWWfsw/libsasl21/include -I/opt/TWWfsw/libopenssl10/include -I/opt/TWWfsw/libdb47/include -c dn2id.c -o dn2id.o cc: "dn2id.c", line 168: error 1539: Cannot do arithmetic with pointers to objects of unknown size. cc: "dn2id.c", line 181: error 1539: Cannot do arithmetic with pointers to objects of unknown size. cc: "dn2id.c", line 249: error 1539: Cannot do arithmetic with pointers to objects of unknown size. cc: "dn2id.c", line 262: error 1539: Cannot do arithmetic with pointers to objects of unknown size. cc: "dn2id.c", line 402: error 1539: Cannot do arithmetic with pointers to objects of unknown size. cc: "dn2id.c", line 798: error 1539: Cannot do arithmetic with pointers to objects of unknown size. gmake[3]: *** [dn2id.lo] Error 1 gmake[3]: Leaving directory `/opt/build/openldap-2.4.36/servers/slapd/back-mdb'

9 years, 11 months

1
0
0 / 0

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs October 2013