openldap-bugs August 2013

openldap-bugs@openldap.org

23 participants
68 discussions

Re: (ITS#7658) LMDB: Crash when mdb_del should ignore data, back trace and patch provided
by hyc＠symas.com 09 Aug '13

09 Aug '13

sog(a)msg.com.mx wrote: > Full_Name: Salvador Ortiz > Version: 24 > OS: Linux > URL: ftp://ftp.msg.com.mx/pub/varios/mdb_del-must-ignore-data.patch > Submission from: (NULL) (187.162.45.111) > > > According to the documentation, mdb_del should ignore the data parameter if the > dbi was not opened with MDB_DUPSORT, but this check is missing causing a > segfault deep in mdb_cursor_set when a NULL dcmp is called. > > I discovered this in the following back trace: > > #0 0x0000000000000000 in ?? () > #1 0x00007ffff1080f69 in mdb_cursor_set (mc=mc@entry=0x7fffffffd220, > key=key@entry=0x7fffffffd5f0, data=0x7fffffffd210, op=MDB_GET_BOTH, > exactp=exactp@entry=0x7fffffffd20c) at mdb.c:5149 > #2 0x00007ffff10862af in mdb_del (txn=txn@entry=0xe60a10, dbi=1, > key=key@entry=0x7fffffffd5f0, data=data@entry=0x7fffffffd600) at mdb.c:7164 > ... > > A simple patch against gitorious's mdb.master provided. Thanks, committed. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#7658) LMDB: Crash when mdb_del should ignore data, back trace and patch provided
by sog＠msg.com.mx 09 Aug '13

09 Aug '13

Full_Name: Salvador Ortiz Version: 24 OS: Linux URL: ftp://ftp.msg.com.mx/pub/varios/mdb_del-must-ignore-data.patch Submission from: (NULL) (187.162.45.111) According to the documentation, mdb_del should ignore the data parameter if the dbi was not opened with MDB_DUPSORT, but this check is missing causing a segfault deep in mdb_cursor_set when a NULL dcmp is called. I discovered this in the following back trace: #0 0x0000000000000000 in ?? () #1 0x00007ffff1080f69 in mdb_cursor_set (mc=mc@entry=0x7fffffffd220, key=key@entry=0x7fffffffd5f0, data=0x7fffffffd210, op=MDB_GET_BOTH, exactp=exactp@entry=0x7fffffffd20c) at mdb.c:5149 #2 0x00007ffff10862af in mdb_del (txn=txn@entry=0xe60a10, dbi=1, key=key@entry=0x7fffffffd5f0, data=data@entry=0x7fffffffd600) at mdb.c:7164 ... A simple patch against gitorious's mdb.master provided.

1 0

Re: (ITS#7657) Alias dereferencing with MDB slow compared with BDB
by quanah＠zimbra.com 08 Aug '13

08 Aug '13

--On August 8, 2013 5:12:50 PM +0100 Mark Cairney <Mark.Cairney(a)ed.ac.uk> wrote: > Hi Quanah, > > I've transferred one of the nodes to back-hdb and the performance appears > to broadly match back-bdb. > > Is there a simple test I can perform to confirm that it is indeed using > back-hdb? > > For completeness here's the result of my timed search: > > bash-4.1$ time ldapsearch -H ldaps://rowan.authorise.is.ed.ac.uk:636 -b > 'ou=staff,ou=law,ou=hss,dc=authorise,dc=ed,dc=ac,dc=uk' -a always > "(&(eduniSchoolCode=S26)(eduniCategory=101)(eduniIDStatus=300))" > ># numResponses: 425 ># numEntries: 424 > > real 0m0.902s > user 0m0.054s > sys 0m0.025s No, that's enough, thanks. So it is MDB specific then. ;) --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

Re: (ITS#7657) Alias dereferencing with MDB slow compared with BDB
by Mark.Cairney＠ed.ac.uk 08 Aug '13

08 Aug '13

Hi Quanah, I've transferred one of the nodes to back-hdb and the performance appears to broadly match back-bdb. Is there a simple test I can perform to confirm that it is indeed using back-hdb? For completeness here's the result of my timed search: bash-4.1$ time ldapsearch -H ldaps://rowan.authorise.is.ed.ac.uk:636 -b 'ou=staff,ou=law,ou=hss,dc=authorise,dc=ed,dc=ac,dc=uk' -a always "(&(eduniSchoolCode=S26)(eduniCategory=101)(eduniIDStatus=300))" # numResponses: 425 # numEntries: 424 real 0m0.902s user 0m0.054s sys 0m0.025s On 08/08/13 16:45, Quanah Gibson-Mount wrote: > --On August 8, 2013 4:34:32 PM +0100 Mark Cairney > <Mark.Cairney(a)ed.ac.uk> wrote: > > >>> back-bdb or back-hdb? "BDB" is just the storage database for those two >>> backends. >> >> >> back-bdb. > > Thanks. I would note that back-mdb is based on back-hdb, and that can > have significant differences in performance profile for some operations > as compared to back-bdb. If it were possible, I'd be curious to know if > you see the same issue with back-hdb as you do with back-mdb. > > --Quanah > -- /**************************** Mark Cairney ITI UNIX Section Information Services University of Edinburgh Tel: 0131 650 6565 Email: Mark.Cairney(a)ed.ac.uk *******************************/ The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.

1 0

Re: (ITS#7657) Alias dereferencing with MDB slow compared with BDB
by quanah＠zimbra.com 08 Aug '13

08 Aug '13

--On August 8, 2013 4:34:32 PM +0100 Mark Cairney <Mark.Cairney(a)ed.ac.uk> wrote: >> back-bdb or back-hdb? "BDB" is just the storage database for those two >> backends. > > > back-bdb. Thanks. I would note that back-mdb is based on back-hdb, and that can have significant differences in performance profile for some operations as compared to back-bdb. If it were possible, I'd be curious to know if you see the same issue with back-hdb as you do with back-mdb. --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

Re: (ITS#7657) Alias dereferencing with MDB slow compared with BDB
by Mark.Cairney＠ed.ac.uk 08 Aug '13

08 Aug '13

On 08/08/13 16:27, Howard Chu wrote: > > How many aliases? > In that OU 435, in total across the whole database: 29179. -- /**************************** Mark Cairney ITI UNIX Section Information Services University of Edinburgh Tel: 0131 650 6565 Email: Mark.Cairney(a)ed.ac.uk *******************************/ The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.

1 0

Re: (ITS#7657) Alias dereferencing with MDB slow compared with BDB
by Mark.Cairney＠ed.ac.uk 08 Aug '13

08 Aug '13

On 08/08/13 16:32, Quanah Gibson-Mount wrote: > > > --On August 8, 2013 3:02:51 PM +0000 Mark.Cairney(a)ed.ac.uk wrote: > >> Full_Name: Mark Cairney >> Version: 2.4.35-RE24 >> OS: SL 6.4 64-bit >> URL: ftp://ftp.openldap.org/incoming/ >> Submission from: (NULL) (129.215.200.23) >> >> >> Having migrated the database of our production service running 2.4.35 >> from BDB to MDB alias de-referencing stopped working completely (this has >> already been reported in ITS# 7557). > > back-bdb or back-hdb? "BDB" is just the storage database for those two > backends. back-bdb. > > Thanks, > Quanah > -- /**************************** Mark Cairney ITI UNIX Section Information Services University of Edinburgh Tel: 0131 650 6565 Email: Mark.Cairney(a)ed.ac.uk *******************************/ The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.

1 0

Re: (ITS#7657) Alias dereferencing with MDB slow compared with BDB
by quanah＠zimbra.com 08 Aug '13

08 Aug '13

--On August 8, 2013 3:02:51 PM +0000 Mark.Cairney(a)ed.ac.uk wrote: > Full_Name: Mark Cairney > Version: 2.4.35-RE24 > OS: SL 6.4 64-bit > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (129.215.200.23) > > > Having migrated the database of our production service running 2.4.35 > from BDB to MDB alias de-referencing stopped working completely (this has > already been reported in ITS# 7557). back-bdb or back-hdb? "BDB" is just the storage database for those two backends. Thanks, Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

Re: (ITS#7657) Alias dereferencing with MDB slow compared with BDB
by hyc＠symas.com 08 Aug '13

08 Aug '13

Mark.Cairney(a)ed.ac.uk wrote: > Full_Name: Mark Cairney > Version: 2.4.35-RE24 > OS: SL 6.4 64-bit > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (129.215.200.23) > > > Having migrated the database of our production service running 2.4.35 from BDB > to MDB alias de-referencing stopped working completely (this has already been > reported in ITS# 7557). > > Upgrading to the RE 24 now has alias derefencing working but it is considerably > slower than BDB was for the same search. > > To confirm this I downgraded one of our nodes (we run 4 nodes in multi-master) > and on that node the search results are once more acceptable: > > With MDB: > time ldapsearch -H ldaps://alder.authorise.is.ed.ac.uk:636 -b > 'ou=staff,ou=law,ou=hss,dc=authorise,dc=ed,dc=ac,dc=uk' -a always > "(&(eduniSchoolCode=S26)(eduniCategory=101)(eduniIDStatus=300))" > > # numResponses: 425 > # numEntries: 424 > > real 0m26.490s > user 0m0.028s > sys 0m0.032s > > > with BDB: > > time ldapsearch -H ldaps://oak.authorise.is.ed.ac.uk:636 -b > 'ou=staff,ou=law,ou=hss,dc=authorise,dc=ed,dc=ac,dc=uk' -a always > "(&(eduniSchoolCode=S26)(eduniCategory=101)(eduniIDStatus=300))" > > real 0m2.605s > user 0m0.054s > sys 0m0.006s > > If there's anything additional I can supply (log files, strace output) let me > know. > > Our OpenLDAP DB currently has around 400,000 user accounts if that's relevant. How many aliases? -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#7657) Alias dereferencing with MDB slow compared with BDB
by Mark.Cairney＠ed.ac.uk 08 Aug '13

08 Aug '13

Full_Name: Mark Cairney Version: 2.4.35-RE24 OS: SL 6.4 64-bit URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (129.215.200.23) Having migrated the database of our production service running 2.4.35 from BDB to MDB alias de-referencing stopped working completely (this has already been reported in ITS# 7557). Upgrading to the RE 24 now has alias derefencing working but it is considerably slower than BDB was for the same search. To confirm this I downgraded one of our nodes (we run 4 nodes in multi-master) and on that node the search results are once more acceptable: With MDB: time ldapsearch -H ldaps://alder.authorise.is.ed.ac.uk:636 -b 'ou=staff,ou=law,ou=hss,dc=authorise,dc=ed,dc=ac,dc=uk' -a always "(&(eduniSchoolCode=S26)(eduniCategory=101)(eduniIDStatus=300))" # numResponses: 425 # numEntries: 424 real 0m26.490s user 0m0.028s sys 0m0.032s with BDB: time ldapsearch -H ldaps://oak.authorise.is.ed.ac.uk:636 -b 'ou=staff,ou=law,ou=hss,dc=authorise,dc=ed,dc=ac,dc=uk' -a always "(&(eduniSchoolCode=S26)(eduniCategory=101)(eduniIDStatus=300))" real 0m2.605s user 0m0.054s sys 0m0.006s If there's anything additional I can supply (log files, strace output) let me know. Our OpenLDAP DB currently has around 400,000 user accounts if that's relevant.

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs August 2013