October 2013 - openldap-bugs

by hyc＠symas.com

h.b.furuseth(a)usit.uio.no wrote: > I wrote: >> OTOH if you add a bunch of slightly smaller nodes, mdb will put >> most of them in separate pages anyway without MDB_APPEND. > > ...because mdb_page_split() has been wasteful since 48ef27b6f5c804eca6a9 > "ITS#7385 fix mdb_page_split (again)". When a txn put()s ascending > keys with nodes of the same size, the new item goes in the fullest page. > > E.g. put data size 1010 with 'int' keys 1,2,3... to an MDB_INTEGERKEY > DB. As the txn progresses, (page: #key #key...) get distributed thus: > > Page 2: #1. > Page 2: #1 #2. > Page 2: #1 #2 #3. > Page 2: #1. Page 3: #2 #3 #4. > Page 2: #1. Page 3: #2. Page 5: #3 #4 #5. > Page 2: #1. Page 3: #2. Page 5: #3. Page 6: #4 #5 #6. > > 2/3 wasted space. Descending put() work better: > > Page 2: #6. > Page 2: #5 #6. > Page 2: #4 #5 #6. > Page 2: #3 #4. Page 3: #5 #6. > Page 2: #2 #3 #4. Page 3: #5 #6. > Page 2: #2 #1. Page 3: #5 #6. Page 5: #3 #4. > > Ascending put() with datasize 1348, so only 2 nodes fit in a page: > > Page 2: #1. > Page 2: #1 #2. > Page 2: #1. Page 3: #2 #3. > Page 2: #1. Page 3: #2. Page 5: #3 #4. > > Half of the space is wasted. Descending order does not help. > > page_split() cannot know which split is best in this case. But it'll > help to guess that the next put() key sometimes will be near this one, > and ensure that the node with the new key will be the smallest. That > will also avoid touching the old page when the nodes are that large, > since the "split" will keep all old nodes in the old page. Fixed now in mdb.master. On a large DB, the previous code used 3276587 pages in slapadd -q, and the new code uses 3272633 pages. It's only a 0.13% savings in this case, it seems the frequency of these insert patterns is quite rare. The runtime is also 1.1% faster going from real 41m35.8s user 50m57.6s sys 5m11.4s to real 41m1.7s user 50m25.8s sys 4m55.3s -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

9 years, 11 months

1
0
0 / 0

Re: (ITS#7723) slapd crashes on multi core machines if a search request is *immediately* followed by an unbind

by hyc＠symas.com

hyc(a)symas.com wrote: > michael.vishchers(a)7p-group.com wrote: >> Full_Name: Michael Vishchers >> Version: 2.4.36 >> OS: RHEL 6.3 >> URL: ftp://ftp.openldap.org/incoming/ >> Submission from: (NULL) (109.41.86.33) >> >> >> The bug mentioned in ITS #7716 unfortunately still persists in 2.4.36. >> >> We now found a way to reproduce it: If you want anyone to spend time on this: Send the stack trace from the crash. Also send your slapd config, sample data, and the actual query used in your client. >> >> - make sure that threads can run freely (i.e., provide enough cpus) > >> - start slapd with the rwm overlay >> - run a client loop that >> -- binds as a valid user >> -- sends a valid search request that is *immediately* (i.e., don't wait for any >> answers) followed by an unbind request >> >> after several iterations (with 2.4.23 it was between 2 and 7, with 2.4.36 it was >> about 30) slapd crashes in malloc.c >> >> > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

9 years, 11 months

1
0
0 / 0

Re: (ITS#7723) slapd crashes on multi core machines if a search request is *immediately* followed by an unbind

by hyc＠symas.com

michael.vishchers(a)7p-group.com wrote: > Full_Name: Michael Vishchers > Version: 2.4.36 > OS: RHEL 6.3 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (109.41.86.33) > > > The bug mentioned in ITS #7716 unfortunately still persists in 2.4.36. > > We now found a way to reproduce it: > > - make sure that threads can run freely (i.e., provide enough cpus) If your client loop is running as a single thread then the number of CPUs and running threads should be irrelevant. If the number of threads and CPUs matter then you're misusing the API. Closing this ITS. > - start slapd with the rwm overlay > - run a client loop that > -- binds as a valid user > -- sends a valid search request that is *immediately* (i.e., don't wait for any > answers) followed by an unbind request > > after several iterations (with 2.4.23 it was between 2 and 7, with 2.4.36 it was > about 30) slapd crashes in malloc.c > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

9 years, 11 months

1
0
0 / 0

Re: (ITS#7724) Invalid FREE suspected in unbind.c

by hyc＠symas.com

michael.vishchers(a)7p-group.com wrote: > Full_Name: Michael Vishchers > Version: 2.4.36 > OS: RHEL 6.3 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (109.41.86.33) > > > This may be connected to ITS #7723: > > in unbind.c, there is new code (newer than 2.4.23, i.e.) that takes care not to > free the ld struct if some other thread is using it (line 82ff) > > Unfortunately, in line 102, after decrementing the ref count in this struct, > this struct is freed nonetheless. The struct is freed because it should only be used by one thread. Read draft-zeilenga-ldap-c-api-concurrency. Closing this ITS. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

9 years, 11 months

1
0
0 / 0

Re: (ITS#7720) 2.4.36 build failure on HP-UX

by h.b.furuseth＠usit.uio.no

On 2013-10-08 19:42, hyc(a)symas.com wrote: > Thanks for the report. Interesting, this should also have been flagged > as > illegal in gcc but wasn't. Fixed now in master. void* pointer arithmetic is a gcc extension, it's treated as char*. gcc warns about it if you use -pedantic or -Wpointer-arith. -- Hallvard

9 years, 11 months

1
0
0 / 0

(ITS#7724) Invalid FREE suspected in unbind.c

by michael.vishchers＠7p-group.com

Full_Name: Michael Vishchers Version: 2.4.36 OS: RHEL 6.3 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (109.41.86.33) This may be connected to ITS #7723: in unbind.c, there is new code (newer than 2.4.23, i.e.) that takes care not to free the ld struct if some other thread is using it (line 82ff) Unfortunately, in line 102, after decrementing the ref count in this struct, this struct is freed nonetheless.

9 years, 11 months

1
0
0 / 0

(ITS#7723) slapd crashes on multi core machines if a search request is *immediately* followed by an unbind

by michael.vishchers＠7p-group.com

Full_Name: Michael Vishchers Version: 2.4.36 OS: RHEL 6.3 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (109.41.86.33) The bug mentioned in ITS #7716 unfortunately still persists in 2.4.36. We now found a way to reproduce it: - make sure that threads can run freely (i.e., provide enough cpus) - start slapd with the rwm overlay - run a client loop that -- binds as a valid user -- sends a valid search request that is *immediately* (i.e., don't wait for any answers) followed by an unbind request after several iterations (with 2.4.23 it was between 2 and 7, with 2.4.36 it was about 30) slapd crashes in malloc.c

9 years, 11 months

1
0
0 / 0

Re: (ITS#7710) contextCSN values not updated by internal non-replicated ops

by quanah＠zimbra.com

--On Thursday, October 10, 2013 11:22 AM +0000 michael(a)stroeder.com wrote: > On Thu, 10 Oct 2013 11:05:28 GMT hyc(a)symas.com wrote >> michael(a)stroeder.com wrote: >> > With current master and re24 which have the fix for this ITS I now see >> > seg faults in test023-refint test057-memberof-refint. >> >> Fixed now in master. > > Thanks. These tests are passing now. > > @Quanah: Would be nice if this last fix could be also ported to RE24. > Thanks in advance. Not particularly necessary to make requests like this. I read all ITS mail and commit traffic. --Quanah -- Quanah Gibson-Mount Architect - Server Zimbra Software, LLC -------------------- Zimbra :: the leader in open source messaging and collaboration

9 years, 11 months

1
0
0 / 0

Re: (ITS#7694) cldap fails with IPv6 due to wrong size sockaddr

by stefw＠redhat.com

On 10.10.2013 13:59, Howard Chu wrote: > Stef Walter wrote: >> On 10.10.2013 12:59, Howard Chu wrote: >>> stefw(a)redhat.com wrote: >>>> Full_Name: Stef Walter >>>> Version: 2.4.35 >>>> OS: Fedora 19 >>>> Submission from: (NULL) (46.5.2.70) >>>> >>>> >>>> Connectionless LDAP (ie: cldap enabled with -DLDAP_CONNECTIONLESS) is >>>> broken for >>>> IPv6 for current versions of openldap. Tested with version 2.4.35 >>>> >>>> It's not clear if this ever worked properly. >>> >>> No, clearly not, the code was written and deprecated before IPv6 >>> existed. Nobody should be using this code today. >> >> Interesting. FWIW, the code is packaged by RHEL and Fedora, and is in >> use by several projects. > > Can you list any of these, offhand? The original spec, RFC1798, is long > obsoleted. There is no such thing as CLDAP in LDAPv3. Support in > OpenLDAP was first removed back in 2000. (commit > 25a9f7427ddc1b584a721ceb0e12690a96d3639e ) > Any apps using this must be quite ancient code and in serious need of a > rewrite. Well, there's still lots of libldap client code around to support LDAP over UDP. Guarded with LDAP_CONNECTIONLESS #defines, and one can use "cldap://xxxx" urls with ldap_initialize() and do basic cldap searches and so on. Windows Server is accessed via CLDAP during discovery. Although there is normative documentation for this, it's easier to understand via these descriptions: http://wiki.wireshark.org/MS-CLDAP https://fedorahosted.org/sssd/wiki/DesignDocs/ActiveDirectoryDNSSites#Sen... So things like samba, IPA, realmd, adcli, and so on ... use and support cldap for talking with AD. I know Samba has reimplemented cldap but the others use libldap for this. Cheers, Stef

9 years, 11 months

1
0
0 / 0

Re: (ITS#7694) cldap fails with IPv6 due to wrong size sockaddr

by hyc＠symas.com

Stef Walter wrote: > On 10.10.2013 12:59, Howard Chu wrote: >> stefw(a)redhat.com wrote: >>> Full_Name: Stef Walter >>> Version: 2.4.35 >>> OS: Fedora 19 >>> Submission from: (NULL) (46.5.2.70) >>> >>> >>> Connectionless LDAP (ie: cldap enabled with -DLDAP_CONNECTIONLESS) is >>> broken for >>> IPv6 for current versions of openldap. Tested with version 2.4.35 >>> >>> It's not clear if this ever worked properly. >> >> No, clearly not, the code was written and deprecated before IPv6 >> existed. Nobody should be using this code today. > > Interesting. FWIW, the code is packaged by RHEL and Fedora, and is in > use by several projects. Can you list any of these, offhand? The original spec, RFC1798, is long obsoleted. There is no such thing as CLDAP in LDAPv3. Support in OpenLDAP was first removed back in 2000. (commit 25a9f7427ddc1b584a721ceb0e12690a96d3639e ) Any apps using this must be quite ancient code and in serious need of a rewrite. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

9 years, 11 months

1
0
0 / 0

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs October 2013