openldap-bugs

openldap-bugs@openldap.org

1 participants
20961 discussions

Re: (ITS#7655) segfault during initial mirror of multimaster delta replication
by quanah＠zimbra.com 05 Aug '13

05 Aug '13

--On Sunday, August 04, 2013 4:27 PM +0000 hans.freitag(a)entiretec.com wrote: > Full_Name: Hans Freitag > Version: 2.4.35 and 33 > OS: SLES 11SP2 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (193.200.138.3) > > > I have a Multimaster Delta replication setup here with bdb on a 18 GB > Database. > > After a crash due to a full disk I made a new database on one node ans > started over. > > The empty node started to replicate, from the full one but after a while > (approx. 2GB) it crashed with a segfault: > > Aug 4 11:45:32 mhr-dd-lda-01 kernel: [52189.476209] slapd[10158]: > segfault at 20 ip 00007ff97ebfabc0 sp 00007ff6e57e6b38 error 4 in > libc-2.11.1.so[7ff97eb79000+155000] > > So i thought, maybe it is not e good Idea to put in a package for SP2 in a > machine running SP1 so my first attempt to solve was an upgrade. After the > upgrade I got this: > > Aug 4 12:46:29 mhr-dd-lda-01 kernel: [ 1414.757587] slapd[3704]: > segfault at 20 ip 00007fc82eee6182 sp 00007fc592e0acf0 error 4 in > slapd[7fc82ee7a000+1e6000] > > So I created a brandnew openldap RPM 2.4.35 rpm to try out if the problem > is maybe related to the 2.4.33 version I am running. But fail: > > Aug 4 13:47:19 mhr-dd-lda-01 kernel: [ 5063.074410] slapd[8749]: > segfault at 20 ip 00007fcbc1b537dc sp 00007fc92624fb88 error 4 in > slapd[7fcbc1ac8000+1ea000] > > At the moment I deactivated the accesslogging on the node which seems to > work. I will know for sure in a few hours. ;-) I can try to reproduce > that on a backup node next week. Whenn all the main nodes are up and > running again. :) I would suggest you build with debugging symbols, enable core files, and provide a backtrace of the problem. What you have provided does not give any useful information for debugging purposes. You also fail to state the backend you are using (back-bdb or back-hdb). For information on how to provide a backtrace: <http://www.openldap.org/faq/data/cache/59.html> Regards, Quanah -- Quanah Gibson-Mount Lead Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

(ITS#7655) segfault during initial mirror of multimaster delta replication
by hans.freitag＠entiretec.com 04 Aug '13

04 Aug '13

Full_Name: Hans Freitag Version: 2.4.35 and 33 OS: SLES 11SP2 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (193.200.138.3) I have a Multimaster Delta replication setup here with bdb on a 18 GB Database. After a crash due to a full disk I made a new database on one node ans started over. The empty node started to replicate, from the full one but after a while (approx. 2GB) it crashed with a segfault: Aug 4 11:45:32 mhr-dd-lda-01 kernel: [52189.476209] slapd[10158]: segfault at 20 ip 00007ff97ebfabc0 sp 00007ff6e57e6b38 error 4 in libc-2.11.1.so[7ff97eb79000+155000] So i thought, maybe it is not e good Idea to put in a package for SP2 in a machine running SP1 so my first attempt to solve was an upgrade. After the upgrade I got this: Aug 4 12:46:29 mhr-dd-lda-01 kernel: [ 1414.757587] slapd[3704]: segfault at 20 ip 00007fc82eee6182 sp 00007fc592e0acf0 error 4 in slapd[7fc82ee7a000+1e6000] So I created a brandnew openldap RPM 2.4.35 rpm to try out if the problem is maybe related to the 2.4.33 version I am running. But fail: Aug 4 13:47:19 mhr-dd-lda-01 kernel: [ 5063.074410] slapd[8749]: segfault at 20 ip 00007fcbc1b537dc sp 00007fc92624fb88 error 4 in slapd[7fcbc1ac8000+1ea000] At the moment I deactivated the accesslogging on the node which seems to work. I will know for sure in a few hours. ;-) I can try to reproduce that on a backup node next week. Whenn all the main nodes are up and running again. :)

1 0

ITS#7599 libldap and libclntsh
by hyc＠symas.com 02 Aug '13

02 Aug '13

This is an Oracle bug, please contact Oracle for a fix. Their library should not be exporting symbols from the ldap_* namespace; this space has been owned by libldap since the 1990s. Closing this ITS. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#7654) [PATCH] back-meta/config.c: #include "ldif.h"
by fumiyas＠osstech.co.jp 02 Aug '13

02 Aug '13

Full_Name: SATOH Fumiyasu Version: master OS: URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (240b:10:2640:bf0:222:5fff:fe95:3cda) >From 9d009b4890586354e94be92f4ddb2462e9d18158 Mon Sep 17 00:00:00 2001 From: SATOH Fumiyasu <fumiyas(a)osstech.co.jp> Date: Fri, 2 Aug 2013 18:20:05 +0900 Subject: [PATCH] back-meta/config.c: #include "ldif.h" --- servers/slapd/back-meta/config.c | 1 + 1 file changed, 1 insertion(+) diff --git a/servers/slapd/back-meta/config.c b/servers/slapd/back-meta/config.c index 0be9ed8..a23f5ef 100644 --- a/servers/slapd/back-meta/config.c +++ b/servers/slapd/back-meta/config.c @@ -30,6 +30,7 @@ #include "slap.h" #include "config.h" #include "lutil.h" +#include "ldif.h" #include "../back-ldap/back-ldap.h" #include "back-meta.h" -- 1.8.3.2

1 0

Re: (ITS#7651) LMDB: Uncontrolled database when opened from multiple processes
by h.b.furuseth＠usit.uio.no 01 Aug '13

01 Aug '13

Howard Chu writes: > h.b.furuseth(a)usit.uio.no wrote: >> I'd call this two doc bugs: >> >> Something should mention that once a page has been used, the file will >> never shrink below that page. (Can't quite say that the file never >> shrinks, since WRITEMAP and Windows will set it to exact re-configurable >> mapsize in some cases. If I remember correctly.) > > You cannot shrink the mapsize below that last used page, and the last used > page is a non-decreasing number. Yes. Duh, that's already roughly stated under mdb_env_set_mapsize(). >> mdb_env_copy() and mdb_env_copyfd() should document that >> >> @node This call can trigger siginficant file size growth if run in >> parallel with write transactions, because it employs a read-only >> transaction. See long-lived transactions under Caveats. I'll add this one though. >> I wonder that means these functions should have a flag parameter. >> For some databases, users might prefer a _write_ transaction while >> reading the DB, blocking writes while copying. > > No one has ever wanted a hot-backup function that halts writes. You > might as well not do a hot-backup at all. Yes, nevermind about a mdb_env_copy() flag. An mdb_txn_begin timeout would make it a viable choice - but if that is implemented someday, those two behaviors together could go in an MDB_env flag. -- Hallvard

1 0

Re: (ITS#7651) LMDB: Uncontrolled database when opened from multiple processes
by hyc＠symas.com 31 Jul '13

31 Jul '13

h.b.furuseth(a)usit.uio.no wrote: > I'd call this two doc bugs: > > Something should mention that once a page has been used, the file will > never shrink below that page. (Can't quite say that the file never > shrinks, since WRITEMAP and Windows will set it to exact re-configurable > mapsize in some cases. If I remember correctly.) You cannot shrink the mapsize below that last used page, and the last used page is a non-decreasing number. > mdb_env_copy() and mdb_env_copyfd() should document that > > @node This call can trigger siginficant file size growth if run in > parallel with write transactions, because it employs a read-only > transaction. See long-lived transactions under Caveats. > > I wonder that means these functions should have a flag parameter. > For some databases, users might prefer a _write_ transaction while > reading the DB, blocking writes while copying. No one has ever wanted a hot-backup function that halts writes. You might as well not do a hot-backup at all. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7651) LMDB: Uncontrolled database when opened from multiple processes
by h.b.furuseth＠usit.uio.no 31 Jul '13

31 Jul '13

I'd call this two doc bugs: Something should mention that once a page has been used, the file will never shrink below that page. (Can't quite say that the file never shrinks, since WRITEMAP and Windows will set it to exact re-configurable mapsize in some cases. If I remember correctly.) mdb_env_copy() and mdb_env_copyfd() should document that @node This call can trigger siginficant file size growth if run in parallel with write transactions, because it employs a read-only transaction. See long-lived transactions under Caveats. I wonder that means these functions should have a flag parameter. For some databases, users might prefer a _write_ transaction while reading the DB, blocking writes while copying. -- Hallvard

1 0

Re: (ITS#7651) LMDB: Uncontrolled database when opened from multiple processes
by hyc＠symas.com 31 Jul '13

31 Jul '13

Dimitrij Denissenko wrote: > Makes sense, but out of interest: shouldn't the file size drop again, > when locked pages are released? No. All allocated pages get reused internally, so there's no point giving them back to the OS. > > On 31/07/13 15:34, Howard Chu wrote: >> dimitrij.denissenko(a)blacksquaremedia.com wrote: >>> Hi again, >>> >>> I was able to recreate the scenario in a test case. I must have mixed >>> something up previously. After more tests it seems that the memory >>> growth only appears when transactions are reset within the child. >>> >>> Here's the full code: >>> https://gist.github.com/dim/6122131 >>> >>> Output on my machine: >>> >>> $ rm -f testdb/* && mkdir -p testdb && make mtest8 && ./mtest8 >>> gcc -pthread -O2 -g -W -Wall -Wno-unused-parameter -Wbad-function-cast >>> -c mtest8.c >>> gcc -pthread -O2 -g -W -Wall -Wno-unused-parameter -Wbad-function-cast >>> mtest8.o liblmdb.a -o mtest8 >>> NO FORK : 1699840 >>> FORK & OPEN : 1699840 >>> FORK & COPY : 6225920 >> >> This behavior is expected. Your child process is holding a read txn >> while it performs the env_copyfd(). During that time writers are not >> able to reuse old data pages, so all writes consume new pages. When >> the copyfd completes and releases its read txn the writers will resume >> reusing old pages. >> >> There's no bug here, this ITS will be closed. >> > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7651) LMDB: Uncontrolled database when opened from multiple processes
by dimitrij.denissenko＠blacksquaremedia.com 31 Jul '13

31 Jul '13

Makes sense, but out of interest: shouldn't the file size drop again, when locked pages are released? On 31/07/13 15:34, Howard Chu wrote: > dimitrij.denissenko(a)blacksquaremedia.com wrote: >> Hi again, >> >> I was able to recreate the scenario in a test case. I must have mixed >> something up previously. After more tests it seems that the memory >> growth only appears when transactions are reset within the child. >> >> Here's the full code: >> https://gist.github.com/dim/6122131 >> >> Output on my machine: >> >> $ rm -f testdb/* && mkdir -p testdb && make mtest8 && ./mtest8 >> gcc -pthread -O2 -g -W -Wall -Wno-unused-parameter -Wbad-function-cast >> -c mtest8.c >> gcc -pthread -O2 -g -W -Wall -Wno-unused-parameter -Wbad-function-cast >> mtest8.o liblmdb.a -o mtest8 >> NO FORK : 1699840 >> FORK & OPEN : 1699840 >> FORK & COPY : 6225920 > > This behavior is expected. Your child process is holding a read txn > while it performs the env_copyfd(). During that time writers are not > able to reuse old data pages, so all writes consume new pages. When > the copyfd completes and releases its read txn the writers will resume > reusing old pages. > > There's no bug here, this ITS will be closed. >

1 0

Re: (ITS#7651) LMDB: Uncontrolled database when opened from multiple processes
by hyc＠symas.com 31 Jul '13

31 Jul '13

dimitrij.denissenko(a)blacksquaremedia.com wrote: > Hi again, > > I was able to recreate the scenario in a test case. I must have mixed > something up previously. After more tests it seems that the memory > growth only appears when transactions are reset within the child. > > Here's the full code: > https://gist.github.com/dim/6122131 > > Output on my machine: > > $ rm -f testdb/* && mkdir -p testdb && make mtest8 && ./mtest8 > gcc -pthread -O2 -g -W -Wall -Wno-unused-parameter -Wbad-function-cast > -c mtest8.c > gcc -pthread -O2 -g -W -Wall -Wno-unused-parameter -Wbad-function-cast > mtest8.o liblmdb.a -o mtest8 > NO FORK : 1699840 > FORK & OPEN : 1699840 > FORK & COPY : 6225920 This behavior is expected. Your child process is holding a read txn while it performs the env_copyfd(). During that time writers are not able to reuse old data pages, so all writes consume new pages. When the copyfd completes and releases its read txn the writers will resume reusing old pages. There's no bug here, this ITS will be closed. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs