openldap-bugs November 2009

openldap-bugs@openldap.org

40 participants
183 discussions

Re: (ITS#6396) slow reads on slaves after many changes in master
by rolnas＠gmail.com 23 Nov '09

23 Nov '09

On 2009.11.23, at 20:12, Quanah Gibson-Mount wrote: > --On Monday, November 23, 2009 8:09 PM +0200 rolnas(a)gmail.com wrote: >=20 >> P.S. I found myself cachesize < total number of entries. Probably I >> lowered it trying to workaround this problem. >=20 > Yes, please raise it. Also, please try the current CVS as I noted. = Also, what BDB version of software are you using? I raised (twice) to: cachesize 20000 idlcachesize 100000 but it didn't help. My test (ldapsearch of entire DB) usually takes ~2s, after changes (the = same sequence of deletes/adds) - ~18s on master, ~60s on mirror, ~140s = on slave. After restart, again ~2s. BDB software is 4.6.21.4 (from FreeBSD ports as OpenLDAP itself). I have no time (sorry) to try compile and run current CVS version. Regards, Rolandas Naujikas > Thanks! >=20 > --Quanah >=20 > -- >=20 > Quanah Gibson-Mount > Principal Software Engineer > Zimbra, Inc > -------------------- > Zimbra :: the leader in open source messaging and collaboration

1 0

Re: (ITS#6396) slow reads on slaves after many changes in master
by quanah＠zimbra.com 23 Nov '09

23 Nov '09

--On Monday, November 23, 2009 8:09 PM +0200 rolnas(a)gmail.com wrote: > P.S. I found myself cachesize < total number of entries. Probably I > lowered it trying to workaround this problem. Yes, please raise it. Also, please try the current CVS as I noted. Also, what BDB version of software are you using? Thanks! --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

(ITS#6398) quiet valgrind (tool -F parsing)
by richton＠nbcs.rutgers.edu 23 Nov '09

23 Nov '09

Full_Name: Aaron Richton Version: HEAD OS: irrelevant URL: Submission from: (NULL) (128.6.31.135) Quiets a valgrind "definitely lost" leak complaint. I submit this patch as public domain. --- slapcommon.c~ 2009-11-23 11:22:05.000000000 -0500 +++ slapcommon.c 2009-11-23 11:22:37.000000000 -0500 @@ -763,6 +763,11 @@ conffile = NULL; } + if ( confdir != NULL ) { + ch_free( confdir ); + confdir = NULL; + } + if ( ldiffile != NULL ) { ch_free( ldiffile ); ldiffile = NULL;

1 0

Re: (ITS#6396) slow reads on slaves after many changes in master
by rolnas＠gmail.com 23 Nov '09

23 Nov '09

> ... > Can you reproduce this behavior with current CVS checkout of the = OPENLDAP_REL_ENG_2_4 branch? I'll note that the entry cache has been = broken for most of RE2.4, and it is specifically fixed in CVS and will = be in the upcoming 2.4.20 release. >=20 > You fail to note in this ITS any of the following information, which = would be useful: Sorry, It was my first submission. > (a) Total number of entries in the database ~15000 (14738) >=20 >> =46rom slapd.conf/cn=3Dconfig: > (b) cachesize 10000 > (c) idlcachesize 50000 > (d) dncachesize Not set, I found somewhere to better don't touch it. >> =46rom the BDB database: >=20 > (e) output of du -c -h *.bdb in the database directory 448K associatedDomain.bdb 288K carLicense.bdb 7.9M cn.bdb 7.1M displayName.bdb 4.4M dn2id.bdb 320K employeeNumber.bdb 528K entryCSN.bdb 544K entryUUID.bdb 464K gidNumber.bdb 3.1M givenName.bdb 30M id2entry.bdb 52K ipHostNumber.bdb 8.0K ipNetworkNumber.bdb 8.0K krb5PrincipalName.bdb 44K macAddress.bdb 336K mail.bdb 120K member.bdb 132K memberUid.bdb 1.3M objectClass.bdb 16K ou.bdb 8.0K sambaDomainName.bdb 60K sambaGroupType.bdb 336K sambaPrimaryGroupSID.bdb 11M sambaSID.bdb 3.7M sn.bdb 20K st.bdb 3.7M uid.bdb 304K uidNumber.bdb 75M total > (f) Contents of your DB_CONFIG file # 100MB cache set_cachesize 0 104857600 1 # 1MB cache for tables and index file names set_lg_regionmax 1048576 # 10MB log files (> 4*set_lg_bsize) set_lg_max 10485760 # 2MB memory cache for log files set_lg_bsize 2097152 # autoremove old log files set_flags DB_LOG_AUTOREMOVE I want add more to this info, if it is relevant: checkpoint 10240 5 cachefree 1000 overlay syncprov syncprov-checkpoint 1000 5 syncprov-sessionlog 10000 Regards, Rolandas Naujikas P.S. I found myself cachesize < total number of entries. Probably I = lowered it trying to workaround this problem. > Thanks, > Quanah >=20 > -- >=20 > Quanah Gibson-Mount > Principal Software Engineer > Zimbra, Inc > -------------------- > Zimbra :: the leader in open source messaging and collaboration

1 0

Re: (ITS#6396) slow reads on slaves after many changes in master
by quanah＠zimbra.com 23 Nov '09

23 Nov '09

--On Monday, November 23, 2009 9:02 AM +0000 rolnas(a)gmail.com wrote: > Full_Name: Rolandas Naujikas > Version: 2.4.19 > OS: FreeBSD 7.1 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (217.147.36.211) > > > We have a setup of 3 slapd: 2 of them running in mirror mode and last one > as a read-only slave. All entries could fit in the cache (my goal is good > read performance). > > Usually when master receive only small changes (modify + several adds) - > read performance is OK. > > When I do ~5% content change (doing a lot of deletes and adds in short > period), then read performance drops sharply - on master not so much > (2x), on slaves 10-20x. In the same time CPU load on slaves when doing > read is becoming high (almost 100%), on master - normal. > > It rests in such situation until I restart slave instances, after that > all such performance problems disappears. > > By comparison version 2.4.16 didn't show such behavior and for some time > (until 2.4.19 was not declared stable) we used this older version. > Can you reproduce this behavior with current CVS checkout of the OPENLDAP_REL_ENG_2_4 branch? I'll note that the entry cache has been broken for most of RE2.4, and it is specifically fixed in CVS and will be in the upcoming 2.4.20 release. You fail to note in this ITS any of the following information, which would be useful: (a) Total number of entries in the database >From slapd.conf/cn=config: (b) cachesize (c) idlcachesize (d) dncachesize >From the BDB database: (e) output of du -c -h *.bdb in the database directory (f) Contents of your DB_CONFIG file Thanks, Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

(ITS#6397) accesslog + syncprov crash
by h.b.furuseth＠usit.uio.no 23 Nov '09

23 Nov '09

Full_Name: Hallvard B Furuseth Version: HEAD, RE24 OS: Linux URL: ftp://ftp.openldap.org/incoming/Hallvard-Furuseth-091123.tgz Submission from: (NULL) (129.240.6.233) Submitted by: hallvard slapd dumps core in test043-delta-syncrepl with this patch to excite accesslog. While I didn't expect good behavior with that change, I didn't expect a coredump either. --- tests/data/slapd-deltasync-master.conf~ 2009-04-06 17:26:37 +0200 +++ tests/data/slapd-deltasync-master.conf 2009-11-23 15:09:56 +0100 @@ -40,4 +40,5 @@ directory @TESTDIR@/db.1.b rootdn "cn=Manager,dc=example,dc=com" +index reqStart eq #bdb#index objectClass eq #bdb#index entryUUID,entryCSN eq @@ -78,4 +79,5 @@ logops writes logsuccess true +logpurge 00:00:05 00:00:03 #monitor#database monitor Happens ca every 2nd test043 run with RE24 --enable-accesslog. Output with backtrace and testrun/ attached.

1 0

(ITS#6396) slow reads on slaves after many changes in master
by rolnas＠gmail.com 23 Nov '09

23 Nov '09

Full_Name: Rolandas Naujikas Version: 2.4.19 OS: FreeBSD 7.1 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (217.147.36.211) We have a setup of 3 slapd: 2 of them running in mirror mode and last one as a read-only slave. All entries could fit in the cache (my goal is good read performance). Usually when master receive only small changes (modify + several adds) - read performance is OK. When I do ~5% content change (doing a lot of deletes and adds in short period), then read performance drops sharply - on master not so much (2x), on slaves 10-20x. In the same time CPU load on slaves when doing read is becoming high (almost 100%), on master - normal. It rests in such situation until I restart slave instances, after that all such performance problems disappears. By comparison version 2.4.16 didn't show such behavior and for some time (until 2.4.19 was not declared stable) we used this older version.

1 0

Re: (ITS#6390) test020 seg faults
by rhafer＠suse.de 23 Nov '09

23 Nov '09

Am Freitag 20 November 2009 19:45:53 schrieb quanah(a)zimbra.com: > --On Friday, November 20, 2009 3:53 PM +0000 quanah(a)zimbra.com wrote: > > --On Friday, November 20, 2009 12:35 PM +0000 rhafer(a)suse.de wrote: > >> Hm, I get the following warning during the make run: > >> > >> =2E./../../servers/slapd/syncrepl.c:2861 is: > >> comma =3D ber_bvrchr( &ptr, ',' ); > >> with ber_bvrchr being a macro around memrchr(). So it seems that > >> memrchr() = is=20 > >> undefined in syncrepl.c. > > > > Please don't add secondary issues to an ITS. File a new one. > > Unless you're saying that is the issue, which isn't clear to me in what you > wrote. :P I thought sending my mails against this ITS would be sufficient to show that I am talking about this issue :). I'll try to make that a bit more obvious next time. -- regards, Ralf

1 0

(ITS#6395) Web site release readme needs BDB version compatibility update
by battery＠writeme.com 22 Nov '09

22 Nov '09

Full_Name: Matt Kassawara Version: 2.4.18 OS: Linux URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (24.56.179.224) The release readme posted on the OpenLDAP web site doesn't explicitly indicate compatibility with BDB 4.8. http://www.openldap.org/software/release/readme.html

1 0

Re: (ITS#6300) Add kqueue support to slapd
by hyc＠symas.com 22 Nov '09

22 Nov '09

bduncan(a)apple.com wrote: > On Sep 22, 2009, at 1:02 PM, Quanah Gibson-Mount wrote: > >> --On Tuesday, September 22, 2009 7:23 PM +0000 bduncan(a)apple.com >> wrote: >> >>> Subject should have been: "PATCH - Added kqueue support to slapd" to >>> make clear this ITS tracks a patch that has been submitted. >> >> What OS(es) has this kqueue support been tested with? For example, >> the last time I looked at kqueue on OSX 10.5, it was very marginal, >> and missing most of the important pieces, which meant it wouldn't >> work for OpenLDAP or Postfix, etc. > > It was tested on OS X 10.6. Should work on all BSDs; nothing OSX- > specific about the kqueue usage in the patch. As far as kqueue > support in OS X 10.5, from what I see, most everything is there. > Certainly all of the kqueue features in this patch are supported in OS > X 10.5. To expand on this: there are a lot of comments around the web that kqueue was fairly broken at least up to 10.5. E.g. this: http://lists.schmorp.de/pipermail/libev/2008q4/000523.html Since we also use a pipe for the wake_sds[] in our event loop, if kqueue only supports sockets, then it cannot be used reliably in slapd. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs November 2009