openldap-bugs

openldap-bugs@openldap.org

1 participants
20960 discussions

(ITS#6400) Different-sized CSNs confuse accesslog
by h.b.furuseth＠usit.uio.no 23 Nov '09

23 Nov '09

Full_Name: Hallvard B Furuseth Version: OS: URL: Submission from: (NULL) (129.240.6.233) Submitted by: hallvard Given CSNs of different lengths, overlays/accesslog.c:log_old_lookup() remembers truncated versions of the longer CSNs as "max CSN". The remembered length shrinks but never grows as CSNs change. I think the test if ( len > pd->csn.bv_len ) len = pd->csn.bv_len; should be if ( len > LDAP_PVT_CSNSTR_BUFSIZE ) len = LDAP_PVT_CSNSTR_BUFSIZE; since as far as I can tell pd->csn.bv_val is always csnbuf from accesslog_purge(). The compare just below must be fixed to match. It is buggy now anyway, CSNs of different lengths can compare equal.

1 0

Re: (ITS#6398) quiet valgrind (tool -F parsing)
by h.b.furuseth＠usit.uio.no 23 Nov '09

23 Nov '09

richton(a)nbcs.rutgers.edu writes: > Quiets a valgrind "definitely lost" leak complaint. > I submit this patch as public domain. Thanks. Fixed in HEAD. -- Hallvard

1 0

Re: (ITS#6397) accesslog + syncprov crash
by h.b.furuseth＠usit.uio.no 23 Nov '09

23 Nov '09

Poking a bit more in the backtrace (with a new coredump, sorry) shows that the filter given to test_filter() is NULL because *ss = { s_base = {bv_len = 6, bv_val = 0x1f94c5a0 "cn=log"}, s_eid = 1, s_op = 0x1f948310 { o_hdr = 0x1f948480 {all nulls}, o_controls = 0x1f9485c8 [(void *) 0x0]}, s_rid = 1, s_sid = -1, s_flags = 1, s_inuse = 1, <fields with null values omitted> } in frame 1, syncprov_matchops (op=0x42ec6690, opc=0x1f958ca0, saveit=1) at syncprov.c:1304: rc = test_filter( &op2, e, ss->s_op->ors_filter ); I have ss == si->si_ops != sm->sm_op. *si = { si_ops = 0x1f951c20, si_ctxcsn = 0x1f74fa70, si_contextdn = {bv_len = 6, bv_val = 0x1f4e4310 "cn=log"}, si_sids = 0x1f74fae0 [0], si_numcsns = 1, si_numops = 49, si_nopres = 1, si_usehint = 1, si_mods = 0x1fa615b0 {avl_data = 0x1fa61d20}, si_ops_mutex = { __data = {__lock = 1, __owner = 26790, __nusers = 1}, __size = "\001\000\000\000\000\000\000\000\246h\000\000\001", '\0' <repeats 26 times>, __align = 1}} Following si->si_mods: sml_mod.sm_type is garbage, is that relevant? *((modtarget *) si->si_mods->avl_data)->mt_mods->mi_op = { o_hdr = 0x42ec6800, o_tag = 102 <LDAP_REQ_MODIFY>, o_time = 1259003422, o_tincr = 0, o_bd = 0x42ec62d0, o_req_dn = {bv_len = 6, bv_val = 0x1f4e4400 "cn=log"}, o_req_ndn = {bv_len = 6, bv_val = 0x1f4e4310 "cn=log"}, o_request.oq_modify = { rs_mods = { rs_modlist = 0x42ec64c0 { sml_mod = { sm_desc = 0x1f4a46e0 <entryCSN>, sm_values = 0x42ec6500 &[ {bv_len = 40, bv_val = 0x42ec6530 "20091123191016.381756Z#000000#000#000000\311_\214\031="}, BER_BVNULL], sm_numvals = 1, sm_op = 2, sm_flags = 1, sm_type = {bv_len = 4222451713, bv_val = 0x42ec6680 ""}}}, rs_no_opattrs = 1 '\001'}}, o_no_schema_check = 1 '\001', o_ctrlflag = '\0' <repeats 14 times>, "\002", '\0' <repeats 16 times>, o_controls = 0x42ec6948 &[(void *) 0x0], o_authz = { sai_dn = {bv_len = 28, bv_val = 0x1f4e46f0 "cn=Manager,dc=example,dc=com"}, sai_ndn = {bv_len = 28, bv_val = 0x1f4e4d90 "cn=manager,dc=example,dc=com"}}, o_callback = 0x1f958c80, o_csn = {bv_len = 40, bv_val = 0x42ec6530 "20091123191016.381756Z#000000#000#000000\311_\214\031="}, o_extra = {slh_first = 0x1f958d28}} (gdb) backtrace #0 0x0000000000452066 in test_filter (op=0x42ec5bf0, e=0x1f577e48, f=0x0) at filterentry.c:69 #1 0x0000000000529cd1 in syncprov_matchops (op=0x42ec6690, opc=0x1f958ca0, saveit=1) at syncprov.c:1304 #2 0x000000000052c524 in syncprov_op_mod (op=0x42ec6690, rs=0x42ec6620) at syncprov.c:2064 #3 0x00000000004b3f0f in overlay_op_walk (op=0x42ec6690, rs=0x42ec6620, which=op_modify, oi=0x1f4e47e0, on=0x1f4e49c0) at backover.c:659 #4 0x00000000004b41a1 in over_op_func (op=0x42ec6690, rs=0x42ec6620, which=op_modify) at backover.c:721 #5 0x00000000004b42b7 in over_op_modify (op=0x42ec6690, rs=0x42ec6620) at backover.c:760 #6 0x0000000000521c67 in accesslog_purge (ctx=0x42ec6d60, arg=0x1f4e4760) at accesslog.c:699 #7 0x00000000005478ed in ldap_int_thread_pool_wrapper (xpool=0x1f4ae8a0) at tpool.c:685 #8 0x0000003d1a4064a7 in start_thread () from /lib64/libpthread.so.0 #9 0x0000003d198d3c2d in clone () from /lib64/libc.so.6 -- Hallvard

1 0

(ITS#6399) Documentation error for "referral" result code 10
by chris.czub＠gmail.com 23 Nov '09

23 Nov '09

Full_Name: Chris Czub Version: 2.4 OS: CentOS URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (208.82.64.120) http://www.openldap.org/doc/admin24/appendix-ldap-result-codes.html#{{refer… The referral section refers to section 4.1.10, which is non-existent in the documentation.

1 0

Re: (ITS#6396) slow reads on slaves after many changes in master
by quanah＠zimbra.com 23 Nov '09

23 Nov '09

--On Monday, November 23, 2009 8:38 PM +0200 rolnas(a)gmail.com wrote: > I raised (twice) to: > > cachesize 20000 > idlcachesize 100000 Ok, make sure dncachesize is set to zero as well. ;) > BDB software is 4.6.21.4 (from FreeBSD ports as OpenLDAP itself). > > I have no time (sorry) to try compile and run current CVS version. Well, as I said, there are a number of known problems with the entry cache in 2.4.19 which are fixed in CVS. So when you find the time, I'd suggest building the CVS version and using it instead. If it still occurs there, the developers will probably want additional information. --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

Re: (ITS#6396) slow reads on slaves after many changes in master
by rolnas＠gmail.com 23 Nov '09

23 Nov '09

On 2009.11.23, at 20:12, Quanah Gibson-Mount wrote: > --On Monday, November 23, 2009 8:09 PM +0200 rolnas(a)gmail.com wrote: >=20 >> P.S. I found myself cachesize < total number of entries. Probably I >> lowered it trying to workaround this problem. >=20 > Yes, please raise it. Also, please try the current CVS as I noted. = Also, what BDB version of software are you using? I raised (twice) to: cachesize 20000 idlcachesize 100000 but it didn't help. My test (ldapsearch of entire DB) usually takes ~2s, after changes (the = same sequence of deletes/adds) - ~18s on master, ~60s on mirror, ~140s = on slave. After restart, again ~2s. BDB software is 4.6.21.4 (from FreeBSD ports as OpenLDAP itself). I have no time (sorry) to try compile and run current CVS version. Regards, Rolandas Naujikas > Thanks! >=20 > --Quanah >=20 > -- >=20 > Quanah Gibson-Mount > Principal Software Engineer > Zimbra, Inc > -------------------- > Zimbra :: the leader in open source messaging and collaboration

1 0

Re: (ITS#6396) slow reads on slaves after many changes in master
by quanah＠zimbra.com 23 Nov '09

23 Nov '09

--On Monday, November 23, 2009 8:09 PM +0200 rolnas(a)gmail.com wrote: > P.S. I found myself cachesize < total number of entries. Probably I > lowered it trying to workaround this problem. Yes, please raise it. Also, please try the current CVS as I noted. Also, what BDB version of software are you using? Thanks! --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

(ITS#6398) quiet valgrind (tool -F parsing)
by richton＠nbcs.rutgers.edu 23 Nov '09

23 Nov '09

Full_Name: Aaron Richton Version: HEAD OS: irrelevant URL: Submission from: (NULL) (128.6.31.135) Quiets a valgrind "definitely lost" leak complaint. I submit this patch as public domain. --- slapcommon.c~ 2009-11-23 11:22:05.000000000 -0500 +++ slapcommon.c 2009-11-23 11:22:37.000000000 -0500 @@ -763,6 +763,11 @@ conffile = NULL; } + if ( confdir != NULL ) { + ch_free( confdir ); + confdir = NULL; + } + if ( ldiffile != NULL ) { ch_free( ldiffile ); ldiffile = NULL;

1 0

Re: (ITS#6396) slow reads on slaves after many changes in master
by rolnas＠gmail.com 23 Nov '09

23 Nov '09

> ... > Can you reproduce this behavior with current CVS checkout of the = OPENLDAP_REL_ENG_2_4 branch? I'll note that the entry cache has been = broken for most of RE2.4, and it is specifically fixed in CVS and will = be in the upcoming 2.4.20 release. >=20 > You fail to note in this ITS any of the following information, which = would be useful: Sorry, It was my first submission. > (a) Total number of entries in the database ~15000 (14738) >=20 >> =46rom slapd.conf/cn=3Dconfig: > (b) cachesize 10000 > (c) idlcachesize 50000 > (d) dncachesize Not set, I found somewhere to better don't touch it. >> =46rom the BDB database: >=20 > (e) output of du -c -h *.bdb in the database directory 448K associatedDomain.bdb 288K carLicense.bdb 7.9M cn.bdb 7.1M displayName.bdb 4.4M dn2id.bdb 320K employeeNumber.bdb 528K entryCSN.bdb 544K entryUUID.bdb 464K gidNumber.bdb 3.1M givenName.bdb 30M id2entry.bdb 52K ipHostNumber.bdb 8.0K ipNetworkNumber.bdb 8.0K krb5PrincipalName.bdb 44K macAddress.bdb 336K mail.bdb 120K member.bdb 132K memberUid.bdb 1.3M objectClass.bdb 16K ou.bdb 8.0K sambaDomainName.bdb 60K sambaGroupType.bdb 336K sambaPrimaryGroupSID.bdb 11M sambaSID.bdb 3.7M sn.bdb 20K st.bdb 3.7M uid.bdb 304K uidNumber.bdb 75M total > (f) Contents of your DB_CONFIG file # 100MB cache set_cachesize 0 104857600 1 # 1MB cache for tables and index file names set_lg_regionmax 1048576 # 10MB log files (> 4*set_lg_bsize) set_lg_max 10485760 # 2MB memory cache for log files set_lg_bsize 2097152 # autoremove old log files set_flags DB_LOG_AUTOREMOVE I want add more to this info, if it is relevant: checkpoint 10240 5 cachefree 1000 overlay syncprov syncprov-checkpoint 1000 5 syncprov-sessionlog 10000 Regards, Rolandas Naujikas P.S. I found myself cachesize < total number of entries. Probably I = lowered it trying to workaround this problem. > Thanks, > Quanah >=20 > -- >=20 > Quanah Gibson-Mount > Principal Software Engineer > Zimbra, Inc > -------------------- > Zimbra :: the leader in open source messaging and collaboration

1 0

Re: (ITS#6396) slow reads on slaves after many changes in master
by quanah＠zimbra.com 23 Nov '09

23 Nov '09

--On Monday, November 23, 2009 9:02 AM +0000 rolnas(a)gmail.com wrote: > Full_Name: Rolandas Naujikas > Version: 2.4.19 > OS: FreeBSD 7.1 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (217.147.36.211) > > > We have a setup of 3 slapd: 2 of them running in mirror mode and last one > as a read-only slave. All entries could fit in the cache (my goal is good > read performance). > > Usually when master receive only small changes (modify + several adds) - > read performance is OK. > > When I do ~5% content change (doing a lot of deletes and adds in short > period), then read performance drops sharply - on master not so much > (2x), on slaves 10-20x. In the same time CPU load on slaves when doing > read is becoming high (almost 100%), on master - normal. > > It rests in such situation until I restart slave instances, after that > all such performance problems disappears. > > By comparison version 2.4.16 didn't show such behavior and for some time > (until 2.4.19 was not declared stable) we used this older version. > Can you reproduce this behavior with current CVS checkout of the OPENLDAP_REL_ENG_2_4 branch? I'll note that the entry cache has been broken for most of RE2.4, and it is specifically fixed in CVS and will be in the upcoming 2.4.20 release. You fail to note in this ITS any of the following information, which would be useful: (a) Total number of entries in the database >From slapd.conf/cn=config: (b) cachesize (c) idlcachesize (d) dncachesize >From the BDB database: (e) output of du -c -h *.bdb in the database directory (f) Contents of your DB_CONFIG file Thanks, Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs