openldap-devel

openldap-devel@openldap.org

1418 discussions

Re: commit: ldap/servers/slapd syncrepl.c
by hyc＠symas.com 02 Dec '08

02 Dec '08

If anyone has some free time it would be a good idea to extend tests 17, 18, and 19 for these cases. Modify an entry (some non-RDN attribute) and then rename the entry. We should test Refresh when both ops have already occurred to the entry, as well as Refresh when each op occurs in a separate refresh pass. And as noted in -bugs, we should test with an entry using a single-valued RDN attribute. On Wed, Dec 03, 2008 at 03:49:54AM +0000, hyc(a)OpenLDAP.org wrote: > Update of /repo/OpenLDAP/pkg/ldap/servers/slapd > > Modified Files: > syncrepl.c 1.431 -> 1.432 > > Log Message: > More for prev commit - if doing both modrdn and modify at once, only > set the CSN on the modify op. > > > CVS Web URLs: > http://www.openldap.org/devel/cvsweb.cgi/servers/slapd/ > http://www.openldap.org/devel/cvsweb.cgi/servers/slapd/syncrepl.c > > Changes are generally available on cvs.openldap.org (and CVSweb) > within 30 minutes of being committed.

2 1

OpenLDAP stuck deep in BDB?
by Quanah Gibson-Mount 02 Dec '08

02 Dec '08

I've seen this now with two different masters, using BDB 4.2.52 + patches, which we long considered stable: Thread 3 (Thread 1115703648 (LWP 31507)): #0 0x0000003fca90bd9c in __pread_nocancel () from /lib64/tls/libpthread.so.0 #1 0x0000002a97348edc in __os_io (dbenv=0xdc3100, op=1, fhp=0xf7d2d0, pgno=71531, pagesize=4096, buf=0x2cb08f8cf0 "_3", niop=0x42681b70) at ../dist/../os/os_rw.c:55 #2 0x0000002a973405ea in __memp_pgread (dbmfp=0xeb5780, mutexp=0x2cbba53150, bhp=0x2cb08f8c58, can_create=0) at ../dist/../mp/mp_bh.c:219 #3 0x0000002a973413f7 in __memp_fget (dbmfp=0xeb5780, pgnoaddr=0x42681cc4, flags=0, addrp=0x42681cc8) at ../dist/../mp/mp_fget.c:580 #4 0x0000002a972d18b3 in __bam_search (dbc=0xdb1dc40, root_pgno=Variable "root_pgno" is not available. ) at ../dist/../btree/bt_search.c:307 #5 0x0000002a972c6e27 in __bam_c_search (dbc=0xdb1dc40, root_pgno=0, key=0x42681f40, flags=28, exactp=0x42681df4) at ../dist/../btree/bt_cursor.c:2547 #6 0x0000002a972c7b7e in __bam_c_get (dbc=0xdb1dc40, key=0x42681f40, data=0x42681f20, flags=28, pgnop=0x42681e64) at ../dist/../btree/bt_cursor.c:962 #7 0x0000002a9730f51b in __db_c_get (dbc_arg=0x28ea700, key=0x42681f40, data=0x42681f20, flags=28) at ../dist/../db/db_cam.c:643 #8 0x0000002a973177d7 in __db_c_get_pp (dbc=0x28ea700, key=0x42681f40, data=0x42681f20, flags=28) at ../dist/../db/db_iface.c:1836 #9 0x0000002a97183dc2 in bdb_dn2id (op=0x428025e0, dn=0x42682028, ei=0x42682010, locker=44, lock=0x42681fd0) at dn2id.c:315 #10 0x0000002a971896fd in bdb_cache_find_ndn (op=0x428025e0, locker=44, ndn=0x291c370, res=0x42682310) at cache.c:341 #11 0x0000002a97189e39 in bdb_cache_find_id (op=0x428025e0, tid=0x0, id=25145, eip=0x42682310, islocked=0, locker=44, lock=0x42682250) at cache.c:716 #12 0x0000002a9717b1c7 in bdb_search (op=0x428025e0, rs=0x42802570) at search.c:696 #13 0x0000002a975921eb in syncprov_findcsn (op=0x5ccd900, mode=FIND_PRESENT) at syncprov.c:681 #14 0x0000002a9759690a in syncprov_op_search (op=0x5ccd900, rs=0x42803d60) at syncprov.c:2074 #15 0x000000000049b935 in overlay_op_walk (op=0x5ccd900, rs=0x42803d60, which=op_search, oi=0xded8c0, on=0xded700) at backover.c:640 #16 0x000000000049bb91 in over_op_func (op=0x5ccd900, rs=0x42803d60, which=op_search) at backover.c:702 #17 0x000000000049bc27 in over_op_search (op=0x5ccd900, rs=0x42803d60) at backover.c:724 #18 0x000000000042f1d8 in fe_op_search (op=0x5ccd900, rs=0x42803d60) at search.c:355 #19 0x000000000042ecac in do_search (op=0x5ccd900, rs=0x42803d60) at search.c:217 #20 0x000000000042bd9c in connection_operation (ctx=0x42803e90, arg_v=0x5ccd900) at connection.c:1133 #21 0x000000000042c28c in connection_read_thread (ctx=0x42803e90, argv=0x1e) at connection.c:1261 ---Type <return> to continue, or q <return> to quit--- #22 0x0000002a956c7c77 in ldap_int_thread_pool_wrapper (xpool=0x8a1f00) at tpool.c:478 #23 0x0000003fca90610a in start_thread () from /lib64/tls/libpthread.so.0 #24 0x0000003fca0c68c3 in clone () from /lib64/tls/libc.so.6 #25 0x0000000000000000 in ?? () Has anyone else seen anything like this? It seems there's possibly a bug inside BDB 4.2.52 that gets hit occasionally. --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

ITS#5835 insights?
by Quanah Gibson-Mount 02 Dec '08

02 Dec '08

Does anyone have any thoughts on ITS#5835? The server keeps hitting the assert in connection.c: static void connection_close( Connection *c ) { ber_socket_t sd = AC_SOCKET_INVALID; assert( connections != NULL ); assert( c != NULL ); /* ITS#4667 we may have gotten here twice */ if ( c->c_conn_state == SLAP_C_INVALID ) return; assert( c->c_struct_state == SLAP_C_USED ); --> assert( c->c_conn_state == SLAP_C_CLOSING ); Which was called by connections_timeout_idle in connection.c: if( difftime( c->c_activitytime+global_idletimeout, now) < 0 ) { /* close it */ connection_closing( c, "idletimeout" ); --> connection_close( c ); i++; With the modified debug logging Howard checked in, I see the following steps taking place: Dec 2 03:06:05 ldap slapd[8292]: connection_closing: readying conn=3049 sd=28 for close so we know it made it into connection_closing. Then we see: Dec 2 03:06:05 ldap slapd[8292]: connection_resched: attempting closing conn=3049 sd=28 which means it made it to connection_resched, which calls connection_close: Debug( LDAP_DEBUG_CONNS, "connection_resched: " "attempting closing conn=%lu sd=%d\n", conn->c_connid, sd, 0 ); connection_close( conn ); Doesn't this directly conflict with line 242 above, which also calls connection_close on the connection? I also see this comment in the connection_closing code: /* ITS#4667 this may allow another thread to drop into * connection_resched / connection_close before we * finish, but that's OK. */ I'm honestly not sure this is really "OK". The only place I see connection_resched called is from connection_operation, and I don't see exactly how I'ved ended up in there in this code path. --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

sd= vs fd=?
by Quanah Gibson-Mount 01 Dec '08

01 Dec '08

I was looking at connection.c, and we intermix sd= and fd= in our connection debug logs, even though they are being used for the same thing as best I can tell. Is there any reason not to just standardize on fd? --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

slapadd performance degradation from 2.3.43 to 2.4.12
by Ralf Haferkamp 26 Nov '08

26 Nov '08

Hi, While doing a few slapadd testruns comparing the RE23 with the RE24 version I ran into a strange issue. I ran test with different LDIFs (100k, 500k and 1000k Entries) and especially with the 500k and 1000k LDIF, slapadd from 2.3.43 was significantly faster than the 2.4.12 version. 2.3.43 loaded the 500k database in 13m54s while it took 33m49s with 2.4.12. The 1000k testcase to 29m41s on 2.3 (still faster than the 500k on 2.4). I didn't finish the 2.4 run with the 1000k database I stopped it after about an hour. I used exactly the same configuration on exactly the same hardware/os for the tests (a HP Proliant DL 580 G3 with 4 3.33GHz Xeons, 8GB of RAM, SLES10-SP2, ext3 filesystem). BerkelyDB Version was 4.5.20 with the following DB_CONFIG: set_cachesize 0 4294967295 1 set_lg_regionmax 262144 set_lg_bsize 2097152 cachesize in slapd.conf was large enough to hold the entire database and tool- threads was set to 4. I did some profiling (with valgrinds callgrind tool) to find out where all the time is spend and it revealed that 2.4 spend a significantly larger amount of systime in the pwrite() function than 2.3. Most of that seemed to come from the bdb_tool_trickle_task() that calls libdb's memp_trickle() function. Just to verify this I ran a testbuild with the trickle_task disabled(). And slapadd's performance was back to a normal level, comparable to the 2.3.43 release. AFAIK the trickle_task() was introduced into 2.4 to increase slapadd throughput but has exactly the opposite effect on my test system. Did anybody else make similar experiences? Or do you see anything thats obviously wrong with my testcases? -- Ralf

4 17

moduleload <statically linked module>
by Hallvard B Furuseth 25 Nov '08

25 Nov '08

Could we make "moduleload <statically linked module>" valid if the module name contains no "." or directory separator? Or introduce a new "module" keyword which works that way? That would simplify slapd.conf scripting, and documentation examples could simply include moduleload and would be correct without any caveats about whether the modules are statically linked. Also, moduleload seems to take an undocumented argument list which is passed to the module init routine. See module.c:module_load(). That ought to work whether or not the module is dynamically loaded. I don't think it does now. -- Hallvard

2 1

OpenLDAP 2.4.x: change of shared library version every release
by Xin LI 24 Nov '08

24 Nov '08

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, I just noticed that 2.4.12 and 2.4.13 both bumped the shared library version. Did we actually changed the API that makes the version bump necessary? Cheers, - -- Xin LI <delphij(a)delphij.net> http://www.delphij.net/ FreeBSD - The Power to Serve! -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (FreeBSD) iEYEARECAAYFAkkrL8EACgkQi+vbBBjt66A08gCfT6uIMZA4x9C+depPBG9NJBhp f4YAn3ilz2KxAZSTZgWnEcwAtPHCZMnr =XmW4 -----END PGP SIGNATURE-----

2 1

OpenLDAP 3.0 Roadmap
by Gavin Henry 24 Nov '08

24 Nov '08

To start the new thread. Thanks. -- Kind Regards, Gavin Henry. OpenLDAP Engineering Team. E ghenry(a)OpenLDAP.org Community developed LDAP software. http://www.openldap.org/project/

1 0

Leak in syncprov_matchops()?
by Pierangelo Masarati 23 Nov '08

23 Nov '08

==30484== 1,371 bytes in 43 blocks are definitely lost in loss record 12 of 13 ==30484== at 0x40053C0: malloc (vg_replace_malloc.c:149) ==30484== by 0x8258EFB: ber_memalloc_x (memory.c:226) ==30484== by 0x80A743C: ch_malloc (ch_malloc.c:54) ==30484== by 0x81747E5: bdb_entry_get (id2entry.c:421) ==30484== by 0x8104832: overlay_entry_get_ov (backover.c:365) ==30484== by 0x81FF53A: syncprov_matchops (syncprov.c:1151) ==30484== by 0x820175E: syncprov_op_mod (syncprov.c:1870) ==30484== by 0x8104F47: overlay_op_walk (backover.c:660) ==30484== by 0x810517C: over_op_func (backover.c:722) ==30484== by 0x81052AA: over_op_delete (backover.c:774) ==30484== by 0x80FA708: syncrepl_entry (syncrepl.c:2283) ==30484== by 0x80F5AC5: do_syncrep2 (syncrepl.c:877) ==30484== by 0x80F72C2: do_syncrepl (syncrepl.c:1301) ==30484== by 0x8086439: connection_read_thread (connection.c:1218) ==30484== by 0x8222E50: ldap_int_thread_pool_wrapper (tpool.c:663) ==30484== by 0xDEB46A: start_thread (in /lib/libpthread-2.5.so) ==30484== by 0x423FDBD: clone (in /lib/libc-2.5.so) This was the result of a concurrency test while testing MMR. Not sure about how to reproduce it, but I've seen lots of deadlock resolutions, so probably there's some exception we're not handling fine with respect to memory allocation (HEAD as of today). p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it ----------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Fax: +39 0382 476497 Email: ando(a)sys-net.it -----------------------------------

2 3

valgrind complains about use of uninitialized var
by Pierangelo Masarati 22 Nov '08

22 Nov '08

==30484== Conditional jump or move depends on uninitialised value(s) ==30484== at 0x81740C2: bdb_entry_release (id2entry.c:271) Apparently, boi_flags is not initialized. Where should it be initialized, and how, is still obscure to me (HEAD as of today). p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it ----------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Fax: +39 0382 476497 Email: ando(a)sys-net.it -----------------------------------

2 1

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-devel