openldap-devel

openldap-devel@openldap.org

1416 discussions

ITS#5835 insights?
by Quanah Gibson-Mount 02 Dec '08

02 Dec '08

Does anyone have any thoughts on ITS#5835? The server keeps hitting the assert in connection.c: static void connection_close( Connection *c ) { ber_socket_t sd = AC_SOCKET_INVALID; assert( connections != NULL ); assert( c != NULL ); /* ITS#4667 we may have gotten here twice */ if ( c->c_conn_state == SLAP_C_INVALID ) return; assert( c->c_struct_state == SLAP_C_USED ); --> assert( c->c_conn_state == SLAP_C_CLOSING ); Which was called by connections_timeout_idle in connection.c: if( difftime( c->c_activitytime+global_idletimeout, now) < 0 ) { /* close it */ connection_closing( c, "idletimeout" ); --> connection_close( c ); i++; With the modified debug logging Howard checked in, I see the following steps taking place: Dec 2 03:06:05 ldap slapd[8292]: connection_closing: readying conn=3049 sd=28 for close so we know it made it into connection_closing. Then we see: Dec 2 03:06:05 ldap slapd[8292]: connection_resched: attempting closing conn=3049 sd=28 which means it made it to connection_resched, which calls connection_close: Debug( LDAP_DEBUG_CONNS, "connection_resched: " "attempting closing conn=%lu sd=%d\n", conn->c_connid, sd, 0 ); connection_close( conn ); Doesn't this directly conflict with line 242 above, which also calls connection_close on the connection? I also see this comment in the connection_closing code: /* ITS#4667 this may allow another thread to drop into * connection_resched / connection_close before we * finish, but that's OK. */ I'm honestly not sure this is really "OK". The only place I see connection_resched called is from connection_operation, and I don't see exactly how I'ved ended up in there in this code path. --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

sd= vs fd=?
by Quanah Gibson-Mount 01 Dec '08

01 Dec '08

I was looking at connection.c, and we intermix sd= and fd= in our connection debug logs, even though they are being used for the same thing as best I can tell. Is there any reason not to just standardize on fd? --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

slapadd performance degradation from 2.3.43 to 2.4.12
by Ralf Haferkamp 26 Nov '08

26 Nov '08

Hi, While doing a few slapadd testruns comparing the RE23 with the RE24 version I ran into a strange issue. I ran test with different LDIFs (100k, 500k and 1000k Entries) and especially with the 500k and 1000k LDIF, slapadd from 2.3.43 was significantly faster than the 2.4.12 version. 2.3.43 loaded the 500k database in 13m54s while it took 33m49s with 2.4.12. The 1000k testcase to 29m41s on 2.3 (still faster than the 500k on 2.4). I didn't finish the 2.4 run with the 1000k database I stopped it after about an hour. I used exactly the same configuration on exactly the same hardware/os for the tests (a HP Proliant DL 580 G3 with 4 3.33GHz Xeons, 8GB of RAM, SLES10-SP2, ext3 filesystem). BerkelyDB Version was 4.5.20 with the following DB_CONFIG: set_cachesize 0 4294967295 1 set_lg_regionmax 262144 set_lg_bsize 2097152 cachesize in slapd.conf was large enough to hold the entire database and tool- threads was set to 4. I did some profiling (with valgrinds callgrind tool) to find out where all the time is spend and it revealed that 2.4 spend a significantly larger amount of systime in the pwrite() function than 2.3. Most of that seemed to come from the bdb_tool_trickle_task() that calls libdb's memp_trickle() function. Just to verify this I ran a testbuild with the trickle_task disabled(). And slapadd's performance was back to a normal level, comparable to the 2.3.43 release. AFAIK the trickle_task() was introduced into 2.4 to increase slapadd throughput but has exactly the opposite effect on my test system. Did anybody else make similar experiences? Or do you see anything thats obviously wrong with my testcases? -- Ralf

4 17

moduleload <statically linked module>
by Hallvard B Furuseth 25 Nov '08

25 Nov '08

Could we make "moduleload <statically linked module>" valid if the module name contains no "." or directory separator? Or introduce a new "module" keyword which works that way? That would simplify slapd.conf scripting, and documentation examples could simply include moduleload and would be correct without any caveats about whether the modules are statically linked. Also, moduleload seems to take an undocumented argument list which is passed to the module init routine. See module.c:module_load(). That ought to work whether or not the module is dynamically loaded. I don't think it does now. -- Hallvard

2 1

OpenLDAP 2.4.x: change of shared library version every release
by Xin LI 24 Nov '08

24 Nov '08

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, I just noticed that 2.4.12 and 2.4.13 both bumped the shared library version. Did we actually changed the API that makes the version bump necessary? Cheers, - -- Xin LI <delphij(a)delphij.net> http://www.delphij.net/ FreeBSD - The Power to Serve! -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (FreeBSD) iEYEARECAAYFAkkrL8EACgkQi+vbBBjt66A08gCfT6uIMZA4x9C+depPBG9NJBhp f4YAn3ilz2KxAZSTZgWnEcwAtPHCZMnr =XmW4 -----END PGP SIGNATURE-----

2 1

OpenLDAP 3.0 Roadmap
by Gavin Henry 24 Nov '08

24 Nov '08

To start the new thread. Thanks. -- Kind Regards, Gavin Henry. OpenLDAP Engineering Team. E ghenry(a)OpenLDAP.org Community developed LDAP software. http://www.openldap.org/project/

1 0

Leak in syncprov_matchops()?
by Pierangelo Masarati 23 Nov '08

23 Nov '08

==30484== 1,371 bytes in 43 blocks are definitely lost in loss record 12 of 13 ==30484== at 0x40053C0: malloc (vg_replace_malloc.c:149) ==30484== by 0x8258EFB: ber_memalloc_x (memory.c:226) ==30484== by 0x80A743C: ch_malloc (ch_malloc.c:54) ==30484== by 0x81747E5: bdb_entry_get (id2entry.c:421) ==30484== by 0x8104832: overlay_entry_get_ov (backover.c:365) ==30484== by 0x81FF53A: syncprov_matchops (syncprov.c:1151) ==30484== by 0x820175E: syncprov_op_mod (syncprov.c:1870) ==30484== by 0x8104F47: overlay_op_walk (backover.c:660) ==30484== by 0x810517C: over_op_func (backover.c:722) ==30484== by 0x81052AA: over_op_delete (backover.c:774) ==30484== by 0x80FA708: syncrepl_entry (syncrepl.c:2283) ==30484== by 0x80F5AC5: do_syncrep2 (syncrepl.c:877) ==30484== by 0x80F72C2: do_syncrepl (syncrepl.c:1301) ==30484== by 0x8086439: connection_read_thread (connection.c:1218) ==30484== by 0x8222E50: ldap_int_thread_pool_wrapper (tpool.c:663) ==30484== by 0xDEB46A: start_thread (in /lib/libpthread-2.5.so) ==30484== by 0x423FDBD: clone (in /lib/libc-2.5.so) This was the result of a concurrency test while testing MMR. Not sure about how to reproduce it, but I've seen lots of deadlock resolutions, so probably there's some exception we're not handling fine with respect to memory allocation (HEAD as of today). p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it ----------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Fax: +39 0382 476497 Email: ando(a)sys-net.it -----------------------------------

2 3

valgrind complains about use of uninitialized var
by Pierangelo Masarati 22 Nov '08

22 Nov '08

==30484== Conditional jump or move depends on uninitialised value(s) ==30484== at 0x81740C2: bdb_entry_release (id2entry.c:271) Apparently, boi_flags is not initialized. Where should it be initialized, and how, is still obscure to me (HEAD as of today). p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it ----------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Fax: +39 0382 476497 Email: ando(a)sys-net.it -----------------------------------

2 1

Strange issue with contextCSN
by Pierangelo Masarati 22 Nov '08

22 Nov '08

I'm running concurrency tests of MMR, and I see some strange issues: 1) loss of sync after multi-concurrent load (multiple concurrent ops on each server, and modifications to the same data subset on all servers, in order to trigger conflicts). I'm still trying to see if there is any pattern or clue about what failed (like finding some explanation in the logs). This happens once in a while after many operations. I don't expect this to be necessarily a bug; it might be the consequence of conflicts. Of course, it would be nice if slapd allows to clearly identify where the conflict occurred, to support manual resolution. 2) loss of sync after single-concurrent load (multiple concurrent ops on a single server). This is really inesplicable (to me), as there should be no conflict. The only possible explanation I see (but need to investigate further) is that an entry is added on a server, sync'd to another one and, in the meanwhile, deleted on the first one before its own sync gets back. This happens very seldom. 3) whay puzzled me a bit is that when I load a single server, I'd expect to end up with a single contextCSN containing the SID of that server. This is correct for the server I load, but the others, even when they get correctly sync'd, contain a contextCSN for each server in the MMR pool, and the contextCSN with the other SIDs don't get propagated to the server that was loaded. It's not clear why those CSNs are generated, and how they get into the loop and propagate between servers that do not receive direct modifications. 4) another thing that puzzled me a bit is that in some cases, when all servers are loaded, and the contextCSNs are one for each SID and the same in all of the servers, they are sorted randomly, and differently; for example: bash-3.2$ diff -u testrun/server2.out testrun/server3.out --- testrun/server2.out 2008-11-22 17:17:28.000000000 +0100 +++ testrun/server3.out 2008-11-22 17:17:28.000000000 +0100 @@ -2497,8 +2497,8 @@ associatedDomain: example.com entryCSN: 20081122161630.753152Z#000000#001#000000 contextCSN: 20081122161708.935242Z#000000#001#000000 -contextCSN: 20081122161658.195350Z#000000#002#000000 contextCSN: 20081122161658.193983Z#000000#003#000000 +contextCSN: 20081122161658.195350Z#000000#002#000000 Not a big deal (except for the need to sort values to compare them), but I'd expect them to be exactly in the same order... I'm going to pack my suite of tests and put them on ftp.openldap.org (and eventually add them to OpenLDAP's test suite, specifically meant to test MMR), but first I need to polish them a little bit and enucleate those that present issues, in order to open specific ITSes. p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it ----------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Fax: +39 0382 476497 Email: ando(a)sys-net.it -----------------------------------

1 0

LDAP_OPT_X_SASL_NOCANON breaks the tests
by Hallvard B Furuseth 20 Nov '08

20 Nov '08

e.g. test000-rootdse: "... Waiting 5 seconds for slapd to start... Could not set LDAP_OPT_X_SASL_NOCANON off" -- Hallvard

2 1

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-devel