openldap-bugs

openldap-bugs@openldap.org

1 participants
21000 discussions

Re: (ITS#6710) Mods already refreshed on a forwarding server is lost by its consumer
by hyc＠symas.com 21 Nov '10

21 Nov '10

rein(a)OpenLDAP.org wrote: > On 11/15/10 16:39 , rein(a)OpenLDAP.org wrote: >> New syncprov consumers connecting to a forwarding server and presenting an >> apperently up-to-date cookie will loose any mods that have already taken place >> on the forwarding server if it itself is refreshing from its provider. This >> should not be a problem if the forwarding server have a sufficiently large >> syncprov log, but a fix for servers without is coming. > The currently committed fix for this its leaves one problem open. If a > forwarding server restarts in the middle of the refresh phase, after > making some changes but before updating the csn, new consumers > connecting with an apparently up to date csn set after the server comes > up again will not know that the context is dirty and will loose these > changes. The same problem arise if the server restarts between the time > a locally initiated delete operation is performed in the database and > the accompanying csn set is saved. > > A fix could be to always assume the context is dirty after start up, and > thereby forcing all clients to undergo the refresh phase even if they > are in sync until some operation that updates the csn set is performed. > An unnecessary refresh is probably better than loosing changes.. I haven't had the opportunity to review these patches yet, but the bug description sounds a little flaky to me. The original design is this: when a consumer requests a refresh from the provider, the provider uses a snapshot of its current contextCSN. All changes from the consumer's cookie to this snapshot (inclusive) are sent to the consumer. Any changes currently in progress that have not yet been committed will be skipped until the next refresh. Nothing is lost, it's simply delayed, and that's in accordance with syncrepl's "eventual convergence" model. Likewise, the provider only updates its contextCSN when a change is fully committed. syncprov should NOT need to defer any consumer while it has outstanding mods. There is no reason that would be needed by the sync protocol. Servers only update their contextCSN after an entire refresh has completed. If a downstream consumer connects while a forwarder is still refreshing, the consumer should receive nothing. (Or, it should only receive the changes between its cookie and the server's committed contextCSN, if any.) You may very well have found bugs in the implementation, but it sounds to me like you've changed the overall behavior to something outside of the original design. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#6718) Send new-cookie messages to all consumers
by rein＠OpenLDAP.org 21 Nov '10

21 Nov '10

Full_Name: Rein Tollevik Version: CVS head OS: Irrelevant URL: Submission from: (NULL) (84.215.59.121) Submitted by: rein When syncprov updates its csn as a result of syncrepl updating its, syncprov currently sends NEW_COOKIE messages only to those consumers that replicate the entry where the csn is stored. It should be sent to all consumers. A fix is coming. -- Rein Tollevik Basefarm AS

1 0

Re: (ITS#6710) Mods already refreshed on a forwarding server is lost by its consumer
by rein＠OpenLDAP.org 21 Nov '10

21 Nov '10

On 11/15/10 16:39 , rein(a)OpenLDAP.org wrote: > New syncprov consumers connecting to a forwarding server and presenting an > apperently up-to-date cookie will loose any mods that have already taken place > on the forwarding server if it itself is refreshing from its provider. This > should not be a problem if the forwarding server have a sufficiently large > syncprov log, but a fix for servers without is coming. The currently committed fix for this its leaves one problem open. If a forwarding server restarts in the middle of the refresh phase, after making some changes but before updating the csn, new consumers connecting with an apparently up to date csn set after the server comes up again will not know that the context is dirty and will loose these changes. The same problem arise if the server restarts between the time a locally initiated delete operation is performed in the database and the accompanying csn set is saved. A fix could be to always assume the context is dirty after start up, and thereby forcing all clients to undergo the refresh phase even if they are in sync until some operation that updates the csn set is performed. An unnecessary refresh is probably better than loosing changes.. Rein

1 0

(ITS#6717) syncprov performs unnecessary presentations/refreshes.
by cmikk＠qwest.net 19 Nov '10

19 Nov '10

Full_Name: Chris Mikkelson Version: 2.4.23 OS: FreeBSD URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (204.147.85.37) I have a multimaster setup, and some read-only slaves mirroring the data with syncrepl. In some circumstances, I've noticed that restarting a slave server leads to a full presentation, even though the masters have an adequately sized sessionlog. This makes refreshOnly synchronization prohibitively expensive. It appears that the problem is as follows: The syncprov overlay uses the minimum CSN from the client cookie to decide where to start synchronization, regardless of whether that CSN has changed relative to the same SID's CSN on the provider. For example: Provider state: CSN: <1 second ago>...#001#... CSN: <1 year ago>.....#002#... sessionlog mincsn: <1 hour ago>...#001#... Consumer state: CSN: <1 minute ago>...#001#... CSN: <1 year ago>.....#002#... <-- same as SID 2 CSN on provider The syncprov overly will take the mincsn from the consumer cookie (the <1 year ago> one above) and compare it to the sessionlog mincsn when deciding whether to do a full presentation. Since the SID 2 CSN hasn't changed in the above example, the consumer is up to date on changes recorded on SID 2, so the SID 1 CSN from the cookie should be used. To reproduce: * Set up slapd servers ID 1 and 2 as multimaster nodes, and a third slapd server replicating from server 1. * Insert one entry into server ID 2, and restart servers 1 and 2. * Insert entries into Server ID 1. * Perform enough modifications / deletions on Server ID 1 to cause entries to be purged from the sessionlog (see ITS #6716). * Wait for slave to catch up, then stop the slave. * Do any modification on Server ID 1 (only need to change the SID 1 contextCSN) * Restart slave. Server will present entries to the slave rather than replaying the session log.

1 0

(ITS#6716) syncprov sessionlog 'sl_mincsn' not assigned a value.
by cmikk＠qwest.net 19 Nov '10

19 Nov '10

Full_Name: Chris Mikkelson Version: 2.4.23 OS: FreeBSD URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (204.147.85.37) On line 2875 of servers/slapd/overlays/syncprov.c (release 2.4.23), the sessionlog mincsn is initialized to a zero-length value. if ( !sl ) { sl = ch_malloc( sizeof( sessionlog ) + LDAP_PVT_CSNSTR_BUFSIZE ); sl->sl_mincsn.bv_val = (char *)(sl+1); sl->sl_mincsn.bv_len = 0; sl->sl_num = 0; sl->sl_head = sl->sl_tail = NULL; ldap_pvt_thread_mutex_init( &sl->sl_mutex ); si->si_logs = sl; } >From there, sl->sl_mincsn isn't touched until log entries are purged from the sessionlog on line 1522: while ( sl->sl_num > sl->sl_size ) { se = sl->sl_head; sl->sl_head = se->se_next; strcpy( sl->sl_mincsn.bv_val, se->se_csn.bv_val ); sl->sl_mincsn.bv_len = se->se_csn.bv_len; ch_free( se ); sl->sl_num--; } So, after the sessionlog has its first entry recorded and before it has its first entry received, sl->sl_mincsn is a zero-length value, which means the following comparison on line 2536: if ( sl->sl_num > 0 && ber_bvcmp( &mincsn, &sl->sl_mincsn ) >= 0 ) { do_present = 0; will evaluate to true in cases where it should not. A quick fix would be to compare directly against the CSN of the sessionlog's head: --- syncprov.c.orig 2010-11-17 14:07:26.000000000 -0600 +++ syncprov.c 2010-11-19 16:50:58.000000000 -0600 @@ -2533,7 +2533,7 @@ /* Are there any log entries, and is the consumer state * present in the session log? */ - if ( sl->sl_num > 0 && ber_bvcmp( &mincsn, &sl->sl_mincsn ) >= 0 ) { + if ( sl->sl_num > 0 && ber_bvcmp( &mincsn, &sl->sl_head->se_csn ) >= 0 ) { do_present = 0; /* mutex is unlocked in playlog */ syncprov_playlog( op, rs, sl, srs, ctxcsn, numcsns, sids );

1 0

Re: ITS#6713
by masarati＠aero.polimi.it 19 Nov '10

19 Nov '10

DevMan wrote: > Please let me know the upgradation procedure. <http://www.openldap.org/doc/admin24/install.html> In general, <http://www.openldap.org/doc/admin24/index.html> p.

1 0

Re: ITS#6713
by devman1624＠yahoo.com 19 Nov '10

19 Nov '10

Please let me know the upgradation procedure. Thx DevMan Sent from my iPhone On Nov 18, 2010, at 3:33 PM, Pierangelo Masarati <masarati(a)aero.polimi.it> w= rote: > Please note that the current release is 2.4.23; since 2.4.16 many issues h= ave been fixed. If the problem persists after upgrading, please make sure y= ou provide useful information, as illustrated here <http://www.openldap.org/= faq/data/cache/56.html> and in subordinate links. >=20 > p.

1 0

(ITS#6715) Broken 'sortvals' for attr with 1 value
by h.b.furuseth＠usit.uio.no 19 Nov '10

19 Nov '10

Full_Name: Hallvard B Furuseth Version: 2.4.23 OS: URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (129.240.6.233) Submitted by: hallvard slapd.conf:'sortvals ATTR' sometimes complains attributeType ATTR value #1 provided more than once when ATTR has one value, because slapd/modify.c:slap_sort_vals() then leaves the result value 'rc' uninitialized. Fixing.

1 0

(ITS#6714) Patch - Mozilla NSS - implement full non-blocking semantics
by rmeggins＠redhat.com 18 Nov '10

18 Nov '10

Full_Name: Rich Megginson Version: 2.4.23 (current CVS HEAD) OS: RHEL6 URL: ftp://ftp.openldap.org/incoming/openldap-2.4.23-use-non-blocking-semantics-… Submission from: (NULL) (76.113.111.209) This patch implements full non-blocking semantics for the MozNSS crypto implementation in the same manner as the openssl and gnutls implementations. SSL_ForceHandshake can be called repeatedly until it gets enough data to complete. One wrinkle is that, when SSL_ForceHandshake returns PR_WOULD_BLOCK_ERROR, we have no way of knowing if it needs data for a read or a write (a la openssl SSL_ERROR_WANT_WRITE and SSL_ERROR_WANT_READ). In order to keep track of that, we use the io_flag in the tls_data object - if the last operation called was a read/recv, we set the io_flag to TLSM_READ, and similar for send/write and TLSM_WRITE. This way, the upflags function knows how to set the sbiod needs_read and needs_write flags appropriately. I also added special handling for the common case where the client uses ldapTOOL -Z and the TLS fails from the client side (e.g. bogus CA cert file). In this case, tlsm_session_accept will get a plain LDAP message beginning with LBER_SEQUENCE instead of a valid SSL header. The code can short circuit the process and return a more meaningful error message. The tlsm_is_io_ready() function is now obsolete - keeping track of the io_flag, plus using the function SSL_DataPending(), removes the need for the former function. I added a convenience function tlsm_get_pvt_tls_data() to get the tls_data associated with the tlsm_session (secure PRFileDesc). This patch file is derived from OpenLDAP Software. All of the modifications to OpenLDAP Software represented in the following patch(es) were developed by Red Hat. Red Hat has not assigned rights and/or interest in this work to any party. I, Rich Megginson am authorized by Red Hat, my employer, to release this work under the following terms. Red Hat hereby place the following modifications to OpenLDAP Software (and only these modifications) into the public domain. Hence, these modifications may be freely used and/or redistributed for any purpose with or without attribution and/or other notice.

1 0

ITS#6712
by masarati＠aero.polimi.it 18 Nov '10

18 Nov '10

Thanks for the report. "hasSubordinates" is generated, not stored in the database; this may explain why multiple values appear despite its "single-valuedness". I'm investigating... p.

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs