openldap-bugs

openldap-bugs@openldap.org

1 participants
20967 discussions

Re: (ITS#5376) Full sync fails with delta-syncrepl, refreshAndPersist, and high modifications
by hyc＠symas.com 13 Feb '08

13 Feb '08

quanah(a)zimbra.com wrote: > With a build of 2.3 with the (adjusted) patches from HEAD, it looks like > this issue is resolved. I fully sync'd a replica from the master, and > after it was done, I did a slapcat of them. > > A direct diff shows a lot of entries out of order (because the replica gets > them in multiple refresh phases), however, if I sort the two ldif files, > and do a diff, I get: > > [zimbra@smtp-2 tmp]$ sort slave.ldif>slave.ldif.s > [zimbra@smtp-2 tmp]$ sort master.ldif>master.ldif.s > [zimbra@smtp-2 tmp]$ diff -u master.ldif.s slave.ldif.s > [zimbra@smtp-2 tmp]$ > > No diff. So I believe this is fixed. Sound correct? Sounds good to me. I guess this should go into both RE23 and RE24. -- -- Howard Chu Chief Architect, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#5370) race condition in slap_op_time()
by h.b.furuseth＠usit.uio.no 13 Feb '08

13 Feb '08

hyc(a)symas.com writes: > Except that this will take a lot more restructuring in accesslog. Going with > the quick fix for now. Looks like I didn't notice the first use of o_tincr in accesslog:-( I guess accesslog can set an "I need o_tincr" flag somewhere, which slapd will then obey. -- Hallvard

1 0

Re: (ITS#5376) Full sync fails with delta-syncrepl, refreshAndPersist, and high modifications
by quanah＠zimbra.com 12 Feb '08

12 Feb '08

--On February 13, 2008 3:14:37 AM +0000 quanah(a)zimbra.com wrote: > > > --On February 13, 2008 3:03:46 AM +0000 quanah(a)zimbra.com wrote: > >> >> And, with the patch for ITS#5282, all replication stops (probably a good >> thing). > > More specifically, the replica stops accepting any new changes, since it > can't proceed on the MODs to a non-existent entry. The master of course > continues to accept and record changes, and any non-problematic replicas > continue to receive changes. With a build of 2.3 with the (adjusted) patches from HEAD, it looks like this issue is resolved. I fully sync'd a replica from the master, and after it was done, I did a slapcat of them. A direct diff shows a lot of entries out of order (because the replica gets them in multiple refresh phases), however, if I sort the two ldif files, and do a diff, I get: [zimbra@smtp-2 tmp]$ sort slave.ldif >slave.ldif.s [zimbra@smtp-2 tmp]$ sort master.ldif >master.ldif.s [zimbra@smtp-2 tmp]$ diff -u master.ldif.s slave.ldif.s [zimbra@smtp-2 tmp]$ No diff. So I believe this is fixed. Sound correct? --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

Re: (ITS#5376) Full sync fails with delta-syncrepl, refreshAndPersist, and high modifications
by quanah＠zimbra.com 12 Feb '08

12 Feb '08

--On February 13, 2008 3:03:46 AM +0000 quanah(a)zimbra.com wrote: > > And, with the patch for ITS#5282, all replication stops (probably a good > thing). More specifically, the replica stops accepting any new changes, since it can't proceed on the MODs to a non-existent entry. The master of course continues to accept and record changes, and any non-problematic replicas continue to receive changes. --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

Re: (ITS#5376) Full sync fails with delta-syncrepl, refreshAndPersist, and high modifications
by quanah＠zimbra.com 12 Feb '08

12 Feb '08

--On February 13, 2008 2:57:39 AM +0000 quanah(a)zimbra.com wrote: > Full_Name: Quanah Gibson-Mount > Version: 2.3.40 > OS: > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (24.23.156.219) > > > If you start a replica with an empty DB, using delta-sync replication, > and the master is receiving a high rate of changes, the Refresh phase > will skip entries that have been modified. When it moves to the > "persist" phase, delta-syncrepl will send modification diffs to those > entries rather than the full entries, so entries end up missing, having > never been sent in a full form. And, with the patch for ITS#5282, all replication stops (probably a good thing). --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

(ITS#5376) Full sync fails with delta-syncrepl, refreshAndPersist, and high modifications
by quanah＠zimbra.com 12 Feb '08

12 Feb '08

Full_Name: Quanah Gibson-Mount Version: 2.3.40 OS: URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (24.23.156.219) If you start a replica with an empty DB, using delta-sync replication, and the master is receiving a high rate of changes, the Refresh phase will skip entries that have been modified. When it moves to the "persist" phase, delta-syncrepl will send modification diffs to those entries rather than the full entries, so entries end up missing, having never been sent in a full form.

1 0

Re: (ITS#5339) wrong referral from back-bdb
by hyc＠symas.com 12 Feb '08

12 Feb '08

h.b.furuseth(a)usit.uio.no wrote: > Howard Chu writes: >>> Both RFCs disagree. >> They are wrong. Or at least, under-specified. In X.501 section 17.3 >> "Directory Distribution Model" it's quite clear that all of the >> components of a distributed directory must belong to a single DIT. > > Which is not true in the LDAP world, and I don't know about today's > X.500 world. Nameflow died and Dante was unable to resurrect it. Maybe > the X.500 world has also switched to 'dc' structure, I don't know. > > Anyway, LDAP is not X.500. RFC4510 Section 2 "Relationship to X.500" This technical specification defines LDAP in terms of [X.500] as an X.500 access mechanism. An LDAP server MUST act in accordance with the X.500 (1993) series of International Telecommunication Union - Telecommunication Standardization (ITU-T) Recommendations when providing the service. > Remember, LDAP referrals need not even be > LDAP URLs. My guess is that's one reason for the difference from X.500: > When you can even return a HTTP URL, why should you _not_ be allowed > to return a far smaller change from the original URL? As I've said many times before, LDAP referrals are a poorly designed piece of junk. The fact that the spec leaves it open to return other types of URLs but nobody in the world knows what that should actually mean to an LDAP client is just further proof of the abysmal lack of thought that went into the design. > Also LDAP started out a lot sloppier than X.500. That's self-evident. ;) > Getting some response > which hopefully resembled what you wanted was better than none:-) Getting a wrong answer is worse than getting no answer, because the client will just assume that everything is OK and keep going, rather than reporting an error to the user. >> And again, the service model requires the directory to store the >> client's data without any alteration. If the client wants to create >> the entry "cn=me,o=we" either that exact entry must be created, or the >> operation must fail. > > Well, the operation does fail in the original server. Then the client > may try another operation at another server... I don't know if that's a > valid interpretation of the service model or not. In any case, it's a > detail compared to the mess No, in this case the client is not being told to retry the *same* operation in another server, it's being told to try a *very different* operation in another server. Once again, the client made a specific request to create an entry with a specific DN. If the server can't honor the request as specified then it should fail and the operation should halt. Anyway, the very existence of LDAP referrals is a violation of the service model. X.501(1993) section 18.2: It is a requirement of the Directory that, for particular modes of user interaction, the distribution of the directory be rendered transparent, thereby giving the effect that the whole of the DIB appears to be within each and every DSA. LDAPv1 had no referrals. When they were introduced in LDAPv2 it's clear that nobody knew what they were doing, or nobody wanted to tackle the glaring absence of an analogue to X.500 DSP. They should never have been introduced. We're stuck with them for now, but we can at least try to make them make sense. -- -- Howard Chu Chief Architect, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#5370) race condition in slap_op_time()
by hyc＠symas.com 12 Feb '08

12 Feb '08

hyc(a)symas.com wrote: > h.b.furuseth(a)usit.uio.no wrote: >> However as far as I can tell, only the accesslog overlay uses >> op->o_tincr, the value which needs the mutex. And accesslog >> calls slap_op_time() itself when it needs that value. >> So maybe we should remove the op->o_tincr field. Other calls >> to slap_op_time() can be replaced with slap_get_time(). > > Sounds like a good idea. It was somewhat redundant with the CSN counter > (except that this was for all operations, and CSN was only for write > operations) and it never became generally useful. (Shifted to microsecond > timestamps in 2.4, because tincr isn't useful across multiple servers.) I > guess it makes more sense to make it specific to accesslog. > Except that this will take a lot more restructuring in accesslog. Going with the quick fix for now. -- -- Howard Chu Chief Architect, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#5374) accesslog+hdb crash with test043-delta-syncrepl
by quanah＠zimbra.com 12 Feb '08

12 Feb '08

--On February 12, 2008 6:12:08 PM +0000 quanah(a)zimbra.com wrote: > (and it passed 10 times for me successfully). At Hallvard's request, I patched accesslog thusly: diff -u accesslog.c.orig accesslog.c --- accesslog.c.orig 2008-02-12 10:18:16.000000000 -0800 +++ accesslog.c 2008-02-12 10:21:46.000000000 -0800 @@ -914,6 +914,7 @@ ldap_pvt_thread_mutex_lock( &li->li_log_mutex ); old = li->li_old; li->li_old = NULL; + assert(li->li_unlock == (int) ldap_pvt_thread_self()); li->li_unlock = 0; ldap_pvt_thread_mutex_unlock( &li->li_op_mutex ); } @@ -1262,7 +1263,8 @@ * overlays like refint to keep working. */ ldap_pvt_thread_mutex_lock( &li->li_op_mutex ); - li->li_unlock = 1; + /* li->li_unlock = 1; */ + li->li_unlock = (int) ldap_pvt_thread_self(); if ( li->li_oldf && ( op->o_tag == LDAP_REQ_DELETE || op->o_tag == LDAP_REQ_MODIFY )) { int rc; Multiple runs of the test continued to be successful. --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

Re: (ITS#5374) accesslog+hdb crash with test043-delta-syncrepl
by quanah＠zimbra.com 12 Feb '08

12 Feb '08

(and it passed 10 times for me successfully). --On February 12, 2008 5:59:22 PM +0000 quanah(a)zimbra.com wrote: > > > --On February 12, 2008 12:11:50 PM +0000 h.b.furuseth(a)usit.uio.no wrote: > >> Full_Name: Hallvard B Furuseth >> Version: RE23 >> OS: Linux >> URL: >> Submission from: (NULL) (129.240.6.233) >> Submitted by: hallvard >> >> >> /configure \ >> --disable-dynamic --disable-aci --enable-crypt --enable-lmpasswd \ >> --enable-spasswd --enable-rlookups --enable-slapi --enable-wrappers \ >> --enable-backends --disable-sql --enable-overlays \ >> CPPFLAGS='-DLDAP_THREAD_DEBUG -DLDAP_RDWR_DEBUG' > > I built as: > > LD_LIBRARY_PATH="/usr/local/lib" CC=gcc CXX=g++ CFLAGS='-g -O0' > CXXFLAGS='-g -O0' CPPFLAGS='-DLDAP_THREAD_DEBUG -DLDAP_RDWR_DEBUG' sh > configure --datadir='${prefix}/lib' --libexecdir='${prefix}/lib' > --sharedstatedir='${prefix}/lib' --prefix=/usr/local > --disable-ipv6 --with-cyrus-sasl --with-tls > --enable-dynamic --enable-slapd --enable-modules > --enable-spasswd --enable-rewrite > --enable-rlookups --enable-wrappers > --enable-backends=mod --disable-shell > --disable-sql --enable-overlays=mod --enable-slapi=yes > > > Thread package selected is: > > -pthread > > s,@LTHREAD_LIBS@, -pthread,;t t > > > --Quanah > > > -- > > Quanah Gibson-Mount > Principal Software Engineer > Zimbra, Inc > -------------------- > Zimbra :: the leader in open source messaging and collaboration > > -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs