openldap-bugs

openldap-bugs@openldap.org

1 participants
20961 discussions

(ITS#6489) replication problem - new valuable information, nearly solved
by frank.offermanns＠caseris.de 14 Sep '10

14 Sep '10

Hello, after digging very deep to find out, what the problem could be, I found interesting facts: 1) With our new client-DLL the master/slave replication problem seems to be solved. (our old DLL wrote each attribute separately, our new writes all in a row), but 2) So I switched back to Master/Master in hope that this works also, but I found out the following. - my client-DLL first create a new user entry and then it does a add with all attributes. - at the second master server I activated replication log and saw the following short message (the important part is surounded with ________): "do_syncrep2: rid=001 cookie=rid=001,sid=001,csn=20100914081312.596142Z#000000#00 1#000000 syncrepl_entry: rid=001 LDAP_RES_SEARCH_ENTRY(LDAP_SYNC_ADD) syncrepl_entry: rid=001 be_search (0) syncrepl_entry: rid=001 cn=10008,o=BAD_CLIENT3,ou=users,o=caesar slap_queue_csn: queing 09369668 20100914081312.596142Z#000000#001#000000 syncprov_matchops: skipping original sid 001 slap_graduate_commit_csn: removing 09377b10 20100914081312.596142Z#000000#001#00 0000 syncrepl_entry: rid=001 be_add cn=10008,o=BAD_CLIENT3,ou=users,o=caesar (0) slap_queue_csn: queing 09369668 20100914081312.596142Z#000000#001#000000 syncprov_matchops: skipping original sid 001 slap_graduate_commit_csn: removing 09377b10 20100914081312.596142Z#000000#001#00 0000 do_syncrep2: rid=001 cookie= syncrepl_entry: rid=001 LDAP_RES_SEARCH_ENTRY(LDAP_SYNC_MODIFY) __________________________________________________________________ dn_callback : new entry is older than ours cn=10008,o=BAD_CLIENT3,ou=users,o=cae sar ours 20100914081312.596142Z#000000#001#000000, new 20100914081312.100780Z#00 0000#001#000000 ___________________________________________________________________ syncrepl_entry: rid=001 be_search (0) syncrepl_entry: rid=001 cn=10008,o=BAD_CLIENT3,ou=users,o=caesar syncrepl_entry: rid=001 entry unchanged, ignored (cn=10008,o=BAD_CLIENT3,ou=user s,o=caesar)" Is it a bug, or a result of a bad time synchronization (I only have windows standard time synchronization) But if it would be a time synchronization problem, in recent posts I asked when the time synchronization is important. I got the answer (if understood correctly) that the time synchronization only matters if concurrent write operations are made. So it should'nt be an issue here, since I made my write operations only at one master. Best regards, Frank

1 0

Re: (ITS#6639) syncrepl failing using SASL/GSSAPI
by hyc＠symas.com 12 Sep '10

12 Sep '10

Bill MacAllister wrote: > Howard, > > The patched packages where installed last night on the production OpenLDAP > master with two of the replicas in the failing state. Once the patched > slapd was started the two problem replicas quickly caught up and everything > looks good now. Thanks for the confirmation. I'll commit a final fix to HEAD shortly. > > Thanks again for your help. > > Bill > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#6639) syncrepl failing using SASL/GSSAPI
by whm＠stanford.edu 12 Sep '10

12 Sep '10

--On Thursday, September 09, 2010 10:43:06 PM -0700 Howard Chu <hyc(a)symas.com> wrote: > whm(a)stanford.edu wrote: >> --On Friday, September 03, 2010 01:23:17 AM -0700 Bill MacAllister<whm(a)stanford.edu> wrote: >> >> The problem with the database was only coincidental. Restoring the database >> got the failing replica past the problem replication event. >> >> In the replica pool of 6 servers we have seen the problem on there of the >> servers. In thinking about this more it is unlikely that it is a slave >> problem since the slaves have been in use for about 6 weeks and we did >> not see the problem. Only when we changed the master to 2.4.23 did we >> see the problem. I have captured a master debug log of the problem >> event. It is at http://www.stanford.edu/~whm/files/master-debug.txt. >> >> Bill >> > Please try with this patch: > > Index: sasl.c > =================================================================== > RCS file: /repo/OpenLDAP/pkg/ldap/libraries/libldap/sasl.c,v > retrieving revision 1.79 > diff -u -r1.79 sasl.c > --- sasl.c 13 Apr 2010 20:17:56 -0000 1.79 > +++ sasl.c 10 Sep 2010 05:42:22 -0000 > @@ -733,8 +733,9 @@ > return ret; > } else if ( p->buf_out.buf_ptr != p->buf_out.buf_end ) { > /* partial write? pretend nothing got written */ > - len2 = 0; > p->flags |= LDAP_PVT_SASL_PARTIAL_WRITE; > + sock_errset(EAGAIN); > + len2 = -1; > } > > /* return number of bytes encoded, not written, to ensure Howard, The patched packages where installed last night on the production OpenLDAP master with two of the replicas in the failing state. Once the patched slapd was started the two problem replicas quickly caught up and everything looks good now. Thanks again for your help. Bill -- Bill MacAllister Infrastructure Delivery Group, Stanford University

1 0

Re: (ITS#6565) DNSSRV referrals: chaining without search base
by masarati＠aero.polimi.it 10 Sep '10

10 Sep '10

> I've checked a simple configuration (global instance of chain overlay; > request for non existing entry, resulting in slapd returning the default > referral), and nothing strange appeared (the offending instruction is > executed). The issue might be specific to your configuration. You'll > probably need to post more details. I've checked a "real" case: DNSSRV-located DSAs, one responding and one not. Things seem to work as expected, no cores and valgrind shows no issue. In any case, I have committed a few modifications to HEAD code (plugs a potential leak). Please test and either provide more details about the core or more details about the configuration. p.

1 0

Re: (ITS#6645) Yet Another LDIF Wrap Patch
by masarati＠aero.polimi.it 10 Sep '10

10 Sep '10

1 0

Re: (ITS#6645) Yet Another LDIF Wrap Patch
by masarati＠aero.polimi.it 10 Sep '10

10 Sep '10

> hyc(a)symas.com writes: >> Not thrilled with the magic constant 2^31-1. Seems like you should have >> just >> used "if ( wrap && len > wrap )" instead of a word-size-dependent magic. > > The code just needs to be a little less clever:-) Pass -1 to a > 'ber_len_t wrap'. The constant (ber_len_t)-1 will prevent wrapping: > "if (len > (ber_len_t)-1)" always fails. Thanks. > But if we are going to have smarter LDIF output, I'm more interested in > an option to avoid base64 encoding of valid UTF-8 strings. Also one > to force base64-encoding for caller-specified attributes. Or just > a bitflag to ldif_sput() after the caller checks the attribute type. See ldif_must_b64_encode_register() :) We already use it to force encoding of olcRootPW and other password-like attrs in proxy backends. p.

1 0

Re: (ITS#6645) Yet Another LDIF Wrap Patch
by h.b.furuseth＠usit.uio.no 10 Sep '10

10 Sep '10

hyc(a)symas.com writes: > Not thrilled with the magic constant 2^31-1. Seems like you should have just > used "if ( wrap && len > wrap )" instead of a word-size-dependent magic. The code just needs to be a little less clever:-) Pass -1 to a 'ber_len_t wrap'. The constant (ber_len_t)-1 will prevent wrapping: "if (len > (ber_len_t)-1)" always fails. But if we are going to have smarter LDIF output, I'm more interested in an option to avoid base64 encoding of valid UTF-8 strings. Also one to force base64-encoding for caller-specified attributes. Or just a bitflag to ldif_sput() after the caller checks the attribute type. -- Hallvard

1 0

Re: (ITS#6646) back-hdb dncache disregarded in some searches
by ondrej.kuznik＠acision.com 10 Sep '10

10 Sep '10

On 09/10/2010 11:19 AM, hyc(a)symas.com wrote: > This fact is already documented in the manpage. This ITS will be closed. I disagree. The manpage states: "It should be noted that the DN cache is allowed to temporarily grow beyond the configured size. It does this if many entries are locked when it tries to do a purge, because that means they're legitimately in use. Also, the DN cache never purges entries that have cached children, so depending on the shape of the DIT, it could have lots of cached DNs over the defined limit." However, in the last case with the sizelimit, the DN cache grows over its configured size permanently. They do not seem to be legitimately in use and if they are, it is because the code that used them told noone it stopped doing so as the sizelimit has been reached or connection severed. All those entries are leafs, so no children either. From all this I deduced that either the code or documentation are missing something. And I will repeat myself, but two searches that differ only in their search bases behave _very_ differently. If one search iterates though the database and uses and frees the entries it processes emitting and the other one gets all entries first and parses and releases them afterwards, and if something interrupts it, does not release some. But yet again, I am only slowly gaining familiarity with OpenLDAP's immerse codebase and thus have no authority to speak about code I am nowhere near familiar with. Still, I am eager to learn as much as I can, so I'll be glad for some pointers. Thanks, Ondra This e-mail and any attachment is for authorised use by the intended recipient(s) only. It may contain proprietary material, confidential information and/or be subject to legal privilege. It should not be copied, disclosed to, retained or used by, any other party. If you are not an intended recipient then please promptly delete this e-mail and any attachment and all copies and inform the sender. Thank you.

1 0

Re: (ITS#6639) syncrepl failing using SASL/GSSAPI
by hyc＠symas.com 10 Sep '10

10 Sep '10

whm(a)stanford.edu wrote: > --On Thursday, September 09, 2010 10:43:06 PM -0700 Howard Chu<hyc(a)symas.com> wrote: > >> whm(a)stanford.edu wrote: >>> --On Friday, September 03, 2010 01:23:17 AM -0700 Bill MacAllister<whm(a)stanford.edu> wrote: >>> >>> The problem with the database was only coincidental. Restoring the database >>> got the failing replica past the problem replication event. >>> >>> In the replica pool of 6 servers we have seen the problem on there of the >>> servers. In thinking about this more it is unlikely that it is a slave >>> problem since the slaves have been in use for about 6 weeks and we did >>> not see the problem. Only when we changed the master to 2.4.23 did we >>> see the problem. I have captured a master debug log of the problem >>> event. It is at http://www.stanford.edu/~whm/files/master-debug.txt. >>> >>> Bill >>> >> Please try with this patch: >> >> Index: sasl.c >> =================================================================== >> RCS file: /repo/OpenLDAP/pkg/ldap/libraries/libldap/sasl.c,v >> retrieving revision 1.79 >> diff -u -r1.79 sasl.c >> --- sasl.c 13 Apr 2010 20:17:56 -0000 1.79 >> +++ sasl.c 10 Sep 2010 05:42:22 -0000 >> @@ -733,8 +733,9 @@ >> return ret; >> } else if ( p->buf_out.buf_ptr != p->buf_out.buf_end ) { >> /* partial write? pretend nothing got written */ >> - len2 = 0; >> p->flags |= LDAP_PVT_SASL_PARTIAL_WRITE; >> + sock_errset(EAGAIN); >> + len2 = -1; >> } >> >> /* return number of bytes encoded, not written, to ensure > > I have applied the patch and the debian packages built fine. I have > installed the new packages on the master servers in our dev and test > environments. Initial tests show that basic replication is working. > Of course, this problem did not exhibit itself in our test > environments. I will install it in our production environment Friday > evening and let you know what happens. > > Am I correct in assuming that the root of the problem is on the providers? Yes. Really, the problem will occur when either a client or server writes a large buffer thru a SASL encrypted connection, but this is more likely to happen when a server is sending responses. > I am asking because I was not planning installing the new version on the > consumers right away since 'emergency' changes to production software > require special approval. I would like to upgrade the providers in our > next maintenance window which is next Thursday. If you think I should > do it with the update of the master let me know. It's probably not urgent for the consumers, just bear in mind that any clients querying them thru SASL could also run into similar issues. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#6639) syncrepl failing using SASL/GSSAPI
by whm＠stanford.edu 10 Sep '10

10 Sep '10

--On Thursday, September 09, 2010 10:43:06 PM -0700 Howard Chu <hyc(a)symas.com> wrote: > whm(a)stanford.edu wrote: >> --On Friday, September 03, 2010 01:23:17 AM -0700 Bill MacAllister<whm(a)stanford.edu> wrote: >> >> The problem with the database was only coincidental. Restoring the database >> got the failing replica past the problem replication event. >> >> In the replica pool of 6 servers we have seen the problem on there of the >> servers. In thinking about this more it is unlikely that it is a slave >> problem since the slaves have been in use for about 6 weeks and we did >> not see the problem. Only when we changed the master to 2.4.23 did we >> see the problem. I have captured a master debug log of the problem >> event. It is at http://www.stanford.edu/~whm/files/master-debug.txt. >> >> Bill >> > Please try with this patch: > > Index: sasl.c > =================================================================== > RCS file: /repo/OpenLDAP/pkg/ldap/libraries/libldap/sasl.c,v > retrieving revision 1.79 > diff -u -r1.79 sasl.c > --- sasl.c 13 Apr 2010 20:17:56 -0000 1.79 > +++ sasl.c 10 Sep 2010 05:42:22 -0000 > @@ -733,8 +733,9 @@ > return ret; > } else if ( p->buf_out.buf_ptr != p->buf_out.buf_end ) { > /* partial write? pretend nothing got written */ > - len2 = 0; > p->flags |= LDAP_PVT_SASL_PARTIAL_WRITE; > + sock_errset(EAGAIN); > + len2 = -1; > } > > /* return number of bytes encoded, not written, to ensure I have applied the patch and the debian packages built fine. I have installed the new packages on the master servers in our dev and test environments. Initial tests show that basic replication is working. Of course, this problem did not exhibit itself in our test environments. I will install it in our production environment Friday evening and let you know what happens. Am I correct in assuming that the root of the problem is on the providers? I am asking because I was not planning installing the new version on the consumers right away since 'emergency' changes to production software require special approval. I would like to upgrade the providers in our next maintenance window which is next Thursday. If you think I should do it with the update of the master let me know. Bill -- Bill MacAllister Infrastructure Delivery Group, Stanford University

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs