openldap-bugs

openldap-bugs@openldap.org

1 participants
20966 discussions

Re: (ITS#5454) syncrepl refreshAndPersist stops receiving
by hyc＠symas.com 11 Apr '08

11 Apr '08

hyc(a)symas.com wrote: >>> It also occurs to me that we probably don't even need to manipulate the >>> slapd runqueue in persist mode, when si->si_conn is already set. I.e., >>> in that case we can only have gotten here because of a listener event, >>> and not because of a runqueue schedule. >> That is probably true. > > Right, my patch avoids that now. Having looked at it again I think replacing > the trylock with a regular lock is fine. OK, reverted the trylock (which was put there in rev 1.376). I unfortunately don't remember what prompted that commit; certainly there was no ITS associated with it. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#5463) syncrepl stops or pauses
by rein＠basefarm.no 11 Apr '08

11 Apr '08

marian.eichholz(a)freenet.ag skrev: > Full_Name: > Version: 2.4.8 > OS: Linux 2.6.23.13 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (194.97.7.65) > > The replication may suddenly stop, and we see no reason for that. Sometimes it > recovers by itself, sometimes it recovers only when the consumer slapd is > stopped and restarted. > Is there a "golden path" to debug this issue? This sounds a lot like the problems we had, see ITS#5454. Replication has been running smootly since we implemented the patch included there. An alternative version is implemented in the CVS head release, but that lacks a change from trylock to real mutex lock before the race condition is fully closed. We experienced a lot of replication problems after upgrading from a slurpd based replication in 2.3.x to syncrepl in 2.4.8. We are currently using the CVS head version (as it was about a week ago), and most of our problems has been solved there :-) You should consider the cvs version, as it has many syncrepl and syncprov fixes. -- Rein Tollevik Basefarm AS

1 0

(ITS#5464) ucred guarded by __USE_GNU in glibc 2.8
by rhafer＠suse.de 11 Apr '08

11 Apr '08

Full_Name: Ralf Haferkamp Version: Any OS: Linux (glibc 2.8) URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (85.8.71.17) Newer glibc Version guards the definition of struct ucred (e.g. used in liblutil/getpeereid.c) by __USE_GNU, hence it is only available when _GNU_SOURCE is defined. The matching glibc changelog only say "Cleanup namespace.". Here's the diff: http://sourceware.org/cgi-bin/cvsweb.cgi/libc/sysdeps/unix/sysv/linux/bits/…

1 0

(ITS#5463) syncrepl stops or pauses
by marian.eichholz＠freenet.ag 11 Apr '08

11 Apr '08

Full_Name: Version: 2.4.8 OS: Linux 2.6.23.13 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (194.97.7.65) Were still migrating von slurpd to syncrepl. We have a 2.3-slurpd-fed provider under OpenLDAP 2.4.8 used by currently two OpenLDAP 2.4.8 consumers with "refreshAndPersist" and a retry-interval of "180 +" (see below) The database is fairly large (8 Mio objects, 15 GB BDB files) and the consumers have a syncrepled database (pulled once, then cloned). There are ongoing updates "each second". We see two issues: When a consumer (re-) connects, the replication for the running consumer pauses until it continues on both nodes. This could become a problem, where a single consumer can be a SPOF for the whole cluster. The replication may suddenly stop, and we see no reason for that. Sometimes it recovers by itself, sometimes it recovers only when the consumer slapd is stopped and restarted. Is there a "golden path" to debug this issue? For the logs: We have "sync" as log level. Consumer 1 (rid 000) [...] Apr 10 16:15:18 0 slapd[27548]: connection_input: conn=192 deferring operation: binding [...] Apr 11 08:45:21 0 slapd[27548]: connection_input: conn=5182 deferring operation: binding Apr 11 08:50:11 0 slapd[27548]: connection_input: conn=5207 deferring operation: binding Apr 11 08:51:09 0 slapd[27548]: connection_input: conn=5212 deferring operation: binding Apr 11 08:53:05 0 slapd[27548]: connection_input: conn=5222 deferring operation: binding Apr 11 08:54:03 0 slapd[27548]: connection_input: conn=5228 deferring operation: binding Consumer 2 (rid 002): [...] Apr 11 02:00:16 2 slapd[22272]: connection_input: conn=3151 deferring operation: binding [...] Apr 11 08:52:09 2 slapd[22272]: connection_input: conn=5224 deferring operation: binding Apr 11 08:53:07 2 slapd[22272]: connection_input: conn=5229 deferring operation: binding Provider (filtered for last syncprov_sendresp entries per rid) Apr 10 16:06:43 1 slapd[27917]: syncprov_sendresp: cookie=rid=000,csn=2008041014 0643.069039Z#000000#000#000000 Apr 11 01:57:58 1 slapd[27917]: syncprov_sendresp: cookie=rid=002,csn=2008041023 5758.696324Z#000000#000#000000 I see no indicators for failing operations, missing resources or any other trouble, incept for the "connection_input" entries. Any idea how to get sync replication running smoothly and reliably? Thank You very much in advance, of course! OK, here are configuraiton snippets for provider and consumer, if it helps. To be honest: I am not quite shure about a reasonable size for syncprov-sessionlog, even after reading the slapo-syncprov. The actual number is meant as "big, but finite". Consumer: --------- syncrepl rid=2 provider="ldap://provider:389" type=refreshAndPersist bindmethod=simple binddn=... credentials=... searchbase=... retry="180 +" provider: --------- database bdb dbnosync cachesize 200000 suffix ... rootdn ... updatedn ... rootpw ... directory /var/ldap/db lastmod on index objectClass eq ... index entryCSN,entryUUID eq checkpoint 512 15 overlay syncprov syncprov-sessionlog 300000 database monitor

1 0

(ITS#5462) Allow setting of RANDOM_FILE for gnutls
by pogma＠thewrittenword.com 10 Apr '08

10 Apr '08

Full_Name: Peter O'Gorman Version: 2.4.8 OS: multiple URL: ftp://ftp.openldap.org/incoming/the-written-word-080410.patch Submission from: (NULL) (24.76.165.223) gcrypt-1.4.x and later allow the egd socket path to be set. This patch allows the conf file option to work when building openldap with gnutls on systems that do not have a /dev/random.

1 0

Re: (ITS#5451) syncprov_checkpoint deadlock bugfix?
by rein＠tollevik.no 10 Apr '08

10 Apr '08

On Wed, 9 Apr 2008, h.b.furuseth(a)usit.uio.no wrote: > Does this help? From my fiddling with ITS#5340 (REP_ENTRY_MODIFIABLE). > I do not understand syncprov's handling of REP_ENTRY_MUSTRELEASE though. > (For one thing it seems to assume that REP_ENTRY_MUSTRELEASE is set if > and only if rs.sr_entry->e_private != NULL. Which is possibly true with > back-bdb but seems a shaky assumption in general.) Almost, except that it triggers an abort.. Using be_entry_release_r() was not correct, as it ended up calling entry_free(). It works if I changes it to use overlay_entry_release_ov() instead, as in the alternative patch at the end. I have put a test script that shows this deadlock on: ftp://ftp.openldap.org/incoming/test053-syncprov-glue It deadlocks without this patch, it and the rest of the test suite succeed with it :-) I haven't tested it in production though.. Rein Index: OpenLDAP/servers/slapd/overlays/syncprov.c diff -u OpenLDAP/servers/slapd/overlays/syncprov.c:1.1.1.14 OpenLDAP/servers/slapd/overlays/syncprov.c:1.14 --- OpenLDAP/servers/slapd/overlays/syncprov.c:1.1.1.14 Thu Apr 10 17:31:24 2008 +++ OpenLDAP/servers/slapd/overlays/syncprov.c Thu Apr 10 17:34:36 2008 @@ -2389,7 +2396,14 @@ if ( !ap ) { if ( !(rs->sr_flags & REP_ENTRY_MODIFIABLE) ) { - rs->sr_entry = entry_dup( rs->sr_entry ); + Entry *new = entry_dup( rs->sr_entry ); + if ( rs->sr_flags & REP_ENTRY_MUSTRELEASE ) { + overlay_entry_release_ov( op, rs->sr_entry, 0, (slap_overinst *)op->o_bd->bd_info ); + rs->sr_flags ^= REP_ENTRY_MUSTRELEASE; + } else if ( rs->sr_flags & REP_ENTRY_MUSTBEFREED ) { + entry_free( rs->sr_entry ); + } + rs->sr_entry = new; rs->sr_flags |= REP_ENTRY_MODIFIABLE|REP_ENTRY_MUSTBEFREED; a = attr_find( rs->sr_entry->e_attrs,

1 0

Re: (ITS#5340) REP_ENTRY_MODIFIABLE bug in dynlist
by h.b.furuseth＠usit.uio.no 10 Apr '08

10 Apr '08

Howard Chu writes: > I have the feeling that MUSTBEFREED and MUSTRELEASE should be > consolidated into a single flag. be_entry_release_r() should simply > call entry_free() if e->e_private is NULL. Maybe. But consider translucent - it grabs incoming entries and remembers them. It can "steal" a MUSTBEFREED entry (though it doesn't now), but must duplicate a MUSTRELEASE entry so that it won't hold on to a lock in an underlying database. -- Hallvard

1 0

Re: (ITS#5451) syncprov_checkpoint deadlock bugfix?
by h.b.furuseth＠usit.uio.no 10 Apr '08

10 Apr '08

Howard Chu writes: > h.b.furuseth(a)usit.uio.no wrote: >> Does this help? From my fiddling with ITS#5340 (REP_ENTRY_MODIFIABLE). >> I do not understand syncprov's handling of REP_ENTRY_MUSTRELEASE though. >> (For one thing it seems to assume that REP_ENTRY_MUSTRELEASE is set if >> and only if rs.sr_entry->e_private != NULL. Which is possibly true with >> back-bdb but seems a shaky assumption in general.) > > That is a requirement, yes. If you implement a backend that has > cache/locking constraints on its entries and you don't provide a > pointer from the entry back to your cache state, then you're an idiot. OTOH you can send an entry without setting MUSTRELEASE, and release it after sending instead. In particular if you didn't about the undocumented sr_flags and their purpose. Or if you don't use be_release for unlocking but just to clean up e_private. ... have I gone blind? I don't see back-bdb setting that flag. Did I just read ITS#3671 and trust it to have updated bdb? -- Hallvard

1 0

Re: (ITS#5459) threw assertion in attr_dup
by richton＠nbcs.rutgers.edu 10 Apr '08

10 Apr '08

> Is this entry still in your database now? Clearly this condition should never > happen - if zero-length values are valid for the syntax, then it should have > been preserved in the (normalized) a_nvals. If not, then it should not have > been in the a_vals in the first place. No, it got deleted last night. But from the core I see: at_syntax_oid = 0x100495d80 "1.3.6.1.4.1.1466.115.121.1.26" IA5String, so empty string is valid. (And makes no logical sense in this situation, but that's for another day...)

1 0

Re: (ITS#5340) REP_ENTRY_MODIFIABLE bug in dynlist
by hyc＠symas.com 10 Apr '08

10 Apr '08

h.b.furuseth(a)usit.uio.no wrote: > - REP_ENTRY_MUSTRELEASE and REP_ENTRY_MUSTBEFREED: > > Callers that set these flags when sending an entry, *allow* the > callee to release/free rs->sr_entry. > Callers cannot depend on it being release/freed, but some do anyway. > > It should simplify things to make the send functions always obey > this flag, but I'm not sure how much cleanup that requires. > There is partial code for that in slap_send_search_entry(), along > with a FIXME note. > > Free MUSTBEFREED-entries with entry_free(), and MUSTRELEASE-entries > with be_entry_release_r(). There is no "use be_entry_release_w()" > flag, but no OpenLDAP code sends write-locked entries anyway. I have the feeling that MUSTBEFREED and MUSTRELEASE should be consolidated into a single flag. be_entry_release_r() should simply call entry_free() if e->e_private is NULL. > + REP_ENTRY_MUSTRELEASE: > > Created for ITS#3671 to ensure that network hangs don't delay > cache entry unlocks: > > Modules should release their cache entry locks before sending > anything. They can't do that with cached entries they send in > rs->sr_entry, so instead they should set REP_ENTRY_MUSTRELEASE. > Conversely, anything which does network I/O (and other potentially > slooow ops I guess) should first obey this flag. > > Several overlays don't obey, nor some other code for sending an > entry when it decides to send an error response instead. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs