openldap-bugs

openldap-bugs@openldap.org

1 participants
21001 discussions

Re: (ITS#5443) Multiple identical attibutes break syncrepl process fatally?
by ando＠sys-net.it 28 Mar '08

28 Mar '08

marian.eichholz(a)freenet.ag wrote: > We use openldap as mail service directory with some 8 Mio objects on several > replicas. > For openldap 2.4.x we have to migrate from slurpd to syncrepl. > We got a working syncrepl provider als slurpd consumer (slapadd -q, 36 hrs) > > So I try to get a blank DB up by syncrepl only (yes, it is not at all > performant, but informative) > > The process kind of breaks after a couple of minutes and some 44.000 objects > (8.000.000 expected). Tracing it on the consumer side (-d 16384), I see > something like this after an entry: > > syncrepl_message_to_entry: rid=001 mods check (forwardto: value #4 provided more > than once) > > Indeed, the entry to come has three "forwardto:" Attributes with the same value > (and other forwardto-attributes, too). This makes no greate sense at the > application level, but until now it has been perfectly OK for the directory, and > the LDAP-API did not complain about the attribute modification, neither did the > slurpd. It is a violation of RFC 4512, section 2.2, which OpenLDAP 2.4 conforms to. > This leads to some questions and suggestions: > > - the provider does not log anything with -d 16384, no error, no nothing. Could > it do some useful logging about successful and failing replication sessions? What's -16384? Since OpenLDAP 2.3 you can use strings to identify each log subsystem (16384 == 0x4000 == "sync"). The error occurs when the consumer tries to manipulate the data it receives. The producer has nothing to do with it, since it assumes that data contained in it already passed sanity checks when they were stored. How incorrect data got stored into the producer is a totally different business, and the producer-side replication process should not muck with it. > - the consumer does not log anything that can explain, why the remaining objects > are not read, either. A bit of warning/logging could help the hopeful admin, > probably. A sync error occurred, which prevented sync'in from continuing. This error is logged by the "sync" subsystem. As far as I understand from reading the code, the error (at least, a replication error) should be logged also by the "any" subsystem, which means that as soon as any logging is enabled, you get a message logged. > - why is one problematic object lethal for the whole rest of the objects, since > future modifications keep to be incorporated? Is this lack of robustness more a > bug or a feature? If inconsistent data is received, synchronization is supposed to stop. In fact, continuing may result in an inconsistent state. The fact that the stop is caused by a real error, and the fact that fixing the error allows synchronization to recover doesn't sound like lack of robustness to me. It sounds more about wisdom. > - are identical attributes really forbidden with LDAP? RFC 4512, Section 2.2 > - what could one do, to prevent unskillful "editors" of the master node to kill > the replication processes for the whole replication cluster? Besides adding a > checking/filtering API layer, of course. Slapd has sanity checks for this. Slapadd doesn't, since it is supposed to be operated only with consistent data, as resulting from slapcat. You might have slapadd'ed inconsistent data to the producer. In the end, I don't see how this ITS involves a bug in synchronization software. The fact your producer got corrupted by inconsistent data might have been caused by a bug in the software, however your analysis does not give a clear indication of how it happened. If it happened by slapadd, then it's a known (and desired, and documented) limitation of the software. Unless you can reproduce it, I'd consider this ITS closed. p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it --------------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Email: pierangelo.masarati(a)sys-net.it ---------------------------------------

1 0

(ITS#5443) Multiple identical attibutes break syncrepl process fatally?
by marian.eichholz＠freenet.ag 28 Mar '08

28 Mar '08

Full_Name: Marian Eichholz Version: 2.4.8 OS: Linux URL: Submission from: (NULL) (194.97.7.65) We use openldap as mail service directory with some 8 Mio objects on several replicas. For openldap 2.4.x we have to migrate from slurpd to syncrepl. We got a working syncrepl provider als slurpd consumer (slapadd -q, 36 hrs) So I try to get a blank DB up by syncrepl only (yes, it is not at all performant, but informative) The process kind of breaks after a couple of minutes and some 44.000 objects (8.000.000 expected). Tracing it on the consumer side (-d 16384), I see something like this after an entry: syncrepl_message_to_entry: rid=001 mods check (forwardto: value #4 provided more than once) Indeed, the entry to come has three "forwardto:" Attributes with the same value (and other forwardto-attributes, too). This makes no greate sense at the application level, but until now it has been perfectly OK for the directory, and the LDAP-API did not complain about the attribute modification, neither did the slurpd. This leads to some questions and suggestions: - the provider does not log anything with -d 16384, no error, no nothing. Could it do some useful logging about successful and failing replication sessions? - the consumer does not log anything that can explain, why the remaining objects are not read, either. A bit of warning/logging could help the hopeful admin, probably. - why is one problematic object lethal for the whole rest of the objects, since future modifications keep to be incorporated? Is this lack of robustness more a bug or a feature? - are identical attributes really forbidden with LDAP? - what could one do, to prevent unskillful "editors" of the master node to kill the replication processes for the whole replication cluster? Besides adding a checking/filtering API layer, of course. Thank You in advance. Please let me know, if I can provide You with something useful information about our issue(s).

1 0

(ITS#5442) slapd_rq not locked before use bugfix
by rein＠basefarm.no 28 Mar '08

28 Mar '08

Full_Name: Rein Tollevik Version: CVS head OS: linux and solaris URL: ftp://ftp.openldap.org/incoming/slapd_rq_lock.patch Submission from: (NULL) (81.93.160.250) I was seeing random failures of the test050-syncrepl-multimaster test. One of the failures was that it went into a tight loop traversing a circular runqueue it had managed to create in slapd_rq.task_list. It seems as this was caused by missing mutex locks around accesses to slapd_rq, which the patch uploaded to ftp://ftp.openldap.org/incoming/slapd_rq_lock.patch fixes. Before I applied this patch the test failed after being run a few times, with it it has now passed 100 times and is still counting. Rein Tollevik Basefarm AS

1 0

(ITS#5441) Avoid unneccessary time() syscall in auditlog overlay
by rein＠basefarm.no 28 Mar '08

28 Mar '08

Full_Name: Rein Tollevik Version: CVS head OS: linux and solaris URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (81.93.160.250) Below is a minor enhancement to the auditlog overlay, don't fetch the time() until it is known that it is needed. Which it isn't on all those search responses that passes through it.. Rein Tollevik Basefarm AS Index: OpenLDAP/servers/slapd/overlays/auditlog.c diff -u OpenLDAP/servers/slapd/overlays/auditlog.c:1.1.1.8 OpenLDAP/servers/slapd/overlays/auditlog.c:1.2 --- OpenLDAP/servers/slapd/overlays/auditlog.c:1.1.1.8 Sat Mar 22 16:48:17 2008 +++ OpenLDAP/servers/slapd/overlays/auditlog.c Wed Mar 26 22:58:55 2008 @@ -74,7 +74,7 @@ Modifications *m; struct berval *b, *who = NULL; char *what, *suffix; - long stamp = slap_get_time(); + time_t stamp; int i; if ( rs->sr_err != LDAP_SUCCESS ) return SLAP_CB_CONTINUE; @@ -125,6 +125,7 @@ return SLAP_CB_CONTINUE; } + stamp = slap_get_time(); fprintf(f, "# %s %ld %s%s%s\n", what, stamp, suffix, who ? " " : "", who ? who->bv_val : "");

1 0

Re: (ITS#5439) syncprov race condition seg. fault
by h.b.furuseth＠usit.uio.no 28 Mar '08

28 Mar '08

rein(a)basefarm.no writes: > It now looks to me as if the entire rs->sr_entry was released an > reused, (...) Could this be ITS#5340? -- Hallvard

1 0

Re: (ITS#5340) REP_ENTRY_MODIFIABLE bug in dynlist
by h.b.furuseth＠usit.uio.no 28 Mar '08

28 Mar '08

ando(a)sys-net.it wrote: [about REP_ENTRY_MUSTRELEASE] > it is not > clear what happens when a callback chain is interrupted by > slap_null_cb() or similar, without getting to slap_send_search_entry(). > This seems to indicate that callbacks should always provide a last > resort means to release the resources they set; if read-only, by keeping > track of what they sent; if modifiable, by freeing the contents of > rs->sr_* if not NULL, setting it to NULL to prevent further cleanup. That sounds cumbersome, I hope slapd could take care of that somehow. But I don't see how the be_release() code can work now. It sounds like be->be_release() functions must check (how?) that the entry was created by 'be', and otherwise pass it on to the next overlay/backend or otherwise to entry_free(). Might involve mucking with op->o_bd and sr_entry->e_private, I suppose. Except maybe I'm missing some existing magic since slapd doesn't regularly crash... be->be_release() does receive entries that were not created by 'be' or at least not with be->be_fetch(), see openldap-devel thread 'slapd API' in mar 2008. > Similarly, the existence of REP_ENTRY_MUSTBEFREED is not totally clear: > in principle as soon as REP_ENTRY_MODIFYABLE is set, it should imply > REP_ENTRY_MUSTBEFREED; the only difference in the semantics of the two > is that REP_ENTRY_MODIFYABLE without REP_ENTRY_MUSTBEFREED implies that > the callback that set the former will take care of freeing the entry; > however, other callbacks may further modify it, so freeing temporary > data should probably be left to the final handler. That's not my impression. MODIFIABLE would be that other modules than the creator can modify the entry - but the creator might still be the one who will free it. MUSTBEFREED is that the entry must be entry_free()ed - the creator will not do it (or not do it unless that flag is set). So, if I'm getting this right... A backend must expect an entry to change or be freed if it sends the entry with REP_ENTRY_<MUSTRELEASE, MUSTBEFREED or MODIFIABLE>, or if it passes through a slap_callback.sc_response. back-ldif does not: it uses e_name/e_nname _after_ sending with with REP_ENTRY_MODIFIABLE. Nor overlay retcode, it calls retcode_entry_response(,,,rs->sr_entry) which makes pointers into sr_entry without checking if those flags are set. If I'm getting that code correctly. I haven't tested. Others apparent problems (also not tested, I've just browsed the code): Overlays that obey and reset MUSTBEFREED or MUSTRELEASE, do not necessarily clear or set MODIFIABLE when setting a new entry. translucent does not, even though it is a careful one which obeys both MUSTBEFREED and MUSTRELEASE. I'm not sure when the code must clear the entry_related flags. Some code which sets new entries seem to assume they are zero, but some code which sets sr_entry=NULL does not clear the flags. There are sr_flags bits which are not about the entry, in particular REP_MATCHED_MUSTBEFREED and REP_MATCHED_MUSTBEFREED. Looks like these flags can get lost when but some code sets sr_flags = <something> rather sr_flags |= <something> or sr_flags ~= ~<something>. -- Hallvard

1 0

Re: ITS#5391 [Fwd: [Berkeley DB Announce] Berkeley DB 4.6.21.1 - Patch Announcement]
by quanah＠zimbra.com 27 Mar '08

27 Mar '08

--On Thursday, March 27, 2008 10:44 PM +0000 bdb(a)oss.oracle.com wrote: > I know you're not using BDB 4.6 with OpenLDAP 2.3, so this patch isn't > strictly relevant to this ITS. It just raises the possibility that there > is a BDB bug involved. Looking at the code for 4.2.52, the mp_fopen.c code is essentially identical to what this patch fixes, so I'd guess it could affect 2.3 releases using bdb 4.2->4.5. And I'm guessing there'll be no patch from Oracle for those much older releases. --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

ITS#5391 [Fwd: [Berkeley DB Announce] Berkeley DB 4.6.21.1 - Patch Announcement]
by bdb＠oss.oracle.com 27 Mar '08

27 Mar '08

I know you're not using BDB 4.6 with OpenLDAP 2.3, so this patch isn't strictly relevant to this ITS. It just raises the possibility that there is a BDB bug involved. -------- Original Message -------- Subject: [Berkeley DB Announce] Berkeley DB 4.6.21.1 - Patch Announcement Date: Thu, 27 Mar 2008 18:03:40 -0400 From: bdb(a)oss.oracle.com Reply-To: bdb(a)oss.oracle.com Organization: Oracle Corporation To: <bdb(a)oss.oracle.com> A patch for Berkeley DB release 4.6.21 that fixes a race condition between checkpoint and DB->close() which can result in the checkpoint thread self-deadlocking (SR [#15692]) is now available. We recommend all users of Berkeley DB apply this patch so as to avoid this issue in production. An web page with all patches (currently only one) for Berkeley DB 4.6.21. http://www.oracle.com/technology/products/berkeley-db/xml/update/4.6.21/patc h.4.6.21.html A text file with the patch: http://www.oracle.com/technology/products/berkeley-db/xml/update/4.6.21/patc h.4.6.21.1 regards, the Berkeley DB Team, -greg _____________________________________________________________________ Gregory Burd greg.burd(a)oracle.com Product Manager, Berkeley DB/JE/XML Oracle Corporation _______________________________________________ BDB mailing list BDB(a)oss.oracle.com http://oss.oracle.com/mailman/listinfo/bdb -- -- Howard Chu Chief Architect, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#5440) back-meta opens too many connections in case of error
by ando＠sys-net.it 26 Mar '08

26 Mar '08

ali.pouya(a)free.fr wrote: > Full_Name: Ali Pouya > Version: 2.4.8 > OS: Linux 2.6 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (145.242.11.3) > > > Hi, > I'm testing the back-meta of OpenLdap 2.4.8 in order to upgrade to this > release. > My client opens a persistent LDAP connection to slapd, binds with an ordinary > account (from the local back-bdb) and begins search operations. > > I notice that if a search operation results in an LDAP error (for example error > 32 : No Such Object), then back-meta opens a new connection to the target > directory for the next operation, leaving the "bad" connection open. > > The conn-ttl and idle-timeout parameters do not close the "bad" connections. > These connections remain there until the client ends its connection. > > This saturates the target servers with unused idle connections. > > Is this a bug or a normal behaviour ? > > I precise that The behaviour is not the same if the client binds with rootdn. 1) Can you post your slapd.conf (sanitized)? 2) What is exactly causing the error? What error is actually being returned from all targets? For back-meta, no such object has some special semantics. Best would be if you can provide a setup that allows to reproduce the critical situation with a very simple test, involving as little targets as possible, along with the LDIF required to populate each target and the operation you used. p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it --------------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Email: pierangelo.masarati(a)sys-net.it ---------------------------------------

1 0

(ITS#5440) back-meta opens too many connections in case of error
by ali.pouya＠free.fr 26 Mar '08

26 Mar '08

Full_Name: Ali Pouya Version: 2.4.8 OS: Linux 2.6 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (145.242.11.3) Hi, I'm testing the back-meta of OpenLdap 2.4.8 in order to upgrade to this release. My client opens a persistent LDAP connection to slapd, binds with an ordinary account (from the local back-bdb) and begins search operations. I notice that if a search operation results in an LDAP error (for example error 32 : No Such Object), then back-meta opens a new connection to the target directory for the next operation, leaving the "bad" connection open. The conn-ttl and idle-timeout parameters do not close the "bad" connections. These connections remain there until the client ends its connection. This saturates the target servers with unused idle connections. Is this a bug or a normal behaviour ? I precise that The behaviour is not the same if the client binds with rootdn. Thanks for your help Best regards Ali

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs