openldap-bugs

openldap-bugs@openldap.org

1 participants
20965 discussions

(ITS#5459) threw assertion in attr_dup
by richton＠nbcs.rutgers.edu 09 Apr '08

09 Apr '08

Full_Name: Aaron Richton Version: 2.3.41 OS: Solaris 9 URL: Submission from: (NULL) (68.192.238.168) (dbx) where current thread: t@18 [1] __lwp_kill(0x0, 0x6, 0xffffffffffffffe6, 0x0, 0x0, 0x0), at 0xffffffff7f0a8d4c [2] raise(0x6, 0x0, 0xffffffff457fe390, 0x0, 0x0, 0x0), at 0xffffffff7f058dc0 [3] abort(0x30, 0x0, 0x30, 0x7efefeff, 0x81010100, 0xff0000), at 0xffffffff7f03e688 [4] __assert(0x10023d488, 0x10023d490, 0x8d, 0x1, 0xffffffff457fe758, 0x11b369c59), at 0xffffffff7f03e98c =>[5] attr_dup(a = 0x1174b0470), line 141 in "attr.c" [6] attrs_dup(a = 0x1174b0470), line 166 in "attr.c" [7] hdb_modify_internal(op = 0xffffffff457ff618, tid = 0x11b369b70, modlist = 0x118ea8bd0, e = 0xffffffff457feab8, text = 0xffffffff457fefd8, textbuf = 0xffffffff457feb4c "^X\xea\x8b\xd0", textlen = 256U), line 55 in "modify.c" [8] hdb_modify(op = 0xffffffff457ff618, rs = 0xffffffff457fefb8), line 480 in "modify.c" [9] syncrepl_entry(si = 0x1004b5050, op = 0xffffffff457ff618, entry = 0x118d17870, modlist = 0xffffffff457ff328, syncstate = 2, syncUUID = 0xffffffff457ff3c0, syncCookie_req = 0xffffffff457ff360, syncCSN = 0xffffffff457ff390), line 1987 in "syncrepl.c" [10] do_syncrep2(op = 0xffffffff457ff618, si = 0x1004b5050), line 735 in "syncrepl.c" [11] do_syncrepl(ctx = 0xffffffff457ffc30, arg = 0x1004b5230), line 1095 in "syncrepl.c" [12] ldap_int_thread_pool_wrapper(xpool = 0x10041e700), line 478 in "tpool.c" (dbx) list 130,150 130 131 if ( a->a_nvals != a->a_vals ) { 132 int j; 133 134 tmp->a_nvals = ch_malloc( (i + 1) * sizeof(struct berval) ); 135 for ( j = 0; !BER_BVISNULL( &a->a_nvals[j] ); j++ ) { 136 assert( j < i ); 137 ber_dupbv( &tmp->a_nvals[j], &a->a_nvals[j] ); 138 if ( BER_BVISNULL( &tmp->a_nvals[j] ) ) break; 139 /* FIXME: error? */ 140 } 141 assert( j == i ); 142 BER_BVZERO( &tmp->a_nvals[j] ); 143 144 } else { 145 tmp->a_nvals = tmp->a_vals; 146 } 147 148 } else { 149 tmp->a_vals = NULL; 150 tmp->a_nvals = NULL; Crazy part is that I hit this on two slaves during the same replication cycle ... yet four slaves DIDN'T crash during said replication set. I think that database is hosed on those two slaves; I'll check more thoroughly tomorrow (I'm off site today). Provider is 2.3.39, so perhaps "it fed 2.3.41 garbage" -- but still, 2.3.41 should be able to gracefully handle it? Or maybe an assert() is a graceful handler, but I don't know about that, it kills my slave in the process?

1 0

(ITS#5458)
by mf＠14v.de 09 Apr '08

09 Apr '08

Looks like newline does not count as whitespace. Regards Matthias

1 0

(ITS#5458) whitespace parsing in schema files
by mf＠14v.de 09 Apr '08

09 Apr '08

Full_Name: Matthias Ferdinand Version: 2.4.8 OS: Linux/SuSE10.0 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (194.102.180.118) Platform: SuSE Linux 10.0 OpenLDAP 2.4.8 from pkgsrc (http://www.netbsd.org/docs/pkgsrc/) gcc-4.2.1 (also with OpenLDAP 2.4.7 from Debian Testing) I tried the following simple attribute definition (located at /opt/pkgsrc/etc/openldap/schema/test2.schema): attributetype ( 1.3.6.1.4.1.22558.2.1.1 NAME 'o4vdescription' DESC 'RFC2256: descriptive information' EQUALITY caseIgnoreMatch SUBSTR caseIgnoreSubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15{1024} ) If I leave out the blank preceding the closing paranthesis, so that there is just the newline between the "{1024}" and the ")", slapd refuses to start with some syntax error: AttributeTypeDescription = "(" whsp numericoid whsp ; AttributeType identifier [ "NAME" qdescrs ] ; name used in AttributeType ... and at loglevel 64 some strange syslog entries: Apr 10 00:45:08 vestatest slapd[25297]: line 6 (attributetype ( 1.3.6.1.4.1.22558.2.1.1 NAME 'o4vdescription' DESC 'RFC2256: descriptive information' EQUALITY caseIgnoreMatch SUBSTR caseIgnoreSubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15{1024}) Apr 10 00:45:08 vestatest slapd[25297]: /opt/pkgsrc/etc/openldap/schema/test2.schema: line 6 attributetype: Missing closing parenthesis before amavisBannedFilesLover $ amavisBypassBannedChecks $ amavisBadHeaderLover $ amavisBypassHeaderChecks $ amavisSpamTagLevel $ amavisSpamTag2Level $ amavisSpamKillLevel $ amavisWhiteli Apr 10 00:45:08 vestatest slapd[25297]: /opt/pkgsrc/etc/openldap/slapd.conf: line 13: <include> handler exited with 1! (the amavis entries are from a schema included just before this one) Looks like the parser reads the attribute definition correctly, but is confused afterwards. Regards Matthias

1 0

Re: (ITS#5340) REP_ENTRY_MODIFIABLE bug in dynlist
by h.b.furuseth＠usit.uio.no 09 Apr '08

09 Apr '08

Below I'll summarize SlapReply.sr_flags as I now understand it after un-confusing myself a little, since this ITS is getting rather long. Mostly about how code which receives a filled SlapReply should use it, which I presume is primarily overlays, slap_callbacks, maybe back-relay. About discussion so far: The sr_flags are inconsistent about when which ones may or must be obeyed. E.g. REP_ENTRY_MUST<RELEASE/BEFREED> does not mean the entry will be released/freed, but REP_<MATCHED/REF> does in some functions. This needs to be normalized. We can state that overlays with a bi_entry_release_rw function must keep track of which module should release which entry, like glue does. At least for now. Never mind my worries about which overlay/backend should release an REP_ENTRY_MUSTRELEASE'd sr_entry. That doesn't matter with current OpenLDAP code. As far as I can see glue is the only overlay with a release function, and glue can dispatch on the entry DN. rs.sr_ref bugs, fixing now: - slap_send_ldap_result() forgets to free rs->sr_ref if it does not call send_ldap_response(). - send_ldap_response() can return with sr_ref set to a freed value. I'll make it return with sr_ref to NULL if REP_REF_MUSTBEFREED. ando(a)sys-net.it writes: > Similarly, the existence of REP_ENTRY_MUSTBEFREED is not totally clear: Looked a bit at cvs logs... Since you are the one who introduced MUSTBEFREED and MODIFIABLE (slap.h 1.503 for rwm) I think we can take it the intent of REP_ENTRY_MUSTBEFREED is indeed unclear/forgotten:-) Anyway, I think I've gotten the flags straight now: * REP_<ENTRY, MATCHED, REF>_MUST<RELEASE, BEFREED>: Overlays/callbacks may modify the rs->sr_<entry, text, ref> pointers. If they do, they must release/free the old value if and only if the corresponding MUST<RELEASE/BEFREED> flag is set. They should (re)set the relevant flags to describe the new value. Many overlays don't, or not thoroughly enough. - REP_<MATCHED, REF>_MUSTBEFREED: Callers setting these can depend on send_ldap_response() to free sr_<matched, ref>. That is, "send response/disconnect" calls. These flags do apparently not apply and should not be set when sending an entry or search continuation reference. Dunno why not REP_REF_MUSTBEFREED when sending continuation references. - REP_ENTRY_MUSTRELEASE and REP_ENTRY_MUSTBEFREED: Callers that set these flags when sending an entry, *allow* the callee to release/free rs->sr_entry. Callers cannot depend on it being release/freed, but some do anyway. It should simplify things to make the send functions always obey this flag, but I'm not sure how much cleanup that requires. There is partial code for that in slap_send_search_entry(), along with a FIXME note. Free MUSTBEFREED-entries with entry_free(), and MUSTRELEASE-entries with be_entry_release_r(). There is no "use be_entry_release_w()" flag, but no OpenLDAP code sends write-locked entries anyway. + REP_ENTRY_MUSTRELEASE: Created for ITS#3671 to ensure that network hangs don't delay cache entry unlocks: Modules should release their cache entry locks before sending anything. They can't do that with cached entries they send in rs->sr_entry, so instead they should set REP_ENTRY_MUSTRELEASE. Conversely, anything which does network I/O (and other potentially slooow ops I guess) should first obey this flag. Several overlays don't obey, nor some other code for sending an entry when it decides to send an error response instead. * REP_ENTRY_MODIFIABLE: Overlays and callbacks may modify the entry which rs->sr_entry points at if this flag is set. If they want to modify an incoming sr_entry when REP_ENTRY_MODIFIABLE is not set, they can use entry_dup(), release or free the old entry if sr_flags says so, put the new entry in sr_entry, and maybe set REP_ENTRY_MODIFIABLE | REP_ENTRY_MUSTBEFREED. Thus code which creates a temporary Entry struct and sends it as rs->sr_entry, should set REP_ENTRY_MODIFIABLE if the entry is of no further interest to it (and it has set no pointers into it, e.g. rs->sr_matched). That allows overlays that wish to modify the entry being sent, to use this entry instead of copying it. * REP_NO_<OPERATIONALS, ENTRYDN, SUBSCHEMA>: Set by the chain overlay to "tell the frontend not to add generated operational attributes". Only backend.c reacts to these flags. Dunno if overlays should reset these flags when replacing sr_entry with a "private" entry (with another DN) rather than a dup of the current one, or something. Some overlays set sr_flags to fresh values while others only update some bits in sr_flags, so the effect is likely somewhat ad-hoc. Leaving that worry to someone else. I've been fiddling with some functions to generalize this, but they seem to need a lot of flags or variants for different modes... Should maybe just commit some draft variants with warnings that they may change, and clean up later. After my changes don't cause slapd to crash, anyway:-) -- Hallvard

1 0

Re: (ITS#5408) back-ldif bugs
by h.b.furuseth＠usit.uio.no 09 Apr '08

09 Apr '08

hyc(a)symas.com writes: > h.b.furuseth(a)usit.uio.no wrote: >> - back-ldif escapes the directory separator as \<hex value>. But >> Windows uses "\" as directory separator, so that doesn't help. It >> needs another escape character. >> >> To avoid double hex-escaping of DNs that contain "\", we can pick >> another escape char, e.g. "^", escape that too, and translate "\" to >> it. Thus DN "cn=x^y\2Cz" gets filename "cn=x^5Ey^2Cz.ldif". > > That sounds fine. Is there any reason not to use %, which is already > used for URL encoding? Good point. The filenames I suggested will not be valid URLs but will resemble them, but that can be fixed by using proper URL-encoding of RDNs. Filenames for RDNs with \ in them get a bit longer though. >> - More characters should likely be hex-escaped: >> + ":" (as in "C:\") and "/" on Windows? I've seen programs use the >> latter as directory separator even on Windows. Others? >> + 8-bit chars in case the OS gets clever about charset handling. >> + Control chars. >> >> I don't use Windows myself though. Nor Mac... > > Again, the rules for URL encoding probably already cover any cases we > should worry about. I don't think so. At least "/" is not "unsafe", if I read it right. >> - RDNs ending with ".ldif" should be escaped. Currently "cn=foo.ldif" >> is the name of both RDN "cn=foo"'s entry file and RDN "cn=foo.ldif"'s >> non-leaf directory. > > Perhaps we should escape '.' Hm, true. "." is (was?) a special character on some filesystems. E.g. a file can only have one of them, or the 2nd is followed by a version number. >> - Sorted-value RDNs: No reply? :-) Ok, sort order can be adjusted separately anyway. One error in my previous attempt: bconfig.c mentions "pseudo-indexed" entries (cn=Include{xx}, cn=Module{xx}) which I presume must be sorted by {xx} too, so back-ldif can't treat only "attr={number}val" as indexed. I guess I'll either check both variants or allow {number} anywhere, depending on convenience. -- Hallvard

1 0

Re: (ITS#5408) back-ldif bugs
by hyc＠symas.com 09 Apr '08

09 Apr '08

h.b.furuseth(a)usit.uio.no wrote: > Full_Name: Hallvard B Furuseth > Version: HEAD, RE23, RE24 > OS: Linux > URL: http://folk.uio.no/hbf/OpenLDAP/back-ldif.c > Submission from: (NULL) (129.240.6.233) > Submitted by: hallvard > > > Here are a bunch of back-ldif bugs some questions. I have code for most > of it, but need advice/discussion on functionality changes. > > I've been sitting on it too long and would keep sitting if I waited > until I'd cleaned up everything, so, posting now instead. I enclose an > URL for a *draft* back-ldif/ldif.c. > > > Functionality changes. > > * Some LDIF files/directories with special chars need to be renamed: > > RE24 or RE25 change? If anyone uses back-ldif as a regular database, > they may need to slapcat/slapadd to upgrade past these changes. > > - back-ldif escapes the directory separator as \<hex value>. But > Windows uses "\" as directory separator, so that doesn't help. It > needs another escape character. > > To avoid double hex-escaping of DNs that contain "\", we can pick > another escape char, e.g. "^", escape that too, and translate "\" to > it. Thus DN "cn=x^y\2Cz" gets filename "cn=x^5Ey^2Cz.ldif". That sounds fine. Is there any reason not to use %, which is already used for URL encoding? > - More characters should likely be hex-escaped: > + ":" (as in "C:\") and "/" on Windows? I've seen programs use the > latter as directory separator even on Windows. Others? > + 8-bit chars in case the OS gets clever about charset handling. > + Control chars. > > I don't use Windows myself though. Nor Mac... Again, the rules for URL encoding probably already cover any cases we should worry about. > > - When back-ldif uses OS-specific escaping, we can't move a directory > tree between Windows and Unix hosts if some RDN in the tree contains > characters with OS-specific escaping. > > Should we special-case both Unix' and Windows' directory separators > on both OSes? Then instead there will be more directory trees which > can't be move between OpenLDAP versions, before and after the > change. Assuming anyone uses back-ldif in the first place:-) Might as well break it all at once. > > - The database suffix must be hex-escaped like the rest of the > filenames. > > - RDNs ending with ".ldif" should be escaped. Currently "cn=foo.ldif" > is the name of both RDN "cn=foo"'s entry file and RDN "cn=foo.ldif"'s > non-leaf directory. Perhaps we should escape '.' > > - Sorted-value RDNs: > > + In dn2path() when IX_FSL != IX_DNL (filename '{' != RDN '{'): > IX_FS[LR] must be hex-escaped. They are treated as equivalent to > '{' - '}', thus different RDNs can map to the same filename. > > The "{1}" in path "/foo/cn=x{,cn={1}y" is not recognized. > The IX_DN[LR] (i.e. '{', '}') to IX_FS[LR] translation is a bit > strange: It translates the first '{' it sees, then the next > '}', then the next '{', etc. Any reason not to just translate > all '{' and '}' chars? If so, we should at least reset at > each filename and '+'. > > + Sorting (in ldif_r_enum_tree()): > > Should "a={1}x" sort before or after "a=x"? Currently it sorts > like "a={". "a=" (normally before) or "a=<CHAR_MAX>" (normally > after) would be better. > > RDN "attr=foo{bar}baz" is treated as a sorted value. Can easily > check for '={'<successful strtol parse'> '}' instead. > "attr={0<octal>}val" and "attr={0x<hex>}val" are recognized as > sorted values, should they be? > > Some of this is from ITS#4627 (back-ldif issues). -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#5457) bdb_add() does not always release entries
by h.b.furuseth＠usit.uio.no 09 Apr '08

09 Apr '08

hyc(a)symas.com writes: >> bdb_unlocked_cache_return_entry_r(&bdb->bi_cache, p ); >> (...) > Releasing entries is somewhat irrelevant, since all locks are released when > the transaction commits. Note that this function is a no-op in proto-bdb.h. > All of those statements are just relics from the first entry cache design, > before we switched to using BDB locks for everything. OK. Does that apply to bdb_cache_return_entry_r( bdb, oe, &lock ); after bdb_cache_add( bdb, ei, op->ora_e, &nrdn, rlocker, &lock ); failed TXN_COMMIT( ltid, 0 ); too? bdb_cache_return_entry_r() is not a no-op. -- Hallvard

1 0

Re: (ITS#5451) syncprov_checkpoint deadlock bugfix?
by rein＠tollevik.no 09 Apr '08

09 Apr '08

On Wed, 9 Apr 2008, h.b.furuseth(a)usit.uio.no wrote: > Does this help? From my fiddling with ITS#5340 (REP_ENTRY_MODIFIABLE). > I do not understand syncprov's handling of REP_ENTRY_MUSTRELEASE though. > (For one thing it seems to assume that REP_ENTRY_MUSTRELEASE is set if > and only if rs.sr_entry->e_private != NULL. Which is possibly true with > back-bdb but seems a shaky assumption in general.) I have found out how to reliably reproduce this bug in our master server, and it definitely involves syncprov. I'm now working on reproducing it in a minimalistic test configuration, I'll try your patch there and report back. Rein

1 0

Re: (ITS#5452) seg fault in syncprov_op abandon on master
by hyc＠symas.com 09 Apr '08

09 Apr '08

quanah(a)zimbra.com wrote: > --On Friday, April 04, 2008 1:23 PM +0200 Hallvard B Furuseth > <h.b.furuseth(a)usit.uio.no> wrote: > >> Since it crashes on >> if ( so->s_op->o_connid == op->o_connid&& >> so->s_op->o_msgid == op->orn_msgid ) >> please try >> frame 0 >> print *op >> print *so->s_op >> and maybe >> backtrace full >> while you are at it. So far there don't appear to be any invalid pointers anywhere, so there's no clue why you hit a SEGV here. Can you reproduce this situation using valgrind? -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#5451) syncprov_checkpoint deadlock bugfix?
by h.b.furuseth＠usit.uio.no 09 Apr '08

09 Apr '08

Does this help? From my fiddling with ITS#5340 (REP_ENTRY_MODIFIABLE). I do not understand syncprov's handling of REP_ENTRY_MUSTRELEASE though. (For one thing it seems to assume that REP_ENTRY_MUSTRELEASE is set if and only if rs.sr_entry->e_private != NULL. Which is possibly true with back-bdb but seems a shaky assumption in general.) Index: syncprov.c =================================================================== RCS file: /repo/OpenLDAP/pkg/ldap/servers/slapd/overlays/syncprov.c,v retrieving revision 1.227 diff -u -2 -r1.227 syncprov.c --- syncprov.c 3 Apr 2008 06:43:48 -0000 1.227 +++ syncprov.c 9 Apr 2008 10:29:40 -0000 @@ -2390,7 +2390,13 @@ if ( !ap ) { if ( !(rs->sr_flags & REP_ENTRY_MODIFIABLE) ) { - rs->sr_entry = entry_dup( rs->sr_entry ); - rs->sr_flags |= - REP_ENTRY_MODIFIABLE|REP_ENTRY_MUSTBEFREED; + Entry *new = entry_dup( rs->sr_entry ); + if ( rs->sr_flags & REP_ENTRY_MUSTRELEASE ) { + be_entry_release_r( op, rs->sr_entry ); + } else if ( rs->sr_flags & REP_ENTRY_MUSTBEFREED ) { + entry_free( rs->sr_entry ); + } + rs->sr_entry = new; + rs->sr_flags = (rs->sr_flags & ~REP_ENTRY_MASK) | + REP_ENTRY_MODIFIABLE | REP_ENTRY_MUSTBEFREED; a = attr_find( rs->sr_entry->e_attrs, slap_schema.si_ad_contextCSN ); -- Hallvard

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs