openldap-bugs June 2013

openldap-bugs@openldap.org

19 participants
65 discussions

Re: (ITS#7614) Markup error in slapd.conf.5
by hyc＠symas.com 04 Jun '13

04 Jun '13

Eric S. Raymond wrote: > Howard Chu <hyc(a)symas.com>: >> OK, I ran Docbook 4.5 on the output from doclifter and I see it's >> complaining on one line of slapd.conf.5.xml. It appears doclifter is >> misinterpreting the .br request. Instead of inserting a line break >> it is either ignoring it completely or doing a paragraph break. >> >> I'm going to reject your submitted patch because it is not the >> minimal patch needed to fix the error, nor is the error actually in >> our manpage source, it is in the doclifter tool. > > You have drastically misunderstood what you saw. This isn't an error > in doclifter, it's a result of a basic ontological mismatch between > the source and target markups that doclifter tries to bridge across. > > There are no line breaks in DocBook's XML, structure-oriented world. > The closest thing to one ia special-purpose <sbr/> tag that is only valid > inside command synoposes. Thus there is no "line break" equivalent > that .br can be mapped to. Nor should there be; that is fundamentally > a presentation-level concept. A line-break may well be only a presentation-level concept, but it is also something that a document author explicitly chooses to insert. A tool (or markup language) that ignores an author's explicit wishes is a broken tool. > My patch adds the structural information required for doclifter to > get traction. The requirement for this is rare, and getting more so > as man page maintainers clean up their sources. Please be part of > that solution rather than part of the problem. Your solution is broken or unfit for the purpose if it cannot process simple and correct manpage source. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7614) Markup error in slapd.conf.5
by esr＠thyrsus.com 04 Jun '13

04 Jun '13

Howard Chu <hyc(a)symas.com>: > OK, I ran Docbook 4.5 on the output from doclifter and I see it's > complaining on one line of slapd.conf.5.xml. It appears doclifter is > misinterpreting the .br request. Instead of inserting a line break > it is either ignoring it completely or doing a paragraph break. > > I'm going to reject your submitted patch because it is not the > minimal patch needed to fix the error, nor is the error actually in > our manpage source, it is in the doclifter tool. You have drastically misunderstood what you saw. This isn't an error in doclifter, it's a result of a basic ontological mismatch between the source and target markups that doclifter tries to bridge across. There are no line breaks in DocBook's XML, structure-oriented world. The closest thing to one ia special-purpose <sbr/> tag that is only valid inside command synoposes. Thus there is no "line break" equivalent that .br can be mapped to. Nor should there be; that is fundamentally a presentation-level concept. Thus, one of either ignoring the .br or interpreting it as a paragraph boundary is exactly what doclifter *should* be doing. This is not a bug, it is correct behavior because those are the only alternatives XML-DocBook supports. doclifter has to deduce which interpretation is correct by applying certain parsing and pattern-matching rules. Usually it succeeds, but there are rare (single-digit-percentage) cases in which it fails. Yours is one. I repeat: the error is *not in the tool*. It's a result of attempting to deduce structure out of the troff tag soup that the tag soup does not intrinsically possess. After 11 years of development doclifter does a pretty heroic job of this, but a perfect one is not possible - there's too much chaos ot there. My patch adds the structural information required for doclifter to get traction. The requirement for this is rare, and getting more so as man page maintainers clean up their sources. Please be part of that solution rather than part of the problem. -- <a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

1 0

Re: (ITS#7614) Markup error in slapd.conf.5
by hyc＠symas.com 04 Jun '13

04 Jun '13

Howard Chu wrote: > Eric S. Raymond wrote: >> Howard Chu <hyc(a)symas.com>: >>> For fear of opening up another can of worms, surely slapd.conf.5 is >>> not the only manpage that needs this patch? Other manpages in the >>> distro also use .nf/.fi and .RS/.RE. >>> >>> We need consistency in our source as well. >> >> Agreed. I have analogous patches for slapd-config.5 and slapo-constraint.5. >> Whatever other OpenLDAP pages are in stock Ubuntu are not throwing >> warnings. How would you like these submitted? >> > I just ran doclifter on all 3 of these man pages and the only warning I got > was about the ../Project file. Please tell me how to reproduce the original > error you're reporting because I'm not seeing it. > > Also, how do we fix it so that the content of ../Project is included as it > should be? > OK, I ran Docbook 4.5 on the output from doclifter and I see it's complaining on one line of slapd.conf.5.xml. It appears doclifter is misinterpreting the .br request. Instead of inserting a line break it is either ignoring it completely or doing a paragraph break. I'm going to reject your submitted patch because it is not the minimal patch needed to fix the error, nor is the error actually in our manpage source, it is in the doclifter tool. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7612) {CLEARTEXT} password scheme broken
by Kurt＠OpenLDAP.org 03 Jun '13

03 Jun '13

On Jun 3, 2013, at 1:08 PM, Hallvard Breien Furuseth <h.b.furuseth(a)usit.uio.no> wrote: > ...OTOH perhaps it's too late to change {cleartext} now if > it has an established useful meaning. Could introduce a new > scheme called {plaintext} or {raw} instead for this purpose. {CLEARTEXT} isn't an RFC 2307 hash scheme... it's a string used in configuration and command line cases to indicate that no RFC 2307 hash scheme is used, that is, the password is cleartext. In the code, IIRC, it's referred to as pseudo-scheme for this reason. The restrictions on clear text passwords that look like RFC 2307 hashed password exist because of the conflict between standard LDAPv3 behavior and RFC 2307 behavior. Adding some RFC 2307 plain-text hash scheme doesn't remove the conflict between standard LDAPv3 behavior RFC 2307 hashed passwords, so IMO the restriction should remain, at least by default. If such an option were introduced, the documentation should make clear that disabling the check can be problematic if ever stored hash passwords are to restored (on Bind) to LDAPv3 compliant LDAPv3 passwords over a period of time. -- Kurt

1 0

Re: (ITS#7614) Markup error in slapd.conf.5
by esr＠thyrsus.com 03 Jun '13

03 Jun '13

Howard Chu <hyc(a)symas.com>: > Hm. I use my own man2html http://highlandsun.com/hyc/man2html.c > which gives pretty good looking output for us. Really, if you're > developing a tool that claims to read troff input, it has to > actually do so. I mean, the point of tools such as this is to be > able to convert existing documents without modifying them, isn't it? There's a subtle difference between tools that translate purely at a presentation level and tools that do content analysis. A purely presentation-level translation such as your man2html is indeed less likely to be thrown by weird troff markup. And much of the time it will produce markup that doesn't look bad, especially on a relatively small collection of pages in a consistent house style. But there are things such a tool cannot do that become more important when you are translating a very large corpus of man pages with multiple authors - such as an entire Linux distribution's man page tree. Here is an example: the treatment of file paths in FILES sections. Some man-page authors mark them up as bold text. Some mark them up as italic. And some give them no highlight at all. A purely presentation-level translator will simply translate any font change from troff to HTML. In the resulting output, filenames will have three different visual signatures. Readers will thus have to work a little harder than they should to recognize filenames. A tool that does content analysis, on the other hand, can recognize presentation-level cliches that mean "this is a filename", such as a line in FILES beginning with .B or .I and containing a /, and map it to a DocBook <filename> pair. The DocBook stylesheet will then ensure that all filenames are visually marked in the *same* way in generated HTML, rather than three different ways. Now multiply this effect by all the different things that can be recognized by content - things like Unix error codes, program listings, command synopses, C function prototypes, references to other manual pages. The effect is a higher-quality and more visually uniform translation. The gain in quality increases with the diversity of the author population. For very large collections like an entire Linux distribution's man page tree it is quite significant. The tradeoff is that, while a presentation-level translator will cheerfully produce a visual garble from ill-formed troff, a tool with a real parser and a content analyzer will have a few more cases in which it just can't cope at all. Not *many* cases, mind you; in the decade I've been developing mine, perhaps 2-3%. But these are worth fixing anyway, because they're likely to break third-party man-page readers. Nothing but troff itself interprets troff perfectly. -- <a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

1 0

Re: (ITS#7612) {CLEARTEXT} password scheme broken
by h.b.furuseth＠usit.uio.no 03 Jun '13

03 Jun '13

I wrote: > This does not imply that a scheme can't be used which simply > represents the passwords as-is, nor that slapd or slap tools have > any business stripping away such a {scheme} prefix. In particular > not when that's the only way to represent cleartext passwords > starting with "{letters}". ...OTOH perhaps it's too late to change {cleartext} now if it has an established useful meaning. Could introduce a new scheme called {plaintext} or {raw} instead for this purpose. -- Hallvard

1 0

Re: (ITS#7612) {CLEARTEXT} password scheme broken
by h.b.furuseth＠usit.uio.no 03 Jun '13

03 Jun '13

On 2013-06-03 20:46, Kurt(a)OpenLDAP.org wrote: > Not a bug... > > Clear text passwords appear in userPassword without any RFC 2307 > scheme, as in > > userPassword: secret > > not: > > userPassword: {CLEARTEXT}secret That's backwards. userPassword values without a {scheme} prefix are cleartext passwords. Values with a {scheme} prefix use that scheme. This does not imply that a scheme can't be used which simply represents the passwords as-is, nor that slapd or slap tools have any business stripping away such a {scheme} prefix. In particular not when that's the only way to represent cleartext passwords starting with "{letters}". Though possibly this would mean slapd needs a tweak to how it represents non-prefixed passwords internally, if it currently uses "{cleartext}" to tell itself that. I have not looked yet. -- Hallvard

1 0

Re: (ITS#7612) {CLEARTEXT} password scheme broken
by Kurt＠OpenLDAP.org 03 Jun '13

03 Jun '13

On May 31, 2013, at 2:38 AM, wferi(a)niif.hu wrote: > Full_Name: Ferenc Wágner > Version: 2.4.31 > OS: Debian GNU/Linux squeeze > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (86.101.52.7) > > > I'm trying to store the hypothetical password "{SSHA}" in cleartext, but > slappasswd refuses to help: > > $ /usr/sbin/slappasswd -s {SSHA} -h {CLEARTEXT} > Password verification failed. > > On #openldap hbf suggested that I file an ITS ("work" in the following means > allowing binding): > > hbf: Looks like {CLEARTEXT} itself is broken. I think "userPassword: > {CLEARTEXT}secret" should work, and so that slappasswd -h {CLEARTEXT} -s secret > can output {CLEARTEXT}secret and userPassword: {CLEARTEXT}{SSHA} would be > valid. > > As I agree with him, here it is. > Not a bug... Clear text passwords appear in userPassword without any RFC 2307 scheme, as in userPassword: secret not: userPassword: {CLEARTEXT}secret A cleartext password of {SSHA} is disallowed for what should be obvious reasons. -- Kurt

1 0

Re: (ITS#7614) Markup error in slapd.conf.5
by hyc＠symas.com 03 Jun '13

03 Jun '13

Eric S. Raymond wrote: > Howard Chu <hyc(a)symas.com>: >> The patch changes much more than your bug report mentions. The error >> message you provide is pretty ambiguous, in particular you haven't >> mentioned exactly which markup element is out of order. Without this >> information we can't confirm what you're fixing. > > Globally, I'm trying to fix up the entire Linux manual page corpus so > it can be automatically lifted to clean HTML. I've been working in > this intermittently since 2002, and am not far from the goal. In > approximately 12,000 pages carried in stock Ubuntu, 94% now lift > without errors or warnings. For all but 17 of the 6% remaining I have > fix patches which I'm trying to get merged upstream. > > I entered one of those patches in your bugtracker, but failed to notice > that the error code in my bug database entry for slapd.conf.5 was > incorrect (failed to match the patch). I apologize for this; > full explanation follows. > > My conversion tool is doclifter, which you can read about at > <http://www.catb.org/~esr/doclifter/>. It lifts manual pages to > XML-DocBook, which in turn is very easily rendered to HTML. This > sequence produces better-quality HTML than tools like man2html > can generate, because of the amount of content analysis done > in the doclifter intermediate step. > There are some legal troff constructions that doclifter cannot > parse well. These are the same sorts of things that any man-page > renderer other than groff itself is likely to get wrong; thus, they > are likely to confuse tools such as XMan, TkMan, and Rosetta as well > as doclifter. Hm. I use my own man2html http://highlandsun.com/hyc/man2html.c which gives pretty good looking output for us. Really, if you're developing a tool that claims to read troff input, it has to actually do so. I mean, the point of tools such as this is to be able to convert existing documents without modifying them, isn't it? > Thhe patch I sent you introduces an .EX/.EE macro pair for framing > code examples, a common extension often found on older Unix manual > pages (I believe it originated in DEC Ultrix). This macro > encapsulates a bunch of low-level troff operations that are hard to > lift well in isolation. There's a ruleset in doclifter that recognizes > .EX/.EE, analyzes the example content, and generates <programlisting> > or <literallayout> tags as appropriate. > > In some cases .EX/.EE replaces .RS/.RE pairs; in others it replaces > .nf/.fi pairs, in still others it replaces .TP, and I think there > are a couple of odd combinations of .RS/.RE with .TP in there too. The fact > that it has to mess with all of these is what makes the patch so > complex. But the result is simple: to express the "example" intention > better so it can be structurally translated. > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7614) Markup error in slapd.conf.5
by esr＠thyrsus.com 03 Jun '13

03 Jun '13

Howard Chu <hyc(a)symas.com>: > The patch changes much more than your bug report mentions. The error > message you provide is pretty ambiguous, in particular you haven't > mentioned exactly which markup element is out of order. Without this > information we can't confirm what you're fixing. Globally, I'm trying to fix up the entire Linux manual page corpus so it can be automatically lifted to clean HTML. I've been working in this intermittently since 2002, and am not far from the goal. In approximately 12,000 pages carried in stock Ubuntu, 94% now lift without errors or warnings. For all but 17 of the 6% remaining I have fix patches which I'm trying to get merged upstream. I entered one of those patches in your bugtracker, but failed to notice that the error code in my bug database entry for slapd.conf.5 was incorrect (failed to match the patch). I apologize for this; full explanation follows. My conversion tool is doclifter, which you can read about at <http://www.catb.org/~esr/doclifter/>. It lifts manual pages to XML-DocBook, which in turn is very easily rendered to HTML. This sequence produces better-quality HTML than tools like man2html can generate, because of the amount of content analysis done in the doclifter intermediate step. There are some legal troff constructions that doclifter cannot parse well. These are the same sorts of things that any man-page renderer other than groff itself is likely to get wrong; thus, they are likely to confuse tools such as XMan, TkMan, and Rosetta as well as doclifter. Thhe patch I sent you introduces an .EX/.EE macro pair for framing code examples, a common extension often found on older Unix manual pages (I believe it originated in DEC Ultrix). This macro encapsulates a bunch of low-level troff operations that are hard to lift well in isolation. There's a ruleset in doclifter that recognizes .EX/.EE, analyzes the example content, and generates <programlisting> or <literallayout> tags as appropriate. In some cases .EX/.EE replaces .RS/.RE pairs; in others it replaces .nf/.fi pairs, in still others it replaces .TP, and I think there are a couple of odd combinations of .RS/.RE with .TP in there too. The fact that it has to mess with all of these is what makes the patch so complex. But the result is simple: to express the "example" intention better so it can be structurally translated. -- <a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

1 0

Re: (ITS#7615) NetBSD sem_open limited to 14 characters
by hyc＠symas.com 03 Jun '13

03 Jun '13

Greg(a)Akua.com wrote: > Full_Name: Greg Kerr > Version: 2.4.35 > OS: NetBSD & FreeBSD > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (50.88.130.68) > > > MDB is not usable on NetBSD because mdb.c has > > sprintf(env->me_txns->mti_wmname, "/MDBw%s", hexbuf); > > To create the semaphore name. > > This results in a 21 character name which is beyond the 14 allowed per "man > sem_open" That's unfortunate. In Linux this limit is 251 characters. Seems you'd need to use the btoa algorithm to safely fit a hash into this size limit. Pretty sure we've tested successfully on FreeBSD and MacOSX already, so apparently they don't suffer from this same limit. > "less than 14 characters in length not including the terminating null > character." > > I will make a local patch ... and maybe it's a NetBSD bug - or at least the > package maintainers ... I will report to him. > > At any rate, the failure of mdb with result code 63 (name too long) was very > confusing ... better error reporting in this file would be nice. > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7614) Markup error in slapd.conf.5
by hyc＠symas.com 03 Jun '13

03 Jun '13

esr(a)thyrsus.com wrote: > Full_Name: Eric S. Raymond > Version: > OS: Linux > URL: http://www.catb.org/~esr/doclifter/prepatch/slapd.conf.5.patch > Submission from: (NULL) (2001:470:e34c:0:226:18ff:fea6:dca4) > > > List syntax error. This means .IP, .TP or .RS/.RE markup is garbled. > Common causes include .TP just before a section header, .TP entries > with tags but no bodies, and mandoc lists with no trailing .El. > These confuse doclifter, and may also mess up stricter man-page > browsers like Xman and Rosetta. > > Fix patch at link. The patch changes much more than your bug report mentions. The error message you provide is pretty ambiguous, in particular you haven't mentioned exactly which markup element is out of order. Without this information we can't confirm what you're fixing. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#7615) NetBSD sem_open limited to 14 characters
by Greg＠Akua.com 03 Jun '13

03 Jun '13

Full_Name: Greg Kerr Version: 2.4.35 OS: NetBSD & FreeBSD URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (50.88.130.68) MDB is not usable on NetBSD because mdb.c has sprintf(env->me_txns->mti_wmname, "/MDBw%s", hexbuf); To create the semaphore name. This results in a 21 character name which is beyond the 14 allowed per "man sem_open" "less than 14 characters in length not including the terminating null character." I will make a local patch ... and maybe it's a NetBSD bug - or at least the package maintainers ... I will report to him. At any rate, the failure of mdb with result code 63 (name too long) was very confusing ... better error reporting in this file would be nice.

1 0

(ITS#7614) Markup error in slapd.conf.5
by esr＠thyrsus.com 02 Jun '13

02 Jun '13

Full_Name: Eric S. Raymond Version: OS: Linux URL: http://www.catb.org/~esr/doclifter/prepatch/slapd.conf.5.patch Submission from: (NULL) (2001:470:e34c:0:226:18ff:fea6:dca4) List syntax error. This means .IP, .TP or .RS/.RE markup is garbled. Common causes include .TP just before a section header, .TP entries with tags but no bodies, and mandoc lists with no trailing .El. These confuse doclifter, and may also mess up stricter man-page browsers like Xman and Rosetta. Fix patch at link.

1 0

Re: (ITS#7594) incomplete MDB subdb cursor cleanup
by h.b.furuseth＠usit.uio.no 02 Jun '13

02 Jun '13

Howard Chu writes: >h.b.furuseth(a)usit.uio.no wrote: >> If a cursor is at a clean subDB page, and its current item is deleted >> by another cursor, then MDB_GET_CURRENT returns the deleted item. > > Fixed now in mdb.master. The subcursor on the clean page will be invalidated > when the page is touched. MDB_NEXT/MDB_PREV crash on assert(mc->mc_flags & C_INITIALIZED) since commit e31c7d3b31d8d5073195e31b283e8ccb46bd13cf, with the patch below to the test program. One partial fix: Instead return a new error code MDB_BAD_CURSOR or maybe MDB_UNSPECIFIED. Also make mdb_cursor_<next/prev>() less forgiving of errors, otherwise the user gets different success results for the two cursors again. Branch mdb/its7594 in <http://folk.uio.no/hbf/OpenLDAP/openldap.git> does this. I did not look at how to get the cursors return the same result. mdb.master output with patched test program: With clean page: Name: Key -> Data (mv_data[mv_len]): m2: a[2] -> x[2] mc/del: a[2] -> [0] a.out: mdb.c:4386: mdb_cursor_next: Assertion `mc->mc_flags & 0x01' failed. Branch mdb/its7594 output: With clean page: Name: Key -> Data (mv_data[mv_len]): m2: a[2] -> x[2] mc/del: a[2] -> [0] its7594a.c:48: mdb_cursor_get(m2, &key, &data, MDB_NEXT): MDB_BAD_CURSOR: Cursor is unusable That assert is OK since it's the test program asserting success. --- its7594.c +++ its7594a.c @@ -28,4 +28,6 @@ E(mdb_cursor_put(mc, STR2VAL("a"), STR2VAL("x"), 0)); E(mdb_cursor_put(mc, STR2VAL("a"), STR2VAL("y"), 0)); + E(mdb_cursor_put(mc, STR2VAL("b"), STR2VAL("z"), 0)); + E(mdb_cursor_put(mc, STR2VAL("c"), STR2VAL("w"), 0)); if (argc < 2) { @@ -42,8 +44,8 @@ E(mdb_cursor_del(mc, 0)); - E(mdb_cursor_get(m2, &key, &data, MDB_GET_CURRENT)); - SHOW("m2/del"); /* Should output the same as... */ - E(mdb_cursor_get(mc, &key, &data, MDB_GET_CURRENT)); - SHOW("mc/del"); /* ...this: "a[2] -> [0]". */ + E(mdb_cursor_get(mc, &key, &data, MDB_NEXT)); + SHOW("mc/del"); /* Outputs "a[2] -> [0]"... */ + E(mdb_cursor_get(m2, &key, &data, MDB_NEXT)); + SHOW("m2/del"); /* ...and this should output the same. */ mdb_txn_abort(txn); -- Hallvard

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs June 2013