On Dec 27, 2007, at 10:26 AM, Philip Guenther wrote:

On Thu, 27 Dec 2007, Dieter Kluenter wrote:

...

"Allan E. Johannesen" aej@WPI.EDU writes:

...

...

...

In a program, which had constructed the ** due to a missing first name, it also was found to break the connection to the server...

CCC5:~> ldapsearch '(cn=** VERNON-GERSTENFELD*)'

[...] There should be only one asterisk and no space after a asterisk. See rfc 4515, String representation on search filters.

Umm, no. That filter matches the grammar in RFC 4515.

Note that there is no way one can represent an empty initial or empty final substrings. That's because empty substrings are nonsense. The filter (cn=**) is invalid as it represents an empty initial, an empty any, and an empty final. slapd(8) correctly errors on receipt of any empty substring.

One can argue that the LDAP ASN.1 and the LDAP filter ABNF should have size constraints on substring values. I consider it a minor bug in specification which should be fixed.

IMO, the bug is in ldapsearch(1). It should reject a filter with empty substrings.

-- Kurt

The relevant grammar terms are:

 filter         = LPAREN filtercomp RPAREN
 filtercomp     = and / or / not / item
 item           = simple / present / substring / extensible
 substring      = attr EQUALS [initial] any [final]
 any            = ASTERISK *(assertionvalue ASTERISK)
 assertionvalue = valueencoding
 valueencoding  = 0*(normal / escaped)
 normal         = UTF1SUBSET / UTFMB
 UTF1SUBSET     = %x01-27 / %x2B-5B / %x5D-7F

I.e., it's a substring match with no 'initial', no 'final', and two 'assertionvalue' parts, the first of which is empty (a 'valueencoding' can be zero length) and the second of which is " VERNON-GERSTENFELD".

So, if the text form of the filter is legal, what's breaking? Either the client is sending bad BER or the server is refusing good BER. It looks to me like the latter: the protocol's ASN.1 says that an 'any' substring is an OCTET STRING, which can be zero length, but the substring parser in slapd doesn't accept that: in get_ssa() it treats a zero length substring as an error: if ( value.bv_val == NULL || value.bv_len == 0 ) { rc = LDAP_INVALID_SYNTAX; goto return_error; }

I suggest that Allan file an ITS...

Philip Guenther

On Thu, 27 Dec 2007, Howard Chu wrote:

Kurt Zeilenga wrote:

...

On Dec 27, 2007, at 10:26 AM, Philip Guenther wrote:

...

Umm, no. That filter matches the grammar in RFC 4515.

Note that there is no way one can represent an empty initial or empty final substrings. That's because empty substrings are nonsense. The filter (cn=**) is invalid as it represents an empty initial, an empty any, and an empty final. slapd(8) correctly errors on receipt of any empty substring.

If this has always been an error, then I'm surprised the restriction isn't mentioned at all---not even in prose---in either RFC 2251 or RFC 4511. Note that RFC 4517 _does_ ban empty 'any' parts in SubstringAssertion, so the issue was at least thought about in that context.

I would also contest the claim that empty substrings are nonsense. They may clearly be useless, but they are not logically inconsistent. Indeed, a server can trivially 'support' them by simply ignoring them whenever they appear. The lack of a restriction against them in the specs would imply that a server that fails to do so is non-compliant.

That said, if most servers throw an error on empty substrings, then clients can't send them and expect it to reliably work and (as you suggest) they should be removed from the protocol when it is revised. Perhaps an errata should be filed now.

Size constraints would also fix this stupid wart in the grammar:

Note that although both the <substring> and <present> productions in the grammar above can produce the "attr=*" construct, this construct is used only to denote a presence filter.

Doing _that_ would require more than size constraints on 'any'. To keep 'substring' from matching "attr=*" you need to make substring something like: substring = attr EQUALS substringpat substringpat = initial ASTERISK [any] [final] / ASTERISK any [final] / ASTERISK final initial = substringvalue any = 1*(substringvalue ASTERISK) final = substringvalue substringvalue = 1*(normal / escaped)

(I.e., break out the three cases of non-empty initial, any, and final.)

...

IMO, the bug is in ldapsearch(1). It should reject a filter with empty substrings.

You mean it's a bug in libldap, as that's what's doing the parsing.

I guess Allan needs to go ahead and change the program that's generating these searches to explicitly test for empty parts and suppress them when building the filter. (He should probably also check that it's doing RFC 2254 encoding of the values that it's inserting in the filters while he's at it. It's no good to catch an empty first-name field if the user can ask for a first-name of "*" and generate "(cn=***lastname*)". Generating filters isn't as trivial as one might wish.)

Philip Guenther

...

...

...

...

"kurt" == Kurt Zeilenga kurt@openldap.org writes:

kurt> IMO, the bug is in ldapsearch(1).

The bug was discovered in a program, not ldapsearch, and wouldn't have been noticed other than it reliably terminating the connection to the server, rather than letting the program continue.

The ldap_search_s() returned LDAP_SUCCESS and the ldap_first_entry was where the assert discovered the closed connection.

So, if this is only a bug in ldapsearch, all the programs that call ldap_search will need to not only check for LDAP_SUCCESS, but also for the LDAP * structure being closed.

On Dec 27, 2007, at 9:16 PM, Howard Chu wrote:

So at some point, this was intended to be legal.

Or I just made the change to allow what RFC 4515 allows, without regard to whether allowing it made any sense.

I think it makes no sense for (foo=**) to be valid when (foo:substringsRule:=\2A\2A), substringsRule being foo's substrings rule, is invalid. It also makes no sense to allow empty any substrings but not allow empty initial and final substrings. A substring is a substring is a substring.

I also note that CN is of syntax directoryString, so (cn=**), if a valid assertion, might not evaluate to true for all (or any) values of CN, as one might expect it would.

-- Kurt