7:35 p.m.
I'm currently trying to run "make test" with back-ndb and most of the
comparisons fail even though the backend is behaving correctly.
1) back-ndb always fills in all objectclasses in the objectclass chain,
while the test.ldif files omit most of the superior classes.
2) back-ndb always returns attributes in the order they were defined in
their corresponding objectclass definition, instead of preserving whatever
order they were submitted in.
Any thoughts on how to address this?
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
7:49 p.m.
--On Wednesday, June 04, 2008 7:35 PM -0700 Howard Chu <hyc(a)symas.com>
wrote:
I'm currently trying to run "make test" with back-ndb
and most of the
comparisons fail even though the backend is behaving correctly.
1) back-ndb always fills in all objectclasses in the objectclass
chain, while the test.ldif files omit most of the superior classes.
2) back-ndb always returns attributes in the order they were defined
in their corresponding objectclass definition, instead of preserving
whatever order they were submitted in.
Any thoughts on how to address this?
Hm, for #2, some sort of intelligent sort, since LDAP doesn't support the
concept of ordered results, any test relying on order is fundamentally
flawed. ;) That'd deal with all backends correctly, I would think.
#1 has always somewhat annoyed me, since I'd generally think you would see
all relevant objectClasses included in an objects output. Is there an RFC
covering it either way?
--Quanah
--
Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra :: the leader in open source messaging and collaboration
5:54 a.m.
Quanah Gibson-Mount writes:
Hm, for #2, some sort of intelligent sort, since LDAP doesn't
support the
concept of ordered results, any test relying on order is fundamentally
flawed. ;) That'd deal with all backends correctly, I would think.
ldapsearch -S "" sorts by DN. But such a change cascades into large
changes to the testdata files, and IIRC there are a few tests where
a specific order is expected. Don't quite remember.
So when testing back-ldif I modify scripts/acfilter.sh to sort some LDIF
output files before comparing them. If Perl is present I sort by DN (I
think), and it could sort the lines in each entry too. Probably loses
some possible errors, but not many. E.g. where an LDIF to be compared
consists of the concatenated output of several commands.
Without Perl it just sorts line by line. Better than nothing.
If anyone cares we could write an LDIF-sort program in C.
(I'm about ready to resume my back-ldif changes now that things
seem to pass make test again. Will wait for 2.4.10 to be released to
avoid interfering.)
#1 has always somewhat annoyed me, since I'd generally think you
would
see all relevant objectClasses included in an objects output. Is
there an RFC covering it either way?
Ah, ITS#5517 (add superclass of existing class fails)...
--
Hallvard
7:38 a.m.
Hallvard B Furuseth wrote:
Quanah Gibson-Mount writes:
> Hm, for #2, some sort of intelligent sort, since LDAP doesn't support the
> concept of ordered results, any test relying on order is fundamentally
> flawed. ;) That'd deal with all backends correctly, I would think.
ldapsearch -S "" sorts by DN. But such a change cascades into large
changes to the testdata files, and IIRC there are a few tests where
a specific order is expected. Don't quite remember.
Right, the glue tests expect a particular order. Don't recall if any others do.
So when testing back-ldif I modify scripts/acfilter.sh to sort some
LDIF
output files before comparing them. If Perl is present I sort by DN (I
think), and it could sort the lines in each entry too. Probably loses
some possible errors, but not many. E.g. where an LDIF to be compared
consists of the concatenated output of several commands.
Without Perl it just sorts line by line. Better than nothing.
If anyone cares we could write an LDIF-sort program in C.
I think awk would be a better choice, since we already use it. I'll look into
that.
(I'm about ready to resume my back-ldif changes now that things
seem to pass make test again. Will wait for 2.4.10 to be released to
avoid interfering.)
Cool, would be nice to get the back-ldif ITS's closed.
> #1 has always somewhat annoyed me, since I'd generally think
you would
> see all relevant objectClasses included in an objects output. Is
> there an RFC covering it either way?
Ah, ITS#5517 (add superclass of existing class fails)...
Yeah, looks like another mess. In back-ndb we have to explicitly store all
values, because that's pretty much the only way to be able to find them again
if searching on superclasses etc. (I guess the alternative would be to rewrite
any filters to OR in all superclasses, but that would be more work on the
search side of things, which we obviously want to avoid.)
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
5:20 p.m.
Howard Chu wrote:
Hallvard B Furuseth wrote:
> Quanah Gibson-Mount writes:
>> Hm, for #2, some sort of intelligent sort, since LDAP doesn't support the
>> concept of ordered results, any test relying on order is fundamentally
>> flawed. ;) That'd deal with all backends correctly, I would think.
> ldapsearch -S "" sorts by DN. But such a change cascades into large
> changes to the testdata files, and IIRC there are a few tests where
> a specific order is expected. Don't quite remember.
Right, the glue tests expect a particular order. Don't recall if any others do.
> So when testing back-ldif I modify scripts/acfilter.sh to sort some LDIF
> output files before comparing them. If Perl is present I sort by DN (I
> think), and it could sort the lines in each entry too. Probably loses
> some possible errors, but not many. E.g. where an LDIF to be compared
> consists of the concatenated output of several commands.
> Without Perl it just sorts line by line. Better than nothing.
> If anyone cares we could write an LDIF-sort program in C.
I think awk would be a better choice, since we already use it. I'll look into
that.
This works with gawk:
#! /bin/sh
# $OpenLDAP: pkg/ldap/tests/scripts/acfilter.sh,v 1.13 2008/01/07 23:20:16
kurt Exp $
grep -v '^#' | awk 'BEGIN{FS="\n";RS=""}
{j=0;
for (i=1; i<=NF; i++){
if ($i ~ /^ /){
x[j] = x[j] "\n" $i;
} else {
j++; x[j] = $i
}
}
print x[1];
delete x[1];
j=asort(x);
for (i=1; i<=j; i++){
print x[i];
}
delete x;
print ""
}'
(all as a single line of text)
Unfortunately the asort() function is a GNU extension, so I think we have to
pipe to the sort command instead:
grep -v '^#' | awk 'BEGIN{FS="\n";RS=""}{j=0; for (i=1;
i<=NF; i++){ if ($i ~
/^ /) { x[j] = x[j] "\n" $i; } else { j++; x[j] = $i }} print x[1]; for (i=2;
i<=j; i++){print x[i] | "sort +0";} delete x; close("sort +0");
print ""}'
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
5:25 p.m.
Howard Chu wrote:
This works with gawk:
#! /bin/sh
# $OpenLDAP: pkg/ldap/tests/scripts/acfilter.sh,v 1.13 2008/01/07 23:20:16
kurt Exp $
grep -v '^#' | awk 'BEGIN{FS="\n";RS=""}
{j=0;
for (i=1; i<=NF; i++){
if ($i ~ /^ /){
x[j] = x[j] "\n" $i;
} else {
j++; x[j] = $i
}
}
print x[1];
delete x[1];
j=asort(x);
for (i=1; i<=j; i++){
print x[i];
}
delete x;
print ""
}'
(all as a single line of text)
Unfortunately the asort() function is a GNU extension, so I think we have to
pipe to the sort command instead:
grep -v '^#' | awk 'BEGIN{FS="\n";RS=""}{j=0; for (i=1;
i<=NF; i++){ if ($i ~
/^ /) { x[j] = x[j] "\n" $i; } else { j++; x[j] = $i }} print x[1]; for (i=2;
i<=j; i++){print x[i] | "sort +0";} delete x; close("sort +0");
print ""}'
Oops. Except piping to sort loses the field boundaries, so continuation lines
are messed up.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
12:31 a.m.
Howard Chu wrote:
Howard Chu wrote:
> (all as a single line of text)
>
> Unfortunately the asort() function is a GNU extension, so I think we
> have to
> pipe to the sort command instead:
>
> grep -v '^#' | awk 'BEGIN{FS="\n";RS=""}{j=0; for
(i=1; i<=NF; i++){
> if ($i ~
> /^ /) { x[j] = x[j] "\n" $i; } else { j++; x[j] = $i }} print x[1];
> for (i=2;
> i<=j; i++){print x[i] | "sort +0";} delete x; close("sort
+0"); print
> ""}'
Oops. Except piping to sort loses the field boundaries, so continuation
lines are messed up.
Frankly I ask myself why bothering with such a mess? I know it's hard to
introduce another binary dependency but I'd recommend to use a decent
(readable) scripting language. Because having to maintain code like the
example above will hold people back from writing detailed tests.
Ciao, Michael.
2:21 a.m.
Howard Chu writes:
Oops. Except piping to sort loses the field boundaries, so
continuation lines are messed up.
Also the POSIX folks in their wisdom normalized the arguments to sort.
Thus GNU sort no longer accepts sort +0, you must say sort -k 1.
And of course comparison obeys the current locale, which may do
"interesting" things to the ordering. But then, so does awk. And
several perl versions have had broken locale support. Maybe
defines.sh should hunt down $LC_*, $LANG etc and unset them:-(
Anyway, I don't expect we lose much by dropping proper compares if
perl is absent. So I've been doing something like this.
Though that was for back-ldif where only some test outputs need
to be sorted.
# With arguments -S backend,backend,...backend, also sort the input for
# those backends: By entries+attrs if possible, just by lines otherwise.
# That's a poor man's testdata verification for backends that return
# entries in a different order than the test expects. It allows us
# to at least get some verification.
#
if test "$BACKEND" = null ; then
# Do not output anything to compare for null backend
:
else
case $1,$2, in
-S*,$BACKEND,*)
grep -v '^#' |
if (perl -e 0) >/dev/null 2>&1 ; then
# There is a Perl in $PATH
perl -ne '
if (/^ /) { $entry[-1] .= $_; }
elsif (/^\r?$/) { &entry; }
else { push @entry, $_; }
sub entry {
$dn = shift(@entry);
push @out, join "", ($dn || ""), sort @entry;
@entry = ();
}
END {
&entry;
print join "\n", sort @out;
}'
else
sort
fi
;;
*)
grep -v '^#'
;;
esac
fi
--
Hallvard
5:05 a.m.
Howard Chu wrote:
I'm currently trying to run "make test" with back-ndb
and most of the
comparisons fail even though the backend is behaving correctly.
1) back-ndb always fills in all objectclasses in the objectclass
chain, while the test.ldif files omit most of the superior classes.
2) back-ndb always returns attributes in the order they were defined
in their corresponding objectclass definition, instead of preserving
whatever order they were submitted in.
Any thoughts on how to address this?
Well, my first reaction is that a test suite based on shell-scripts
makes it hard to really be LDAPv3 compliant.
The complete solution is to have schema-aware LDAP clients for testing.
Obviously you need a decent programming language for the tests.
Fine-grained error checking/handling is another issue with the
shell-based approach.
And I wonder whether it's worth to think about speeding up the test
suite by clustering tests so that not every trivial test script has to
bother with slapd configuration and loading test data. I'm aware that
isolating tests from each other has some advantages too though.
Ciao, Michael.
5583
days inactive
5596
days old
8 comments
4 participants
participants (4)
-
Hallvard B Furuseth -
Howard Chu -
Michael Ströder -
Quanah Gibson-Mount