asserts and manadatory build instructions (was ITS#8240)
by Michael Ströder
hyc(a)symas.com wrote in ITS#8240:
> Our patch response was too hasty. There is no OpenLDAP bug here, the real
> issue is production binaries being built with asserts enabled instead of
> compiling with -DNDEBUG. That's an issue for packagers and distros to resolve.
> Closing this ITS, not an OpenLDAP bug.
Maybe I missed something. But this is the first time I've heard about -DNDEBUG
being mandatory when compiling binary packages for production use. Does it
have other effects?
And what are general rules for assert statements in OpenLDAP code?
In my own (Python) code assert statements are supposed to be only triggered if
something goes wrong *internally* (type issues etc.). If somebody manages to
trigger an assert statement with invalid input from "outside" I always
consider this to be a serious bug revealing insufficient error handling even
though e.g. web2ldap just logs the exception but won't crash. YMMV, but please
clarify.
I also wonder whether there are more mandatory rules for building packages and
where I can find them.
Please don't get me wrong: My inquiry is in good faith to avoid unnecessary
ITS based on misunderstanding.
Ciao, Michael.
1 year, 6 months
Please review 2.5 plan (non-development items)
by Ondřej Kuzník
Hi,
I've prepared a plan what the project wants to achieve as part of the
2.5 stream apart from core OpenLDAP development that I intend to send to
-technical for wider discussion and as a call for participation.
It has been suggested to me that people might want to comment/propose
changes to it so attaching a draft here. Please let me know what you
think or if you agree in broad terms it is fit to be circulated more
widely.
Thanks,
--
Ondřej Kuzník
Senior Software Engineer
Symas Corporation http://www.symas.com
Packaged, certified, and supported LDAP solutions powered by OpenLDAP
3 years, 8 months
Test suite overhaul
by Ondřej Kuzník
This is a follow-up to my email to -technical, this time more into the
future of OpenLDAP's test suite and tools we could build to make
managing OpenLDAP deployments much easier for everyone.
Historically, there has been a decent coverage in the test suite and
that's what's being run before anyone pushes, but it's not been enough
to capture some of the issues. It also takes ages to finish, and is very
limited in what can be tested, mostly because we can only rely on POSIX
shell and command line tools.
That means a lot of boilerplate (and since shell functions are not
available, that boilerplate has to be copy-pasted everywhere) and we can
never actively do two things at the same time, let alone submit several
operations over the same connection. There is no way we can get notified
when things happen (servers have started up, all members think they're
in sync, ...) so we have to rely on polling and manual delays which
slows it down significantly, or introduces false positives on hosts
where these delays are too short. The tests can't be run in parallel
either, not from the same build environment.
None of the above can be resolved without a major overhaul. I've been
thinking about all of that for a while now and want to get started
improving the situation.
If we switched to an actual programming language to write the tests in,
something like Python 3, Golang or Rust, we could build a support
library including a harness to manage slapd through, tools to configure
and interact with the server in a high level way (continue as soon as
the server has started up, or once an MMR environment settles, ...). All
of this should then make it much easier to write new tests and maintain
and understand existing ones.
I have already started the part that people without intimate experience
with OpenLDAP internals would struggle with - a loadable module that
knows how to connect back to the program that started it and informs it
of the port it's listening on and when it has become ready to process
traffic. That code is here:
https://github.com/mistotebe/openldap/tree/mod_harness
To see it in action, just run `nc -l -p <port>`, then start slapd with
`-h ldap://localhost:0` (yes, an explicit zero) and make sure this is in
slapd's configuration:
```
moduleload path/to/mod_harness
backend harness
host localhost
port <your port above>
```
I would appreciate if people joined to help define the testing
infrastructure. My preference is Python 3 as the host language, but
anything that can express concurrency well and has a good LDAP client
implementation should be acceptable if there are enough volunteers. We
will also have to look into defining the protocol the test suite would
use to communicate with the slapd module as the existing plain text
protocol is just a placeholder for easy experimentation.
This would do well for testing in the 2.5 lifecycle. In the future, I
want to extend this to support fine grained testing where the inner
workings of slapd are much more transparent to the test code. Eventually
going as far as making it possible to unit-test any OpenLDAP code. The
first step in that direction is probably going to be learning how to
host a C compiler within the loadable module so we can compile, link and
inject code and callbacks into slapd as and when needed to meld the test
suite with the server much better. Clang (the LLVM C/C++ compiler) seems
to be capable of this.
All of the above will introduce new helper APIs, so in the end it will
overlap with efforts to develop new user-facing tools to configure,
monitor and manage OpenLDAP deployments:
- user friendly tools to manage cn=config
- high-level tools to set up and manage common replication topologies
- tools to monitor replication progress
- many more as we progress with making OpenLDAP user friendly...
(Please don't turn this thread into another flame war on slapd.conf vs.
cn=config, start another one if you really can't resist.)
Again, thanks for being brave and getting this far, let us know what you
think. This is mostly vaporware yet, giving everyone a great opportunity
to shape the project and leave their mark - remember that without your
help, it will take a long while before it's ready!
Regards,
--
Ondřej Kuzník
Senior Software Engineer
Symas Corporation http://www.symas.com
Packaged, certified, and supported LDAP solutions powered by OpenLDAP
3 years, 10 months
libldap cyrus.c and windows (RE24 release candidate)
by Quanah Gibson-Mount
Having an issue with cyrus.c on windows.
I keep hitting:
openldap/libraries/libldap/cyrus.c: In function 'ldap_int_sasl_bind':
openldap/libraries/libldap/cyrus.c:392:19: error: 'HOST_NAME_MAX'
undeclared (first use in this function); did you mean 'FILENAME_MAX'?
char my_hostname[HOST_NAME_MAX + 1];
^~~~~~~~~~~~~
FILENAME_MAX
Yet this code compiles just fine:
$ cat test.c
#include <limits.h>
void main() {
char my_hostname[HOST_NAME_MAX + 1];
}
In cyrus.c, we have:
#ifdef HAVE_CYRUS_SASL
...
#ifdef HAVE_LIMITS_H
#include <limits.h>
#endif
...
in config.log, it has:
#define HAVE_CYRUS_SASL 1
and
#define HAVE_LIMITS_H 1
So the limits.h include file should be included.
Any thoughts?
--Quanah
--
Quanah Gibson-Mount
Product Architect
Symas Corporation
Packaged, certified, and supported LDAP solutions powered by OpenLDAP:
<http://www.symas.com>
3 years, 10 months
Drop support for GNUTLS and libnss in 2.5?
by Michael Ströder
HI!
IMHO OpenLDAP project should drop support for building against GNUTLS
and libnss. Support for these seems to be largely non-existent and it's
a waste of time, especially since there is no build pipeline and no
automatic testing for all the variants.
The support for libnss was done by RedHat for the unified crypto project
which is AFAICS obsolete. Does anybody maintain the stuff?
The support for GNUTLS was requested by Debian folks because of OpenSSL
licensing paranoia. Does anybody maintain the stuff?
The question is whether this is still revelavant with OpenSSL 3.0.0
moving to Apache-2.0 license [1]. [2] says APL-2.0 is not compatible
with GPLv2 though.
Ciao, Michael.
[1] https://www.openssl.org/blog/blog/categories/license/
[2] https://www.gnu.org/licenses/license-list.html#apache2
3 years, 10 months
back_ldap / TLS Issues with OPENLDAP_REL_ENG_2_4_48
by Nikos Voutsinas
Hi all,
In the view of the new openldap release, I ran some tests by using the
current snapshot of the OPENLDAP_REL_ENG_2_4_48 tree and based on my
findings It seems that this build breaks the back_ldap backend when it is
used with a remote ldaps:/// server.
In particular, the following snippet of proxy bind configuration, which
works on the same system, with the same remote ldaps:/// server /
certificate and the 2.4.47 release, fails with the engineering release of
2.4.48. The testing environment was a Debian (Stable/Buster) and Openldap
was compiled with the Debian's gnu TLS libs. Based on my previous
experience I would have bet that this is a GNU TLS issue, however this
seems to be a different case considering that the error happens only with
the switch from the 2.4.47 to 2.4.48. Could this be another side effect of
the related to ITS#8427 fixes?
dn: cn=module{0},cn=config
changetype: modify
add: olcModuleLoad
olcModuleLoad: {2}back_ldap
olcModuleLoad: rwm
dn: olcOverlay={0}rwm,olcDatabase={-1}frontend,cn=config
changetype: add
objectClass: olcOverlayConfig
objectClass: olcRwmConfig
olcOverlay: rwm
olcRwmRewrite: rwm-rewriteEngine "on"
olcRwmRewrite: rwm-rewriteContext "bindDN"
olcRwmRewrite: rwm-rewriteRule "^academicID=([^,]+),ou=People,dc=acme"
"academicID=$1,cn=authn" ":@I"
dn: olcDatabase={3}ldap,cn=config
changetype: add
objectClass: olcDatabaseConfig
objectClass: olcLDAPConfig
olcDatabase: {3}ldap
olcAccess: to * by * manage
olcSuffix: cn=authn
olcRootDN: cn=admin,cn=authn
olcRootPW: {SSHA}<REMOVED>
olcDbURI: ldaps://remote-authn.acme.foo:636
dn: olcOverlay={0}rwm,olcDatabase={3}ldap,cn=config
changetype: add
objectClass: olcOverlayConfig
objectClass: olcRwmConfig
olcOverlay: rwm
olcRwmRewrite: rwm-rewriteEngine "on"
olcRwmRewrite: rwm-rewriteContext "bindDN"
olcRwmRewrite: rwm-rewriteRule "^academicID=([^,]+),cn=authn"
"academicID=$1,ou=People,dc=acme" ":@I"
The debug output shows the following:
5d32a159 <<< dnPrettyNormal:
<academicID=E2Q4KXGLNSPLB25T8TLLT5,ou=People,dc=acme>,
<academicID=e2q4kxglnsplb25t8tllt5,ou=people,dc=acme>
ldap_create
ldap_url_parse_ext(ldaps://remote-authn.acme.foo:636)
5d32a159 =>ldap_back_getconn: conn=1000 op=0: lc=0x7f10ac12abc0 inserted
refcnt=1 rc=0
ldap_sasl_bind
ldap_send_initial_request
ldap_new_connection 1 1 0
ldap_int_open_connection
ldap_connect_to_host: TCP remote-authn.acme.foo:636
ldap_new_socket: 16
ldap_prepare_socket: 16
ldap_connect_to_host: Trying <IP of remote-authn.acme.foo>:636
ldap_pvt_connect: fd: 16 tm: -1 async: 0
attempting to connect:
connect success
tls_write: want=337, written=337
0000: 16 03 01 01 4c 01 00 01 48 03 03 57 00 4d a5 80
....L...H..W.M..
0010: d4 4b 71 8e 08 62 4f 7a b6 a9 4f 20 cd e3 04 9b .Kq..bOz..O
....
0020: 04 91 54 e8 78 9d 20 44 cd bd b3 00 00 3a 13 02 ..T.x.
D.....:..
0030: 13 03 13 01 13 04 c0 2c cc a9 c0 ad c0 0a c0 2b
.......,.......+
....
....
TLS: peer cert untrusted or revoked (0x42)
TLS: can't connect: (unknown error code).
5d32a169 send_ldap_result: conn=1000 op=0 p=3
5d32a169 send_ldap_result: err=52 matched="" text="Proxy operation retry
failed"
5d32a169 send_ldap_response: msgid=1 tag=97 err=52
Best Regards,
Nikos
3 years, 10 months
Issues ISC dhcpd using libldap of OpenLDAP 2.4.48
by Michael Ströder
HI!
I've tried the following without success:
Disabled Link Time Optimization (LTO), recently introduced for openSUSE
Tumbleweed, in packages openldap2 and dhcp-server. Just to make sure it
does not cause any harm.
Upstream update to dhcp 4.4.1 without any of openSUSE's back-port patches.
dhcpd does not even contact the LDAP server. It seems there's some call
into libldap failing and this is handled by dhcpd as connection error.
FWIW: I've attached the strace output of the following command which
does not say much:
LDAPNOINIT=1 /usr/sbin/dhcpd -T
I have no clues where to look now and I have to give up.
Could someone review the source of dhcpd?
The relevant files are ldap*.c:
https://source.isc.org/cgi-bin/gitweb.cgi?p=dhcp.git;a=tree;f=server
Ciao, Michael.
------------------------- dhcpd.conf ------------------------------
ldap-server "127.0.0.1";
ldap-port 389;
ldap-username "cn=host.example.com,ou=DHCP,example,dc=com";
ldap-password "host-secret";
ldap-base-dn "ou=DHCP,example,dc=com";
ldap-ssl off;
ldap-method dynamic;
3 years, 10 months
NO-USER-MODIFICATION for attribute type description memberOf
by Michael Ströder
HI!
Currently attribute type description memberOf does not have
NO-USER-MODIFICATION.
memberof.c contains a commented line:
/* "NO-USER-MODIFICATION " */ /* add? */
But it's my understanding that if the memberof overlay is responsible
maintaining this attribute NO-USER-MODIFICATION should be added.
Any objections against adding it?
Ciao, Michael.
3 years, 10 months
Persistent failures of test050
by Quanah Gibson-Mount
I've noticed that when running test050 in a loop, it often fails, even
after increasing the sleep timeout defaults. Where it fails in the test is
inconsistent and which servers differ is inconsistent as well. I'm
concerned we may have a regression or perhaps long standing issue here that
needs addressing. I'll continue to investigate and see if I can get more
details on what the issue(s) are. In my latest run I see:
.....
Using ldapmodify to add/modify/delete entries from server 1...
iteration 1
iteration 2
iteration 3
iteration 4
iteration 5
iteration 6
iteration 7
iteration 8
iteration 9
iteration 10
Waiting 10 seconds for servers to resync...
Using ldapsearch to read all the entries from server 1...
Using ldapsearch to read all the entries from server 2...
Using ldapsearch to read all the entries from server 3...
Using ldapsearch to read all the entries from server 4...
Comparing retrieved entries from server 1 and server 2...
Comparing retrieved entries from server 1 and server 3...
test failed - server 1 and server 3 databases differ
Failed after 3 of 500 iterations
[build@freebsd12 ~/git/openldap-2-4/tests/testrun]$ diff -u server1.out
server3.out
--- server1.out 2019-06-22 18:23:54.933600000 +0000
+++ server3.out 2019-06-22 18:23:55.049209000 +0000
@@ -1,3 +1,8 @@
+dn: cn=Add-Mod-Del,dc=example,dc=com
+cn: Add-Mod-Del
+objectClass: organizationalRole
+description: guinea pig
+
dn: cn=All Staff,ou=Groups,dc=example,dc=com
member: cn=Manager,dc=example,dc=com
member: cn=Barbara Jensen,ou=Information Technology
Division,ou=People,dc=exam
--Quanah
--
Quanah Gibson-Mount
Product Architect
Symas Corporation
Packaged, certified, and supported LDAP solutions powered by OpenLDAP:
<http://www.symas.com>
3 years, 11 months