11:28 p.m.
Hello,
I have a strange behaviour regarding TLS encryption with an LDAP server.
Everything works like a charm for a while, and without any sign, the
server begins to not respond for TLS traffic. As the server is partially
open on internet, I force TLS, so it is very annoying for us.
I change a lot of parameters, I already read several thread about that
(and more specially, the one with exactly the same error message as me,
where it was solved by specifying the same ciphers in slapd.conf and
ldap.conf, but it doesn't work for me ...)
You will find all my parameters below, hope I forget nothing. I can
provide more log files with and without the problem on demand.
The ldap server is used by apache, postfix, saslauthd, pam_ldap,
nss_ldap ...
Thanks in advance if someone can found a solution for me !!!
Best regards
Denis Sacchet
===================
Here are all the information I can give you :
@(#) $OpenLDAP: slapd 2.3.30 (Mar 9 2007 05:43:02) $
on a Debian Etch server, here are the link information for slapd:
linux-gate.so.1 => (0xffffe000)
libldap_r-2.3.so.0 => /usr/lib/libldap_r-2.3.so.0 (0xb7f41000)
liblber-2.3.so.0 => /usr/lib/liblber-2.3.so.0 (0xb7f35000)
libiodbc.so.2 => /usr/lib/libiodbc.so.2 (0xb7eed000)
libslp.so.1 => /usr/lib/libslp.so.1 (0xb7ede000)
libsasl2.so.2 => /usr/lib/libsasl2.so.2 (0xb7ec8000)
libssl.so.0.9.8 => /usr/lib/i686/cmov/libssl.so.0.9.8 (0xb7e89000)
libcrypto.so.0.9.8 => /usr/lib/i686/cmov/libcrypto.so.0.9.8
(0xb7d4f000)
libcrypt.so.1 => /lib/tls/i686/cmov/libcrypt.so.1 (0xb7d21000)
libresolv.so.2 => /lib/tls/i686/cmov/libresolv.so.2 (0xb7d0d000)
libpthread.so.0 => /lib/tls/i686/cmov/libpthread.so.0 (0xb7cfb000)
libltdl.so.3 => /usr/lib/libltdl.so.3 (0xb7cf4000)
libwrap.so.0 => /lib/libwrap.so.0 (0xb7cec000)
libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0xb7bbb000)
libdl.so.2 => /lib/tls/i686/cmov/libdl.so.2 (0xb7bb7000)
libnsl.so.1 => /lib/tls/i686/cmov/libnsl.so.1 (0xb7ba0000)
libz.so.1 => /usr/lib/libz.so.1 (0xb7b8c000)
/lib/ld-linux.so.2 (0xb7f88000)
The same for ldapsearch :
linux-gate.so.1 => (0xffffe000)
libldap-2.3.so.0 => /usr/lib/libldap-2.3.so.0 (0xb7f8d000)
liblber-2.3.so.0 => /usr/lib/liblber-2.3.so.0 (0xb7f81000)
libsasl2.so.2 => /usr/lib/libsasl2.so.2 (0xb7f6a000)
libssl.so.0.9.8 => /usr/lib/i686/cmov/libssl.so.0.9.8 (0xb7f2b000)
libcrypto.so.0.9.8 => /usr/lib/i686/cmov/libcrypto.so.0.9.8
(0xb7df1000)
libcrypt.so.1 => /lib/tls/i686/cmov/libcrypt.so.1 (0xb7dc3000)
libresolv.so.2 => /lib/tls/i686/cmov/libresolv.so.2 (0xb7db0000)
libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0xb7c7f000)
libdl.so.2 => /lib/tls/i686/cmov/libdl.so.2 (0xb7c7a000)
libz.so.1 => /usr/lib/libz.so.1 (0xb7c66000)
/lib/ld-linux.so.2 (0xb7fca000)
A part of my slapd.conf (no acl, no pass :) ) :
include /etc/ldap/schema/core.schema
include /etc/ldap/schema/cosine.schema
include /etc/ldap/schema/rfc2307bis.schema
include /etc/ldap/schema/inetorgperson.schema
include /etc/ldap/schema/mozillaabpersonalpha.schema
include /etc/ldap/schema/evolutionperson.schema
include /etc/ldap/schema/ouba.schema
include /etc/ldap/schema/samba.schema
pidfile /var/run/slapd/slapd.pid
argsfile /var/run/slapd/slapd.args
modulepath /usr/lib/ldap
moduleload back_bdb
moduleload smbk5pwd
backend bdb
checkpoint 512 30
sizelimit 500
tool-threads 1
security ssf=128
disasllow bind_anon
password-hash {SHA}
TLSCACertificateFile /etc/ssl/certs/<hiddendomain>.pem
TLSCertificateFile /etc/ldap/ssl/ldap.<hiddendomain>.com.crt
TLSCertificateKeyFile /etc/ldap/ssl/ldap.<hiddendomain>.com.key
TLSCipherSuite ALL:!ADH:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv3:+EXP
TLSVerifyClient never
TLSCRLCheck none
TLSRandFile /dev/hwrng
loglevel any
#######################################################################
# <hiddendomain>.com database
database bdb
overlay smbk5pwd
suffix "dc=<hiddendomain>,dc=com"
rootdn "cn=Manager,dc=<hiddendomain>,dc=com"
directory "/var/lib/ldap/<hiddendomain>.com"
dbconfig set_cachesize 0 2097152 0
dbconfig set_lk_max_objects 1500
dbconfig set_lk_max_locks 1500
dbconfig set_lk_max_lockers 1500
index objectClass eq uid uidNumber memberUid gidNumber service
lastmod on
replogfile /var/lib/ldap/<hiddendomain>.com/replog
My ldap.conf file :
TLS_CACERT /etc/ssl/certs/<hiddendomain>.pem
TLSCipherSuite ALL:!ADH:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv3:+EXP
BASE dc=<hiddendomain>, dc=com
URI ldap://ldap.<hiddendomain>.com:389
A trace of ldapsearch when there is the problem :
ldapsearch -D "uid=dsacchet,ou=accounts,dc=<hiddendomain>,dc=com" -h
"ldap.<hiddendomain>.com" -ZZ -W -x -d 9 "(objectClass=*)"
ldap_create
ldap_url_parse_ext(ldap://ldap.<hiddendomain>.com)
ldap_extended_operation_s
ldap_extended_operation
ldap_send_initial_request
ldap_new_connection 1 1 0
ldap_int_open_connection
ldap_connect_to_host: TCP ldap.<hiddendomain>.com:389
ldap_new_socket: 3
ldap_prepare_socket: 3
ldap_connect_to_host: Trying 88.191.47.236:389
ldap_connect_timeout: fd: 3 tm: -1 async: 0
ldap_open_defconn: successful
ldap_send_server_request
ber_scanf fmt ({it) ber:
ber_scanf fmt ({) ber:
ber_flush: 31 bytes to sd 3
ldap_result ld 0x8057558 msgid 1
ldap_chkResponseList ld 0x8057558 msgid 1 all 1
ldap_chkResponseList returns ld 0x8057558 NULL
wait4msg ld 0x8057558 msgid 1 (infinite timeout)
wait4msg continue ld 0x8057558 msgid 1 all 1
** ld 0x8057558 Connections:
* host: ldap.<hiddendomain>.com port: 389 (default)
refcnt: 2 status: Connected
last used: Mon Dec 10 08:21:46 2007
** ld 0x8057558 Outstanding Requests:
* msgid 1, origid 1, status InProgress
outstanding referrals 0, parent count 0
** ld 0x8057558 Response Queue:
Empty
ldap_chkResponseList ld 0x8057558 msgid 1 all 1
ldap_chkResponseList returns ld 0x8057558 NULL
ldap_int_select
read1msg: ld 0x8057558 msgid 1 all 1
ber_get_next
ber_get_next: tag 0x30 len 12 contents:
read1msg: ld 0x8057558 msgid 1 message type extended-result
ber_scanf fmt ({eaa) ber:
read1msg: ld 0x8057558 0 new referrals
read1msg: mark request completed, ld 0x8057558 msgid 1
request done: ld 0x8057558 msgid 1
res_errno: 0, res_error: <>, res_matched: <>
ldap_free_request (origid 1, msgid 1)
ldap_free_connection 0 1
ldap_free_connection: refcnt 1
ldap_parse_extended_result
ber_scanf fmt ({eaa) ber:
ldap_parse_result
ber_scanf fmt ({iaa) ber:
ber_scanf fmt (}) ber:
ldap_msgfree
TLS trace: SSL_connect:before/connect initialization
TLS trace: SSL_connect:SSLv2/v3 write client hello A
TLS trace: SSL_connect:SSLv3 read server hello A
TLS certificate verification: depth: 1, err: 0, subject:
/C=FR/ST=Lorraine/L=Nancy/O=<hiddencompany>/OU=<hiddencompany>/CN=<hiddencompany>
Root C.A./emailAddress=it(a)<hiddendomain>.com, issuer:
/C=FR/ST=Lorraine/L=Nancy/O=<hiddencompany>/OU=<hiddencompany>/CN=<hiddencompany>
Root C.A./emailAddress=it(a)<hiddendomain>.com
TLS certificate verification: depth: 0, err: 0, subject:
/C=FR/ST=Lorraine/L=Nancy/O=<hiddencompany>/OU=<hiddencompany>/CN=smtp.<hiddendomain>.com/emailAddress=it(a)<hiddendomain>.com,
issuer:
/C=FR/ST=Lorraine/L=Nancy/O=<hiddencompany>/OU=<hiddencompany>/CN=<hiddencompany>
Root C.A./emailAddress=it(a)<hiddendomain>.com
TLS trace: SSL_connect:SSLv3 read server certificate A
TLS trace: SSL_connect:SSLv3 read server certificate request A
TLS trace: SSL_connect:SSLv3 read server done A
TLS trace: SSL_connect:SSLv3 write client certificate A
TLS trace: SSL_connect:SSLv3 write client key exchange A
TLS trace: SSL_connect:SSLv3 write change cipher spec A
TLS trace: SSL_connect:SSLv3 write finished A
TLS trace: SSL_connect:SSLv3 flush data
TLS trace: SSL3 alert read:fatal:handshake failure
TLS trace: SSL_connect:failed in SSLv3 read finished A
TLS: can't connect.
ldap_perror
ldap_start_tls: Connect error (-11)
additional info: error:14094410:SSL
routines:SSL3_READ_BYTES:sslv3 alert handshake failure
The same just after a fresh restart :
# ldapsearch -D "uid=dsacchet,ou=accounts,dc=<hiddendomain>,dc=com"
-h "ldap.<hiddendomain>.com" -ZZ -W -x -d 9 "(objectClass=*)"
ldap_create
ldap_url_parse_ext(ldap://ldap.<hiddendomain>.com)
ldap_extended_operation_s
ldap_extended_operation
ldap_send_initial_request
ldap_new_connection 1 1 0
ldap_int_open_connection
ldap_connect_to_host: TCP ldap.<hiddendomain>.com:389
ldap_new_socket: 3
ldap_prepare_socket: 3
ldap_connect_to_host: Trying 88.191.47.236:389
ldap_connect_timeout: fd: 3 tm: -1 async: 0
ldap_open_defconn: successful
ldap_send_server_request
ber_scanf fmt ({it) ber:
ber_scanf fmt ({) ber:
ber_flush: 31 bytes to sd 3
ldap_result ld 0x8057558 msgid 1
ldap_chkResponseList ld 0x8057558 msgid 1 all 1
ldap_chkResponseList returns ld 0x8057558 NULL
wait4msg ld 0x8057558 msgid 1 (infinite timeout)
wait4msg continue ld 0x8057558 msgid 1 all 1
** ld 0x8057558 Connections:
* host: ldap.<hiddendomain>.com port: 389 (default)
refcnt: 2 status: Connected
last used: Mon Dec 10 08:22:20 2007
** ld 0x8057558 Outstanding Requests:
* msgid 1, origid 1, status InProgress
outstanding referrals 0, parent count 0
** ld 0x8057558 Response Queue:
Empty
ldap_chkResponseList ld 0x8057558 msgid 1 all 1
ldap_chkResponseList returns ld 0x8057558 NULL
ldap_int_select
read1msg: ld 0x8057558 msgid 1 all 1
ber_get_next
ber_get_next: tag 0x30 len 12 contents:
read1msg: ld 0x8057558 msgid 1 message type extended-result
ber_scanf fmt ({eaa) ber:
read1msg: ld 0x8057558 0 new referrals
read1msg: mark request completed, ld 0x8057558 msgid 1
request done: ld 0x8057558 msgid 1
res_errno: 0, res_error: <>, res_matched: <>
ldap_free_request (origid 1, msgid 1)
ldap_free_connection 0 1
ldap_free_connection: refcnt 1
ldap_parse_extended_result
ber_scanf fmt ({eaa) ber:
ldap_parse_result
ber_scanf fmt ({iaa) ber:
ber_scanf fmt (}) ber:
ldap_msgfree
TLS trace: SSL_connect:before/connect initialization
TLS trace: SSL_connect:SSLv2/v3 write client hello A
TLS trace: SSL_connect:SSLv3 read server hello A
TLS certificate verification: depth: 1, err: 0, subject:
/C=FR/ST=Lorraine/L=Nancy/O=<hiddencompany>/OU=<hiddencompany>/CN=<hiddencompany>
Root C.A./emailAddress=it(a)<hiddendomain>.com, issuer:
/C=FR/ST=Lorraine/L=Nancy/O=<hiddencompany>/OU=<hiddencompany>/CN=<hiddencompany>
Root C.A./emailAddress=it(a)<hiddendomain>.com
TLS certificate verification: depth: 0, err: 0, subject:
/C=FR/ST=Lorraine/L=Nancy/O=<hiddencompany>/OU=<hiddencompany>/CN=smtp.<hiddendomain>.com/emailAddress=it(a)<hiddendomain>.com,
issuer:
/C=FR/ST=Lorraine/L=Nancy/O=<hiddencompany>/OU=<hiddencompany>/CN=<hiddencompany>
Root C.A./emailAddress=it(a)<hiddendomain>.com
TLS trace: SSL_connect:SSLv3 read server certificate A
TLS trace: SSL_connect:SSLv3 read server done A
TLS trace: SSL_connect:SSLv3 write client key exchange A
TLS trace: SSL_connect:SSLv3 write change cipher spec A
TLS trace: SSL_connect:SSLv3 write finished A
TLS trace: SSL_connect:SSLv3 flush data
TLS trace: SSL_connect:SSLv3 read finished A
Enter LDAP Password:
--
Denis Sacchet aka. Ouba ("`-/")_.-'"``-._
. . `; -._ )-;-,_`)
"Computers are like air conditionners (v_,)' _ )`-.\ ``-'
They stop working properly when you _.- _..-_/ / ((.'
open Windows !!!" ((,.-' ((,/
8:37 a.m.
Hello!
On Mon, 2007-12-10 Denis Sacchet wrote:
I have a strange behaviour regarding TLS encryption with an LDAP
server.
Everything works like a charm for a while, and without any sign, the
server begins to not respond for TLS traffic. As the server is partially
open on internet, I force TLS, so it is very annoying for us.
I change a lot of parameters, I already read several thread about that
(and more specially, the one with exactly the same error message as me,
where it was solved by specifying the same ciphers in slapd.conf and
ldap.conf, but it doesn't work for me ...)
You will find all my parameters below, hope I forget nothing. I can
provide more log files with and without the problem on demand.
The ldap server is used by apache, postfix, saslauthd, pam_ldap,
nss_ldap ...
Thanks in advance if someone can found a solution for me !!!
[...]
We are experiencing exactly the same issues without having found any solution
yet. Any suggestion highly appreciated!
Thanks,
Fabian Steiner
10:22 a.m.
--On December 10, 2007 5:37:36 PM +0100 Fabian Steiner
<lists(a)fabis-site.net> wrote:
Hello!
We are experiencing exactly the same issues without having found any
solution yet. Any suggestion highly appreciated!
Hopefully all of the Debian specific problems with TLS will go away once
OpenLDAP 2.4 is released and integrated into Debian, since it has GnuTLS
support. People often run into SSL/TLS issues on Debian because it has a
hacked version of OpenLDAP 2.1 libraries linked against GnuTLS, that causes
serious problems when the ldap libraries linked against OpenSSL are also
loaded.
--Quanah
--
Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra :: the leader in open source messaging and collaboration
1:46 p.m.
Quanah Gibson-Mount :
Hopefully all of the Debian specific problems with TLS will go away once
OpenLDAP 2.4 is released and integrated into Debian, since it has GnuTLS
support. People often run into SSL/TLS issues on Debian because it has
a hacked version of OpenLDAP 2.1 libraries linked against GnuTLS, that
causes serious problems when the ldap libraries linked against OpenSSL
are also loaded.
The mix between openssl and gnutls is my first asumption. I tried some
trick in this sense ... I already have problems with /etc/ssl/certs
directory, when I use CACertDirs with a lot of certificates in the
directory, it was awfully long to get the result, so I switch to
CaCertFile instead ... And the problem is only with pam_ldap and not
with ldapsearch (or the contrary, I don't remember), and the pam_ldap
was linked against gnutls and ldapsearch linked against openssl ...
Do you know when the Openldap 2.4 is planned to be available in Debian
and also simply from openldap ?
Buchan Milne says :
So, how are you sure this is TLS-specific, and not just lack of (say)
BDB backend tuning resulting in deferred binds ?
Could you please explain a little bit more your suspicion ? I don't
really see what you point out !
Howard Chu :
The fact that a reboot is required indicates that any problem is not
in any user-level code. Maybe your /dev/random has run out of entropy,
or some other underlying system resource is gone. Maybe strace would
help here.
For my current problem, I also have doubts about /dev/(u)random and
entropy because it was very long to generate a 2048 bits private keys,
as the hardware is a Via C7, I switch to /dev/hwrng which is a very good
random number generator. And the problem is always here ...
To finish, I put some strace and syslog as requested :
http://www.ouba.org/strace.slapd.no.probleme.tls.200712070838
http://www.ouba.org/strace.slapd.probleme.tls.200712070809
http://www.ouba.org/syslog.slapd.tls.problem.200712070804
Thanks all for your help
Best regards
Denis Sacchet
11:32 p.m.
Dear all,
does anyone have any clue for this issue ?
Many thanks in advance
Best regards
Denis Sacchet
Denis Sacchet wrote:
> Quanah Gibson-Mount :
> Hopefully all of the Debian specific problems with TLS will go away
> once OpenLDAP 2.4 is released and integrated into Debian, since it has
> GnuTLS support. People often run into SSL/TLS issues on Debian
> because it has a hacked version of OpenLDAP 2.1 libraries linked
> against GnuTLS, that causes serious problems when the ldap libraries
> linked against OpenSSL are also loaded.
The mix between openssl and gnutls is my first asumption. I tried some
trick in this sense ... I already have problems with /etc/ssl/certs
directory, when I use CACertDirs with a lot of certificates in the
directory, it was awfully long to get the result, so I switch to
CaCertFile instead ... And the problem is only with pam_ldap and not
with ldapsearch (or the contrary, I don't remember), and the pam_ldap
was linked against gnutls and ldapsearch linked against openssl ...
Do you know when the Openldap 2.4 is planned to be available in Debian
and also simply from openldap ?
> Buchan Milne says :
> So, how are you sure this is TLS-specific, and not just lack of (say)
> BDB backend tuning resulting in deferred binds ?
Could you please explain a little bit more your suspicion ? I don't
really see what you point out !
> Howard Chu :
> The fact that a reboot is required indicates that any problem is not
> in any user-level code. Maybe your /dev/random has run out of entropy,
> or some other underlying system resource is gone. Maybe strace would
> help here.
For my current problem, I also have doubts about /dev/(u)random and
entropy because it was very long to generate a 2048 bits private keys,
as the hardware is a Via C7, I switch to /dev/hwrng which is a very good
random number generator. And the problem is always here ...
To finish, I put some strace and syslog as requested :
http://www.ouba.org/strace.slapd.no.probleme.tls.200712070838
http://www.ouba.org/strace.slapd.probleme.tls.200712070809
http://www.ouba.org/syslog.slapd.tls.problem.200712070804
Thanks all for your help
Best regards
Denis Sacchet
--
Denis Sacchet aka. Ouba ("`-/")_.-'"``-._
. . `; -._ )-;-,_`)
"Computers are like air conditionners (v_,)' _ )`-.\ ``-'
They stop working properly when you _.- _..-_/ / ((.'
open Windows !!!" ((,.-' ((,/
1:35 p.m.
Denis Sacchet wrote:
Dear all,
does anyone have any clue for this issue ?
Nothing obvious from the info you've posted.
> Do you know when the Openldap 2.4 is planned to be available in
Debian
> and also simply from openldap ?
OpenLDAP 2.4.6 has been available for more than a month. OpenLDAP 2.4.7 was
just released yesterday. You should probably try building 2.4.7 with GNUtls on
your system and see if things are any better.
--
-- Howard Chu
Chief Architect, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
5:22 a.m.
I already have problems with /etc/ssl/certs directory, when I use
CACertDirs
with a lot of certificates in the directory, it was awfully long to get
the result, so I switch to CaCertFile instead ... And the problem is only
with pam_ldap and not with ldapsearch (or the contrary, I don't remember),
and the pam_ldap was linked against gnutls and ldapsearch linked against
openssl ...
We experienced the same problem. The culprit turned out to be libldap2
(opposed to libldap-2.3-0), which has GNUTLS support patched in by Debian. To the
contrary, ldapsearch is linked against libldap-2.3-0, which uses OpenSSL.
More precisely, the problem is get_ca_list() in libraries/libldap/tls.c. If you profile a
test program like
#include <stdlib.h>
#include <dirent.h>
#include <stdio.h>
#include <sys/param.h>
#include <sys/stat.h>
#include <gnutls/gnutls.h>
gnutls_certificate_credentials_t ca_list;
int main(int argc, char *argv[]) {
DIR *d;
struct dirent *dent;
char ca_file[MAXPATHLEN];
struct stat s;
char *dir;
if (argc != 2) exit(1);
dir = argv[1];
d = opendir(dir);
if (d == NULL) exit(1);
gnutls_global_init();
gnutls_certificate_allocate_credentials(&ca_list);
while ((dent = readdir(d)) != NULL) {
snprintf(ca_file, sizeof ca_file, "%s/%s", dir, dent->d_name);
stat(ca_file, &s);
if (!S_ISREG(s.st_mode)) continue;
gnutls_certificate_set_x509_trust_file(ca_list, ca_file, GNUTLS_X509_FMT_PEM);
}
closedir(d);
}
you see unreasonable amounts of time being spent in memory and string manipulation
routines.
Not being an TLS/SSL expert, I'm wondering why you need to add all those certificates
in the first place. I thought the whole point of all those <subject
hash>.<serial> links in /etc/openssl/certs (or whatever) was that a client could
find a CA certificate simply by hashing the subject.
Another pitfall is that libnss-ldap ignores the tls_cacert{file,dir} directives in
libnss-ldap.conf unless "ssl on" is also given. So, if you set the URL to
ldaps://..., you end up in OpenLDAP using SSL, but with the parameters given in ldap.conf,
not those in libnss-ldap.conf. I've reported this to PADL Soft.
3:45 p.m.
Edgar Fuß <ef(a)math.uni-bonn.de> writes:
> I already have problems with /etc/ssl/certs directory, when I use
CACertDirs
> with a lot of certificates in the directory, it was awfully long to get
> the result, so I switch to CaCertFile instead ... And the problem is only
> with pam_ldap and not with ldapsearch (or the contrary, I don't remember),
> and the pam_ldap was linked against gnutls and ldapsearch linked against
> openssl ...
We experienced the same problem. The culprit turned out to be libldap2 (opposed to
libldap-2.3-0), which has GNUTLS support patched in by Debian. To the contrary, ldapsearch
is linked against libldap-2.3-0, which uses OpenSSL.
More precisely, the problem is get_ca_list() in libraries/libldap/tls.c. If you profile a
test program like
#include <stdlib.h>
#include <dirent.h>
#include <stdio.h>
#include <sys/param.h>
#include <sys/stat.h>
#include <gnutls/gnutls.h>
gnutls_certificate_credentials_t ca_list;
int main(int argc, char *argv[]) {
DIR *d;
struct dirent *dent;
char ca_file[MAXPATHLEN];
struct stat s;
char *dir;
if (argc != 2) exit(1);
dir = argv[1];
d = opendir(dir);
if (d == NULL) exit(1);
gnutls_global_init();
gnutls_certificate_allocate_credentials(&ca_list);
while ((dent = readdir(d)) != NULL) {
snprintf(ca_file, sizeof ca_file, "%s/%s", dir, dent->d_name);
stat(ca_file, &s);
if (!S_ISREG(s.st_mode)) continue;
gnutls_certificate_set_x509_trust_file(ca_list, ca_file, GNUTLS_X509_FMT_PEM);
}
closedir(d);
}
you see unreasonable amounts of time being spent in memory and string manipulation
routines.
Edgar, thanks for providing test code. I profiled and improved GnuTLS
to execute your code faster, on my system the time was reduced from 40
seconds to 0.3 seconds. I found debugging and fixing this problem
interesting so I blogged about it:
http://blog.josefsson.org/2008/02/27/real-world-performance-tuning-with-c...
Not being an TLS/SSL expert, I'm wondering why you need to add
all
those certificates in the first place. I thought the whole point of
all those <subject hash>.<serial> links in /etc/openssl/certs (or
whatever) was that a client could find a CA certificate simply by
hashing the subject.
GnuTLS doesn't support hashed certificate directories. Further, TLS
servers need to send a list of names of trusted certificates to clients,
so the server has to open and parse all local trust roots anyway. Right
now, this is done for clients too, since the relevant code in GnuTLS
doesn't know whether it will be used as a client or server. I hope the
new code will be fast enough so that it isn't a bottle-neck. I suppose
that it could be optimized further, so that it isn't done for clients at
all, but let's not optimize prematurely.
Thanks,
/Simon
11:38 p.m.
Simon Josefsson wrote:
Edgar Fuß<ef(a)math.uni-bonn.de> writes:
>> I already have problems with /etc/ssl/certs directory, when I use CACertDirs
>> with a lot of certificates in the directory, it was awfully long to get
>> the result, so I switch to CaCertFile instead ... And the problem is only
>> with pam_ldap and not with ldapsearch (or the contrary, I don't remember),
>> and the pam_ldap was linked against gnutls and ldapsearch linked against
>> openssl ...
> We experienced the same problem. The culprit turned out to be libldap2 (opposed to
libldap-2.3-0), which has GNUTLS support patched in by Debian. To the contrary, ldapsearch
is linked against libldap-2.3-0, which uses OpenSSL.
>
> More precisely, the problem is get_ca_list() in libraries/libldap/tls.c. If you
profile a test program like
>
> #include<stdlib.h>
> #include<dirent.h>
> #include<stdio.h>
> #include<sys/param.h>
> #include<sys/stat.h>
> #include<gnutls/gnutls.h>
> gnutls_certificate_credentials_t ca_list;
>
> int main(int argc, char *argv[]) {
> DIR *d;
> struct dirent *dent;
> char ca_file[MAXPATHLEN];
> struct stat s;
> char *dir;
>
> if (argc != 2) exit(1);
> dir = argv[1];
> d = opendir(dir);
> if (d == NULL) exit(1);
> gnutls_global_init();
> gnutls_certificate_allocate_credentials(&ca_list);
> while ((dent = readdir(d)) != NULL) {
> snprintf(ca_file, sizeof ca_file, "%s/%s", dir, dent->d_name);
> stat(ca_file,&s);
> if (!S_ISREG(s.st_mode)) continue;
> gnutls_certificate_set_x509_trust_file(ca_list, ca_file, GNUTLS_X509_FMT_PEM);
> }
> closedir(d);
> }
>
> you see unreasonable amounts of time being spent in memory and string manipulation
routines.
Edgar, thanks for providing test code. I profiled and improved GnuTLS
to execute your code faster, on my system the time was reduced from 40
seconds to 0.3 seconds. I found debugging and fixing this problem
interesting so I blogged about it:
http://blog.josefsson.org/2008/02/27/real-world-performance-tuning-with-c...
Yes, callgrind is pretty cool, much more useful than old gprof-style
profilers. Though as you note, the runtimes of any valgrind tool can be pretty
extreme. My tweaked version of FunctionCheck is also useful when you can
afford to compile an instrumented version of your code. Faster runtimes in
exchange for extra compile time - frequently it's a worthwhile tradeoff.
http://highlandsun.com/hyc/#fncchk
> Not being an TLS/SSL expert, I'm wondering why you need to
add all
> those certificates in the first place. I thought the whole point of
> all those<subject hash>.<serial> links in /etc/openssl/certs (or
> whatever) was that a client could find a CA certificate simply by
> hashing the subject.
GnuTLS doesn't support hashed certificate directories. Further, TLS
servers need to send a list of names of trusted certificates to clients,
so the server has to open and parse all local trust roots anyway. Right
now, this is done for clients too, since the relevant code in GnuTLS
doesn't know whether it will be used as a client or server. I hope the
new code will be fast enough so that it isn't a bottle-neck. I suppose
that it could be optimized further, so that it isn't done for clients at
all, but let's not optimize prematurely.
You're already approaching GnuTLS version 2.4. If optimizing now is premature,
when will it actually be time to optimize?
--
-- Howard Chu
Chief Architect, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
3:47 a.m.
Howard Chu <hyc(a)symas.com> writes:
Yes, callgrind is pretty cool, much more useful than old gprof-style
profilers. Though as you note, the runtimes of any valgrind tool can
be pretty extreme. My tweaked version of FunctionCheck is also useful
when you can afford to compile an instrumented version of your
code. Faster runtimes in exchange for extra compile time - frequently
it's a worthwhile tradeoff.
http://highlandsun.com/hyc/#fncchk
Interesting, does the code still run and is useful? The latest comment
about it was from 2005. To be able to use it, I need to set up a useful
benchmark environment first though.
>> Not being an TLS/SSL expert, I'm wondering why you need
to add all
>> those certificates in the first place. I thought the whole point of
>> all those<subject hash>.<serial> links in /etc/openssl/certs (or
>> whatever) was that a client could find a CA certificate simply by
>> hashing the subject.
>
> GnuTLS doesn't support hashed certificate directories. Further, TLS
> servers need to send a list of names of trusted certificates to clients,
> so the server has to open and parse all local trust roots anyway. Right
> now, this is done for clients too, since the relevant code in GnuTLS
> doesn't know whether it will be used as a client or server. I hope the
> new code will be fast enough so that it isn't a bottle-neck. I suppose
> that it could be optimized further, so that it isn't done for clients at
> all, but let's not optimize prematurely.
You're already approaching GnuTLS version 2.4. If optimizing now is
premature, when will it actually be time to optimize?
If anyone wants to contribute by profiling/optimizing something in
GnuTLS, doing it now is fine with me.
/Simon
4:18 a.m.
Simon Josefsson wrote:
Howard Chu<hyc(a)symas.com> writes:
> Yes, callgrind is pretty cool, much more useful than old gprof-style
> profilers. Though as you note, the runtimes of any valgrind tool can
> be pretty extreme. My tweaked version of FunctionCheck is also useful
> when you can afford to compile an instrumented version of your
> code. Faster runtimes in exchange for extra compile time - frequently
> it's a worthwhile tradeoff.
>
> http://highlandsun.com/hyc/#fncchk
Interesting, does the code still run and is useful? The latest comment
about it was from 2005. To be able to use it, I need to set up a useful
benchmark environment first though.
Sure, works fine. After I got call arc timings in, it pretty much had
everything I wanted from it so it hasn't changed since. Works on Linux and
Solaris.
--
-- Howard Chu
Chief Architect, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
9:10 a.m.
Denis Sacchet <ouba(a)ouba.org> writes:
Hello,
I have a strange behaviour regarding TLS encryption with an LDAP
server. Everything works like a charm for a while, and without any
sign, the server begins to not respond for TLS traffic. As the server
is partially open on internet, I force TLS, so it is very annoying for
us.
[...]
A trace of ldapsearch when there is the problem :
ldapsearch -D "uid=dsacchet,ou=accounts,dc=<hiddendomain>,dc=com" -h
"ldap.<hiddendomain>.com" -ZZ -W -x -d 9 "(objectClass=*)"
You are connection to host ldap.domain.com
/C=FR/ST=Lorraine/L=Nancy/O=<hiddencompany>/OU=<hiddencompany>/CN=smtp.<hiddendomain>.com/emailAddress=it(a)<hiddendomain>.com,
but the certificate is issued to host smtp.domain.com, a certificate
verification must fail.
-Dieter
--
Dieter Klünter | Systemberatung
http://www.dkluenter.de
GPG Key ID:8EF7B6C6
8:36 a.m.
Dieter Kluenter a écrit :
You are connection to host ldap.domain.com
>
/C=FR/ST=Lorraine/L=Nancy/O=<hiddencompany>/OU=<hiddencompany>/CN=smtp.<hiddendomain>.com/emailAddress=it(a)<hiddendomain>.com,
>
but the certificate is issued to host smtp.domain.com, a certificate
verification must fail.
In fact, the trace doesn't show it, but I have an alternate domain name
in the certificate, and the CN is set to the FQDN of the server (without
that, OpenLDAP doesn't start), so if you look at the successful
connection, the host and certificate are the same, and it works ...
Thanks for your answer ...
Best regards
Denis
10:10 a.m.
Denis Sacchet wrote:
Hello,
I have a strange behaviour regarding TLS encryption with an LDAP server.
Everything works like a charm for a while, and without any sign, the
server begins to not respond for TLS traffic. As the server is partially
open on internet, I force TLS, so it is very annoying for us.
A trace of ldapsearch when there is the problem :
Show the corresponding slapd debug output for the same situation.
--
-- Howard Chu
Chief Architect, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
12:34 p.m.
Howard Chu wrote:
Denis Sacchet wrote:
> Hello,
>
> I have a strange behaviour regarding TLS encryption with an LDAP server.
> Everything works like a charm for a while, and without any sign, the
> server begins to not respond for TLS traffic. As the server is partially
> open on internet, I force TLS, so it is very annoying for us.
>
> A trace of ldapsearch when there is the problem :
Show the corresponding slapd debug output for the same situation.
Of course, I don't want to hijack the OP's thread but as our problems seem to
be rather similar I can also provide the corresponding slapd log:
[...]
slapd starting
>> slap_listener(ldap:///)ldap_pvt_gethostbyname_a: host=uranos,
r=0
connection_get(11): got connid=0
connection_read(11): checking for input on id=0
ber_get_next
ber_get_next: tag 0x30 len 29 contents:
ber_get_next
ber_get_next on fd 11 failed errno=11 (Resource temporarily unavailable)
do_extended
ber_scanf fmt ({m) ber:
send_ldap_extended: err=0 oid= len=0
send_ldap_response: msgid=1 tag=120 err=0
ber_flush: 14 bytes to sd 11
connection_get(11): got connid=0
connection_read(11): checking for input on id=0
TLS trace: SSL_accept:before/accept initialization
TLS trace: SSL_accept:SSLv3 read client hello A
TLS trace: SSL_accept:SSLv3 write server hello A
TLS trace: SSL_accept:SSLv3 write certificate A
TLS trace: SSL_accept:SSLv3 write certificate request A
TLS trace: SSL_accept:SSLv3 flush data
TLS trace: SSL_accept:error in SSLv3 read client certificate A
TLS trace: SSL_accept:error in SSLv3 read client certificate A
connection_get(11): got connid=0
connection_read(11): checking for input on id=0
TLS trace: SSL3 alert write:fatal:handshake failure
TLS trace: SSL_accept:error in SSLv3 read client certificate B
TLS: can't accept.
TLS: error:140890C7:SSL routines:SSL3_GET_CLIENT_CERTIFICATE:peer did not
return a certificate s3_srvr.c:2471
connection_read(11): TLS accept failure error=-1 id=0, closing
[...]
Thanks,
Fabian
12:39 p.m.
Fabian Steiner wrote:
Of course, I don't want to hijack the OP's thread but as our
problems seem to
be rather similar I can also provide the corresponding slapd log:
This looks like a simple configuration error; you have slapd configured to
require client certificates and the client didn't send one. Either you need to
configure the client with a certificate, or you need to relax the requirement
on the server.
[...]
slapd starting
>>> slap_listener(ldap:///)ldap_pvt_gethostbyname_a: host=uranos, r=0
connection_get(11): got connid=0
connection_read(11): checking for input on id=0
ber_get_next
ber_get_next: tag 0x30 len 29 contents:
ber_get_next
ber_get_next on fd 11 failed errno=11 (Resource temporarily unavailable)
do_extended
ber_scanf fmt ({m) ber:
send_ldap_extended: err=0 oid= len=0
send_ldap_response: msgid=1 tag=120 err=0
ber_flush: 14 bytes to sd 11
connection_get(11): got connid=0
connection_read(11): checking for input on id=0
TLS trace: SSL_accept:before/accept initialization
TLS trace: SSL_accept:SSLv3 read client hello A
TLS trace: SSL_accept:SSLv3 write server hello A
TLS trace: SSL_accept:SSLv3 write certificate A
TLS trace: SSL_accept:SSLv3 write certificate request A
TLS trace: SSL_accept:SSLv3 flush data
TLS trace: SSL_accept:error in SSLv3 read client certificate A
TLS trace: SSL_accept:error in SSLv3 read client certificate A
connection_get(11): got connid=0
connection_read(11): checking for input on id=0
TLS trace: SSL3 alert write:fatal:handshake failure
TLS trace: SSL_accept:error in SSLv3 read client certificate B
TLS: can't accept.
TLS: error:140890C7:SSL routines:SSL3_GET_CLIENT_CERTIFICATE:peer did not
return a certificate s3_srvr.c:2471
connection_read(11): TLS accept failure error=-1 id=0, closing
--
-- Howard Chu
Chief Architect, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
12:59 p.m.
Howard Chu wrote:
Fabian Steiner wrote:
> Of course, I don't want to hijack the OP's thread but as our problems
> seem to be rather similar I can also provide the corresponding slapd log:
This looks like a simple configuration error; you have slapd configured to
require client certificates and the client didn't send one. Either you need
to configure the client with a certificate, or you need to relax the
requirement on the server.
[...]
In fact, this was also our first assumption after having analyzed the output
for the very first time but due to our configuration this should't happen:
[...]
TLSCertificateFile /etc/ssl-certs/ldap.crt
TLSCertificateKeyFile /etc/ssl-certs/ldap.key
TLSCACertificateFile /etc/ssl-certs/ca.crt
TLSVerifyClient never
[...]
Moreover, this wouldn't explain why it /does/ work for some time (as far as
our case is concerned it works as long as slapd isn't restarted). Once the
problem has occured the server has to be rebooted in order to ensure a
working setup again :-(
Thanks,
Fabian
1:02 p.m.
Fabian Steiner wrote:
Howard Chu wrote:
> Fabian Steiner wrote:
>> Of course, I don't want to hijack the OP's thread but as our problems
>> seem to be rather similar I can also provide the corresponding slapd log:
> This looks like a simple configuration error; you have slapd configured to
> require client certificates and the client didn't send one. Either you need
> to configure the client with a certificate, or you need to relax the
> requirement on the server.
> [...]
In fact, this was also our first assumption after having analyzed the output
for the very first time but due to our configuration this should't happen:
[...]
TLSCertificateFile /etc/ssl-certs/ldap.crt
TLSCertificateKeyFile /etc/ssl-certs/ldap.key
TLSCACertificateFile /etc/ssl-certs/ca.crt
TLSVerifyClient never
[...]
Moreover, this wouldn't explain why it /does/ work for some time (as far as
our case is concerned it works as long as slapd isn't restarted). Once the
problem has occured the server has to be rebooted in order to ensure a
working setup again :-(
The fact that a reboot is required indicates that any problem is not in any
user-level code. Maybe your /dev/random has run out of entropy, or some other
underlying system resource is gone. Maybe strace would help here.
--
-- Howard Chu
Chief Architect, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
1:46 p.m.
Howard Chu wrote:
The fact that a reboot is required indicates that any problem is not
in any
user-level code. Maybe your /dev/random has run out of entropy, or some
other underlying system resource is gone. Maybe strace would help here.
Thanks for pointing me into the right direction. I can remember having
problems with Apache and mod_ssl in the past as they lacked of entropy with
no keyboard and mouse being attached to the machine. I have just installed
rng-tools and hope that this might solve the issues.
I'll report on it as soon as I can confirm that this was the cause.
Cheers,
Fabian
12:21 p.m.
On Monday 10 December 2007 09:28:38 Denis Sacchet wrote:
Hello,
I have a strange behaviour regarding TLS encryption with an LDAP server.
Everything works like a charm for a while, and without any sign, the
server begins to not respond for TLS traffic. As the server is partially
open on internet, I force TLS, so it is very annoying for us.
security ssf=128
So, how are you sure this is TLS-specific, and not just lack of (say) BDB
backend tuning resulting in deferred binds ?
Regards,
Buchan
5766
days inactive
5845
days old
openldap-software@openldap.org
19 comments
8 participants
participants (8)
-
Buchan Milne -
Denis Sacchet -
Dieter Kluenter -
Edgar Fuß -
Fabian Steiner -
Howard Chu -
Quanah Gibson-Mount -
Simon Josefsson