Hi to all,
when I setup the loadbalancer lloadd via slapd.conf everything is working fine. Here my slapd.conf ----------------- TLSCertificateFile /opt/symas/etc/openldap/example-net-cert.pem TLSCertificateKeyFile /opt/symas/etc/openldap/example-net-key.pem TLSCACertificateFile /opt/symas/etc/openldap/cacert.pem
pidfile /var/symas/run/slapd.pid argsfile /var/symas/run/slapd.args
loglevel 256
modulepath /opt/symas/lib/openldap moduleload lloadd.la
backend lload
listen "ldap://:1389 ldaps://:1636"
feature proxyauthz
TLSShareSlapdCTX true
bindconf bindmethod=simple network-timeout=5 binddn=uid=lloadd,ou=users,dc=example,dc=net credentials=geheim tls_cacert="/opt/symas/etc/openldap/cacert.pem" tls_cert="/opt/symas/etc/openldap/example-net-cert.pem" tls_key="/opt/symas/etc/openldap/example-net-key.pem"
tier roundrobin backend-server uri=ldaps://provider01.example.net retry=5000 max-pending-ops=50 conn-max-pending=10 numconns=10 bindconns=5 backend-server uri=ldaps://provider02.example.net retry=5000 max-pending-ops=50 conn-max-pending=10 numconns=10 bindconns=5
database monitor rootdn cn=monitor rootpw geheim
-----------------
As soon as I change to cn=config with the following configuration: ----------------- dn: cn=config objectClass: olcGlobal cn: config olcLogLevel: stats olcPidFile: /var/symas/run/slapd.pid olcArgsFile: /var/symas/run/slapd.args olcToolThreads: 1 olcTLSCACertificateFile: /opt/symas/etc/openldap/cacert.pem olcTLSCertificateFile: /opt/symas/etc/openldap/example-net-cert.pem olcTLSCertificateKeyFile: /opt/symas/etc/openldap/example-net-key.pem
dn: cn=schema,cn=config objectClass: olcSchemaConfig cn: schema
dn: cn=module{0},cn=config objectClass: olcModuleList cn: module{0} olcModulePath: /opt/symas/lib/openldap olcModuleLoad: lloadd.la olcModuleLoad: argon2.la
dn: olcDatabase={-1}frontend,cn=config objectClass: olcDatabaseConfig objectClass: olcFrontendConfig olcDatabase: {-1}frontend olcSizeLimit: 500 olcAccess: {0}to * by dn.exact=gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth manage by * break olcAccess: {1}to dn="" by * read olcAccess: {2}to dn.base="cn=subschema" by * read olcPasswordHash: {ARGON2}
dn: olcDatabase={0}config,cn=config objectClass: olcDatabaseConfig olcDatabase: {0}config olcRootDN: cn=admin,cn=config #olcRootPW: geheim olcRootPW: {ARGON2}$argon2i$v=19$m=4096,t=3,p=1$cXdlcnJ0enV6dWlvMTIz$G/l0lynf7ygdz0tG+E7S1fBibsFs/L80AUSisiGl/v4 olcAccess: {0}to * by dn.exact=gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth manage
dn: olcBackend={0}lload,cn=config objectClass: olcBackendConfig objectClass: olcBkLloadConfig olcBackend: {0}lload olcBkLloadBindconf: bindmethod=simple timeout=0 network-timeout=5 binddn="uid=lloadd,ou=users,dc=example,dc=net" credentials="geheim" keepalive=0:0:0 tcp-user-timeout=0 tls_cert="/opt/symas/etc/openldap/example-net-cert.pem" tls_key="/opt/symas/etc/openldap/example-net-key.pem" tls_cacert="/opt/symas/etc/openldap/cacert.pem" olcBkLloadIOThreads: 1 olcBkLloadListen: ldap://:1389 olcBkLloadListen: ldaps://:1636 olcBkLloadSockbufMaxClient: 16777215 olcBkLloadSockbufMaxUpstream: 16777215 olcBkLloadMaxPDUPerCycle: 10 olcBkLloadIOTimeout: 10000 olcBkLloadFeature: proxyauthz olcBkLloadTLSCRLCheck: none olcBkLloadVerifyClient: never olcBkLloadTLSProtocolMin: 0.0 olcBkLloadTLSShareSlapdCTX: TRUE olcBkLloadClientMaxPending: 0 olcBkLloadWriteCoherence: 0
dn: cn={0}tier 1,olcBackend={0}lload,cn=config objectClass: olcBkLloadTierConfig cn: {0}tier 1 olcBkLloadTierType: roundrobin
dn: cn={0}server 1,cn={0}tier 1,olcBackend={0}lload,cn=config objectClass: olcBkLloadBackendConfig cn: {0}server 1 olcBkLloadBackendUri: ldaps://provider01.example.net olcBkLloadNumconns: 10 olcBkLloadBindconns: 5 olcBkLloadRetry: 5000 olcBkLloadMaxPendingOps: 50 olcBkLloadMaxPendingConns: 10 olcBkLloadStartTLS: critical olcBkLloadWeight: 1
dn: cn={1}server 2,cn={0}tier 1,olcBackend={0}lload,cn=config objectClass: olcBkLloadBackendConfig cn: {1}server 2 olcBkLloadBackendUri: ldaps://provider02.example.net olcBkLloadNumconns: 10 olcBkLloadBindconns: 5 olcBkLloadRetry: 5000 olcBkLloadMaxPendingOps: 50 olcBkLloadMaxPendingConns: 10 olcBkLloadWeight: 1
dn: olcDatabase={1}monitor,cn=config objectClass: olcDatabaseConfig olcDatabase: {1}monitor olcAccess: {0}to dn.subtree="cn=monitor" by dn.exact=cn=admin,cn=config read
----------------- The slapd is stating and with "ss -tlpn" I see port 1636 and 1389 as listen (next to 636 and 389) I git the following errormessage when I try to contect the ldap-server via the loadbalancer.
------------------- ldap_bind: Server is unavailable (52) additional info: no connections available
-------------------
Did I miss sommthing? I also try to translate the working slapd.conf with slaptest, but the result is the same.
Stefan
On Mon, Dec 04, 2023 at 11:40:29AM +0100, Stefan Kania wrote:
Hi to all,
when I setup the loadbalancer lloadd via slapd.conf everything is working fine. Here my slapd.conf [...]
As soon as I change to cn=config with the following configuration: [...]
The slapd is stating and with "ss -tlpn" I see port 1636 and 1389 as listen (next to 636 and 389) I git the following errormessage when I try to contect the ldap-server via the loadbalancer.
ldap_bind: Server is unavailable (52) additional info: no connections available
Did I miss sommthing? I also try to translate the working slapd.conf with slaptest, but the result is the same.
Hi Stefan, the configurations certainly look equivalent, but no connections to provider1/2 are being established ("no connections available" to use), can you see any errors in the logs that would show why that is?
Regards,
Hello Ondrej,
if I get: -------------- root@loadbalancer01:~# ldapsearch -x -D uid=repl-user,ou=users,dc=example,dc=net -W Enter LDAP Password: ldap_bind: Server is unavailable (52) additional info: no connections available --------------
The log on the loadbalancer is showing: ------------- Dez 04 14:19:33 loadbalancer01 slapd[883]: operation_init: received a new operation, bind request with msgid=1 for client connid=1 Dez 04 14:19:33 loadbalancer01 slapd[883]: request_bind: connid=1, msgid=1 no available connection found Dez 04 14:19:33 loadbalancer01 slapd[883]: operation_init: received a new operation, unbind request with msgid=2 for client connid=1 Dez 04 14:19:33 loadbalancer01 slapd[883]: handle_one_request: received unbind, closing client connid=1 -------------
On the ldap-server I see, nothing in the log:
Next thing I did was ldapsearch with "-d 3" and I got: ------------- TLS trace: SSL_connect:SSL negotiation finished successfully TLS trace: SSL_connect:SSL negotiation finished successfully TLS trace: SSL_connect:SSLv3/TLS read server session ticket tls_read: want=5, got=5 0000: 17 03 03 00 3f ....?
tls_read: want=63, got=63 0000: 15 70 78 36 2f bb aa 06 f3 34 d7 dc c7 40 c7 f1 .px6/....4...@.. 0010: a0 74 0c 31 20 5f 50 15 6a e9 33 55 10 8a 6d a1 .t.1 _P.j.3U..m. 0020: 29 ad 3a ba a8 1e d7 e8 72 e1 3d 17 5f c3 fe d0 ).:.....r.=._... 0030: 4a 94 08 e3 b5 cc 56 03 ac a1 f4 76 e9 30 31 J.....V....v.01 ldap_read: want=8, got=8 0000: 30 84 00 00 00 28 02 01 0....(..
ldap_read: want=38, got=38 0000: 01 61 84 00 00 00 1f 0a 01 34 04 00 04 18 6e 6f .a.......4....no 0010: 20 63 6f 6e 6e 65 63 74 69 6f 6e 73 20 61 76 61 connections ava 0020: 69 6c 61 62 6c 65 ilable
ber_get_next: tag 0x30 len 40 contents: ldap_find_request_by_msgid: msgid 1, lr 0x562310953bc0 lr->lr_refcnt = 1 read1msg: ld 0x56231090e7f0 msgid 1 message type bind ber_scanf fmt ({eAA) ber: read1msg: ld 0x56231090e7f0 0 new referrals read1msg: mark request completed, ld 0x56231090e7f0 msgid 1 request done: ld 0x56231090e7f0 msgid 1 res_errno: 52, res_error: <no connections available>, res_matched: <> ldap_return_request: lrx 0x562310953bc0, lr 0x562310953bc0 ldap_return_request: lrx->lr_msgid 1, lrx->lr_refcnt is now 0, lr is still present ldap_free_request (origid 1, msgid 1) ldap_free_request_int: lr 0x562310953bc0 msgid 1 removed ldap_do_free_request: asked to free lr 0x562310953bc0 msgid 1 refcnt 0 ldap_parse_result ber_scanf fmt ({iAA) ber: ber_scanf fmt (}) ber: ldap_msgfree ldap_err2string ldap_bind: Server is unavailable (52) additional info: no connections available ldap_free_connection 1 1 ldap_send_unbind ber_flush2: 7 bytes to sd 3 tls_write: want=29, written=29 0000: 17 03 03 00 18 e8 81 d9 3d 8a 61 51 f0 8d 3d c8 ........=.aQ..=. 0010: 93 9a c7 ef aa 3a 65 15 a5 d7 6f 97 66 .....:e...o.f ldap_write: want=7, written=7 0000: 30 05 02 01 02 42 00 0....B.
tls_write: want=24, written=24 0000: 17 03 03 00 13 92 92 4f 5a b9 79 a9 b3 2b 3e 38 .......OZ.y..+>8 0010: 53 a2 03 7f 8f cf 85 76 S......v
TLS trace: SSL3 alert write:warning:close notify ldap_free_connection: actually freed
-------------
i first tough it could be some TLS problem but as you see TLS is ok.
Now I checked what I see on the both ldap-servers when restarting slapd, and I see nothing. So no connection is established for the proxy authentication on slapd start.
If I switch to slapd.conf it works fine on both ldap-servers. ------------- Dez 04 14:27:20 provider02 slapd[501]: conn=1047 fd=21 ACCEPT from IP=192.168.56.24:59358 (IP=0.0.0.0:636) Dez 04 14:27:20 provider02 slapd[501]: conn=1047 fd=21 TLS established tls_ssf=256 ssf=256 tls_proto=TLSv1.3 tls_cipher=TLS_AES_256_GCM_SHA384 Dez 04 14:27:20 provider02 slapd[501]: conn=1047 op=0 BIND dn="uid=lloadd,ou=users,dc=example,dc=net" method=128 Dez 04 14:27:20 provider02 slapd[501]: conn=1047 op=0 BIND dn="uid=lloadd,ou=users,dc=example,dc=net" mech=SIMPLE bind_ssf=0 ssf=256 ------------- There must be something wrong with the bind configuration.
Stefan
Am 04.12.23 um 11:52 schrieb Ondřej Kuzník:
On Mon, Dec 04, 2023 at 11:40:29AM +0100, Stefan Kania wrote:
Hi to all,
when I setup the loadbalancer lloadd via slapd.conf everything is working fine. Here my slapd.conf [...]
As soon as I change to cn=config with the following configuration: [...]
The slapd is stating and with "ss -tlpn" I see port 1636 and 1389 as listen (next to 636 and 389) I git the following errormessage when I try to contect the ldap-server via the loadbalancer.
ldap_bind: Server is unavailable (52) additional info: no connections available
Did I miss sommthing? I also try to translate the working slapd.conf with slaptest, but the result is the same.
Hi Stefan, the configurations certainly look equivalent, but no connections to provider1/2 are being established ("no connections available" to use), can you see any errors in the logs that would show why that is?
Regards,
Now I did a check with tcpdump. Starting tcpdump on both systems I see, that the tcp connection is established. But now packages send when doing a ldapsearch.
Am 04.12.23 um 11:52 schrieb Ondřej Kuzník:
On Mon, Dec 04, 2023 at 11:40:29AM +0100, Stefan Kania wrote:
Hi to all,
when I setup the loadbalancer lloadd via slapd.conf everything is working fine. Here my slapd.conf [...]
As soon as I change to cn=config with the following configuration: [...]
The slapd is stating and with "ss -tlpn" I see port 1636 and 1389 as listen (next to 636 and 389) I git the following errormessage when I try to contect the ldap-server via the loadbalancer.
ldap_bind: Server is unavailable (52) additional info: no connections available
Did I miss sommthing? I also try to translate the working slapd.conf with slaptest, but the result is the same.
Hi Stefan, the configurations certainly look equivalent, but no connections to provider1/2 are being established ("no connections available" to use), can you see any errors in the logs that would show why that is?
Regards,
On Mon, Dec 04, 2023 at 02:34:57PM +0100, Stefan Kania wrote:
Hello Ondrej,
if I get:
root@loadbalancer01:~# ldapsearch -x -D uid=repl-user,ou=users,dc=example,dc=net -W Enter LDAP Password: ldap_bind: Server is unavailable (52) additional info: no connections available
The log on the loadbalancer is showing:
Dez 04 14:19:33 loadbalancer01 slapd[883]: operation_init: received a new operation, bind request with msgid=1 for client connid=1 Dez 04 14:19:33 loadbalancer01 slapd[883]: request_bind: connid=1, msgid=1 no available connection found Dez 04 14:19:33 loadbalancer01 slapd[883]: operation_init: received a new operation, unbind request with msgid=2 for client connid=1 Dez 04 14:19:33 loadbalancer01 slapd[883]: handle_one_request: received unbind, closing client connid=1
On the ldap-server I see, nothing in the log:
Next thing I did was ldapsearch with "-d 3" and I got: [...]
i first tough it could be some TLS problem but as you see TLS is ok.
Now I checked what I see on the both ldap-servers when restarting slapd, and I see nothing. So no connection is established for the proxy authentication on slapd start.
If I switch to slapd.conf it works fine on both ldap-servers.
Dez 04 14:27:20 provider02 slapd[501]: conn=1047 fd=21 ACCEPT from IP=192.168.56.24:59358 (IP=0.0.0.0:636) Dez 04 14:27:20 provider02 slapd[501]: conn=1047 fd=21 TLS established tls_ssf=256 ssf=256 tls_proto=TLSv1.3 tls_cipher=TLS_AES_256_GCM_SHA384 Dez 04 14:27:20 provider02 slapd[501]: conn=1047 op=0 BIND dn="uid=lloadd,ou=users,dc=example,dc=net" method=128 Dez 04 14:27:20 provider02 slapd[501]: conn=1047 op=0 BIND dn="uid=lloadd,ou=users,dc=example,dc=net" mech=SIMPLE bind_ssf=0 ssf=256
There must be something wrong with the bind configuration.
Hi Stefan, are you trying to use the load balancer before it has had a chance to establish its own connections?
Can you provide logs from before you start interacting with lloadd with your client to see whether it's even had a chance to establish them? Loglevel at least 'conns'. lloadd's connection management is not reactive, it has to establish (some) connections to upstreams before anything can be proxied. Until any useable connections exist, every operation that cannot be processed locally will be rejected.
Thanks,
Am 04.12.23 um 15:00 schrieb Ondřej Kuzník:
Hi Stefan, are you trying to use the load balancer before it has had a chance to establish its own connections?
Hi Ondrej,
setting loglevel to "conns" on both, the ldap-servers and the loadbalancer I'm getting no entry in the log on the two ldap-server but on the loadbalancer I see: --------------- Dez 04 15:33:54 loadbalancer01 slapd[1519]: lload_listener: listen=14, new connection fd=29 Dez 04 15:33:54 loadbalancer01 slapd[1519]: lload_connection_init: connection connid=4 allocated for socket fd=29 peername=IP=192.168.56.24:50978 Dez 04 15:33:54 loadbalancer01 slapd[1519]: client_tls_handshake_cb: connid=4 finished Dez 04 15:33:54 loadbalancer01 slapd[1519]: connection_read_cb: connection connid=4 ready to read Dez 04 15:33:54 loadbalancer01 slapd[1519]: operation_init: received a new operation, bind request with msgid=1 for client connid=4 Dez 04 15:33:54 loadbalancer01 slapd[1519]: request_bind: connid=4, msgid=1 no available connection found Dez 04 15:33:54 loadbalancer01 slapd[1519]: connection_write_cb: considering writing to live connid=4 what=0 Dez 04 15:33:54 loadbalancer01 slapd[1519]: connection_write_cb: have something to write to connection connid=4 Dez 04 15:33:54 loadbalancer01 slapd[1519]: handle_pdus: re-enabled read event on connid=4 Dez 04 15:33:54 loadbalancer01 slapd[1519]: connection_read_cb: suspended read event on connid=4 Dez 04 15:33:54 loadbalancer01 slapd[1519]: connection_read_cb: connection connid=4 ready to read Dez 04 15:33:54 loadbalancer01 slapd[1519]: operation_init: received a new operation, unbind request with msgid=2 for client connid=4 Dez 04 15:33:54 loadbalancer01 slapd[1519]: handle_one_request: received unbind, closing client connid=4 Dez 04 15:33:54 loadbalancer01 slapd[1519]: client_unlink: removing client connid=4 Dez 04 15:33:54 loadbalancer01 slapd[1519]: connection_read_cb: suspended read event on connid=4 Dez 04 15:33:54 loadbalancer01 slapd[1519]: client_destroy: destroying client connid=4
---------------
Doing the same with loadbalancer configured via slapd.conf I see:
----on the ldap-server-------- Dez 04 15:38:47 provider01 slapd[505]: daemon: activity on 1 descriptor Dez 04 15:38:47 provider01 slapd[505]: daemon: activity on: Dez 04 15:38:47 provider01 slapd[505]: 21r Dez 04 15:38:47 provider01 slapd[505]: Dez 04 15:38:47 provider01 slapd[505]: daemon: read active on 21 Dez 04 15:38:47 provider01 slapd[505]: daemon: epoll: listen=7 active_threads=0 tvp=zero Dez 04 15:38:47 provider01 slapd[505]: daemon: epoll: listen=8 active_threads=0 tvp=zero Dez 04 15:38:47 provider01 slapd[505]: daemon: epoll: listen=9 active_threads=0 tvp=zero Dez 04 15:38:47 provider01 slapd[505]: daemon: epoll: listen=10 active_threads=0 tvp=zero Dez 04 15:38:47 provider01 slapd[505]: daemon: epoll: listen=11 active_threads=0 tvp=zero Dez 04 15:38:47 provider01 slapd[505]: conn=1201 op=0 BIND dn="uid=repl-user,ou=users,dc=example,dc=net" method=128 Dez 04 15:38:47 provider01 slapd[505]: daemon: activity on 1 descriptor Dez 04 15:38:47 provider01 slapd[505]: daemon: activity on: Dez 04 15:38:47 provider01 slapd[505]: Dez 04 15:38:47 provider01 slapd[505]: daemon: epoll: listen=7 active_threads=0 tvp=zero Dez 04 15:38:47 provider01 slapd[505]: daemon: epoll: listen=8 active_threads=0 tvp=zero Dez 04 15:38:47 provider01 slapd[505]: daemon: epoll: listen=9 active_threads=0 tvp=zero Dez 04 15:38:47 provider01 slapd[505]: daemon: epoll: listen=10 active_threads=0 tvp=zero Dez 04 15:38:47 provider01 slapd[505]: daemon: epoll: listen=11 active_threads=0 tvp=zero Dez 04 15:38:47 provider01 slapd[505]: conn=1201 op=0 BIND dn="uid=repl-user,ou=users,dc=example,dc=net" mech=SIMPLE bind_ssf=0 ssf=256 Dez 04 15:38:47 provider01 slapd[505]: conn=1201 op=0 RESULT tag=97 err=0 qtime=0.000017 etime=0.013379 text=
------------------------------
and
-----on the loadbalancer ---- Dez 04 15:38:47 loadbalancer01 slapd[1623]: operation_init: received a new operation, bind request with msgid=1 for client connid=30 Dez 04 15:38:47 loadbalancer01 slapd[1623]: handle_bind_response: received response for bind request msgid=1 by client connid=30, result=0 Dez 04 15:38:47 loadbalancer01 slapd[1623]: forward_final_response: connid=4 msgid=1 finishing up with a request for client connid=30 Dez 04 15:38:47 loadbalancer01 slapd[1623]: operation_init: received a new operation, search request with msgid=2 for client connid=30 Dez 04 15:38:47 loadbalancer01 slapd[1623]: forward_final_response: connid=1 msgid=2 finishing up with a request for client connid=30 Dez 04 15:38:47 loadbalancer01 slapd[1623]: operation_init: received a new operation, unbind request with msgid=3 for client connid=30 Dez 04 15:38:47 loadbalancer01 slapd[1623]: handle_one_request: received unbind, closing client connid=30 -----------------------------
On Mon, Dec 04, 2023 at 03:42:34PM +0100, Stefan Kania wrote:
Am 04.12.23 um 15:00 schrieb Ondřej Kuzník:
Hi Stefan, are you trying to use the load balancer before it has had a chance to establish its own connections?
Hi Ondrej,
setting loglevel to "conns" on both, the ldap-servers and the loadbalancer I'm getting no entry in the log on the two ldap-server but on the loadbalancer I see:
Dez 04 15:33:54 loadbalancer01 slapd[1519]: lload_listener: listen=14, new connection fd=29 Dez 04 15:33:54 loadbalancer01 slapd[1519]: lload_connection_init: connection connid=4 allocated for socket fd=29 peername=IP=192.168.56.24:50978 Dez 04 15:33:54 loadbalancer01 slapd[1519]: client_tls_handshake_cb: connid=4 finished Dez 04 15:33:54 loadbalancer01 slapd[1519]: connection_read_cb: connection connid=4 ready to read Dez 04 15:33:54 loadbalancer01 slapd[1519]: operation_init: received a new operation, bind request with msgid=1 for client connid=4 Dez 04 15:33:54 loadbalancer01 slapd[1519]: request_bind: connid=4, msgid=1 no available connection found Dez 04 15:33:54 loadbalancer01 slapd[1519]: connection_write_cb: considering writing to live connid=4 what=0 Dez 04 15:33:54 loadbalancer01 slapd[1519]: connection_write_cb: have something to write to connection connid=4 Dez 04 15:33:54 loadbalancer01 slapd[1519]: handle_pdus: re-enabled read event on connid=4 Dez 04 15:33:54 loadbalancer01 slapd[1519]: connection_read_cb: suspended read event on connid=4 Dez 04 15:33:54 loadbalancer01 slapd[1519]: connection_read_cb: connection connid=4 ready to read Dez 04 15:33:54 loadbalancer01 slapd[1519]: operation_init: received a new operation, unbind request with msgid=2 for client connid=4 Dez 04 15:33:54 loadbalancer01 slapd[1519]: handle_one_request: received unbind, closing client connid=4 Dez 04 15:33:54 loadbalancer01 slapd[1519]: client_unlink: removing client connid=4 Dez 04 15:33:54 loadbalancer01 slapd[1519]: connection_read_cb: suspended read event on connid=4 Dez 04 15:33:54 loadbalancer01 slapd[1519]: client_destroy: destroying client connid=4
Are you saying there are really no messages at all between lloadd startup and your client connecting to it?
I will say it again: lloadd does not open any connections in response to client activity, they are established as part of its operation and until you have at least one, requests will be rejected with 52 Unavailable.
You want to figure out why no connections are being opened (successfully).
Regards,
Am 04.12.23 um 16:19 schrieb Ondřej Kuzník:
I will say it again: lloadd does not open any connections in response to client activity, they are established as part of its operation and until you have at least one, requests will be rejected with 52 Unavailable.
I Know: Starting the loadbalancer service the bind user (binddn=uid=lloadd,ou=users,dc=example,dc=net) will establish the connection to the ldap-server the user who will connect from a client will use these connection to "talk" to the ldap-server with it's own credentials. I can follow this using slapd.conf.
Using cn=config (now with "conns" as loglevel) I see on the loadbalancer: --------------- Dez 04 19:06:52 loadbalancer01 systemd[1]: Starting symas-openldap-server.service - Symas OpenLDAP Server Daemon... Dez 04 19:06:52 loadbalancer01 slapd[2102]: @(#) $OpenLDAP: slapd 2.6.6 (Aug 8 2023 21:23:03) $ openldap Dez 04 19:06:52 loadbalancer01 slapd[2102]: slapd starting Dez 04 19:06:52 loadbalancer01 slapd[2102]: daemon: added 4r listener=(nil) Dez 04 19:06:52 loadbalancer01 slapd[2102]: daemon: added 7r listener=0x562b0de0d550 Dez 04 19:06:52 loadbalancer01 slapd[2102]: daemon: added 8r listener=0x562b0de0d640 Dez 04 19:06:52 loadbalancer01 slapd[2102]: daemon: added 9r listener=0x562b0de0d790 Dez 04 19:06:52 loadbalancer01 slapd[2102]: daemon: added 10r listener=0x562b0de0d970 Dez 04 19:06:52 loadbalancer01 slapd[2102]: daemon: added 11r listener=0x562b0de0da60 Dez 04 19:06:52 loadbalancer01 systemd[1]: Started symas-openldap-server.service - Symas OpenLDAP Server Daemon. Dez 04 19:06:52 loadbalancer01 slapd[2102]: daemon: epoll: listen=7 active_threads=0 tvp=zero Dez 04 19:06:52 loadbalancer01 slapd[2102]: daemon: epoll: listen=8 active_threads=0 tvp=zero Dez 04 19:06:52 loadbalancer01 slapd[2102]: daemon: epoll: listen=9 active_threads=0 tvp=zero Dez 04 19:06:52 loadbalancer01 slapd[2102]: daemon: epoll: listen=10 active_threads=0 tvp=zero Dez 04 19:06:52 loadbalancer01 slapd[2102]: daemon: epoll: listen=11 active_threads=0 tvp=zero Dez 04 19:06:52 loadbalancer01 slapd[2102]: daemon: activity on 1 descriptor Dez 04 19:06:52 loadbalancer01 slapd[2102]: daemon: activity on: Dez 04 19:06:52 loadbalancer01 slapd[2102]: Dez 04 19:06:52 loadbalancer01 slapd[2102]: daemon: epoll: listen=7 active_threads=0 tvp=zero Dez 04 19:06:52 loadbalancer01 slapd[2102]: daemon: epoll: listen=8 active_threads=0 tvp=zero ... --------------- The last messages will repeat until I stop slapd.
On the ldap-server (also "conns" as loglevel) I see nothing at all.
If I look with "ss" on the loadbalancer I see: --------------- root@loadbalancer01:~# ss -tln | tail -n +1 State Recv-Q Send-Q Local Address:Port Peer Address:PortProcess LISTEN 0 1024 0.0.0.0:1389 0.0.0.0:* LISTEN 0 2048 0.0.0.0:389 0.0.0.0:* LISTEN 0 128 0.0.0.0:22 0.0.0.0:* LISTEN 0 1024 0.0.0.0:1636 0.0.0.0:* LISTEN 0 2048 0.0.0.0:636 0.0.0.0:* ---------------
Then I tried to read the config with slapcat and I got: --------------------- root@loadbalancer01:~# slapcat -n0 lload_open_listener: bind(3) failed errno=98 (Address already in use) lload_open_listener: bind(3) failed errno=98 (Address already in use) lload_open_listener: failed on ldap://:1389 olcBkLloadListen: value #0: could not open a listener for ldap://:1389 config error processing olcBackend={0}lload,cn=config: could not open a listener for ldap://:1389 slapcat: bad configuration file! ---------------------
But looking at my configuration I see: --------- olcBkLloadListen: ldap://:1389 olcBkLloadListen: ldaps://:1636 ---------
If I change the order of ldap and ldaps I get the same only with ldaps. If I remove on line I get the message with the remaining protocol.
Btw I get all the same errors when I convert the slapd.conf with slaptest and use the result to start the loadbalancer. So I can't find an error here.
At the moment I don't know where to look next
Stefan
Hi Ondrej,
I restarted with a new test. Now I'm having 2 loadbalancer one is configured via cn=config and one over slapd.conf. Both are configured exactly the same. Same binduser, same ldap-server same everything. For my test I started tcpdump on the loadbalancer and on the two ldap-server.
Starting the loadbalancer which is configured via slapd.conf I can see all the packages on both, the ldap-servers and the loadbalancer.
Doing the same test with the loadbalancer configured via cn=config I see absolutely nothing, no package is send.
When I set the loglevel to any, I can see that slapd is reading the configuration from cn=config, but I can't see any error. Slapd is running but no connection to any of the ldap-server is established.
Next thing I did was starting the slapd over the commandline with strace on both systems strace /opt/symas/lib/slapd -f /opt/symas/etc/openldap/slapd.conf 2>start-mit-strace
and
strace /opt/symas/lib/slapd -F /opt/symas/etc/openldap/slapd.d 2>start-mit-strace
The result for the server with slapd.conf is showning: --------------- connect(10, {sa_family=AF_INET6, sin6_port=htons(1389), sin6_flowinfo=htonl(0), inet_pton(AF_INET6, "::", &sin6_addr), sin6_scope_id=0}, 28) = 0 connect(10, {sa_family=AF_INET, sin_port=htons(1389), sin_addr=inet_addr("0.0.0.0")}, 16) = 0 bind(10, {sa_family=AF_INET, sin_port=htons(1389), sin_addr=inet_addr("0.0.0.0")}, 16) = 0 bind(11, {sa_family=AF_INET6, sin6_port=htons(1389), sin6_flowinfo=htonl(0), inet_pton(AF_INET6, "::", &sin6_addr), sin6_scope_id=0}, 28) = 0 ... connect(12, {sa_family=AF_INET6, sin6_port=htons(1636), sin6_flowinfo=htonl(0), inet_pton(AF_INET6, "::", &sin6_addr), sin6_scope_id=0}, 28) = 0 connect(12, {sa_family=AF_INET, sin_port=htons(1636), sin_addr=inet_addr("0.0.0.0")}, 16) = 0 bind(12, {sa_family=AF_INET, sin_port=htons(1636), sin_addr=inet_addr("0.0.0.0")}, 16) = 0 bind(13, {sa_family=AF_INET6, sin6_port=htons(1636), sin6_flowinfo=htonl(0), inet_pton(AF_INET6, "::", &sin6_addr), sin6_scope_id=0}, 28) = 0 ---------------
The same search in the result on the loadbalancer configured via cn=config is showing nothing.
I don't know where else I can search. It must be possible to configure the loadbalancer via cn=config.
On both loadbalancer "ss -tlpn" is showing the port 389 636 1389 1636 as listing.
Trying to connect with "telnet <IP> 1636" to both, only on the loadbalancer configured via slapd.conf I can see packages arriving in tcpdump.
There is NO firewall at all running on both systems!
Any idea?
Am 04.12.23 um 14:51 schrieb Stefan Kania:
Now I did a check with tcpdump. Starting tcpdump on both systems I see, that the tcp connection is established. But now packages send when doing a ldapsearch.
--On Wednesday, December 6, 2023 8:11 PM +0100 Stefan Kania stefan@kania-online.de wrote:
Hi Ondrej,
I restarted with a new test. Now I'm having 2 loadbalancer one is configured via cn=config and one over slapd.conf. Both are configured exactly the same. Same binduser, same ldap-server same everything. For my test I started tcpdump on the loadbalancer and on the two ldap-server.
Out of curiosity -
If you define a:
database config
section in slapd.conf, and then make it so you can connect to the config db and dump it via ldapsearch, does it match your cn=config database you're working from? or have the same issue if you use that dump as the configuration?
--Quanah
Am 06.12.23 um 22:12 schrieb Quanah Gibson-Mount:
--On Wednesday, December 6, 2023 8:11 PM +0100 Stefan Kania stefan@kania-online.de wrote:
Hi Ondrej,
I restarted with a new test. Now I'm having 2 loadbalancer one is configured via cn=config and one over slapd.conf. Both are configured exactly the same. Same binduser, same ldap-server same everything. For my test I started tcpdump on the loadbalancer and on the two ldap-server.
Out of curiosity -
If you define a:
database config
section in slapd.conf, and then make it so you can connect to the config db and dump it via ldapsearch, does it match your cn=config database you're working from? or have the same issue if you use that dump as the configuration?
I added: ---------- database config rootdn "cn=admin,cn=config" rootpw config ---------- to the slapd.conf. After adding slapd is still working with slapd.conf then I converted the slapd.conf with: slaptest -F /opt/symas/etc/openldap/slapd.d -f /opt/symas/etc/openldap/slapd.conf
without any error. I can start slapd but:
It's exactly the same. The slapd starts, I can see the listening ports with "ss -tlpn" But ldapsearch is still showing: ------------ ldapsearch -x -D uid=repl-user,ou=users,dc=example,dc=net -w geheim ldap_bind: Server is unavailable (52) additional info: no connections available
------------ These are exactly the same messages as before.
Stefan
--Quanah
--On Thursday, December 7, 2023 5:23 PM +0100 Stefan Kania stefan@kania-online.de wrote:
I added:
database config rootdn "cn=admin,cn=config" rootpw config
to the slapd.conf. After adding slapd is still working with slapd.conf then I converted the slapd.conf with: slaptest -F /opt/symas/etc/openldap/slapd.d -f /opt/symas/etc/openldap/slapd.conf
My question was more, once you add the database config block, if you ldapsearch the cn=config database it generates, does it match what you get from slatpest conversion.
--Quanah
Am 07.12.23 um 17:40 schrieb Quanah Gibson-Mount:
My question was more, once you add the database config block, if you ldapsearch the cn=config database it generates, does it match what you get from slatpest conversion.
Now I understand :-). so that's what I did now
adding ---------- database config rootdn "cn=admin,cn=config" rootpw config ----------
to slapd.conf and start slapd with slapd.conf then I did:
ldapsearch -x -D cn=admin,cn=config -w geheim -b cn=config -H ldap://loadbalancer01.example.net -LLL > slapd-conf.ldif
Then:
rm -rf /opt/symas/etc/openldap/slapd.d/* slapadd -n0 -F /opt/symas/etc/openldap/slapd.d/ -l slapd-conf.ldif chown -R openldap: /opt/symas/etc/openldap/slapd.d/
switch to start slapd via cn=config
then: systemctl restart symas-openldap-server.service
Slapd has started. But the behavior is the same. Same error message as before. ----------- ldapsearch -x -D uid=repl-user,ou=users,dc=example,dc=net -w geheim ldap_bind: Server is unavailable (52) additional info: no connections available -----------
Stefan
On Wed, Dec 06, 2023 at 08:11:52PM +0100, Stefan Kania wrote:
Hi Ondrej,
I restarted with a new test. Now I'm having 2 loadbalancer one is configured via cn=config and one over slapd.conf. Both are configured exactly the same. Same binduser, same ldap-server same everything. For my test I started tcpdump on the loadbalancer and on the two ldap-server.
Starting the loadbalancer which is configured via slapd.conf I can see all the packages on both, the ldap-servers and the loadbalancer.
Doing the same test with the loadbalancer configured via cn=config I see absolutely nothing, no package is send.
Hi Stefan, thanks for the info, I believe I've tracked it down and filed ITS#10142, if you're able to test patches, can you give MR!663 a go?
https://git.openldap.org/openldap/openldap/-/merge_requests/663
Thanks,
Am 11.12.23 um 18:10 schrieb Ondřej Kuzník:
On Wed, Dec 06, 2023 at 08:11:52PM +0100, Stefan Kania wrote:
Hi Ondrej,
I restarted with a new test. Now I'm having 2 loadbalancer one is configured via cn=config and one over slapd.conf. Both are configured exactly the same. Same binduser, same ldap-server same everything. For my test I started tcpdump on the loadbalancer and on the two ldap-server.
Starting the loadbalancer which is configured via slapd.conf I can see all the packages on both, the ldap-servers and the loadbalancer.
Doing the same test with the loadbalancer configured via cn=config I see absolutely nothing, no package is send.
Hi Stefan, thanks for the info, I believe I've tracked it down and filed ITS#10142, if you're able to test patches, can you give MR!663 a go?
https://git.openldap.org/openldap/openldap/-/merge_requests/663
Thanks,
Hi Ondrej,
sorry, but I'm not a developer, I just use OpenLDAP. I'm always try to help if I can test something. I took a look at the merge-request but for me it only strange code ;-). I think I have to compile openldap and before that I have to,somehow, get the patch into the source. Right?
For me it's interesting what is wrong and how did you fixed it. If you have new packages (best for me would be Debian) I can test it before the next release.
Stefan
-- Stefan Kania Landweg 13 25693 St. Michaelisdonn
Signieren jeder E-Mail hilft Spam zu reduzieren und schützt Ihre Privatsphäre. Ein kostenfreies Zertifikat erhalten Sie unter https://www.dgn.de/dgncert/index.html
openldap-technical@openldap.org
-
Ondřej Kuzník -
Quanah Gibson-Mount -
Stefan Kania