Hi,
I understood from slapd-ldap(5) description of "idle-timeout" that cached connections towards remote LDAP server would be automatically dropped after <time> seconds.
Problem: cached connections that are idle do not get dropped.
Questions:
(1) Is this expected?
(2) Are idle connections kept due to limitation in the implementation: when connection is idle, back-ldap does not have a trigger that could be used to drop idle connections?
Background:
While experimenting with this, it seems that idle timeout is only checked when there is new activity towards the cached connection i.e. connection needs to become active before idle timeout is checked. If the connection just remains idle, nothing will happen.
I'm trying to study the timeout handling in back-ldap code, and I believe I found relevant code at the end of ldap_back_getconn() in bind.c. It will eventually trigger unbind and disconnect, but only when new activity happens after the idle period is reached. I did not find other paths that could trigger unbind of cached connection.
--On Thursday, November 26, 2020 8:48 AM +0000 Tero Saarni tero.saarni@est.tech wrote:
Hi,
I understood from slapd-ldap(5) description of "idle-timeout" that cached connections towards remote LDAP server would be automatically dropped after <time> seconds.
Problem: cached connections that are idle do not get dropped.
General note: This was a bug that has been fixed and will be in the next OpenLDAP release (ITS#9400).
--Quanah
--
Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
--On Monday, November 30, 2020 10:19 AM -0800 Quanah Gibson-Mount quanah@symas.com wrote:
--On Thursday, November 26, 2020 8:48 AM +0000 Tero Saarni tero.saarni@est.tech wrote:
Hi,
I understood from slapd-ldap(5) description of "idle-timeout" that cached connections towards remote LDAP server would be automatically dropped after <time> seconds.
Problem: cached connections that are idle do not get dropped.
General note: This was a bug that has been fixed and will be in the next OpenLDAP release (ITS#9400).
Err, sorry, ITS#9400 was for retry.
This is ITS#9197, still needs fixing (back-ldap likely needs a task to check for idle connections).
--Quanah
--
Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
"Tero Saarni" tero.saarni@est.tech schrieb am 26.11.2020 um 09:48 in
Nachricht 20201126084819.798.54576@hypatia.openldap.org:
Hi,
I understood from slapd-ldap(5) description of "idle-timeout" that cached connections towards remote LDAP server would be automatically dropped after <time> seconds.
Problem: cached connections that are idle do not get dropped.
It may depend on the version, but for us it worked. How did you check that it doesn't work?
Questions:
(1) Is this expected?
(2) Are idle connections kept due to limitation in the implementation: when connection is idle, back-ldap does not have a trigger that could be used to drop idle connections?
Background:
While experimenting with this, it seems that idle timeout is only checked when there is new activity towards the cached connection i.e. connection needs to become active before idle timeout is checked. If the connection just remains idle, nothing will happen.
I'm trying to study the timeout handling in back-ldap code, and I believe I found relevant code at the end of ldap_back_getconn() in bind.c. It will eventually trigger unbind and disconnect, but only when new activity happens after the idle period is reached. I did not find other paths that could trigger unbind of cached connection.
-- Tero
Ulrich Windl wrote:
It may depend on the version, but for us it worked. How did you check that it doesn't work?
I have tested by setting up "idassert-bind" configuration which results in cached connection. I set idle-timeout to 5 (seconds). I did packet capture with Wireshark to see that the connection is still up.
To test I have executed:
1. execute search from client 2. wait for the duration of idle-timeout to expire 3. observe from LDAP protocol capture that the connection is still up
Secondly I tested:
1. execute search from client 2. wait for the duration of idle-timeout to expire 3. execute search from client again 4. observe that unbind and TCP connection tear down happened immediately after step 3
I also tested that search before idle timeout extends the timeout:
1. execute search from client 2. execute search from client again before idle-timeout 3. observe that nothing happens 4. execute search after idle-timeout has passed when measuring the time from step 2. 5. observe that unbind and TCP connection tear down happened immediately after step 4
I also tried to find the function within back-ldap code that compares the idle timeout value. This is the only place in the code I found idle timeout being compared https://git.openldap.org/openldap/openldap/-/blob/master/servers/slapd/back-... and it is in function ldap_back_getconn(). I attached gdb to slapd and breakpoint to the function. Debugger shows that it does not seem to get called unless there is activity from client to proxy i.e. not during connection from proxy to remote LDAP remains idle.
I tested only with recent releases and git master, not with very old versions since they are bit harder to compile with modern distros. But I have compared the code from a random historical release. It seems to be the same as today.
Quanah also replied "back-ldap likely needs a task to check for idle connections" so I'm bit puzzled if this has worked previously. Maybe ldap_back_getconn() can be called in some other scenario also without having traffic from client towards the proxy?
--On Tuesday, December 1, 2020 8:20 AM +0000 Tero Saarni tero.saarni@est.tech wrote:
I tested only with recent releases and git master, not with very old versions since they are bit harder to compile with modern distros. But I have compared the code from a random historical release. It seems to be the same as today.
Quanah also replied "back-ldap likely needs a task to check for idle connections" so I'm bit puzzled if this has worked previously. Maybe ldap_back_getconn() can be called in some other scenario also without having traffic from client towards the proxy?
Howard specifically said the following while I was discussing with him:
----------- The current idletimeout code in there is pretty useless. It checks the timestamp the next time a conn is referenced, so if it's never referenced, the idle timeout never has any effect. If the conn *is* referenced - you should just use the conn, instead of killing it. -----------
So generally, if a load balancer or other traffic shaper is in use that closes connections silently, set a keepalive. Overall the idle timeout has little purpose for back-ldap connections.
Regards, Quanah
--
Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
Quanah Gibson-Mount wrote:
--On Tuesday, December 1, 2020 8:20 AM +0000 Tero Saarni tero.saarni@est.tech wrote:
I tested only with recent releases and git master, not with very old versions since they are bit harder to compile with modern distros. But I have compared the code from a random historical release. It seems to be the same as today.
Quanah also replied "back-ldap likely needs a task to check for idle connections" so I'm bit puzzled if this has worked previously. Maybe ldap_back_getconn() can be called in some other scenario also without having traffic from client towards the proxy?
Howard specifically said the following while I was discussing with him:
The current idletimeout code in there is pretty useless. It checks the timestamp the next time a conn is referenced, so if it's never referenced, the idle timeout never has any effect. If the conn *is* referenced - you should just use the conn, instead of killing it.
So generally, if a load balancer or other traffic shaper is in use that closes connections silently, set a keepalive. Overall the idle timeout has little purpose for back-ldap connections.
Thinking about it some more, there is a valid use case - if you know that a firewall will silently close connections after some time, you can set the idletimeout to a shorter time to prevent it from trying to use a connection that would have died.
Howard Chu wrote:
So generally, if a load balancer or other traffic shaper is in use that closes connections silently, set a keepalive. Overall the idle timeout has little purpose for back-ldap connections.
Thinking about it some more, there is a valid use case - if you know that a firewall will silently close connections after some time, you can set the idletimeout to a shorter time to prevent it from trying to use a connection that would have died.
From what I can see, proxy will still try to use the connection that has died. ldap_back_getconn() just marks the connection for deletion. From the comment in bind.c:
/* let it be used, but taint/delete it so that no-one else can look it up any further */
Since the TCP connection does not exist, the remote server will just respond TCP RST and thanks to retry fix #9400 a new connection will be created immediately and the operation will succeed.
I wonder if anyone is already looking at adding a task to check for idle connections? If not, I could try, though I'm unsure if that would result in anything and I would certainly need some hand-holding :)
-- Tero
Quanah Gibson-Mount quanah@symas.com schrieb am 01.12.2020 um 21:15 in
Nachricht <8A3F8DDDE068E83FD6E7561D@[192.168.1.156]>:
‑‑On Tuesday, December 1, 2020 8:20 AM +0000 Tero Saarni tero.saarni@est.tech wrote:
I tested only with recent releases and git master, not with very old versions since they are bit harder to compile with modern distros. But I have compared the code from a random historical release. It seems to be the same as today.
Quanah also replied "back‑ldap likely needs a task to check for idle connections" so I'm bit puzzled if this has worked previously. Maybe ldap_back_getconn() can be called in some other scenario also without having traffic from client towards the proxy?
Howard specifically said the following while I was discussing with him:
‑‑‑‑‑‑‑‑‑‑‑ The current idletimeout code in there is pretty useless. It checks the timestamp the next time a conn is referenced, so if it's never referenced, the idle timeout never has any effect. If the conn *is* referenced ‑ you should just use the conn, instead of killing it. ‑‑‑‑‑‑‑‑‑‑‑
So generally, if a load balancer or other traffic shaper is in use that closes connections silently, set a keepalive. Overall the idle timeout has
little purpose for back‑ldap connections.
Hi!
Having written an app myself that had the same problem, I just added a timeout thread that watches the time of last activity for each registered connection (which is a thread in my app). If the last activity is too old, the connection is terminated. In OpenLDAP the monitor database shows there is a monitorConnectionActivityTime, so I can imagine this could be fixed ;-)
Regards, Ulrich
Regards, Quanah
‑‑
Quanah Gibson‑Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
openldap-technical@openldap.org
-
Howard Chu -
Quanah Gibson-Mount -
Tero Saarni -
Ulrich Windl