Hi,
As a user of slapd-ldap I've bumped into few corner cases related to handling retries and timeouts [1][2][3][4]. I think it demonstrates how non-trivial problem proxying really is, even if it might seem quite simple for casual user at first. While working with a patch for [1] I was wondering following:
My use case:
I have many proxies in the network: one per Kubernetes cluster, but large number of clusters in the network. I'd like to reduce the number of long- running connections to centralized server to the absolute minimum. The number of concurrent TCP connections handled by the remote LDAP server is the bottleneck. Optimally, all connections should be dropped as soon as client is done with the LDAP query.
Question:
Would it be possible to disable all (or only some) caching and retry logic and instead have the proxy mirror the behavior of the clients and remote server:
(1) Disconnect the client connection when corresponding remote connection got disconnected
(2) Disconnect the connection to the remote server when the client disconnects from the proxy (or if remote connection is shared between many clients: disconnect when last client disconnects)
In other words, delegate the complications back to the remote server and clients, instead of trying to solve them at the proxy.
Could this simplify the proxy?
What would be the performance implications? In my use case the concurrent TCP connections towards remote server would reduce, but the number of individual connections could increase due to (2).
Best regards, Tero
[1] Idle and connection timeout implementation https://bugs.openldap.org/show_bug.cgi?id=9197%C2%A0 [2] crash if rebinding after retry fails https://bugs.openldap.org/show_bug.cgi?id=9288 [3] retry fails after remote server disconnected https://bugs.openldap.org/show_bug.cgi?id=9400 [4] rebind-as-user credentials lost after retrying remote connection https://bugs.openldap.org/show_bug.cgi?id=9468
Tero Saarni wrote:
Hi,
As a user of slapd-ldap I've bumped into few corner cases related to handling retries and timeouts [1][2][3][4]. I think it demonstrates how non-trivial problem proxying really is, even if it might seem quite simple for casual user at first. While working with a patch for [1] I was wondering following:
My use case:
I have many proxies in the network: one per Kubernetes cluster, but large number of clusters in the network. I'd like to reduce the number of long- running connections to centralized server to the absolute minimum. The number of concurrent TCP connections handled by the remote LDAP server is the bottleneck. Optimally, all connections should be dropped as soon as client is done with the LDAP query.
In any heavily loaded environment you'll find that connection establishment becomes serious overhead in itself. Thus it's better to aim for longer lived connections that get reused as much as possible.
Howard Chu wrote:
In any heavily loaded environment you'll find that connection establishment becomes serious overhead in itself. Thus it's better to aim for longer lived connections that get reused as much as possible.
Surely, in an environment where there is high number of clients per proxy and only few proxies connected to a central server.
I would still like to ask your opinion on the first case, where remote server disconnects, but proxy does not disconnect clients. I would assume this should not be common, but instead present an exceptional case? (server timeouting too long sessions, connectivity issue, server crash...)
Proxy tries its best to create an illusion for clients that the connection to remote server is still up while it is not. When it is re-established, the proxy replays bind on behalf of the client. This obviously must happen with the same credentials as the initial bind from client, in order to present same kind of session with the same client privileges. Option rebind-as-user=true addresses this, but it does not work due to https://bugs.openldap.org/show_bug.cgi?id=9468. I assume this should then be fixed by storing client credentials somewhere else than the structure representing the (lost) remote connection?
I'm puzzled about what should happen in the same scenario but rebind-as-user=false? From security perspective, it can be a plus that proxy does not keep client credentials in memory for extended period. But on the other hand, if the replayed bind then would change into anonymous bind, wouldn't the client likely fail in strange ways as it suddenly became unprivileged to execute operations?
-- Tero
Tero Saarni wrote:
Howard Chu wrote:
In any heavily loaded environment you'll find that connection establishment becomes serious overhead in itself. Thus it's better to aim for longer lived connections that get reused as much as possible.
Surely, in an environment where there is high number of clients per proxy and only few proxies connected to a central server.
I would still like to ask your opinion on the first case, where remote server disconnects, but proxy does not disconnect clients. I would assume this should not be common, but instead present an exceptional case? (server timeouting too long sessions, connectivity issue, server crash...)
Yes, it's assumed that these are rare cases.
Proxy tries its best to create an illusion for clients that the connection to remote server is still up while it is not.
When it is re-established, the proxy replays bind on behalf of the client. This obviously must happen with the same credentials as the initial bind from client, in order to present same kind of session with the same client privileges. Option rebind-as-user=true addresses this, but it does not work due to https://bugs.openldap.org/show_bug.cgi?id=9468. I assume this should then be fixed by storing client credentials somewhere else than the structure representing the (lost) remote connection?
I'm puzzled about what should happen in the same scenario but rebind-as-user=false?
The rebind-as-user option was originally only used when chasing referrals. With the current code, your choices are to use proxyAuthz to assert a user identity on reconnect, or simply fail instead of reconnecting. Personally I'd lean towards the latter. For shared connections retrying is still appropriate, because we only share connections for special cases, like rootdn usage where we already know we can establish the correct credentials.
As for storing the credentials elsewhere - that becomes moot if you choose to just fail the connection. Otherwise, I'd just pull the creds out of the lost connection before freeing it, to save in the newly created connection.
From security perspective, it can be a plus that proxy does not keep client credentials in memory for extended period. But on the other hand, if the replayed bind then would change into anonymous bind, wouldn't the client likely fail in strange ways as it suddenly became unprivileged to execute operations?
-- Tero
Howard Chu wrote:
The rebind-as-user option was originally only used when chasing referrals. With the current code, your choices are to use proxyAuthz to assert a user identity on reconnect, or simply fail instead of reconnecting. Personally I'd lean towards the latter. For shared connections retrying is still appropriate, because we only share connections for special cases, like rootdn usage where we already know we can establish the correct credentials.
Ok, thanks.
I would like to work with this issue (in context of [1]).
To summarize my understanding: when proxy currently would retry non-shared connection, it can fail instead and disconnect the client. Client can then continue by re-establishing connection and binding again.
-- Tero
-
Howard Chu -
Tero Saarni