https://bugs.openldap.org/show_bug.cgi?id=10549
Issue ID: 10549 Summary: lloadd rejects proxied connections: proxyp() cannot read PROXY header on non-blocking socket Product: OpenLDAP Version: 2.6.13 Hardware: x86_64 OS: Linux Status: UNCONFIRMED Keywords: needs_review Severity: normal Priority: --- Component: lloadd Assignee: bugs@openldap.org Reporter: tim.wagner@freenet.de Target Milestone: ---
# Overview
When an lloadd listener is configured with the PROXY protocol enabled, a large fraction of incoming connections are dropped before they are accepted as clients. We're using HAProxy 3.4 (asme with 3.3, 3.2) to distribute the connections across 3 lloadd proxies. The log of lloadd fills with:
``` proxyp(<fd>): header read failed 11 (Resource temporarily unavailable) lload_listener: proxyp(<fd>) failed ```
The root cause is that `proxyp()` is written for a *blocking* socket, but the socket lloadd hands it is *non-blocking*.
`proxyp()` (`servers/slapd/proxyp.c`) reads the PROXY header with a single synchronous `tcp_read()` that expects the whole header at once, and its retry loop only handles `EINTR`:
```c /* servers/slapd/proxyp.c:73 */ do { ret = tcp_read( SLAP_FD2SOCK( sfd ), (char *)&pph, sizeof(pph) ); } while ( ret == -1 && errno == EINTR );
if ( ret == -1 ) { ... "header read failed %d (%s)" ... /* EAGAIN lands here */ return 0; } ```
In lloadd, the accept callback runs on a non-blocking fd. The listener is created via libevent without `LEV_OPT_LEAVE_SOCKETS_BLOCKING`:
```c /* servers/lloadd/daemon.c:945 */ listener = evconnlistener_new( listener_base, lload_listener, ls, LEV_OPT_THREADSAFE|LEV_OPT_DEFERRED_ACCEPT, SLAPD_LISTEN_BACKLOG, ls->ls_sd ); ```
libevent's `evconnlistener` sets every accepted socket non-blocking unless that flag is set. `lload_listener()` then calls `proxyp( s, from, &dummy )` (`daemon.c:788`) synchronously, without ever making the fd blocking. So whenever the full PROXY header is not already buffered in the kernel at the moment the accept callback fires, `tcp_read()` returns -1/`EAGAIN`, `proxyp()` returns 0, and lloadd closes the connection.
slapd is not affected: it `accept()`s the fd itself and calls `proxyp()` while the fd is still blocking (only the *listening* socket is made non-blocking, `servers/slapd/daemon.c:2624`; Linux `accept()` does not inherit `O_NONBLOCK`), so the single-shot reads block until the header arrives.
# Actual Results
Many connections are refused with (we see 7-12/s on every ldap proxy):
``` proxyp(<fd>): header read failed 11 (Resource temporarily unavailable) lload_listener: proxyp(<fd>) failed ```
The PROXY header is never parsed; the connection is closed.
# Expected Results
`proxyp()` should wait for the PROXY header to arrive (up to a timeout) on a non-blocking socket, parse it, and let the connection proceed — matching the behavior lloadd users get from slapd.
# Additional Information (my speculation and assumptions)
1. **Primary trigger.** The specific errno-11 message requires an open, non-blocking fd with *zero* bytes buffered. A peer that completes the TCP handshake but has not yet sent the header produces exactly this. `TCP_DEFER_ACCEPT` (from `LEV_OPT_DEFERRED_ACCEPT`) reduces the race but does not remove it: its retransmit-based timeout delivers a data-less established connection, and health checkers/scanners that connect and stay silent hit the same path. (A peer that connects and immediately sends FIN would instead yield `ret == 0` → the "header read insufficient data" branch, not EAGAIN.)
2. **Latent secondary bug (same root cause).** Even when some bytes are present but not the full header, the non-blocking `tcp_read()` returns a short read, hitting the "header read insufficient data" branch (`proxyp.c:85`). A PROXY header split across TCP segments is therefore also rejected. The same applies to the address (`proxyp.c:129`) and options (`proxyp.c:206`) reads.
3. **Suggested fix direction.** Harden `proxyp()` for non-blocking sockets: on `EAGAIN`, `poll()`/`select()` for readability with a bounded timeout and retry; accumulate the header/address/options across partial reads instead of assuming a single `tcp_read()` delivers each field in full. This fixes both the EAGAIN rejection and the split-header rejection, and keeps proxyp() correct for both slapd (blocking) and lloadd (non-blocking) callers. (An alternative — temporarily setting the fd blocking around the proxyp() call in `lload_listener()` — is simpler but blocks the accept thread on a slow/malicious peer and still needs a timeout.)
If you nee additional information please let me know.
https://bugs.openldap.org/show_bug.cgi?id=10549
--- Comment #1 from Ondřej Kuzník ondra@mistotebe.net --- AFAIK the proxy protocol states that the header must arrive in one block so non-blocking reading exploits that. HAProxy are the people who wrote the spec, so I'm surprised that they are the ones violating it?
https://bugs.openldap.org/show_bug.cgi?id=10549
--- Comment #2 from Tim tim.wagner@freenet.de --- I think it's not a violation of the PROXY protocol happen here (e.g. fragmented header). The PROXY protocol header is received in one chunk right after the 3-wy handshake, but "too late" - the buffer is read slightly before.
From PROXY Protocol specs: ``` The receiver may apply a short timeout and decide to abort the connection if the protocol header is not seen within a few seconds (at least 3 seconds to cover a TCP retransmit). ```
An example out of a tcpdump with relative timings: 11.201180 C->S SYN 11.201208 S->C SYN,ACK 11.201424 C->S ACK <- bare ACK completes handshake, carries NO data 11.201524 S->C FIN <- lloadd closes 100us later, before any data arrives 11.201546 C->S 28 bytes <- PROXY v2 PROXY/TCPv4 header arrives -- too late 11.201581 S->C RST 11.201618 C->S 31 bytes <- LDAP StartTLS ExtendedRequest (1.3.6.1.4.1.1466.20037) 11.201626 S->C RST
I can provide the tcpdump/pcap anonymized if necessary.
https://bugs.openldap.org/show_bug.cgi?id=10549
--- Comment #3 from Ondřej Kuzník ondra@mistotebe.net --- On Mon, Jul 27, 2026 at 09:21:46AM +0000, openldap-its@openldap.org wrote:
I think it's not a violation of the PROXY protocol happen here (e.g. fragmented header). The PROXY protocol header is received in one chunk right after the 3-wy handshake, but "too late" - the buffer is read slightly before.
From PROXY Protocol specs:
The receiver may apply a short timeout and decide to abort the connection if the protocol header is not seen within a few seconds (at least 3 seconds to cover a TCP retransmit).An example out of a tcpdump with relative timings: 11.201180 C->S SYN 11.201208 S->C SYN,ACK 11.201424 C->S ACK <- bare ACK completes handshake, carries NO data 11.201524 S->C FIN <- lloadd closes 100us later, before any data arrives 11.201546 C->S 28 bytes <- PROXY v2 PROXY/TCPv4 header arrives -- too late 11.201581 S->C RST 11.201618 C->S 31 bytes <- LDAP StartTLS ExtendedRequest (1.3.6.1.4.1.1466.20037) 11.201626 S->C RST
Right, on Linux, using TCP_DEFER_ACCEPT for PROXY sockets looks tempting, except there is no universally available equivalent for *BSDs.
We can't block the rest of the balancer's listener for seconds, so the only alternative seems to be splitting the accept/proxyp/setup phases into their own stages. Not ideal but might be necessary.
I would note that running a proxy in front of lloadd doesn't buy you much if anything, lloadd cannot and will not pass this information to its backends (we route requests on a per-operation basis, not per connection as all traditional proxies do). The support exists only because of reports that some clouds (Amazon?) force the use of PROXY protocol.
Regards,
https://bugs.openldap.org/show_bug.cgi?id=10549
Quanah Gibson-Mount quanah@openldap.org changed:
What |Removed |Added ---------------------------------------------------------------------------- Target Milestone|--- |2.7.1 Keywords|needs_review |