Hello,
I'm trying to use slapd-meta (openldap version 2.4.28) as a proxy between a web application LDAP authentication request and a MS AD server. The idea being that if we have multiple LDAP authentication clients we can add another target with suffix massage to redirect requests to the appropriate LDAP server. We were forced to use a dedicated proxy because we were told the MS server's firewall requires an actual IP, not a FQDN, and we don't have a stable ip (we're using a load balancer service which only provides a FQDN).
In general, it looked like this solution would work well. My problem is that the connection between the slapd-meta proxy and the MS server does not seem very robust. I would be extremely grateful if someone could help me understand where I might start looking to diagnose what is wrong.
#################### Problem Summary ##################### The problem I'm seeing was reported as an "Operations Error" from the command line. Initially, I did not have idle-timeout set at all, and this error would occur reliably after 15 minutes of inactivity, and was not recoverable (I needed to restart the slapd server to correct the problem). I suspected that the cause was that the connection was remaining idle too long and getting invalidated by the MS LDAP server (which I don't have access to).
To address this problem, I set idle-timeout to 10m (MS AD apparently has a default timeout of 15m). I have set up a cron job to "ping" the MS server every minute in an attempt to keep the connection open (although I'm not sure if it actually uses the same connection, so this may not be doing much). I also set the value of the nretries configuration directive to 20.
These adjustments seemed to help significantly. With constant activity (authentication through the web application), everything appears to work great. However I still occasionally get the "operations error," which causes my web application to reject the initial authentication even with correct credentials. At this point it is most reliably reproduced by inducing more than 15 minutes of inactivity on the slapd-meta proxy, and then attempting a login. After an initial failed login attempt, the problem seems to correct for itself (i.e. a second login attempt through the web application with the same credentials will succeed).
I am at all times able to connect directly to the LDAP server with no trouble. It is only connections through the slapd-meta proxy I have set up that (sometimes) fail. It may also be worth noting that I don't see any of these problems with an openldap test server that I set up. Thanks so much in advance for any help or suggestions.
################################## Problem Details ##################################### Below is an excerpt from my syslog from a failed authentication request initiated by my web application:
-------------------- log output from syslog ------------------------------------------ Aug 28 20:33:26 my-proxy-server slapd[10330]: daemon: epoll: listen=8 active_threads=0 tvp=NULL Aug 28 20:33:26 my-proxy-server slapd[10330]: daemon: epoll: listen=9 active_threads=0 tvp=NULL Aug 28 20:33:26 my-proxy-server slapd[10330]: daemon: epoll: listen=10 active_threads=0 tvp=NULL Aug 28 20:33:26 my-proxy-server slapd[10330]: conn=1094 op=1 do_search Aug 28 20:33:26 my-proxy-server slapd[10330]: >>> dnPrettyNormal: <dc=AAA,dc=YYY,dc=ZZZ> Aug 28 20:33:26 my-proxy-server slapd[10330]: <<< dnPrettyNormal: <dc=AAA,dc=YYY,dc=ZZZ>, <dc=AAA,dc=YYY,dc=ZZZ> Aug 28 20:33:26 my-proxy-server slapd[10330]: SRCH "dc=AAA,dc=YYY,dc=ZZZ" 2 3 Aug 28 20:33:26 my-proxy-server slapd[10330]: 0 0 0 Aug 28 20:33:26 my-proxy-server slapd[10330]: begin get_filter Aug 28 20:33:26 my-proxy-server slapd[10330]: EQUALITY Aug 28 20:33:26 my-proxy-server slapd[10330]: end get_filter 0 Aug 28 20:33:26 my-proxy-server slapd[10330]: filter: (?SAMACCOUNTNAME=xxx) Aug 28 20:33:26 my-proxy-server slapd[10330]: => get_ctrls Aug 28 20:33:26 my-proxy-server slapd[10330]: => get_ctrls: oid="2.16.840.1.113730.3.4.2" (noncritical) Aug 28 20:33:26 my-proxy-server slapd[10330]: <= get_ctrls: n=1 rc=0 err="" Aug 28 20:33:26 my-proxy-server slapd[10330]: attrs: Aug 28 20:33:26 my-proxy-server slapd[10330]: Aug 28 20:33:26 my-proxy-server slapd[10330]: conn=1094 op=1 SRCH base="dc=AAA,dc=YYY,dc=ZZZ" scope=2 deref=3 filter="(?SAMACCOUNTNAME=xxx)" Aug 28 20:33:26 my-proxy-server slapd[10330]: conn=1094 op=1 meta_back_getconn: mc=0x7fafbc004500 conn=ROOTDN expired (tainted). Aug 28 20:33:26 my-proxy-server slapd[10330]: conn=1094 op=1: meta_back_getconn[1] Aug 28 20:33:26 my-proxy-server slapd[10330]: conn=1094 op=1 meta_back_getconn: candidates=1 conn=ROOTDN fetched Aug 28 20:33:26 my-proxy-server slapd[10330]: conn=1094 op=1 >>> meta_back_search_start[1] Aug 28 20:33:26 my-proxy-server slapd[10330]: conn=1094 op=1 >>> meta_search_dobind_init[1] Aug 28 20:33:26 my-proxy-server slapd[10330]: conn=1094 op=1 <<< meta_search_dobind_init[1]=1 Aug 28 20:33:26 my-proxy-server slapd[10330]: ==> rewrite_context_apply [depth=1] string='dc=AAA,dc=YYY,dc=ZZZ' Aug 28 20:33:26 my-proxy-server slapd[10330]: ==> rewrite_rule_apply rule='((.+),)?dc=AAA,[ ]?dc=YYY,[ ]?dc=ZZZ$' string='dc=AAA,dc=YYY,dc=ZZZ' [1 pass(es)] Aug 28 20:33:26 my-proxy-server slapd[10330]: ==> rewrite_context_apply [depth=1] res={0,'ou=Org Unit,dc=AAA,dc=BBB,dc=CCC,dc=DDD'} Aug 28 20:33:26 my-proxy-server slapd[10330]: [rw] searchBase: "dc=AAA,dc=YYY,dc=ZZZ" -> "ou=Org Unit,dc=AAA,dc=BBB,dc=CCC,dc=DDD" Aug 28 20:33:26 my-proxy-server slapd[10330]: ==> rewrite_context_apply [depth=1] string='(SAMACCOUNTNAME=xxx)' Aug 28 20:33:26 my-proxy-server slapd[10330]: ==> rewrite_context_apply [depth=1] res={0,'NULL'} Aug 28 20:33:26 my-proxy-server slapd[10330]: [rw] searchFilter: "(SAMACCOUNTNAME=xxx)" -> "(SAMACCOUNTNAME=xxx)" Aug 28 20:33:26 my-proxy-server slapd[10330]: conn=1094 op=1 meta_back_retry[1]: retrying URI="ldap://ms-ldap-server.AAA.BBB.CCC.DDD" DN="cn=bind-manager,ou=Org Unit,dc=AAA,dc=BBB,dc=CCC,dc=DDD". Aug 28 20:33:26 my-proxy-server slapd[10330]: >>> conn=1094 op=1 meta_back_bind_op_result[1] Aug 28 20:33:26 my-proxy-server slapd[10330]: <<< conn=1094 op=1 meta_back_bind_op_result[1] err=0 Aug 28 20:33:26 my-proxy-server slapd[10330]: conn=1094 op=1 meta_back_retry[1]: meta_back_single_dobind=0 Aug 28 20:33:26 my-proxy-server slapd[10330]: conn=1094 op=1 <<< meta_back_search_start[1]=1 Aug 28 20:33:26 my-proxy-server slapd[10330]: conn=1094 op=1 meta_back_search: ncandidates=1 cnd=" *" Aug 28 20:33:26 my-proxy-server slapd[10330]: conn=1094 op=1 meta_back_search[1] match="" err=1 (Operations error). Aug 28 20:33:26 my-proxy-server slapd[10330]: send_ldap_result: conn=1094 op=1 p=3 Aug 28 20:33:26 my-proxy-server slapd[10330]: send_ldap_result: err=1 matched="" text="" Aug 28 20:33:26 my-proxy-server slapd[10330]: send_ldap_response: msgid=2 tag=101 err=1 Aug 28 20:33:26 my-proxy-server slapd[10330]: conn=1094 op=1 SEARCH RESULT tag=101 err=1 nentries=0 text=
######################### slapd.conf ######################## # This is the main slapd configuration file. See slapd.conf(5) for more # info on the configuration options.
####################################################################### # Global Directives:
# Features to permit #allow bind_v2
# Schema and objectClass definitions include /etc/ldap/schema/core.schema include /etc/ldap/schema/cosine.schema include /etc/ldap/schema/nis.schema include /etc/ldap/schema/inetorgperson.schema
# Where the pid file is put. The init.d script # will not stop the server if you change this. pidfile /var/run/slapd/slapd.pid
# List of arguments that were passed to the server argsfile /var/run/slapd/slapd.args
# Read slapd.conf(5) for possible values # should reduce this once we're live loglevel any
# Where the dynamically loaded modules are stored modulepath /usr/lib/ldap moduleload back_meta.la moduleload back_ldap.la
# The maximum number of entries that is returned for a search operation sizelimit 500
# The tool-threads parameter sets the actual amount of cpu's that is used # for indexing. tool-threads 1
# experiment trying to figure out error below: # ber_get_next on fd 12 failed errno=34 (Numerical result out of range) #sockbuf_max_incoming_auth 5000000000
# Specific Backend Directives for 'other': # Backend specific directives apply to this backend until another # 'backend' directive occurs backend meta
# this is broken - causes slapd to terminate #TLSCipherSuite ALL:!NULL #TLSCipherSuite ECDHE-RSA-AES256-SHA #TLSCipherSuite SECURE256
# TLS Security settings TLSCACertificateFile /etc/ldap/cert/private/ca.my-proxy-server.YYY.ZZZ.pem TLSCertificateFile /etc/ldap/cert/private/ca.my-proxy-server.YYY.ZZZ.pem TLSCertificateKeyFile /etc/ldap/cert/private/private.my-proxy-server.YYY.ZZZ.pem #TLSVerifyClient try # require TLS (secure) connection #security tls=1
######################################################### # CUSTOM META SLAPD #################################################
database meta suffix "dc=YYY,dc=ZZZ" rootdn "cn=XXX,dc=YYY,dc=ZZZ" rootpw *****
# The following parameters were added to trouble-shoot the unstable connection w/ MS LDAP server idle-timeout 10m nretries 20
####################################################################### # Test LDAP server on staging # Each LDAP server will get its own entry like this # Please document each server #######################################################
# The LDAP server URL. Note the dc=test in the suffix, which allows us to uniquely identify the request as # one that should be passed to our test server. We might use one like dc=AAA for an entry that would pass # requests to the AAA LDAP server uri "ldap://test.YYY.ZZZ/dc=test,dc=YYY,dc=ZZZ"
# Tells the client to use a secure connection with the server. tls start
# More connection specifics: specifies the filename/location of the client certificate if the server requires one # we also specify the DN used to bind, and credentials (password) for the binding DN. idassert-bind tls_cacert=/etc/ldap/cert/test.YYY.ZZZ.crt binddn="cn=admin,dc=YYY,dc=ZZZ" credentials="*****" # NB: There is a parameter that needs to "allow" the server to require a client certificate (tls_reqcert=allow), however it doesn't seem to # work. So we were forced to set this in the ldap.conf (client-side) config file. slapd uses this file to set default values for many # parameters, including this one.
# This removes the "dc=test" specifier before delegating the LDAP request to the test server. suffixmassage "dc=test,dc=YYY,dc=ZZZ" "dc=YYY,dc=ZZZ"
########################################################################## # AAA LDAP Server Entry # Here is the AAA-provided "user context": # OU=Org Unit, DC=AAA, DC=BBB, DC=CCC, DC=DDD # ##########################################################
uri "ldap://ms-ldap-server.AAA.BBB.CCC.DDD/dc=AAA,dc=YYY,dc=ZZZ" tls start idassert-bind tls_cacert=/etc/ldap/cert/AAA.BBB.CCC.DDD.crt binddn="cn=bind-manager,ou=Org Unit,dc=AAA,dc=BBB,dc=CCC,dc=DDD" credentials="******" suffixmassage "dc=AAA,dc=YYY,dc=ZZZ" "ou=Org Unit,dc=AAA,dc=BBB,dc=CCC,dc=DDD" default-target