We've finally reached the point in replacing our old authentication system that I'm attempting to get GSSAPI working with our ldap.uvm.edu system.
We have five systems that are behind the LVS (Linux Virtual System) load balancer.
I've got GSSAPI partially working.
As long as I use the real name of the servers, ldapwhoami will return the correct information. However, when I try to use the load balanced name (ldap.uvm.edu), then the ldapwhoami fails with the following:
$ ldapwhoami -H ldap://ldap.uvm.edu SASL/GSSAPI authentication started ldap_sasl_interactive_bind_s: Invalid credentials (49)
and what I find in syslog on the server that got the traffic is:
SASL [conn=864335] Failure: GSSAPI Error: Miscellaneous failure (Wrong principal in request) conn=864335 op=1 RESULT tag=97 err=49 text=SASL(-13): authentication failure: GSSAPI Failure: gss_accept_sec_context
Our DNS is configured so that ldap.uvm.edu is 132.198.101.196, the PTR for that returns vip1.uvm.edu (which also forward resolves to 132.198.101.196).
I have set the KRB5_KTNAME environment variable to /etc/openldap/ldap.keytab, which contains the following keys ldap/<realname>.uvm.edu -- this is the real name of each of the five servers ldap/ldap.uvm.edu -- which I assume is extraneous ldap/vip1.uvm.edu
The /etc/krb5.keytab holds keys for host/<realname>.uvm.edu, host/ldap.uvm.edu, and host/vip1.uvm.edu. Again, I assume that the entry for host/ldap.uvm.edu is extraneous.
As I'm running on Linux, the 132.198.101.196 address is attached to the loopback interface on each of the ldap servers. Slapd is listening on 0.0.0.0:389 and 0.0.0.0:636.
I'm using OpenLDAP 2.3.43 and (Red Hat's) cyrus-sasl-2.1.19-14 package.
Is this a stupid configuration problem that I've somehow missed in the documentation, a bug that Red Hat hasn't back-ported in cyrus-sasl, or something else?
Thanks,
--On Monday, February 02, 2009 3:13 PM -0500 Francis Swasey Frank.Swasey@uvm.edu wrote:
We've finally reached the point in replacing our old authentication system that I'm attempting to get GSSAPI working with our ldap.uvm.edu system.
Good luck. :) The only way I ever got this working was via software load balancing on round-robin DNS, where the virtual name would resolve to the actual host. My guess is that once you're connected to "ldap.uvm.edu" it gets the IP addr of the real system it's connected to, and that doesn't match ldap.uvm.edu, giving you a mismatch.
--Quanah
--
Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration
Today at 4:16pm, Quanah Gibson-Mount wrote:
--On Monday, February 02, 2009 3:13 PM -0500 Francis Swasey Frank.Swasey@uvm.edu wrote:
We've finally reached the point in replacing our old authentication system that I'm attempting to get GSSAPI working with our ldap.uvm.edu system.
Good luck. :) The only way I ever got this working was via software load balancing on round-robin DNS, where the virtual name would resolve to the actual host.
LVS is a software load balancer. What software load balancer did you use that also used DNS round-robin (or am I completely confused in my understanding of what you just wrote)?
My guess is that once you're connected to "ldap.uvm.edu" it gets the IP addr of the real system it's connected to, and that doesn't match ldap.uvm.edu, giving you a mismatch.
Yeah, that's my guess too of the current failure.
On 3 Feb 2009, at 01:31, Frank Swasey wrote:
Yeah, that's my guess too of the current failure.
The problem is that both the client and the server must have a matching idea of the service principal to use in establishing the GSSAPI connection.
The client will use ldap/ldap.uvm.edu, as that's the only name it knows the server by. However, the server will end up using ldap/ hostname() and therefore the two won't match, and you'll get these errors.
There is a work around for this at the GSSAPI layer, which is to tell the server to trust any principal that exists in the service's keytab. Unfortunately, Cyrus SASL doesn't seem to expose a mechanism for doing this, and so the only way to do so is via a code change to the SASL library.
S.
On 2/3/09 7:45 AM, Simon Wilkinson wrote:
On 3 Feb 2009, at 01:31, Frank Swasey wrote:
Yeah, that's my guess too of the current failure.
The problem is that both the client and the server must have a matching idea of the service principal to use in establishing the GSSAPI connection.
The client will use ldap/ldap.uvm.edu, as that's the only name it knows the server by. However, the server will end up using ldap/hostname() and therefore the two won't match, and you'll get these errors.
There is a work around for this at the GSSAPI layer, which is to tell the server to trust any principal that exists in the service's keytab. Unfortunately, Cyrus SASL doesn't seem to expose a mechanism for doing this, and so the only way to do so is via a code change to the SASL library.
Thanks. I have been able to at least get it to work with ldap/ldap.uvm.edu by setting the sasl-host argument in slapd.conf. So, now I have reversed the failure mode. It works with ldaps://ldap.uvm.edu and fails with ldaps://<realname>.uvm.edu. Which is "OK" for my purposes.
I'd really like to be able to have both work, but perhaps cyrus-sasl will change at some point in the future to support the kind of trickery that really happens out here in the world.
On Tue, 3 Feb 2009, Francis Swasey wrote:
failure mode. It works with ldaps://ldap.uvm.edu and fails with ldaps://<realname>.uvm.edu. Which is "OK" for my purposes.
I'd really like to be able to have both work, but perhaps cyrus-sasl will change at some point in the future to support the kind of trickery that really happens out here in the world.
I'm not so sure I agree with this. In theory, when you're doing load balancing through a VIP, all the real servers should appear absolutely identical. You don't *want* the clients to be able to see which real they're getting. If one of them processes requests as rs1 and a different one doesn't...that's a failure of the load balancing configuration, IMO.
With that said, I'll of course note that in practice, theory and practice don't always agree. But it looks like they do in this case, and I can't see what the negative would be.
[maybeHint: Around here, the reals know who they are individually, but slapd is only configured as the VIP and the VIP name is aliased in /etc/hosts. So we can check end-to-end authentication locally on each real, and it shows up on Nagios appropriately, in the event that one of them gets confused. MOST IMPORTANTLY, this is exactly the same as the USER experience, because we're NOT monitoring "ldap://rs1/" -- that's not what users care about!]
On 2/3/09 12:37 PM, Aaron Richton wrote:
On Tue, 3 Feb 2009, Francis Swasey wrote:
failure mode. It works with ldaps://ldap.uvm.edu and fails with ldaps://<realname>.uvm.edu. Which is "OK" for my purposes.
I'd really like to be able to have both work, but perhaps cyrus-sasl will change at some point in the future to support the kind of trickery that really happens out here in the world.
I'm not so sure I agree with this. In theory, when you're doing load balancing through a VIP, all the real servers should appear absolutely identical. You don't *want* the clients to be able to see which real they're getting. If one of them processes requests as rs1 and a different one doesn't...that's a failure of the load balancing configuration, IMO.
With that said, I'll of course note that in practice, theory and practice don't always agree. But it looks like they do in this case, and I can't see what the negative would be.
I want it to work with GSSAPI as the VIP for the users. I'd also like it to work as the real name so I could start using GSSAPI for the replication (currently using SSL protected simple authentication).
[maybeHint: Around here, the reals know who they are individually, but slapd is only configured as the VIP and the VIP name is aliased in /etc/hosts. So we can check end-to-end authentication locally on each real, and it shows up on Nagios appropriately, in the event that one of them gets confused. MOST IMPORTANTLY, this is exactly the same as the USER experience, because we're NOT monitoring "ldap://rs1/" -- that's not what users care about!]
We have configured nagios to monitor both the real names and the VIP. If one of the real names goes away, we care and respond. If the VIP goes away, we REALLY care (cuz email and a bunch of other stuff just tanked too) and respond even if we have to abandon a shopping cart full of groceries in the checkout line...
Frank
--On Monday, February 02, 2009 8:31 PM -0500 Frank Swasey Frank.Swasey@uvm.edu wrote:
Today at 4:16pm, Quanah Gibson-Mount wrote:
--On Monday, February 02, 2009 3:13 PM -0500 Francis Swasey Frank.Swasey@uvm.edu wrote:
We've finally reached the point in replacing our old authentication system that I'm attempting to get GSSAPI working with our ldap.uvm.edu system.
Good luck. :) The only way I ever got this working was via software load balancing on round-robin DNS, where the virtual name would resolve to the actual host.
LVS is a software load balancer. What software load balancer did you use that also used DNS round-robin (or am I completely confused in my understanding of what you just wrote)?
Something custom that stanford wrote. ;)
http://www.eyrie.org/~eagle/software/lbcd/
But what it doesn't do is make use the LB name when it does the actual bind, as it gets translated into the real host.
Fresh ticket cache:
tribes:~> klist Ticket cache: FILE:/tmp/krb5cc_54046 Default principal: quanah@stanford.edu
Valid starting Expires Service principal 02/03/09 08:33:09 02/04/09 09:33:07 krbtgt/stanford.edu@stanford.edu 02/03/09 08:33:09 02/04/09 09:33:07 afs/ir.stanford.edu@stanford.edu
Kerberos 4 ticket cache: /tmp/tkt54046 klist: You have no tickets cached
ldapsearch:
tribes:~> ldapsearch -h ldap1 uid=quanah uid SASL/GSSAPI authentication started SASL username: quanah@stanford.edu SASL SSF: 56 SASL installing layers ...
Now the ticket cache has ldap/ldap1: tribes:~> klist Ticket cache: FILE:/tmp/krb5cc_54046 Default principal: quanah@stanford.edu
Valid starting Expires Service principal 02/03/09 08:33:09 02/04/09 09:33:07 krbtgt/stanford.edu@stanford.edu 02/03/09 08:33:09 02/04/09 09:33:07 afs/ir.stanford.edu@stanford.edu 02/03/09 08:33:59 02/04/09 09:33:07 ldap/ldap1.stanford.edu@stanford.edu
Kerberos 4 ticket cache: /tmp/tkt54046 klist: You have no tickets cached
And here's the actual record for ldap.stanford.edu:
tribes:~> host -t txt ldap ldap.Stanford.EDU CNAME ldap.best.Stanford.EDU ldap.best.Stanford.EDU TXT " 150/1.000 ldap3.stanford.edu" !!! ldap.best.Stanford.EDU TXT record has zero ttl ldap.best.Stanford.EDU TXT " 150/1.000 ldap1.stanford.edu" !!! ldap.best.Stanford.EDU TXT record has zero ttl ldap.best.Stanford.EDU TXT " 150/1.000 ldap4.stanford.edu" !!! ldap.best.Stanford.EDU TXT record has zero ttl ldap.best.Stanford.EDU TXT " 120/1.000 ldap2.stanford.edu" !!! ldap.best.Stanford.EDU TXT record has zero ttl
--Quanah
--
Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration
openldap-software@openldap.org
-
Aaron Richton -
Francis Swasey -
Frank Swasey
-
Quanah Gibson-Mount -
Simon Wilkinson