All,
I'm not sure if my questions here are getting through to the group (I don't see a copy to myself)
I've diagnosed this issue. The user has a "£" (UK Pound) in his password. Remove it and everything authenticates OK. Are there settings in Apache/LDAP for controlling what characters can be used as passwords?
Adrian
________________________________
From: Adrian Marsh Sent: 14 November 2008 11:49 To: 'openldap-technical@openldap.org' Subject: Debugging a user authentication
Hi All,
Using Apache 2.2, how do I debug the LDAP lookups being made to a 2003 Domain Controller. Ive one user whos failing to authenticate, but all my other users do and Im trying to see who. He authenticates ok, same password via other mechanisms to the DC, but just not via the Apache LDAP lookup.
I'm an LDAP novice so am looking for names of debug tools/methods etc.
Thanks,
Adrian
On Fri, Nov 14, 2008 at 02:49:16PM -0000, Adrian Marsh wrote:
I've diagnosed this issue. The user has a "£" (UK Pound) in his password. Remove it and everything authenticates OK.
"£" is not a 7-bit character. You therefore run into problems with character sets.
Almost everything in LDAP is defined to use UTF-8, but passwords have always been Octet Strings (1) The effect of this is that passwords can contain absolutely anything and it is the client system's problem to be consistent about the character set. That does not work in the general case, as there are probably lots of client systems for each LDAP server and they could all be using different character sets.
Thus your user's "£" character will translate into a different sequence of bytes if the client is using UTF-8 from the sequence you would see from an ISO-8859-1 client.
The only safe solution for most systems at the moment is to ban non-ASCII (7-bit) characters in passwords :-(
(1): The latest LDAP spec introduced pwprep to solve this problem, but hardly anything implements it yet. It will be many years before you can depend on common LDAP clients doing itproperly.
Andrew
Hi Andrew,
Thanks for that explanation. I'm not 100% sure of which route to take. Now we're aware of the issue we can look for it in future. Our password policy doesn't enforce special characters, but doesn't ban them either, so we may re-think this for the future.
Adrian
-----Original Message----- From: Andrew Findlay [mailto:andrew.findlay@skills-1st.co.uk] Sent: 17 November 2008 20:29 To: Adrian Marsh Cc: openldap-technical@openldap.org Subject: Re: Debugging a user authentication
On Fri, Nov 14, 2008 at 02:49:16PM -0000, Adrian Marsh wrote:
I've diagnosed this issue. The user has a "£" (UK Pound) in his password. Remove it and everything authenticates OK.
"£" is not a 7-bit character. You therefore run into problems with character sets.
Almost everything in LDAP is defined to use UTF-8, but passwords have always been Octet Strings (1) The effect of this is that passwords can contain absolutely anything and it is the client system's problem to be consistent about the character set. That does not work in the general case, as there are probably lots of client systems for each LDAP server and they could all be using different character sets.
Thus your user's "£" character will translate into a different sequence of bytes if the client is using UTF-8 from the sequence you would see from an ISO-8859-1 client.
The only safe solution for most systems at the moment is to ban non-ASCII (7-bit) characters in passwords :-(
(1): The latest LDAP spec introduced pwprep to solve this problem, but hardly anything implements it yet. It will be many years before you can depend on common LDAP clients doing itproperly.
Andrew
Andrew Findlay writes:
(1): The latest LDAP spec introduced pwprep to solve this problem, but hardly anything implements it yet. It will be many years before you can depend on common LDAP clients doing itproperly.
It's not just a client-side issue. Most sites store a password hash in their server rather than the cleartext password. That means the client needs to encode password with the same character encoding and preparation as whatever hashed the server-side password. (E.g. the /etc/passwd program.) Or the server needs to prepare cleartext passwords it receives from the client the same way, but it's likely a bad idea for the server to e.g. assume client passwords are latin-1 and convert to UTF-8.
On Tue, Nov 18, 2008 at 02:57:16PM +0100, Hallvard B Furuseth wrote:
Andrew Findlay writes:
(1): The latest LDAP spec introduced pwprep to solve this problem,
I meant SASLprep - RFC4013
but hardly anything implements it yet. It will be many years before you can depend on common LDAP clients doing itproperly.
It's not just a client-side issue. Most sites store a password hash in their server rather than the cleartext password. That means the client needs to encode password with the same character encoding and preparation as whatever hashed the server-side password. (E.g. the
Ideally password hashing should always be done by the server to avoid risks like that. It is not always possible though :-(
/etc/passwd program.) Or the server needs to prepare cleartext passwords it receives from the client the same way, but it's likely a bad idea for the server to e.g. assume client passwords are latin-1 and convert to UTF-8.
A very bad idea indeed! The server has no way to know what character set the client is using. That is why SASLprep has to be applied at the client end.
Andrew
Hallvard B Furuseth wrote:
Andrew Findlay writes:
(1): The latest LDAP spec introduced pwprep to solve this problem, but hardly anything implements it yet. It will be many years before you can depend on common LDAP clients doing itproperly.
It's not just a client-side issue. Most sites store a password hash in their server rather than the cleartext password. That means the client needs to encode password with the same character encoding and preparation as whatever hashed the server-side password. (E.g. the /etc/passwd program.) Or the server needs to prepare cleartext passwords it receives from the client the same way, but it's likely a bad idea for the server to e.g. assume client passwords are latin-1 and convert to UTF-8.
userPassword is an octetString, therefore if the server does any type of character set conversion on its values it is Broken.
Clients should not do any hashing or encoding; they should use the PasswordModify exop and send a plaintext octetString. IMO it is not the LDAP subsystem's job to worry about how that octetString was generated. E.g., if you're using a device whose input mechanism (keyboard, touchscreen, whatever) can only generate 7 bit ASCII characters, that's not our concern. If your password was originally selected using 8 bit data, and your current keyboard cannot generate the relevant octets, you're SOL.
userPassword is a string of *octets* not *characters*...
Howard Chu writes:
[Pulling last line up front]
userPassword is a string of *octets* not *characters*...
This is backwards. That simply means anything can be stored there - so password charset policy, if any, is up to whoever stores userPassword values. As in fact RFC 4519 2.41 paragraph 2 says:
2.41. 'userPassword' (...) The application SHOULD prepare textual strings used as passwords by transcoding them to Unicode, applying SASLprep [RFC4013], and encoding as UTF-8. The determination of whether a password is textual is a local client matter.
userPassword is an octetString, therefore if the server does any type of character set conversion on its values it is Broken.
Which means it's also Broken if it hashes a Simple Bind password before comparing it with userPassword. Since OpenLDAP does anyway though, or if one used another attribute where that wasn't wrong bug, there's no formal reason why the hash function couldn't involve charset conversion.
Certainly it'd be an ugly hack, hopefully going away someday. And I'm not arguing to do it if it's avoidable. But ugly hacks are normal enough when one has to deal with a pre-existing mess, which certainly describes charset issues on many sites.
Also several SASL mechanisms do hash passwords, but this time according to the standard. And if I remember correctly, several specify UTF-8 passwords.
Clients should not do any hashing or encoding; they should use the PasswordModify exop and send a plaintext octetString.
That's perfect from the LDAP point of view, if the LDAP admin is in charge of the world - or at least of the site. But I was talking about the case where the source of userPassword in LDAP is not updates by the users, but another password store such as /etc/passwd, NIS or whatever.
And where the LDAP admin does not control how passwords are prepared before they are hashed there, so that LDAP must accommodate the quirks of that other password store.
IMO it is not the LDAP subsystem's job to worry about how that octetString was generated.
No, it's the LDAP admin's and/or the user's job. If it's the LDAP admin, he faces LDAP clients which send Bind passwords, a source of stored passwords, terminals and users with various character sets - and it's his job to get them to agree on what a password looks like.
E.g., if you're using a device whose input mechanism (keyboard, touchscreen, whatever) can only generate 7 bit ASCII characters, that's not our concern.
If you mean "our" as in OpenLDAP project, indeed not.
If your password was originally selected using 8 bit data, and your current keyboard cannot generate the relevant octets, you're SOL.
But if your password was originally stored using Latin-1 but you're now using a client which sends UTF-8 or vice versa, it may be possible for LDAP to help. Either by hacking the Bind password or by storing two password hashes in userPassword, one for each supported character set.
Hallvard B Furuseth wrote:
Howard Chu writes:
[Pulling last line up front]
userPassword is a string of *octets* not *characters*...
This is backwards. That simply means anything can be stored there
- so password charset policy, if any, is up to whoever stores
userPassword values.
Yupp. And this lead to interop problems.
As in fact RFC 4519 2.41 paragraph 2 says:
2.41. 'userPassword' (...) The application SHOULD prepare textual strings used as passwords by transcoding them to Unicode, applying SASLprep [RFC4013], and encoding as UTF-8. The determination of whether a password is textual is a local client matter.
And that was good progress!
This has been debated on ietf-ldapbis: http://www.openldap.org/lists/ietf-ldapbis/200110/msg00006.html http://www.openldap.org/lists/ietf-ldapbis/200309/msg00026.html (The "References" and "Follow-Ups" links are not complete in this archive. You have to sometimes click on "Next by Date".)
Ciao, Michael.
Michael Ströder writes:
Hallvard B Furuseth wrote:
Howard Chu writes: [Pulling last line up front]
userPassword is a string of *octets* not *characters*...
This is backwards. That simply means anything can be stored there
- so password charset policy, if any, is up to whoever stores
userPassword values.
Yupp. And this lead to interop problems.
Well, yes. In the sense that "LDAP auth won't work at our site" is no interop problem while "it works when clients do it <this way>" is.
Hallvard B Furuseth wrote:
Michael Ströder writes:
Hallvard B Furuseth wrote:
Howard Chu writes: [Pulling last line up front]
userPassword is a string of *octets* not *characters*...
This is backwards. That simply means anything can be stored there
- so password charset policy, if any, is up to whoever stores
userPassword values.
Yupp. And this lead to interop problems.
Well, yes. In the sense that "LDAP auth won't work at our site" is no interop problem while "it works when clients do it <this way>" is.
My original posting back then was triggered by Netscape Communicator 4.x sending the password as ISO-8859-1 (Latin 1) while using UTF-8 for everything else (even with LDAPv2). Well, that's a long time ago but I can imagine that there are LDAP clients out there which still do it like this and (almost correctly) claim to be LDAPv3 compliant.
Ciao, Michael.
Michael Ströder writes:
Hallvard B Furuseth wrote:
Michael Ströder writes:
Hallvard B Furuseth wrote: Yupp. And this lead to interop problems.
Well, yes. In the sense that "LDAP auth won't work at our site" is no interop problem while "it works when clients do it <this way>" is.
My original posting back then was triggered by Netscape Communicator 4.x sending the password as ISO-8859-1 (Latin 1) while using UTF-8 for everything else (even with LDAPv2). Well, that's a long time ago but I can imagine that there are LDAP clients out there which still do it like this and (almost correctly) claim to be LDAPv3 compliant.
Yes. It has led to interop problems.
Hallvard B Furuseth wrote:
Howard Chu writes:
[Pulling last line up front]
userPassword is a string of *octets* not *characters*...
This is backwards.
No.
That simply means anything can be stored there
Yes. I could use a 16 byte binary UUID; the server has no responsibility or preference here.
- so password charset policy, if any, is up to whoever stores
userPassword values. As in fact RFC 4519 2.41 paragraph 2 says:
2.41. 'userPassword' (...) The application SHOULD prepare textual strings used as passwords by transcoding them to Unicode, applying SASLprep [RFC4013], and encoding as UTF-8. The determination of whether a password is textual is a local client matter.
And again - client matter, not server.
userPassword is an octetString, therefore if the server does any type of character set conversion on its values it is Broken.
Which means it's also Broken if it hashes a Simple Bind password before comparing it with userPassword.
Yes. And the fact that RFC2307 made this common practice doesn't change the fact that it's broken.
Since OpenLDAP does anyway though, or if one used another attribute where that wasn't wrong bug, there's no formal reason why the hash function couldn't involve charset conversion.
Again, making things more broken is not a good idea. If we're touching code, it should be to make it *less* broken.
Certainly it'd be an ugly hack, hopefully going away someday. And I'm not arguing to do it if it's avoidable. But ugly hacks are normal enough when one has to deal with a pre-existing mess, which certainly describes charset issues on many sites.
Accomodating pre-existing messes just allows them to perpetuate forever. Adding systems to the mess only increases the mess. At some point you have to say "No more." Otherwise your job as an admin quickly grows beyond any possibility of control.
Also several SASL mechanisms do hash passwords, but this time according to the standard. And if I remember correctly, several specify UTF-8 passwords.
What SASL does is not relevant here; what SASL hands to LDAP is opaque data.
IMO it is not the LDAP subsystem's job to worry about how that octetString was generated.
No, it's the LDAP admin's and/or the user's job. If it's the LDAP admin, he faces LDAP clients which send Bind passwords, a source of stored passwords, terminals and users with various character sets - and it's his job to get them to agree on what a password looks like.
E.g., if you're using a device whose input mechanism (keyboard, touchscreen, whatever) can only generate 7 bit ASCII characters, that's not our concern.
If you mean "our" as in OpenLDAP project, indeed not.
If your password was originally selected using 8 bit data, and your current keyboard cannot generate the relevant octets, you're SOL.
But if your password was originally stored using Latin-1 but you're now using a client which sends UTF-8 or vice versa, it may be possible for LDAP to help. Either by hacking the Bind password or by storing two password hashes in userPassword, one for each supported character set.
That is certainly your option, since userPassword is multivalued. But the client would have to use a regular ldapmodify op to achieve this, since the original character string only exists consistently on the client.
If you really want to fix this, it will take more than the current LDAPv3 spec work has accomplished so far. E.g., you need a new attributeType ("userCredential" perhaps) that you define to be a directory string, not an octet string, so that it's understood that character-set semantics are significant. (And, since character strings in LDAP must be UTF-8, you eliminate that ambiguity permanently. You also eliminate the possibility of using arbitrary octet sequences though, which may leave yet another group of users out in the cold.) You probably use tags to differentiate hash types, so that you don't require clients or servers to muck with the actual password attribute values. (E.g., userCredential;crypt: xyzzy)
openldap-technical@openldap.org
-
Adrian Marsh -
Andrew Findlay -
Hallvard B Furuseth -
Howard Chu -
Michael Ströder