6:49 a.m.
All,
I'm not sure if my questions here are getting through to the group (I don't see a
copy to myself)
I've diagnosed this issue. The user has a "£" (UK Pound) in his password.
Remove it and everything authenticates OK. Are there settings in Apache/LDAP for
controlling what characters can be used as passwords?
Adrian
________________________________
From: Adrian Marsh
Sent: 14 November 2008 11:49
To: 'openldap-technical(a)openldap.org'
Subject: Debugging a user authentication
Hi All,
Using Apache 2.2, how do I debug the LDAP lookups being made to a 2003 Domain Controller.
Ive one user whos failing to authenticate, but all my other users do and Im trying to see
who. He authenticates ok, same password via other mechanisms to the DC, but just not via
the Apache LDAP lookup.
I'm an LDAP novice so am looking for names of debug tools/methods etc.
Thanks,
Adrian
12:28 p.m.
New subject: Debugging a user authentication
On Fri, Nov 14, 2008 at 02:49:16PM -0000, Adrian Marsh wrote:
I've diagnosed this issue. The user has a "£" (UK
Pound) in his password.
Remove it and everything authenticates OK.
"£" is not a 7-bit character. You therefore run into problems with
character sets.
Almost everything in LDAP is defined to use UTF-8, but passwords have
always been Octet Strings (1) The effect of this is that passwords can
contain absolutely anything and it is the client system's problem to be
consistent about the character set. That does not work in the general
case, as there are probably lots of client systems for each LDAP server
and they could all be using different character sets.
Thus your user's "£" character will translate into a different sequence
of bytes if the client is using UTF-8 from the sequence you would see
from an ISO-8859-1 client.
The only safe solution for most systems at the moment is to ban
non-ASCII (7-bit) characters in passwords :-(
(1): The latest LDAP spec introduced pwprep to solve this problem,
but hardly anything implements it yet. It will be many years before
you can depend on common LDAP clients doing itproperly.
Andrew
--
-----------------------------------------------------------------------
| From Andrew Findlay, Skills 1st Ltd |
| Consultant in large-scale systems, networks, and directory services |
| http://www.skills-1st.co.uk/ +44 1628 782565 |
-----------------------------------------------------------------------
2:24 a.m.
New subject: Debugging a user authentication
Hi Andrew,
Thanks for that explanation. I'm not 100% sure of which route to take. Now we're
aware of the issue we can look for it in future. Our password policy doesn't enforce
special characters, but doesn't ban them either, so we may re-think this for the
future.
Adrian
-----Original Message-----
From: Andrew Findlay [mailto:andrew.findlay@skills-1st.co.uk]
Sent: 17 November 2008 20:29
To: Adrian Marsh
Cc: openldap-technical(a)openldap.org
Subject: Re: Debugging a user authentication
On Fri, Nov 14, 2008 at 02:49:16PM -0000, Adrian Marsh wrote:
I've diagnosed this issue. The user has a "£" (UK
Pound) in his password.
Remove it and everything authenticates OK.
"£" is not a 7-bit character. You therefore run into problems with
character sets.
Almost everything in LDAP is defined to use UTF-8, but passwords have
always been Octet Strings (1) The effect of this is that passwords can
contain absolutely anything and it is the client system's problem to be
consistent about the character set. That does not work in the general
case, as there are probably lots of client systems for each LDAP server
and they could all be using different character sets.
Thus your user's "£" character will translate into a different sequence
of bytes if the client is using UTF-8 from the sequence you would see
from an ISO-8859-1 client.
The only safe solution for most systems at the moment is to ban
non-ASCII (7-bit) characters in passwords :-(
(1): The latest LDAP spec introduced pwprep to solve this problem,
but hardly anything implements it yet. It will be many years before
you can depend on common LDAP clients doing itproperly.
Andrew
--
-----------------------------------------------------------------------
| From Andrew Findlay, Skills 1st Ltd |
| Consultant in large-scale systems, networks, and directory services |
| http://www.skills-1st.co.uk/ +44 1628 782565 |
-----------------------------------------------------------------------
5:57 a.m.
New subject: Debugging a user authentication
Andrew Findlay writes:
(1): The latest LDAP spec introduced pwprep to solve this problem,
but hardly anything implements it yet. It will be many years before
you can depend on common LDAP clients doing itproperly.
It's not just a client-side issue. Most sites store a password hash in
their server rather than the cleartext password. That means the client
needs to encode password with the same character encoding and
preparation as whatever hashed the server-side password. (E.g. the
/etc/passwd program.) Or the server needs to prepare cleartext
passwords it receives from the client the same way, but it's likely a
bad idea for the server to e.g. assume client passwords are latin-1 and
convert to UTF-8.
--
Hallvard
6:42 a.m.
New subject: Debugging a user authentication
On Tue, Nov 18, 2008 at 02:57:16PM +0100, Hallvard B Furuseth wrote:
Andrew Findlay writes:
> (1): The latest LDAP spec introduced pwprep to solve this problem,
I meant SASLprep - RFC4013
> but hardly anything implements it yet. It will be many years
before
> you can depend on common LDAP clients doing itproperly.
It's not just a client-side issue. Most sites store a password hash in
their server rather than the cleartext password. That means the client
needs to encode password with the same character encoding and
preparation as whatever hashed the server-side password. (E.g. the
Ideally password hashing should always be done by the server to avoid
risks like that. It is not always possible though :-(
/etc/passwd program.) Or the server needs to prepare cleartext
passwords it receives from the client the same way, but it's likely a
bad idea for the server to e.g. assume client passwords are latin-1 and
convert to UTF-8.
A very bad idea indeed! The server has no way to know what character
set the client is using. That is why SASLprep has to be applied at the
client end.
Andrew
--
-----------------------------------------------------------------------
| From Andrew Findlay, Skills 1st Ltd |
| Consultant in large-scale systems, networks, and directory services |
| http://www.skills-1st.co.uk/ +44 1628 782565 |
-----------------------------------------------------------------------
2:32 p.m.
New subject: Debugging a user authentication
Hallvard B Furuseth wrote:
Andrew Findlay writes:
> (1): The latest LDAP spec introduced pwprep to solve this problem,
> but hardly anything implements it yet. It will be many years before
> you can depend on common LDAP clients doing itproperly.
It's not just a client-side issue. Most sites store a password hash in
their server rather than the cleartext password. That means the client
needs to encode password with the same character encoding and
preparation as whatever hashed the server-side password. (E.g. the
/etc/passwd program.) Or the server needs to prepare cleartext
passwords it receives from the client the same way, but it's likely a
bad idea for the server to e.g. assume client passwords are latin-1 and
convert to UTF-8.
userPassword is an octetString, therefore if the server does any type of
character set conversion on its values it is Broken.
Clients should not do any hashing or encoding; they should use the
PasswordModify exop and send a plaintext octetString. IMO it is not the LDAP
subsystem's job to worry about how that octetString was generated. E.g., if
you're using a device whose input mechanism (keyboard, touchscreen, whatever)
can only generate 7 bit ASCII characters, that's not our concern. If your
password was originally selected using 8 bit data, and your current keyboard
cannot generate the relevant octets, you're SOL.
userPassword is a string of *octets* not *characters*...
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
3:39 a.m.
New subject: Debugging a user authentication
Howard Chu writes:
[Pulling last line up front]
userPassword is a string of *octets* not *characters*...
This is backwards. That simply means anything can be stored there
- so password charset policy, if any, is up to whoever stores
userPassword values. As in fact RFC 4519 2.41 paragraph 2 says:
2.41. 'userPassword'
(...)
The application SHOULD prepare textual strings used as passwords
by transcoding them to Unicode, applying SASLprep [RFC4013], and
encoding as UTF-8. The determination of whether a password is
textual is a local client matter.
userPassword is an octetString, therefore if the server does any
type of character set conversion on its values it is Broken.
Which means it's also Broken if it hashes a Simple Bind password before
comparing it with userPassword. Since OpenLDAP does anyway though, or
if one used another attribute where that wasn't wrong bug, there's no
formal reason why the hash function couldn't involve charset conversion.
Certainly it'd be an ugly hack, hopefully going away someday. And I'm
not arguing to do it if it's avoidable. But ugly hacks are normal
enough when one has to deal with a pre-existing mess, which certainly
describes charset issues on many sites.
Also several SASL mechanisms do hash passwords, but this time according
to the standard. And if I remember correctly, several specify UTF-8
passwords.
Clients should not do any hashing or encoding; they should use the
PasswordModify exop and send a plaintext octetString.
That's perfect from the LDAP point of view, if the LDAP admin is in
charge of the world - or at least of the site. But I was talking about
the case where the source of userPassword in LDAP is not updates by the
users, but another password store such as /etc/passwd, NIS or whatever.
And where the LDAP admin does not control how passwords are prepared
before they are hashed there, so that LDAP must accommodate the quirks
of that other password store.
IMO it is not the LDAP subsystem's job to worry about how that
octetString was generated.
No, it's the LDAP admin's and/or the user's job. If it's the LDAP
admin, he faces LDAP clients which send Bind passwords, a source of
stored passwords, terminals and users with various character sets -
and it's his job to get them to agree on what a password looks like.
E.g., if you're using a device whose input mechanism (keyboard,
touchscreen, whatever) can only generate 7 bit ASCII characters,
that's not our concern.
If you mean "our" as in OpenLDAP project, indeed not.
If your password was originally selected using
8 bit data, and your current keyboard cannot generate the relevant
octets, you're SOL.
But if your password was originally stored using Latin-1 but you're now
using a client which sends UTF-8 or vice versa, it may be possible for
LDAP to help. Either by hacking the Bind password or by storing two
password hashes in userPassword, one for each supported character set.
--
Hallvard
4:44 a.m.
New subject: Debugging a user authentication
Hallvard B Furuseth wrote:
Howard Chu writes:
[Pulling last line up front]
> userPassword is a string of *octets* not *characters*...
This is backwards. That simply means anything can be stored there
- so password charset policy, if any, is up to whoever stores
userPassword values.
Yupp. And this lead to interop problems.
As in fact RFC 4519 2.41 paragraph 2 says:
2.41. 'userPassword'
(...)
The application SHOULD prepare textual strings used as passwords
by transcoding them to Unicode, applying SASLprep [RFC4013], and
encoding as UTF-8. The determination of whether a password is
textual is a local client matter.
And that was good progress!
This has been debated on ietf-ldapbis:
http://www.openldap.org/lists/ietf-ldapbis/200110/msg00006.html
http://www.openldap.org/lists/ietf-ldapbis/200309/msg00026.html
(The "References" and "Follow-Ups" links are not complete in this
archive. You have to sometimes click on "Next by Date".)
Ciao, Michael.
4:58 a.m.
New subject: Debugging a user authentication
Michael Ströder writes:
Hallvard B Furuseth wrote:
>Howard Chu writes:
> [Pulling last line up front]
>> userPassword is a string of *octets* not *characters*...
>
> This is backwards. That simply means anything can be stored there
> - so password charset policy, if any, is up to whoever stores
> userPassword values.
Yupp. And this lead to interop problems.
Well, yes. In the sense that "LDAP auth won't work at our site" is
no interop problem while "it works when clients do it <this way>" is.
--
Hallvard
5:05 a.m.
New subject: Debugging a user authentication
Hallvard B Furuseth wrote:
Michael Ströder writes:
> Hallvard B Furuseth wrote:
>> Howard Chu writes:
>> [Pulling last line up front]
>>> userPassword is a string of *octets* not *characters*...
>> This is backwards. That simply means anything can be stored there
>> - so password charset policy, if any, is up to whoever stores
>> userPassword values.
> Yupp. And this lead to interop problems.
Well, yes. In the sense that "LDAP auth won't work at our site" is
no interop problem while "it works when clients do it <this way>" is.
My original posting back then was triggered by Netscape Communicator 4.x
sending the password as ISO-8859-1 (Latin 1) while using UTF-8 for
everything else (even with LDAPv2). Well, that's a long time ago but I
can imagine that there are LDAP clients out there which still do it like
this and (almost correctly) claim to be LDAPv3 compliant.
Ciao, Michael.
5:14 a.m.
New subject: Debugging a user authentication
Michael Ströder writes:
Hallvard B Furuseth wrote:
>Michael Ströder writes:
>>Hallvard B Furuseth wrote:
>> Yupp. And this lead to interop problems.
>
> Well, yes. In the sense that "LDAP auth won't work at our site" is
> no interop problem while "it works when clients do it <this way>"
is.
My original posting back then was triggered by Netscape Communicator 4.x
sending the password as ISO-8859-1 (Latin 1) while using UTF-8 for
everything else (even with LDAPv2). Well, that's a long time ago but I
can imagine that there are LDAP clients out there which still do it like
this and (almost correctly) claim to be LDAPv3 compliant.
Yes. It has led to interop problems.
--
Hallvard
3:05 p.m.
New subject: Debugging a user authentication
Hallvard B Furuseth wrote:
Howard Chu writes:
[Pulling last line up front]
> userPassword is a string of *octets* not *characters*...
This is backwards.
No.
That simply means anything can be stored there
Yes. I could use a 16 byte binary UUID; the server has no responsibility or
preference here.
- so password charset policy, if any, is up to whoever stores
userPassword values. As in fact RFC 4519 2.41 paragraph 2 says:
2.41. 'userPassword'
(...)
The application SHOULD prepare textual strings used as passwords
by transcoding them to Unicode, applying SASLprep [RFC4013], and
encoding as UTF-8. The determination of whether a password is
textual is a local client matter.
And again - client matter, not server.
> userPassword is an octetString, therefore if the server does any
> type of character set conversion on its values it is Broken.
Which means it's also Broken if it hashes a Simple Bind password before
comparing it with userPassword.
Yes. And the fact that RFC2307 made this common practice doesn't change the
fact that it's broken.
Since OpenLDAP does anyway though, or
if one used another attribute where that wasn't wrong bug, there's no
formal reason why the hash function couldn't involve charset conversion.
Again, making things more broken is not a good idea. If we're touching code,
it should be to make it *less* broken.
Certainly it'd be an ugly hack, hopefully going away someday.
And I'm
not arguing to do it if it's avoidable. But ugly hacks are normal
enough when one has to deal with a pre-existing mess, which certainly
describes charset issues on many sites.
Accomodating pre-existing messes just allows them to perpetuate forever.
Adding systems to the mess only increases the mess. At some point you have to
say "No more." Otherwise your job as an admin quickly grows beyond any
possibility of control.
Also several SASL mechanisms do hash passwords, but this time
according
to the standard. And if I remember correctly, several specify UTF-8
passwords.
What SASL does is not relevant here; what SASL hands to LDAP is opaque data.
> IMO it is not the LDAP subsystem's job to worry about how
that
> octetString was generated.
No, it's the LDAP admin's and/or the user's job. If it's the LDAP
admin, he faces LDAP clients which send Bind passwords, a source of
stored passwords, terminals and users with various character sets -
and it's his job to get them to agree on what a password looks like.
> E.g., if you're using a device whose input mechanism
(keyboard,
> touchscreen, whatever) can only generate 7 bit ASCII characters,
> that's not our concern.
If you mean "our" as in OpenLDAP project, indeed not.
> If your password was originally selected using
> 8 bit data, and your current keyboard cannot generate the relevant
> octets, you're SOL.
But if your password was originally stored using Latin-1 but you're now
using a client which sends UTF-8 or vice versa, it may be possible for
LDAP to help. Either by hacking the Bind password or by storing two
password hashes in userPassword, one for each supported character set.
That is certainly your option, since userPassword is multivalued. But the
client would have to use a regular ldapmodify op to achieve this, since the
original character string only exists consistently on the client.
If you really want to fix this, it will take more than the current LDAPv3 spec
work has accomplished so far. E.g., you need a new attributeType
("userCredential" perhaps) that you define to be a directory string, not an
octet string, so that it's understood that character-set semantics are
significant. (And, since character strings in LDAP must be UTF-8, you
eliminate that ambiguity permanently. You also eliminate the possibility of
using arbitrary octet sequences though, which may leave yet another group of
users out in the cold.) You probably use tags to differentiate hash types, so
that you don't require clients or servers to muck with the actual password
attribute values. (E.g., userCredential;crypt: xyzzy)
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
5315
days inactive
5321
days old
openldap-technical@openldap.org
11 comments
5 participants
participants (5)
-
Adrian Marsh -
Andrew Findlay -
Hallvard B Furuseth -
Howard Chu -
Michael Ströder