Re: (ITS#7413) Some UTF-8 characters are wrongly recognized as identical in multi-valued attributes
by hyc＠symas.com
m.bachmann(a)cms.hu-berlin.de wrote:
> Full_Name: Michail Bachmann
> Version: 2.4.31
> OS: Linux
> URL: ftp://ftp.openldap.org/incoming/
> Submission from: (NULL) (141.20.3.158)
>
>
>
> The UTF-8 character 'FEMININE ORDINAL INDICATOR' (U+00AA) is wrongly treated as
> 'LATIN SMALL LETTER A' (U+0061), so adding a multi-valued attribute where U+00AA
> is replaced with U+0061 (or the other way around) fails:
This works as designed. See the Unicode specification, rules for decomposition.
>
> $ ldapadd -f create.ldif -h localhost -D 'cn=admin,dc=example,dc=com'
>
> ### create.ldif ###
> dn: cn=test,dc=example,dc=com
> objectClass: organizationalRole
> objectClass: top
> cn: test
> street: a street
> ### /create.ldif ###
>
> $ ldapmodify -f modify.ldif -h localhost -D 'cn=admin,dc=example,dc=com'
>
> ### modify.ldif ###
> dn: cn=test,dc=example,dc=com
> changetype: modify
> add: street
> street: ª street
> -
>
> ### /modify.ldif ###
>
> will result in
>
> ldap_modify: Type or value exists (20)
> additional info: modify/add: street: value #0 already exists
>
> It seems like U+00AA is decomposed into '<super>+U+0061' and then the '<super>'
> part is discarded.
>
> Tested with 2.4.31 (Debian) and 2.4.21 (Ubuntu)
>
>
>
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
10 years, 11 months
(ITS#7413) Some UTF-8 characters are wrongly recognized as identical in multi-valued attributes
by m.bachmann＠cms.hu-berlin.de
Full_Name: Michail Bachmann
Version: 2.4.31
OS: Linux
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (141.20.3.158)
The UTF-8 character 'FEMININE ORDINAL INDICATOR' (U+00AA) is wrongly treated as
'LATIN SMALL LETTER A' (U+0061), so adding a multi-valued attribute where U+00AA
is replaced with U+0061 (or the other way around) fails:
$ ldapadd -f create.ldif -h localhost -D 'cn=admin,dc=example,dc=com'
### create.ldif ###
dn: cn=test,dc=example,dc=com
objectClass: organizationalRole
objectClass: top
cn: test
street: a street
### /create.ldif ###
$ ldapmodify -f modify.ldif -h localhost -D 'cn=admin,dc=example,dc=com'
### modify.ldif ###
dn: cn=test,dc=example,dc=com
changetype: modify
add: street
street: ª street
-
### /modify.ldif ###
will result in
ldap_modify: Type or value exists (20)
additional info: modify/add: street: value #0 already exists
It seems like U+00AA is decomposed into '<super>+U+0061' and then the '<super>'
part is discarded.
Tested with 2.4.31 (Debian) and 2.4.21 (Ubuntu)
10 years, 11 months
(ITS#7412) check_password contrib module
by clem.oudot＠gmail.com
Full_Name: Clément OUDOT
Version:
OS: GNU/Linux
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (88.173.78.196)
As said by Guillaume Rousse in ITS#7348 (
http://www.openldap.org/its/index.cgi/Incoming?id=7348;selectid=7348#themesg), I
would like to contribute the check_password module to OpenLDAP.
---
contrib/slapd-modules/README | 3 +
contrib/slapd-modules/check-password/Makefile | 52 +++
contrib/slapd-modules/check-password/README | 146 +++++++++
.../slapd-modules/check-password/check_password.c | 356 +++++++++++++++=
++++++
4 files changed, 557 insertions(+)
create mode 100644 contrib/slapd-modules/check-password/Makefile
create mode 100644 contrib/slapd-modules/check-password/README
create mode 100644 contrib/slapd-modules/check-password/check_password.c=
diff --git a/contrib/slapd-modules/README b/contrib/slapd-modules/README
index db74379..d8005ff 100644
--- a/contrib/slapd-modules/README
+++ b/contrib/slapd-modules/README
@@ -20,6 +20,9 @@ allop (overlay)
autogroup (overlay)
Automated updates of group memberships.
=20
+check_password (plugin)
+ External password quality check module for ppolicy
+
cloak (overlay)
Hide specific attributes unless explicitely requested
=20
diff --git a/contrib/slapd-modules/check-password/Makefile b/contrib/slap=
d-modules/check-password/Makefile
new file mode 100644
index 0000000..42dd18f
--- /dev/null
+++ b/contrib/slapd-modules/check-password/Makefile
@@ -0,0 +1,52 @@
+
+LDAP_SRC =3D ../../..
+LDAP_BUILD =3D ../../..
+LDAP_INC =3D -I$(LDAP_BUILD)/include -I$(LDAP_SRC)/include -I$(LDAP_SRC)=
/servers/slapd
+LDAP_LIB =3D $(LDAP_BUILD)/libraries/libldap_r/libldap_r.la \
+ $(LDAP_BUILD)/libraries/liblber/liblber.la
+
+CRACKLIB_PATH =3D /usr/share/cracklib/pw_dict
+CRACKLIB_INC =3D=20
+CRACKLIB_LIB =3D -lcrack
+
+CONFIG_PATH =3D /etc/openldap/check_password.conf
+
+LIBTOOL =3D $(LDAP_BUILD)/libtool
+CC =3D gcc
+OPT =3D -g -O2 -Wall
+DEFS =3D -DHAVE_CRACKLIB -DCRACKLIB_DICTPATH=3D"\"$(CRACKLIB_PATH)\"" \
+ -DCONFIG_FILE=3D"\"$(CONFIG_PATH)\"" -DDEBUG
+INCS =3D $(LDAP_INC) $(CRACKLIB_INC)
+LIBS =3D $(LDAP_LIB) $(CRACKLIB_LIB)
+
+PROGRAMS =3D check_password.la
+LTVER =3D 0:0:0
+
+prefix=3D/usr/local
+exec_prefix=3D$(prefix)
+ldap_subdir=3D/openldap
+
+libdir=3D$(exec_prefix)/lib
+libexecdir=3D$(exec_prefix)/libexec
+moduledir =3D $(libexecdir)$(ldap_subdir)
+
+.SUFFIXES: .c .o .lo
+
+.c.lo:
+ $(LIBTOOL) --mode=3Dcompile $(CC) $(OPT) $(DEFS) $(INCS) -c $<
+
+all: $(PROGRAMS)
+
+check_password.la: check_password.lo
+ $(LIBTOOL) --mode=3Dlink $(CC) $(OPT) -version-info $(LTVER) \
+ -rpath $(moduledir) -module -o $@ $? $(LIBS)
+
+clean:
+ rm -rf *.o *.lo *.la .libs
+
+install: $(PROGRAMS)
+ mkdir -p $(DESTDIR)$(moduledir)
+ for p in $(PROGRAMS) ; do \
+ $(LIBTOOL) --mode=3Dinstall cp $$p $(DESTDIR)$(moduledir) ; \
+ done
+
diff --git a/contrib/slapd-modules/check-password/README b/contrib/slapd-=
modules/check-password/README
new file mode 100644
index 0000000..10191c2
--- /dev/null
+++ b/contrib/slapd-modules/check-password/README
@@ -0,0 +1,146 @@
+
+check_password.c - OpenLDAP pwdChecker library
+
+2007-06-06 Michael Steinmann <msl(a)calivia.com>
+2008-01-30 Pierre-Yves Bonnetain <py.bonnetain(a)ba-cst.com>
+2009 Clement Oudot <clem.oudot(a)gmail.com> - LTB-project
+2009 Jerome HUET - LTB-project
+
+check_password.c is an OpenLDAP pwdPolicyChecker module used to check th=
e
+strength and quality of user-provided passwords.
+
+This module is used as an extension of the OpenLDAP password policy cont=
rols,
+see slapo-ppolicy(5) section pwdCheckModule.
+
+check_password.c will run a number of checks on the passwords to ensure =
minimum
+strength and quality requirements are met. Passwords that do not meet th=
ese
+requirements are rejected.
+
+
+Password checks
+---------------
+ - passwords shorter than 6 characters are rejected if cracklib is used =
(because
+ cracklib WILL reject them).
+
+ - syntactic checks controls how many different character classes are us=
ed
+ (lower, upper, digit and punctuation characters). The minimum number =
of
+ classes is defined in a configuration file. You ca
The attached patch file is derived from OpenLDAP Software. All of the
modifications to OpenLDAP Software represented in the following patch(es) were
developed by Clément OUDOT clem.oudot(a)gmail.com. I have not assigned rights
and/or interest in this work to any party.
I, Clément OUDOT, hereby place the following modifications to OpenLDAP Software
(and only these modifications) into the public domain. Hence, these
modifications may be freely used and/or redistributed for any purpose with or
without attribution and/or other notice.
10 years, 11 months
(ITS#7411) ldap_get_option.3: Warning from "groff" about a tab in the manual
by bjarniig＠rhi.hi.is
Full_Name: Bjarni Ingi Gislason
Version: 2.4.32
OS: GNU/Linux
URL:
Submission from: (NULL) (130.208.164.54)
Software: openldap
Version: 2.4.32
File: doc/man/ldap_get_option.3
Patch: Yes
From "groff -b -ww -z -mandoc ldap_get_option.3":
groff: ldap_get_option.3: stdin:456: warning: tab character in unquoted macro
argument
#
Output from "nroff" should (usually) have only the left margin adjusted
(".ad l"). Otherwise the space between words can be too long.
Output for a manual should (usually) have the right margin ragged
(".ad l"), as it is not (usually) intended for a book or other professional
printing medium.
#
#
Each sentence should begin in a new line.
a) The space between sentences is different between formatters.
b) Changes in one line do then not affect others, and do therefore not
appear as changed in the output of "diff".
#
Sometimes a small change in space between a roman and an italic
character is needed (use ".RI" or ".IR" macros).
#
Remove spaces at end of lines.
#
Patch:
--- ldap_get_option.3 2012-07-31 16:39:26.000000000 +0000
+++ /tmp/ldap_get_option.3.new 2012-10-09 21:27:10.240593613 +0000
@@ -453,7 +453,7 @@
.BR "char ***" .
The caller must not free or otherwise muck with it.
.TP
-.B LDAP_OPT_X_SASL_NOCANON
+.B LDAP_OPT_X_SASL_NOCANON
Sets/gets the NOCANON flag.
When unset, the hostname is canonicalized.
.BR invalue
10 years, 11 months
Re: (ITS#7348) enhancement: check_password contrib module
by Kurt Zeilenga
This submission was been placed in a suspended state for intellectual property
rights (IPR) reasons.
Unfortunately, I got busy with other things and didn't have time to respond. I
do now.
Generally, as the Foundation requires notices to be provided by authors (or
other right holders), it does not
accept 3rd party submissions. The Foundation views this as a 3rd party
submission as it apparently comes
from someone who has no rights in the submission.
Generally, the Project does not duplication distribution of materials otherwise
made available. This
material appears to be distributed by the LTB Project and the Project sees no
reason to supplant their
effects. The submitter is not listed as a member of the LTB Project and assumes
the submitter is not
authorized to speak on behalf of the LTB Project.
The contribution is missing appropriate IPR Notices. See
<http://www.openldap.org/devel/contributing.html>. These MUST be provided by
the authors (or other
rights holders). IPR notices provided by non-authors will not be considered.
For these reasons, your submission is rejected. If it is the LTB Project's
wish to contribute this material to
the OpenLDAP Project, their Project Leader (who we believed also authored
portions of this material),
working in cooperation with other authors, should personally submit the
materials, with appropriate IPR
notices, and a statement that they wish the OpenLDAP Project to include the
materials in OpenLDAP
Software and shepherd further development.
This ITS will now be closed.
10 years, 11 months
Re: (ITS#7348) enhancement: check_password contrib module
by Guillaume.Rousse＠inria.fr
Ceci est un message signC) cryptographiquement au format MIME.
--------------ms010005020603050107040502
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: quoted-printable
Ping ? Can I have any kind of feedback ?
--------------ms010005020603050107040502
Content-Type: application/pkcs7-signature; name="smime.p7s"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="smime.p7s"
Content-Description: Signature cryptographique S/MIME
MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIJYDCC
BJUwggN9oAMCAQICECu5mlkLgTFbaSg8Mgwue0kwDQYJKoZIhvcNAQEFBQAwOzELMAkGA1UE
BhMCTkwxDzANBgNVBAoTBlRFUkVOQTEbMBkGA1UEAxMSVEVSRU5BIFBlcnNvbmFsIENBMB4X
DTExMDExMTAwMDAwMFoXDTE0MDExMDIzNTk1OVowWDELMAkGA1UEBhMCRlIxDjAMBgNVBAoT
BUlOUklBMRkwFwYDVQQDExBHdWlsbGF1bWUgUk9VU1NFMR4wHAYJKoZIhvcNAQkCFg9yb3Vz
c2VAaW5yaWEuZnIwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDU9hkZ4j+MxXqQ
310Pah2QqpshhbJ8T9bACuYEcIdsZuxCYDUgJUVZJmHZK6IZbX4xiVr85pSHPJYG8z0nXXh4
ZXN0iPGVd2/YFpcqt2rB+KDydyrioXfLNiS6JZeKWEaPhKh/5C6/E898XwxB7VzV4meEDOpJ
4si8pNkCnPFFLMXfH/IG4lFvMj9qwfTIMGymB5/hLnZBm5/pK0Bm6wUESvN6YWFRjZ0ctd5p
bvDG/vZpjQiavHC+r4YSLBgeFUb4W5N6XGXNsxZvUCRc4mC6XaiDaId75klRxFTdDmtynnLl
44mm6KJws158b3YC1kGACTYJaT+01R+D08PsVy2bAgMBAAGjggF2MIIBcjAfBgNVHSMEGDAW
gBRjTUNaGUg/xEbBArq/7g7lgrdmpjAdBgNVHQ4EFgQUT8X4ighZJ1th5zEgoWZwKTXpQcIw
DgYDVR0PAQH/BAQDAgWgMAwGA1UdEwEB/wQCMAAwHQYDVR0lBBYwFAYIKwYBBQUHAwQGCCsG
AQUFBwMCMBgGA1UdIAQRMA8wDQYLKwYBBAGyMQECAh0wPwYDVR0fBDgwNjA0oDKgMIYuaHR0
cDovL2NybC50Y3MudGVyZW5hLm9yZy9URVJFTkFQZXJzb25hbENBLmNybDByBggrBgEFBQcB
AQRmMGQwOgYIKwYBBQUHMAKGLmh0dHA6Ly9jcnQudGNzLnRlcmVuYS5vcmcvVEVSRU5BUGVy
c29uYWxDQS5jcnQwJgYIKwYBBQUHMAGGGmh0dHA6Ly9vY3NwLnRjcy50ZXJlbmEub3JnMCQG
A1UdEQQdMBuBGUd1aWxsYXVtZS5Sb3Vzc2VAaW5yaWEuZnIwDQYJKoZIhvcNAQEFBQADggEB
AEItBMmkv3DtWrx5KhZDlYV+ZLht8pFJTiK84GtnZ0dh+8pUSCzk8QS8Hly9gyiTfmTyZXiy
UpnVXLLatBp3kgOw9ZCbdGj3TB6iOvHzHKaUHq/jdVf/S2CVlkU1ekRa5UnbsxFObZnZRu0G
2Lb4DbbwZYmS9OgGDzueBQ5B64X5yZXUueXnrgJMBa2JCT9QGvs6Px9src1ZyCArWTLGUmu2
NsaOI+WJMXdByLHOPJZp5A52dPPsnwDoyYrNrXPXngXXL0Pz3JPsKCA3lvb52oQkXDjqgm5o
rYdHKhbxJvHG9WozyCppVBgSYve0IVyVfBWXLiCtTWnpY//DA23Hd2cwggTDMIIDq6ADAgEC
AhBz/lf637jFCIF7Zrlr8C3vMA0GCSqGSIb3DQEBBQUAMIGuMQswCQYDVQQGEwJVUzELMAkG
A1UECBMCVVQxFzAVBgNVBAcTDlNhbHQgTGFrZSBDaXR5MR4wHAYDVQQKExVUaGUgVVNFUlRS
VVNUIE5ldHdvcmsxITAfBgNVBAsTGGh0dHA6Ly93d3cudXNlcnRydXN0LmNvbTE2MDQGA1UE
AxMtVVROLVVTRVJGaXJzdC1DbGllbnQgQXV0aGVudGljYXRpb24gYW5kIEVtYWlsMB4XDTA5
MDUxODAwMDAwMFoXDTI4MTIzMTIzNTk1OVowOzELMAkGA1UEBhMCTkwxDzANBgNVBAoTBlRF
UkVOQTEbMBkGA1UEAxMSVEVSRU5BIFBlcnNvbmFsIENBMIIBIjANBgkqhkiG9w0BAQEFAAOC
AQ8AMIIBCgKCAQEAyBXZ9TNqI6GQDc+7BUTDqx9KNYUaIYWgT/jwQOJKQ5v+W7Gwv7RX3HWA
QUtkGvbbT2+P0CVFNfnqy0r6+9rT7UWIEZQ25MyoDe/FPTftFnvjwpWeWDN/Ivv4/+zmvtuu
CmUlIofab4SLRuhAhig/v1YI4krpg6LpIvst+rYoH5HBw3H7U8ArTqQMoW6dVe3s4SSHOgji
DRzkxE3Qyyf6hGTm0ZedViRbk7spLkPiQWo94kpl/JpfWoaHvIfHeYCWmVHGkA9kkZl9EN2s
LAMq4Xhk/s49TvQrUBFL0VjUmwPwf/U7U7BTQ/vFL8QEKRo6rNdV6dEOldE7MX94T64pLQID
AQABo4IBTTCCAUkwHwYDVR0jBBgwFoAUiYJnfcSdJnAAS7RQSHzePa4Ebn0wHQYDVR0OBBYE
FGNNQ1oZSD/ERsECur/uDuWCt2amMA4GA1UdDwEB/wQEAwIBBjASBgNVHRMBAf8ECDAGAQH/
AgEAMBgGA1UdIAQRMA8wDQYLKwYBBAGyMQECAh0wWAYDVR0fBFEwTzBNoEugSYZHaHR0cDov
L2NybC51c2VydHJ1c3QuY29tL1VUTi1VU0VSRmlyc3QtQ2xpZW50QXV0aGVudGljYXRpb25h
bmRFbWFpbC5jcmwwbwYIKwYBBQUHAQEEYzBhMDgGCCsGAQUFBzAChixodHRwOi8vY3J0LnVz
ZXJ0cnVzdC5jb20vVVROQUFBQ2xpZW50X0NBLmNydDAlBggrBgEFBQcwAYYZaHR0cDovL29j
c3AudXNlcnRydXN0LmNvbTANBgkqhkiG9w0BAQUFAAOCAQEABiupUy8T3Fw5FsyGn15Me3L7
7I1Vil6aCv9TTHb0Bj1Qz1fwos+vmYyq/qAZdj6ZAzL6dYM4irtrmqUME7LUG3bmlC5nmFnj
kWwCkJqcyGBLVavKiFqNK+VplQMH0dQO/CQiLlmxY6Rf7dkjcuSczjpcbB9PqQDJHf76f0Ut
ti6E3Q8noFkYTtV2JUX0mSZ522+fI/dDuysPBKOBJiy3ezX5PXdfQCHmfx2lllq90MsWOmy7
YYuK/QQ5RArLLOHLzi4QmBrb4JPtSWRkCCCft6NQ8KLdyrTGfAw9514V3CeG5Do7UloXq6kG
UyudCXNkHAHD/TDShwNv5BUDejlfaDGCAwcwggMDAgEBME8wOzELMAkGA1UEBhMCTkwxDzAN
BgNVBAoTBlRFUkVOQTEbMBkGA1UEAxMSVEVSRU5BIFBlcnNvbmFsIENBAhAruZpZC4ExW2ko
PDIMLntJMAkGBSsOAwIaBQCgggGNMBgGCSqGSIb3DQEJAzELBgkqhkiG9w0BBwEwHAYJKoZI
hvcNAQkFMQ8XDTEyMTAxMDE1MTQ1MFowIwYJKoZIhvcNAQkEMRYEFISpiKvxYJ/cJnbtLJAt
y+oy8hXXMF4GCSsGAQQBgjcQBDFRME8wOzELMAkGA1UEBhMCTkwxDzANBgNVBAoTBlRFUkVO
QTEbMBkGA1UEAxMSVEVSRU5BIFBlcnNvbmFsIENBAhAruZpZC4ExW2koPDIMLntJMGAGCyqG
SIb3DQEJEAILMVGgTzA7MQswCQYDVQQGEwJOTDEPMA0GA1UEChMGVEVSRU5BMRswGQYDVQQD
ExJURVJFTkEgUGVyc29uYWwgQ0ECECu5mlkLgTFbaSg8Mgwue0kwbAYJKoZIhvcNAQkPMV8w
XTALBglghkgBZQMEASowCwYJYIZIAWUDBAECMAoGCCqGSIb3DQMHMA4GCCqGSIb3DQMCAgIA
gDANBggqhkiG9w0DAgIBQDAHBgUrDgMCBzANBggqhkiG9w0DAgIBKDANBgkqhkiG9w0BAQEF
AASCAQBSTpiPHQsxT9U0zme6qRLWCJXDvU6vfyPzeM+6Fb6t9iH9RlXrmNKBPx501HQWCfwf
K/OGGzzNULpU3N3pTp++BRoyPnNPVmIwLe1MedC7SMf/5oL2p8KTGu+VUPd0TXaJihfXwq0+
sThwq9O1464Cfxr+n27k/GSvU3AHsn3j59BqSdGZUXs+Dg/42acuWoV2xYL0ZjyRt8d1ZoU7
3U5TWKTEIAP5zMnSvBfHbfV4HUEIl9AVKtPaaaq4L1aQnByyEG6cN39R4N+LPwQXVW4pBwuI
nKUF8FImmM3tovtb0aINf83d+RFuAhGEUGxLLFgYRGHzxRdCSIC85rOry1C4AAAAAAAA
--------------ms010005020603050107040502--
10 years, 11 months
Re: (ITS#7400) Memberof and Syncrepl incompatibility
by hyc＠symas.com
arun s wrote:
> Hi,
> To try and overcome this issue, we tried two fixes:
>
> 1. Every time a user was deleted from a group, we force-updated the user
> object manually to make sure its entryCSN got updated and it got
> replicated properly. This is an expensive operation and did not scale
> well for big group sizes (10-20k), and did not work out.
This is also one of the reasons the code in memberof.c was reverted. Working
as you suggest *cannot* scale. It makes no difference whether you do it in
your external client or inside slapd, the amount of actual work required to
replicate everything quickly grows out of control. This is why the
documentation states that memberof must be configured on each replica - the
only way to successfully execute the amount of work is to distribute the work
evenly to each replica.
> 2. We then tried to do the same thing in OpenLDAP. We noticed in the
> memberof.c commits that there were a couple of patches to force the
> entryCSN of the user object to get updated. (http://tinyurl.com/8k4qrdj
> and http://tinyurl.com/9akqgfl)These have since been reverted because of
> access log and some replication issues, but for us, speed was a higher
> priority. I reapplied these patches back to the code. This solved the
> member-of replication issue, but we noticed that occasionally under a
> heavy load, there was a sudden surge in OpenLDAP's memory usage going up
> to whatever memory was available and finally crashing.
>
> We have gone back to option (1) though (2) would be the preferred option.
>
> Any help on figuring out why (2) caused the memory bloat would be really
> great. I can provide more details/memory traces if needed.
>
> We will be glad to contribute any fixes once we are able to nail down
> the issue.
Probably this conversation should continue on the openldap-devel mailing list.
It needs some new design work; it is not a simple bugfix.
>
> Thanks,
> Arunkumar
> ------------------------------------------------------------------------------
> *From:* Howard Chu <hyc(a)symas.com>
> *To:* arun s <arunkumar_1123(a)yahoo.com>
> *Cc:* "openldap-its(a)openldap.org" <openldap-its(a)openldap.org>
> *Sent:* Monday, 1 October 2012 6:59 PM
> *Subject:* Re: (ITS#7400) Memberof and Syncrepl incompatibility
>
> arun s wrote:
> > Hi,
> > Yes, I am able to reproduce the issue with 2.4.32
> >
> > Making sense of the logs for the exact reproduction is hard since it needs a
> > lot of operations in a short time. But this will probably help:
> >
> > 1. At the start of the test, the group temp_group existed.
> >
> > 2. I created a user temp_user and added temp_user to temp_group.
> >
> > 3. During replication, the group was replicated first and I got an error 32
> > (NO_SUCH_OBJECT) when it tried to modify the memberOf of the temp_user object
> > (This does not exist in the readslave yet).
> >
> > 4. The temp_user object was replicated next, and as you see, querying it does
> > show a memberOf attribute, proving that this field was replicated. (Note that
> > I have run OpenLDAP with debug as -1 and verified that the replicated data has
> > the memberOf field in it. I can provide this too if needed.)
>
> I see. The current code drops the memberOf attribute if it was not explicitly
> requested in the search. However, by default the consumer requests "+" which
> means "all operational attributes" and so slapd considers memberOf to have
> been requested. We need to reconsider this aspect of the design.
> >
> > 5. The more serious problem occurs when the sequence is reversed and the group
> > has been deleted as a last operation - The user is replicated first, but since
> > the group is deleted, it is never replicated and a stale memberOf entry stays
> > with the user.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
10 years, 11 months
Re: (ITS#7400) Memberof and Syncrepl incompatibility
by arunkumar_1123＠yahoo.com
--1251447850-1386810138-1349783312=:99375
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable
Hi,=0ATo try and overcome this issue, we tried two fixes:=0A=0A1. Every tim=
e a user was deleted from a group, we force-updated the user=A0=0Aobject ma=
nually to make sure its entryCSN got updated and it got=A0=0Areplicated pro=
perly. This is an expensive operation and did not scale=A0=0Awell for big g=
roup sizes (10-20k), and did not work out.=0A=0A2. We then tried to do the =
same thing in OpenLDAP. We noticed in the=A0=0Amemberof.c commits that ther=
e were a couple of patches to force the=A0=0AentryCSN of the user object to=
get updated. (http://tinyurl.com/8k4qrdj=A0=0Aand http://tinyurl.com/9akqg=
fl)These have since been reverted because of=A0=0Aaccess log and some repli=
cation issues, but for us, speed was a higher=A0=0Apriority. I reapplied th=
ese patches back to the code. This solved the=A0=0Amember-of replication is=
sue, but we noticed that occasionally under a=A0=0Aheavy load, there was a =
sudden surge in OpenLDAP's memory usage going up=A0=0Ato whatever memory wa=
s available and finally crashing.=0A=0AWe have gone back to option (1) thou=
gh (2) would be the preferred option.=0A=0AAny help on figuring out why (2)=
caused the memory bloat would be really=A0=0Agreat. I can provide more det=
ails/memory traces if needed.=0A=0AWe will be glad to contribute any fixes =
once we are able to nail down=A0=0Athe issue.=0A=0AThanks,=0AArunkumar=0A=
=0A________________________________=0A From: Howard Chu <hyc(a)symas.com>=0AT=
o: arun s <arunkumar_1123(a)yahoo.com> =0ACc: "openldap-its(a)openldap.org" <op=
enldap-its(a)openldap.org> =0ASent: Monday, 1 October 2012 6:59 PM=0ASubject:=
Re: (ITS#7400) Memberof and Syncrepl incompatibility=0A =0Aarun s wrote:=
=0A> Hi,=0A> Yes, I am able to reproduce the issue with 2.4.32=0A> =0A> Mak=
ing sense of the logs for the exact reproduction is hard since it needs a=
=0A> lot of operations in a short time. But this will probably help:=0A> =
=0A> 1. At the start of the test, the group temp_group existed.=0A> =0A> 2.=
I created a user temp_user and added temp_user to temp_group.=0A> =0A> 3. =
During replication, the group was replicated first and I got an error 32=0A=
> (NO_SUCH_OBJECT) when it tried to modify the memberOf of the temp_user ob=
ject=0A> (This does not exist in the readslave yet).=0A> =0A> 4. The temp_u=
ser object was replicated next, and as you see, querying it does=0A> show a=
memberOf attribute, proving that this field was replicated. (Note that=0A>=
I have run OpenLDAP with debug as -1 and verified that the replicated data=
has=0A> the memberOf field in it. I can provide this too if needed.)=0A=0A=
I see. The current code drops the memberOf attribute if it was not explicit=
ly requested in the search. However, by default the consumer requests "+" w=
hich means "all operational attributes" and so slapd considers memberOf to =
have been requested. We need to reconsider this aspect of the design.=0A> =
=0A> 5. The more serious problem occurs when the sequence is reversed and t=
he group=0A> has been deleted as a last operation - The user is replicated =
first, but since=0A> the group is deleted, it is never replicated and a sta=
le memberOf entry stays=0A> with the user.=0A=0A--=A0 -- Howard Chu=0A=A0 =
CTO, Symas Corp.=A0 =A0 =A0 =A0 =A0 http://www.symas.com=0A=A0 Director, Hi=
ghland Sun=A0 =A0 http://highlandsun.com/hyc/=0A=A0 Chief Architect, OpenLD=
AP=A0 http://www.openldap.org/project/
--1251447850-1386810138-1349783312=:99375
Content-Type: text/html; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable
<html><body><div style=3D"color:#000; background-color:#fff; font-family:ti=
mes new roman, new york, times, serif;font-size:12pt"><div style=3D"font-fa=
mily: 'times new roman', 'new york', times, serif; font-size: 12pt; "><span=
>=0A=0A=0A=0A=0A=0A=0A=0A<div class=3D"p1">Hi,</div></span></div><div><div>=
To try and overcome this issue, we tried two fixes:</div><div><br></div><di=
v>1. Every time a user was deleted from a group, we force-updated the user&=
nbsp;</div><div>object manually to make sure its entryCSN got updated and i=
t got </div><div>replicated properly. This is an expensive operation a=
nd did not scale </div><div>well for big group sizes (10-20k), and did=
not work out.</div><div><br></div><div>2. We then tried to do the same thi=
ng in OpenLDAP. We noticed in the </div><div>memberof.c commits that t=
here were a couple of patches to force the </div><div>entryCSN of the =
user object to get updated. (http://tinyurl.com/8k4qrdj </div><div>and=
http://tinyurl.com/9akqgfl)These have since been reverted because of =
</div><div>access log and some replication issues, but for us, speed was a =
higher </div><div>priority. I reapplied these patches back to the code=
. This solved
the </div><div>member-of replication issue, but we noticed that occas=
ionally under a </div><div>heavy load, there was a sudden surge in Ope=
nLDAP's memory usage going up </div><div>to whatever memory was availa=
ble and finally crashing.</div><div><br></div><div>We have gone back to opt=
ion (1) though (2) would be the preferred option.</div><div><br></div><div>=
Any help on figuring out why (2) caused the memory bloat would be really&nb=
sp;</div><div>great. I can provide more details/memory traces if needed.</d=
iv><div><br></div><div>We will be glad to contribute any fixes once we are =
able to nail down </div><div>the issue.</div><div><br></div><div>Thank=
s,</div><div>Arunkumar</div></div> <div style=3D"font-family: 'times new r=
oman', 'new york', times, serif; font-size: 12pt; "> <div style=3D"font-fam=
ily: 'times new roman', 'new york', times, serif; font-size: 12pt; "> <div =
dir=3D"ltr"> <font size=3D"2" face=3D"Arial"> <hr size=3D"1"> <b><span
style=3D"font-weight:bold;">From:</span></b> Howard Chu <hyc(a)symas.com&=
gt;<br> <b><span style=3D"font-weight: bold;">To:</span></b> arun s <aru=
nkumar_1123(a)yahoo.com> <br><b><span style=3D"font-weight: bold;">Cc:</sp=
an></b> "openldap-its(a)openldap.org" <openldap-its(a)openldap.org> <br> =
<b><span style=3D"font-weight: bold;">Sent:</span></b> Monday, 1 October 20=
12 6:59 PM<br> <b><span style=3D"font-weight: bold;">Subject:</span></b> Re=
: (ITS#7400) Memberof and Syncrepl incompatibility<br> </font> </div> <br>a=
run s wrote:<br>> Hi,<br>> Yes, I am able to reproduce the issue with=
2.4.32<br>> <br>> Making sense of the logs for the exact reproductio=
n is hard since it needs a<br>> lot of operations in a short time. But t=
his will probably help:<br>> <br>> 1. At the start of the test, the g=
roup temp_group existed.<br>> <br>> 2. I created a user temp_user and=
added temp_user to temp_group.<br>> <br>> 3. During replication, the
group was replicated first and I got an error 32<br>> (NO_SUCH_OBJECT) =
when it tried to modify the memberOf of the temp_user object<br>> (This =
does not exist in the readslave yet).<br>> <br>> 4. The temp_user obj=
ect was replicated next, and as you see, querying it does<br>> show a me=
mberOf attribute, proving that this field was replicated. (Note that<br>>=
; I have run OpenLDAP with debug as -1 and verified that the replicated dat=
a has<br>> the memberOf field in it. I can provide this too if needed.)<=
br><br>I see. The current code drops the memberOf attribute if it was not e=
xplicitly requested in the search. However, by default the consumer request=
s "+" which means "all operational attributes" and so slapd considers membe=
rOf to have been requested. We need to reconsider this aspect of the design=
.<br>> <br>> 5. The more serious problem occurs when the sequence is =
reversed and the group<br>> has been deleted as a last operation
- The user is replicated first, but since<br>> the group is deleted, it=
is never replicated and a stale memberOf entry stays<br>> with the user=
.<br><br>-- -- Howard Chu<br> CTO, Symas Corp. &n=
bsp; <a href=3D"http://www.symas.com/" target=3D"_blank">htt=
p://www.symas.com</a><br> Director, Highland Sun <a hre=
f=3D"http://highlandsun.com/hyc/" target=3D"_blank">http://highlandsun.com/=
hyc/</a><br> Chief Architect, OpenLDAP <a href=3D"http://www.op=
enldap.org/project/" target=3D"_blank">http://www.openldap.org/project/</a>=
<br><br><br> </div> </div> </div></body></html>
--1251447850-1386810138-1349783312=:99375--
10 years, 11 months
Re: (ITS#7367) [PATCH] MozNSS: update list of supported cipher suites
by jvcelak＠redhat.com
> >> I will note that Mandriva, at least, continually updates the version
> >> of OpenLDAP they ship, unlike most distributions, so it definitely
> >> isn't all. And my point is, Red Hat could do better, and I'd like to
> >> see them do better. I'd like to see Debian/Ubuntu do better too.
> >> I.e., this isn't specific to Red Hat, but the discussion here is about
> >> Red Hat, and what it can do. I discuss Debian and what it can do
> >> better with the Debian devs on their openldap dev list.
> >
> > Then I'd like to hear what Jan and the other Red Hat OpenLDAP
> > maintainers have to say.
As for Fedora, I usualy rebase the package immediately when the new upstream
version is available. And I do it in the stable and newer releases (for today
it would be stable F17, alpha F18, and Rawhide). And if there is a problem in
older Fedora, I just pick the patch.
So we are definitelly not behind with Fedora -- the last submitted updates can
be found here: https://admin.fedoraproject.org/updates/search/openldap
As for RHEL it is more complicated, the rebases are allowed rarely. We cannot
afford any API/ABI breakage and regressions. and I can imagine that a new
feature in OpenLDAP changes the current behavior or affects some other
component.
I can remind a bug which appeared after rebase to 2.4.31 in Fedora and we do
not want to cause a similar problems in RHEL:
https://bugzilla.redhat.com/show_bug.cgi?id=771484
http://www.openldap.org/its/index.cgi?findid=7167
I would like to see newer OpenLDAP in RHEL because it would be easier for me
to maintain it. But I'm not sure if the advantages will outweight the
disadvantages and if our customers will benefit from it rather than be forced
to deal with new problems.
> Ok. One thing I do with Debian is help triage issues that are reported
> there with the upstream ITS system if the issues do not appear to be due to
> the usage of an old version. If there is a simple way to do that with Red
> Hat, I could help there as well.
When there is a new bug report, I usualy try to reproduce with the package
from RHEL or Fedora. And then with the newest OpenLDAP from git master. If I'm
able to reproduce, I always create a report in your ITS.
Jan
10 years, 11 months