openldap-bugs April 2013

openldap-bugs@openldap.org

27 participants
76 discussions

(ITS#7565) unacceptable growth on delete with back-mdb
by quanah＠OpenLDAP.org 05 Apr '13

05 Apr '13

Full_Name: Quanah Gibson-Mount Version: 2.4.35 OS: Linux 2.6 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (75.111.39.181) With back-mdb, if you add a large number of users to the database (via ldapadd), and then delete them (via ldapmodify), the DB explodes in size. After the initial ldapadd of 30 million users, the DB is 17GB in size. After deleting the 30 million users, the DB is 52GB in size. This is a 3.05 times larger than the DBs initial size. With back-hdb, there is no change in size of the DB (it is static). Clearly this is not a desirable state of affairs. This appears to be a bug with the freelist. After the add is complete, we have: zimbra@zre-ldap001:~/data/ldap/mdb/db$ mdb_stat -f . Freelist Status Tree depth: 3 Branch pages: 19 Leaf pages: 2007 Overflow pages: 5155 Entries: 15879 Free pages: 2612121 My delete run is not yet complete, but it is now: zimbra@zre-ldap001:~/data/ldap/mdb/db$ mdb_stat -f . Freelist Status Tree depth: 3 Branch pages: 99 Leaf pages: 11096 Overflow pages: 20891 Entries: 82932 Free pages: 9381096 (and growing) --Quanah

1 0

Re: (ITS#7563) slapd modifies attribute value of pwdAttribute
by michael＠stroeder.com 05 Apr '13

05 Apr '13

This is a cryptographically signed message in MIME format. --------------ms070208010706030608090702 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Howard, please try yourself first before answering. Dieter is right here. Example: dn: cn=3DDefault Password Policy,ou=3DPolicies,dc=3Dstroeder,dc=3Dde changetype: modify delete: pwdAttribute pwdAttribute: userPassword - add: pwdAttribute pwdAttribute: 2.5.4.35 - Reading it: dn: cn=3DDefault Password Policy,ou=3DPolicies,dc=3Dstroeder,dc=3Dde cn: Default Password Policy [..] pwdAttribute: userPassword [..] I vaguely remember I had to implement a work-around in web2ldap to deal w= ith that when generating delta modification data when the user edits such an = entry. Ciao, Michael. --------------ms070208010706030608090702 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIILHzCC BT8wggQnoAMCAQICDwCmSwABAAIAivjZQ8SBvzANBgkqhkiG9w0BAQUFADB8MQswCQYDVQQG EwJERTEcMBoGA1UEChMTVEMgVHJ1c3RDZW50ZXIgR21iSDElMCMGA1UECxMcVEMgVHJ1c3RD ZW50ZXIgQ2xhc3MgMSBMMSBDQTEoMCYGA1UEAxMfVEMgVHJ1c3RDZW50ZXIgQ2xhc3MgMSBM MSBDQSBJWDAeFw0xMjA2MDYxOTAyMTZaFw0xMzA2MDcxOTAyMTZaMCgxCzAJBgNVBAYTAkRF MRkwFwYDVQQDDBBNaWNoYWVsIFN0csO2ZGVyMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIB CgKCAQEAxXZGav40rnGNLxEggBW94MILWHlfC8a23Jew5U1gPlfRTXOjjzmoaZ1uCyGdgF6M VvuO9T1aTQNGH+OdeGe3P7Tfc/NsLJFJ2wtd8blvhmodUgse2eypiWjNOd4gZuhalBhgsQ0K b5D6/1foghII4E264iZlJ7AJ+UYcO+GxvFWT0YMTbLckgDkZk7c3qwTozdhYvXarvqx+8Ou/ kuxpQQhac/ebzxpu0N+RHSf2KIUS0g0tEGnPtGv6iL+9QNHc4JKo9Y9KKVw3tQy+Re+FQLxB 1fPE5F+qxuD3AUENpOwkMsqWLM94ohtx3CFqLpxfUPrnKFLAHOhHEbByYGvFPwIDAQABo4IC EDCCAgwwgaUGCCsGAQUFBwEBBIGYMIGVMFEGCCsGAQUFBzAChkVodHRwOi8vd3d3LnRydXN0 Y2VudGVyLmRlL2NlcnRzZXJ2aWNlcy9jYWNlcnRzL3RjX2NsYXNzMV9MMV9DQV9JWC5jcnQw QAYIKwYBBQUHMAGGNGh0dHA6Ly9vY3NwLml4LnRjY2xhc3MxLnRjdW5pdmVyc2FsLWkudHJ1 c3RjZW50ZXIuZGUwHwYDVR0jBBgwFoAU6bgoHUbP/M34TpvF7ktg69g7P9EwDAYDVR0TAQH/ BAIwADBKBgNVHSAEQzBBMD8GCSqCFAAsAQEBATAyMDAGCCsGAQUFBwIBFiRodHRwOi8vd3d3 LnRydXN0Y2VudGVyLmRlL2d1aWRlbGluZXMwDgYDVR0PAQH/BAQDAgTwMB0GA1UdDgQWBBS2 KAWfTfgJ/JQ63qLGwTXYLnI+LzBiBgNVHR8EWzBZMFegVaBThlFodHRwOi8vY3JsLml4LnRj Y2xhc3MxLnRjdW5pdmVyc2FsLWkudHJ1c3RjZW50ZXIuZGUvY3JsL3YyL3RjX0NsYXNzMV9M MV9DQV9JWC5jcmwwMwYDVR0lBCwwKgYIKwYBBQUHAwIGCCsGAQUFBwMEBggrBgEFBQcDBwYK KwYBBAGCNxQCAjAfBgNVHREEGDAWgRRtaWNoYWVsQHN0cm9lZGVyLmNvbTANBgkqhkiG9w0B AQUFAAOCAQEAQ3bvVUpEq+cQrLpcogyt5BJNk/WvUvOHqhzyj28M9pg9hcDl1+MYl5qqj6tR GSTLPQZyf287pcmbMwbcTGZO/gbW9v7RYcut6RauWdwKMCUmKC3J4fVfDq9ZETA2WOV68ef4 B3Gzdhghsbp3Rhp5dDmrCVKAHlafm6ZwJrEQ9P76fxnQZzRLgeKpZep5ePH5YHUB3+YaOQvJ FG0bOXvfHhRiRG7/HW2G+yDgjHSxDz8AFzMWL/RFePqZ4pn6T/SM/qU6WEpW39MWyJNoH/Kx QDYK8gGYuesn1ciMCTnjrvZQj0fonGTO4SfWekJRkuGrJ7dYSZRjYbDcWBBkdFLWzzCCBdgw ggTAoAMCAQICDgboAAEAAkqWLSQM/sXJMA0GCSqGSIb3DQEBBQUAMHkxCzAJBgNVBAYTAkRF MRwwGgYDVQQKExNUQyBUcnVzdENlbnRlciBHbWJIMSQwIgYDVQQLExtUQyBUcnVzdENlbnRl ciBVbml2ZXJzYWwgQ0ExJjAkBgNVBAMTHVRDIFRydXN0Q2VudGVyIFVuaXZlcnNhbCBDQSBJ MB4XDTA5MTEwMzE0MDgxOVoXDTI1MTIzMTIxNTk1OVowfDELMAkGA1UEBhMCREUxHDAaBgNV BAoTE1RDIFRydXN0Q2VudGVyIEdtYkgxJTAjBgNVBAsTHFRDIFRydXN0Q2VudGVyIENsYXNz IDEgTDEgQ0ExKDAmBgNVBAMTH1RDIFRydXN0Q2VudGVyIENsYXNzIDEgTDEgQ0EgSVgwggEi MA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC75pBuz2Lp6QuqthDVR+V8XSsncZpozVVt 5KLv5P7yemMRwleKyH3PjmYfZUVL64Biab1GjovFblqVGCrep/EfdRonq20yU+P7TVhiLP8Z 5cegDZotIYhZhM0d8cPIij6w5d4IJM/8QCy6QSOUu4ASiTVItoYE4AFPjLqpmPwcie0fiqHH hpgmHnJla/7PZdkMZEsaCfVDEWBmJuMzVprJPT40anjG5VBLyM2I5DlsUCaeQCy2O3w3sqf1 3dyzUcv03IICuNc63towXA31Qt0TaVNU6YAmQjMepdfMbspmCZ+G8D2+xophEPPR/1vkstst smUMqX0XrLonTUJczglPAgMBAAGjggJZMIICVTCBmgYIKwYBBQUHAQEEgY0wgYowUgYIKwYB BQUHMAKGRmh0dHA6Ly93d3cudHJ1c3RjZW50ZXIuZGUvY2VydHNlcnZpY2VzL2NhY2VydHMv dGNfdW5pdmVyc2FsX3Jvb3RfSS5jcnQwNAYIKwYBBQUHMAGGKGh0dHA6Ly9vY3NwLnRjdW5p dmVyc2FsLUkudHJ1c3RjZW50ZXIuZGUwHwYDVR0jBBgwFoAUkqR1LKSevoFE63n8isWVpesQ dXMwEgYDVR0TAQH/BAgwBgEB/wIBADBSBgNVHSAESzBJMAYGBFUdIAAwPwYJKoIUACwBAQEB MDIwMAYIKwYBBQUHAgEWJGh0dHA6Ly93d3cudHJ1c3RjZW50ZXIuZGUvZ3VpZGVsaW5lczAO BgNVHQ8BAf8EBAMCAQYwHQYDVR0OBBYEFOm4KB1Gz/zN+E6bxe5LYOvYOz/RMIH9BgNVHR8E gfUwgfIwge+ggeyggemGRmh0dHA6Ly9jcmwudGN1bml2ZXJzYWwtSS50cnVzdGNlbnRlci5k ZS9jcmwvdjIvdGNfdW5pdmVyc2FsX3Jvb3RfSS5jcmyGgZ5sZGFwOi8vd3d3LnRydXN0Y2Vu dGVyLmRlL0NOPVRDJTIwVHJ1c3RDZW50ZXIlMjBVbml2ZXJzYWwlMjBDQSUyMEksTz1UQyUy MFRydXN0Q2VudGVyJTIwR21iSCxPVT1yb290Y2VydHMsREM9dHJ1c3RjZW50ZXIsREM9ZGU/ Y2VydGlmaWNhdGVSZXZvY2F0aW9uTGlzdD9iYXNlPzANBgkqhkiG9w0BAQUFAAOCAQEAOcjE m+6+mO5Icm+N53G2DpCM07LBFSGoRpBoX0oE8TrJaIQh2KXmBHVdn9LU8kt3QzLclctgvwJV 0KwcsMUUl5tlCsMPpR3s2Ek5lbWpvvr0HqtW56blAQiINV9nBd1EJFASIkRjefGbV2nOq9Yz UU+N8HA7jq1ROhd/NZZraGhjthwKyfjfHV7PKxGlY+3M0MbTIG+q/GhIfm0euDpFqhKG88e9 ALXr/uoSn3MzeOcoOWjTpW3adtFO4VWVgKbgG7jNrFbvRVlHmFLbOm4msjE5aXWxLiTwpJ2X iF4zKca1vAdAOgw9us90jEtOeiH6GzjNxEMvb7TfeO6Zkuc6HDGCA84wggPKAgEBMIGPMHwx CzAJBgNVBAYTAkRFMRwwGgYDVQQKExNUQyBUcnVzdENlbnRlciBHbWJIMSUwIwYDVQQLExxU QyBUcnVzdENlbnRlciBDbGFzcyAxIEwxIENBMSgwJgYDVQQDEx9UQyBUcnVzdENlbnRlciBD bGFzcyAxIEwxIENBIElYAg8ApksAAQACAIr42UPEgb8wCQYFKw4DAhoFAKCCAhMwGAYJKoZI hvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMTMwNDA1MTYzODUwWjAjBgkq hkiG9w0BCQQxFgQUHxv/K+bf1DIc3hHVPnUWsdJ/pvEwbAYJKoZIhvcNAQkPMV8wXTALBglg hkgBZQMEASowCwYJYIZIAWUDBAECMAoGCCqGSIb3DQMHMA4GCCqGSIb3DQMCAgIAgDANBggq hkiG9w0DAgIBQDAHBgUrDgMCBzANBggqhkiG9w0DAgIBKDCBoAYJKwYBBAGCNxAEMYGSMIGP MHwxCzAJBgNVBAYTAkRFMRwwGgYDVQQKExNUQyBUcnVzdENlbnRlciBHbWJIMSUwIwYDVQQL ExxUQyBUcnVzdENlbnRlciBDbGFzcyAxIEwxIENBMSgwJgYDVQQDEx9UQyBUcnVzdENlbnRl ciBDbGFzcyAxIEwxIENBIElYAg8ApksAAQACAIr42UPEgb8wgaIGCyqGSIb3DQEJEAILMYGS oIGPMHwxCzAJBgNVBAYTAkRFMRwwGgYDVQQKExNUQyBUcnVzdENlbnRlciBHbWJIMSUwIwYD VQQLExxUQyBUcnVzdENlbnRlciBDbGFzcyAxIEwxIENBMSgwJgYDVQQDEx9UQyBUcnVzdENl bnRlciBDbGFzcyAxIEwxIENBIElYAg8ApksAAQACAIr42UPEgb8wDQYJKoZIhvcNAQEBBQAE ggEAFn6FSF038qyMGUYl1VtsoCpXsoQU8AP1E9C6x5iOg3kEK/0hXv0OZq7XuAdDZCryRBxj CzUs25YPHdGANcXOJkNpq8EvYFiIgUFZj3jrc8bl2qFs0bm9YN955CsQIXkG32wVs8QcpTup EnuTssA2/zNePYZLpUKiOjyuEiD0EzfiThJA2UlrFzXGWxSIN+wTfv7hJuIpNttYiKo19PQf ITI6C2+LtiGMnnZ2AREgTgRhkJyrYPGwOFoEKmuwo7f6GKnA1WMa8+LEuceQXvGL9YObolq0 PVd6W+GRxWhXixQMkEk5Jwi7XV3tyRkV45c5ZA79ZiD/ldjV/k+2HcSF7gAAAAAAAA== --------------ms070208010706030608090702--

1 0

(ITS#7564) LMDB library bug: reopening LMDB may fail, fixed-address mmap
by tamas.kenez＠kishonti.net 05 Apr '13

05 Apr '13

Full_Name: Tamás Kenéz Version: LMDB 8eef7a42 OS: windows 7 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (82.141.191.242) Using the LMDB library (commit 8eef7a4275eda) on windows 7: To reproduce: Run mtest multiple times. The test sometimes fails with the with the windows error code 487 (ERROR_INVALID_ADDRESS) in mdb_env_open -> mdb_env_open2 at this line env->me_map = MapViewOfFileEx(mh, flags & MDB_WRITEMAP ? FILE_MAP_WRITE : FILE_MAP_READ, 0, 0, env->me_mapsize, meta.mm_address); The reason is that when windows fails to map at the specified address (meta.mm_address) it returns 0. I tried this quick workaround: if ( !env->me_map ) env->me_map = MapViewOfFileEx(mh, flags & MDB_WRITEMAP ? FILE_MAP_WRITE : FILE_MAP_READ, 0, 0, env->me_mapsize, NULL); which does return valid address, but then it fails later here: else if (meta.mm_address && env->me_map != meta.mm_address) { /* Can happen because the address argument to mmap() is just a * hint. mmap() can pick another, e.g. if the range is in use. * The MAP_FIXED flag would prevent that, but then mmap could * instead unmap existing pages to make room for the new map. */ return EBUSY; /* TODO: Make a new MDB_* error code? */

1 0

Re: (ITS#7515) Nested liblmdb transaction bugs
by hyc＠symas.com 05 Apr '13

05 Apr '13

h.b.furuseth(a)usit.uio.no wrote: > I wrote: >>> Can't use a DBI from an aborted child txn, unlike aborted main txn: >> (...) >> Well, closing the DBI in mdb_txn_abort() caused problems. A fix >> in the opposite direction is easy, if we want DBIs to work that >> way: mdb_env_open() can open the DBI in any parent txns too. > > To get more consistent behavior, if that wasn't clear: Any parent > txn will receive the DBI independently of commit/abort, just like > the MDB_env. Future children of a preexisting txn still differ, > which is probably for the best. Changing that would take locking. > After much discussion, this part has been reverted. The rationale is simple: nothing created inside a TXN may be visible outside the TXN until the TXN successfully commits. On an abort or failed commit, all DBIs opened in the TXN are automatically closed, because technically they never existed. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7563) slapd modifies attribute value of pwdAttribute
by hyc＠symas.com 05 Apr '13

05 Apr '13

dieter(a)dkluenter.de wrote: > Full_Name: > Version: 2.4.33 > OS: openSuSE-12.3-x86_64 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (91.65.235.202) > > > the pwdAttribute type requires a syntax of 1.3.6.1.4.1.1466.115.121.1.38, > according to man slapo-ppolicy and ppolicy.schema. > when adding a policy, the value of pwdAttribute gets changend from OID 2.5.4.35 > to userPassword. You are mistaken. slapd never changes this attribute from what the user stored. > In a replicated system syncrepl complaints about > syncrepl_message_to_entry: rid=001 mods check (pwdAttribute: value #0 invalid > per syntax) do_syncrepl: rid=001 rc 21 retrying. This error will go away if you configure the ppolicy overlay on the consumer. Closing this ITS. > > -Dieter > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#7563) slapd modifies attribute value of pwdAttribute
by dieter＠dkluenter.de 05 Apr '13

05 Apr '13

Full_Name: Version: 2.4.33 OS: openSuSE-12.3-x86_64 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (91.65.235.202) the pwdAttribute type requires a syntax of 1.3.6.1.4.1.1466.115.121.1.38, according to man slapo-ppolicy and ppolicy.schema. when adding a policy, the value of pwdAttribute gets changend from OID 2.5.4.35 to userPassword. In a replicated system syncrepl complaints about syncrepl_message_to_entry: rid=001 mods check (pwdAttribute: value #0 invalid per syntax) do_syncrepl: rid=001 rc 21 retrying. -Dieter

1 0

(ITS#7562) Remove deprecated slapd.replog(5) reference in manpages
by fumiyas＠osstech.co.jp 04 Apr '13

04 Apr '13

Full_Name: SATOH Fumiyasu Version: master OS: URL: https://gist.github.com/fumiyas/5316077 Submission from: (NULL) (219.111.105.97) Some manpages refer deprecated slapd.replog(5).

1 0

(ITS#7561) mdb_drop bugs
by h.b.furuseth＠usit.uio.no 04 Apr '13

04 Apr '13

Full_Name: Hallvard B Furuseth Version: mdb.master 8eef7a4275eda8f2fa2e0d1e67c1d5cbcd91607e OS: Linux x86_64 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (193.69.163.163) Submitted by: hallvard mdb_drop() does not seem to return overflow pages to the freelist. Does not call cursor_del or anything, nor 'ovpages' loop like it has. E.g. the enclosed program which writes some pages and then drops the DB shows 59 free of 62 useed pages with non-overflows, but just 3 free of 38-39 with overflows. #include <lmdb.h> #include <assert.h> #include <stdio.h> #include <stdlib.h> int main(void) { MDB_env *env; MDB_txn *txn; MDB_dbi dbi; int rc, sz, sum; # define DBPATH "test.mdb" # define E(e) ((void) (rc = (e)), assert(!rc)) for (sz = 1500; sz < 9000; sz *= 2) { MDB_val key = {sizeof(sum), &sum}, data = {sz, NULL}; printf("======== datasize %d ========\n", sz); fflush(NULL); remove(DBPATH); E(mdb_env_create(&env)); E(mdb_env_set_maxdbs(env, 1)); E(mdb_env_open(env, DBPATH, MDB_NOSUBDIR, 0666)); E(mdb_txn_begin(env, 0, 0, &txn)); { E(mdb_dbi_open(txn, "foo", MDB_CREATE, &dbi)); for (sum = 0; (sum += sz) < 100000; ) E(mdb_put(txn, dbi, &key, &data, MDB_RESERVE)); E(mdb_txn_commit(txn)); } E(mdb_txn_begin(env, 0, 0, &txn)); { E(mdb_drop(txn, dbi, 1)); E(mdb_txn_commit(txn)); } mdb_env_close(env); system("../mdb_stat -naef " DBPATH " | grep -v ': 0$'"); } return 0; }

1 0

Re: (ITS#7447) backsql and german umlaute
by masarati＠aero.polimi.it 04 Apr '13

04 Apr '13

On 04/04/2013 08:13 AM, tomas.novosad(a)linuxbox.cz wrote: > Hello, > > i got exactly same problem. > > Only the discussed character is different ;-)). > When ThunderBird (or ldapsearch, it doesnt matter) send search query to LDAP with some UTF-8 character, > the result query to DB (PGSQL in this case) is like > (upper(last_name) LIKE '%šEV%') > > where the search parameter is: > %<lower case utf8 character>EV% > > obviously backsql does not correctly handle UTF8 characters. > > I can't find any way how to avoid this. > If only back-sql would leave the upper case conversion on DB - like > this: > (upper(last_name) LIKE upper('%šEV%'T The solution is to augment table ldap_attr_mappings (with non-trivial implications on DN searching and matching) with a field that specifies the encoding for a particular attribute, and convert back and forth any time an operation affects those attributes. Not trivial, but contributions are welcome. p. -- Pierangelo Masarati Associate Professor Dipartimento di Scienze e Tecnologie Aerospaziali Politecnico di Milano

1 0

(ITS#7447) backsql and german umlaute
by tomas.novosad＠linuxbox.cz 03 Apr '13

03 Apr '13

Hello, i got exactly same problem. Only the discussed character is different ;-)). When ThunderBird (or ldapsearch, it doesnt matter) send search query to LDAP with some UTF-8 character, the result query to DB (PGSQL in this case) is like (upper(last_name) LIKE '%šEV%') where the search parameter is: %<lower case utf8 character>EV% obviously backsql does not correctly handle UTF8 characters. I can't find any way how to avoid this. If only back-sql would leave the upper case conversion on DB - like this: (upper(last_name) LIKE upper('%šEV%')) or use ILIKE Anyone has any suggestion how to workaround this? Thanks in advance -- Tomáš Novosad

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs April 2013