- openldap-bugs - openldap.org

(ITS#7867) TLS SHA512
by yann+openldap＠verry.org 31 May '14

31 May '14

Full_Name: Yann Verry Version: 2.4.39 OS: debian/sid URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (2a01:e35:2e6d:c800:d572:aec:1b42:380c) Hi, I would like (CACert sign class3 now with SHA512) to switch my X509 certificate with a signature algorithm SHA512. When I do this openldap bind on SSL port but was unable to provide SSL connection as you can see in error: 2014-05-31T00:20:46.852821+02:00 peach slapd[23997]: >>> slap_listener(ldaps:///) 2014-05-31T00:20:46.853369+02:00 peach slapd[23997]: connection_get(35): got connid=1068 2014-05-31T00:20:46.853891+02:00 peach slapd[23997]: connection_read(35): checking for input on id=1068 2014-05-31T00:20:46.854526+02:00 peach slapd[23997]: connection_read(35): TLS accept failure error=-1 id=1068, closing 2014-05-31T00:20:46.855071+02:00 peach slapd[23997]: connection_close: conn=1068 sd=35 If I fall back to sha256 it works fine How to reproduce ================ - generate self signed with sha256 and sha512: mkdir -p /etc/ldap/ssl cd !$ # priv certtool --generate-privkey --sec-param normal --outfile mypriv_normal.key # self certtool -s --load-privkey mypriv_normal.key --outfile gnutls512_normal.crt --hash SHA512 certtool -s --load-privkey mypriv_normal.key --outfile gnutls256_normal.crt --hash SHA256 # build PEM cat mypriv_normal.key gnutls512_normal.crt > gnutls512_normal.pem cat mypriv_normal.key gnutls256_normal.crt > gnutls256_normal.pem my cn=config: olcTLSCACertificateFile: /etc/ldap/ssl/sslcertificate.pem olcTLSCertificateFile: /etc/ldap/ssl/sslcertificate.pem olcTLSCertificateKeyFile: /etc/ldap/ssl/sslcertificate.pem now just play with symlink. sha256 ------ ln -s gnutls256_normal.pem sslcertificate.pem ; then restart openldap make a client connection: gnutls-cli ldap.verry.org -p 636 Resolving 'ldap.verry.org'... Connecting to '2a01:e35:2e6d:c800:cafe:deca:0:42:636'... - Certificate type: X.509 - Got a certificate list of 1 certificates. - Certificate[0] info: - subject `CN=ldap.verry.org', issuer `CN=ldap.verry.org', RSA key 2432 bits, signed using RSA-SHA256, activated `2014-05-31 08:26:59 UTC', expires `2024-05-28 08:27:03 UTC', SHA-1 fingerprint `600b2a502289644c075d4b3eaf7b1efd38685687' - The hostname in the certificate matches 'ldap.verry.org'. - Peer's certificate issuer is unknown - Peer's certificate is NOT trusted - Version: TLS1.2 - Key Exchange: RSA - Cipher: AES-128-CBC - MAC: SHA1 - Compression: NULL - Handshake was completed - Simple Client Mode: server view, it's OK: 538909bf conn=1007 fd=32 TLS established tls_ssf=256 ssf=256 sha512 ------ rm previous symlink and ln -s gnutls512_normal.pem sslcertificate.pem ; then restart openldap make a connection: gnutls-cli ldap.verry.org -p 636 Resolving 'ldap.verry.org'... Connecting to '2a01:e35:2e6d:c800:cafe:deca:0:42:636'... *** Fatal error: A TLS packet with unexpected length was received. *** Handshake has failed GnuTLS error: A TLS packet with unexpected length was received. server view: TLS: can't accept: Could not negotiate a supported cipher suite.. 538909f9 connection_read(28): TLS accept failure error=-1 id=1000, closing 538909f9 connection_closing: readying conn=1000 sd=28 for close 538909f9 connection_close: conn=1000 sd=28 538909f9 daemon: removing 28 538909f9 conn=1000 fd=28 closed (TLS negotiation failure) gnutls ====== gnutls-cli -l|grep SHA512 MACs: SHA1, MD5, SHA256, SHA384, SHA512, SHA224, UMAC-96, UMAC-128, AEAD Digests: SHA1, MD5, SHA256, SHA384, SHA512, SHA224 PK-signatures: SIGN-RSA-SHA1, SIGN-RSA-SHA1, SIGN-RSA-SHA224, SIGN-RSA-SHA256, SIGN-RSA-SHA384, SIGN-RSA-SHA512, SIGN-RSA-RMD160, SIGN-DSA-SHA1, SIGN-DSA-SHA1, SIGN-DSA-SHA224, SIGN-DSA-SHA256, SIGN-RSA-MD5, SIGN-RSA-MD5, SIGN-RSA-MD2, SIGN-ECDSA-SHA1, SIGN-ECDSA-SHA224, SIGN-ECDSA-SHA256, SIGN-ECDSA-SHA384, SIGN-ECDSA-SHA512 I can provide more information as needed to solve this issue Regards, Yann

1 0

Re: (ITS#7866) [PATCH] Mention default database in tools' man pages
by hyc＠symas.com 30 May '14

30 May '14

jsynacek(a)redhat.com wrote: > Full_Name: Jan Synacek > Version: 2.4.39 > OS: > URL: http://jsynacek.fedorapeople.org/openldap/jsynacek-20140530-0001-Mention-de… > Submission from: (NULL) (209.132.186.34) > > > Many of the slaptools use -b to specify a database. However, there is no mention > (apart from section 10.2.1. "The slapadd program" in the admin guide) in the > documentation that if there are multiple databases in the configuration, the > first one is used when no -b is given. The provided patch tries to improve the > situation by providing an explicit mention of the default database. Seriously? In 15 years no one else has ever remarked on this, why is this even worth discussing? At any rate, the added text is incorrect. Technically, "the first database in the configuration that supports the requested operation" is used. E.g., if back-monitor is the first DB in the config, it is skipped. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#7866) [PATCH] Mention default database in tools' man pages
by jsynacek＠redhat.com 30 May '14

30 May '14

Full_Name: Jan Synacek Version: 2.4.39 OS: URL: http://jsynacek.fedorapeople.org/openldap/jsynacek-20140530-0001-Mention-de… Submission from: (NULL) (209.132.186.34) Many of the slaptools use -b to specify a database. However, there is no mention (apart from section 10.2.1. "The slapadd program" in the admin guide) in the documentation that if there are multiple databases in the configuration, the first one is used when no -b is given. The provided patch tries to improve the situation by providing an explicit mention of the default database.

1 0

Re: (ITS#7863) Invalid DN syntax (34) not written by slapo-accesslog
by pierangelo.masarati＠polimi.it 30 May '14

30 May '14

On 05/30/2014 01:33 AM, hyc(a)symas.com wrote: > pierangelo.masarati(a)polimi.it wrote: >> It now comes to my mind that another perhaps less intrusive chance of >> intervention could be to act at the *response* level. Think of: >> >> - the possibility to stack code in the response chain (not necessarily >> overlays) >> >> - have a way to understand if the response originates *before* the >> frontend was called >> >> In this case, the custom code could re-parse the request to re-detect >> why it failed and handle it (e.g. log custom information on failure >> reason). Not a piece of cake, but probably less intrusive than other >> options. (I'm not sure the raw request buffer is still available at >> this stage, though.) > > Still not something useful for slapo-accesslog - we're storing log info as > LDAP attributes. We can't store reqDN if the DN has invalid syntax. We can't > store reqMod values if the attributetype is unknown or the values are invalid. > When a request fails validation it's literally garbage, and toxic - cannot be > considered safe to preserve in a DB. Sure. Indeed, it could not be associated with accesslog (invalid DN needs to be detected way before a database can be selected). Such a case would belong to some "debugging" layer (e.g. client-server interaction troubleshooting), which BTW I'm convinced the current logging is more than enough for. p. -- Pierangelo Masarati Associate Professor Dipartimento di Scienze e Tecnologie Aerospaziali Politecnico di Milano

1 0

Re: (ITS#7863) Invalid DN syntax (34) not written by slapo-accesslog
by hyc＠symas.com 29 May '14

29 May '14

pierangelo.masarati(a)polimi.it wrote: > It now comes to my mind that another perhaps less intrusive chance of > intervention could be to act at the *response* level. Think of: > > - the possibility to stack code in the response chain (not necessarily > overlays) > > - have a way to understand if the response originates *before* the > frontend was called > > In this case, the custom code could re-parse the request to re-detect > why it failed and handle it (e.g. log custom information on failure > reason). Not a piece of cake, but probably less intrusive than other > options. (I'm not sure the raw request buffer is still available at > this stage, though.) Still not something useful for slapo-accesslog - we're storing log info as LDAP attributes. We can't store reqDN if the DN has invalid syntax. We can't store reqMod values if the attributetype is unknown or the values are invalid. When a request fails validation it's literally garbage, and toxic - cannot be considered safe to preserve in a DB. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7863) Invalid DN syntax (34) not written by slapo-accesslog
by pierangelo.masarati＠polimi.it 29 May '14

29 May '14

On 05/29/2014 09:31 PM, pierangelo.masarati(a)polimi.it wrote: > On 05/29/2014 06:13 PM, michael(a)stroeder.com wrote: >> Howard Chu wrote: >>> michael(a)stroeder.com wrote: >>>> It seems that modify requests which failed due to Invalid DN syntax (34) are >>>> not >>>> written to accesslog-DB. I guess that those requests get abandoned by the >>>> frontend and never reach the backend at all. >>> >>> Correct. >>> >>>> It would be handy to see the invalid modify request in the accesslog-DB though. >>>> >>>> Any chance to achieve this? >>> >>> Not likely. The frontend must call select_backend() based on the incoming DN >>> to determine which backend to invoke, and thus which stack of overlays are >>> involved. If the DN is invalid, no selection can occur. >> >> Hmm, I've done some more tests. Invalid syntax (21) also does not make it >> beyond the frontend into accesslog-DB. >> >> I have no clear opinion on this. Of course the current behaviour is good for >> performance. But sometimes one would like to observe what broken LDAP clients >> sent in a modify request in the past. >> >> Also running with BER loglevel or breaking up the TLS connection with stunnel >> and sniff with Wireshark is not always an option. >> >> Having this configurable would be great. >> >> What's your opinion on this? > > In terms of code architecture, overlays have been designed to work on > validated data (the request must be well formed). To this end, the > notion of "frontend" database was added to make it possible to layer an > overlay *before* backend selection. > > To have a custom piece of code muck with a request before it is > validated (or even before it is parsed) you need to split the code into: > > - parsing, without doing any sanity check > > - allow custom code to step in > > - do validation afterwards > > This does not necessarily imply performance penalty (when no custom code > steps in, the amount of operations would be the same for sane requests; > in case of malformed requests, you would miss the opportunity to bail > out on the first inconsistency). > > However, it would imply significant refactorization of the current code, > without (IMHO) a strong motivation for it: malformed requests are > already handled in terms of response to clients, and somehow logged. > > An (expensive) alternative would be to register a handler at connection > level which duplicates the parsing to anticipate checking for errors > that would be detected before passing control to the frontend. Not the > way I'd go, unless strictly necessary. > > A paranoid but intriguing alternative would be to implement an operation > mode that keeps parsing after the first error is detected, turning the > operation into a no-op. Again, lots of refactorization and error > handling and so on (for example, what if a syntax, an attribute > description or a matching rule is not found? You'd always have to check > that the corresponding pointer is not NULL before using it, whereas > right now we often assume that if execution got there, data must be > consistent. Of course, we could introduce dummy syntaxes > "NonExistingSyntax", ..., but we'd probably need to define their > semantics in such a way that they apply to every case we need to handle > and so). It now comes to my mind that another perhaps less intrusive chance of intervention could be to act at the *response* level. Think of: - the possibility to stack code in the response chain (not necessarily overlays) - have a way to understand if the response originates *before* the frontend was called In this case, the custom code could re-parse the request to re-detect why it failed and handle it (e.g. log custom information on failure reason). Not a piece of cake, but probably less intrusive than other options. (I'm not sure the raw request buffer is still available at this stage, though.) p. -- Pierangelo Masarati Associate Professor Dipartimento di Scienze e Tecnologie Aerospaziali Politecnico di Milano

1 0

Re: (ITS#7863) Invalid DN syntax (34) not written by slapo-accesslog
by pierangelo.masarati＠polimi.it 29 May '14

29 May '14

On 05/29/2014 06:13 PM, michael(a)stroeder.com wrote: > Howard Chu wrote: >> michael(a)stroeder.com wrote: >>> It seems that modify requests which failed due to Invalid DN syntax (34) are >>> not >>> written to accesslog-DB. I guess that those requests get abandoned by the >>> frontend and never reach the backend at all. >> >> Correct. >> >>> It would be handy to see the invalid modify request in the accesslog-DB though. >>> >>> Any chance to achieve this? >> >> Not likely. The frontend must call select_backend() based on the incoming DN >> to determine which backend to invoke, and thus which stack of overlays are >> involved. If the DN is invalid, no selection can occur. > > Hmm, I've done some more tests. Invalid syntax (21) also does not make it > beyond the frontend into accesslog-DB. > > I have no clear opinion on this. Of course the current behaviour is good for > performance. But sometimes one would like to observe what broken LDAP clients > sent in a modify request in the past. > > Also running with BER loglevel or breaking up the TLS connection with stunnel > and sniff with Wireshark is not always an option. > > Having this configurable would be great. > > What's your opinion on this? In terms of code architecture, overlays have been designed to work on validated data (the request must be well formed). To this end, the notion of "frontend" database was added to make it possible to layer an overlay *before* backend selection. To have a custom piece of code muck with a request before it is validated (or even before it is parsed) you need to split the code into: - parsing, without doing any sanity check - allow custom code to step in - do validation afterwards This does not necessarily imply performance penalty (when no custom code steps in, the amount of operations would be the same for sane requests; in case of malformed requests, you would miss the opportunity to bail out on the first inconsistency). However, it would imply significant refactorization of the current code, without (IMHO) a strong motivation for it: malformed requests are already handled in terms of response to clients, and somehow logged. An (expensive) alternative would be to register a handler at connection level which duplicates the parsing to anticipate checking for errors that would be detected before passing control to the frontend. Not the way I'd go, unless strictly necessary. A paranoid but intriguing alternative would be to implement an operation mode that keeps parsing after the first error is detected, turning the operation into a no-op. Again, lots of refactorization and error handling and so on (for example, what if a syntax, an attribute description or a matching rule is not found? You'd always have to check that the corresponding pointer is not NULL before using it, whereas right now we often assume that if execution got there, data must be consistent. Of course, we could introduce dummy syntaxes "NonExistingSyntax", ..., but we'd probably need to define their semantics in such a way that they apply to every case we need to handle and so). My 2c. p. -- Pierangelo Masarati Associate Professor Dipartimento di Scienze e Tecnologie Aerospaziali Politecnico di Milano

1 0

Re: (ITS#7863) Invalid DN syntax (34) not written by slapo-accesslog
by michael＠stroeder.com 29 May '14

29 May '14

Howard Chu wrote: > michael(a)stroeder.com wrote: >> It seems that modify requests which failed due to Invalid DN syntax (34) are >> not >> written to accesslog-DB. I guess that those requests get abandoned by the >> frontend and never reach the backend at all. > > Correct. > >> It would be handy to see the invalid modify request in the accesslog-DB though. >> >> Any chance to achieve this? > > Not likely. The frontend must call select_backend() based on the incoming DN > to determine which backend to invoke, and thus which stack of overlays are > involved. If the DN is invalid, no selection can occur. Hmm, I've done some more tests. Invalid syntax (21) also does not make it beyond the frontend into accesslog-DB. I have no clear opinion on this. Of course the current behaviour is good for performance. But sometimes one would like to observe what broken LDAP clients sent in a modify request in the past. Also running with BER loglevel or breaking up the TLS connection with stunnel and sniff with Wireshark is not always an option. Having this configurable would be great. What's your opinion on this? Ciao, Michael.

1 0

(ITS#7865) man slapo-refint extended with olc-example
by wandel＠b1-systems.de 29 May '14

29 May '14

Full_Name: Michael Wandel Version: 2.4.39 OS: Linux URL: ftp://ftp.openldap.org/incoming/wandel-140529-0001-Manpage-enhancement-for-… Submission from: (NULL) (217.86.177.164) The man pag of the refint Overlay handle only the slapd.conf variant. I add an example for using the conf Backend.

1 0

Re: (ITS#7864) slapo-translucent should honour the manageDSAit control
by michael＠stroeder.com 29 May '14

29 May '14

This is a cryptographically signed message in MIME format. --------------ms080107050802050209040602 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable hyc(a)symas.com wrote: > michael(a)stroeder.com wrote: >> This is a cryptographically signed message in MIME format. >=20 >> jkozyra(a)gmail.com wrote: >>> ManageDSAit is honoured by other overlays like dynlist, retcode, and = un=3D >> ique. The >>> translucent overlay should add this functionality. >> >> Can your please elaborate on the semantics you want to have when using= >> ManageDSAit control. >> >> Originally ManageDSAit control was only meant to manage referral entri= es.=3D >=20 >> It was (ab)used for some other purposes especially *before* Relax Rule= s >> control was defined/implemented. >=20 > In particular, the control changes search behavior to return the actual= entry=20 > for a referral entry, instead of returning a searchReference. In the ot= her=20 > overlays, e.g. dynlist, the control changes search behavior to return t= he=20 > actual entry, instead of trying to expand the list of dynamic members. >=20 > For translucent, the control should return the actual local entry, inst= ead of=20 > trying to return the result of merging local and remote data. I thought the Proxy cache privateDB control (1.3.6.1.4.1.4203.666.11.9.5.= 1) should be used for that. CIao, Michael. --------------ms080107050802050209040602 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIFfzCC BXswggNjoAMCAQICAwxOfTANBgkqhkiG9w0BAQUFADB5MRAwDgYDVQQKEwdSb290IENBMR4w HAYDVQQLExVodHRwOi8vd3d3LmNhY2VydC5vcmcxIjAgBgNVBAMTGUNBIENlcnQgU2lnbmlu ZyBBdXRob3JpdHkxITAfBgkqhkiG9w0BCQEWEnN1cHBvcnRAY2FjZXJ0Lm9yZzAeFw0xMjEw MDIyMDE3MDlaFw0xNDEwMDIyMDE3MDlaMD8xGDAWBgNVBAMUD01pY2hhZWwgU3Ry9mRlcjEj MCEGCSqGSIb3DQEJARYUbWljaGFlbEBzdHJvZWRlci5jb20wggEiMA0GCSqGSIb3DQEBAQUA A4IBDwAwggEKAoIBAQDo2SKth5GhtaDrCyfGtyUG+/hAAa/J52L0NFN4SSRvTtdGf9HfWwwd NCtgae0TVGWk2lKDbXA9d5vmyIiRhuwxd90H6FLErhRBeB9G67qtw87E8WUoXt2DwPQEUTWV hqHpPadlmgFw3+i3TGQQTe3O3W9MMMd4GJNhObem2VGRuCD37OXnzBksTcq0FPJgcWAhe3d/ 0ItOkNWBqgq8Mf3p7WFBhaQ0a27BC/mKtH8fI3kPcS305imPRja69Msq3EwUZBc9ToVp6FRQ NYKjfOBybDUzVkmRZl3H8xutQP2w8Zxb8m5f7Q1BfLLrIFScfYvIDgOERxTCd4lab8+/09XH AgMBAAGjggFEMIIBQDAMBgNVHRMBAf8EAjAAMFYGCWCGSAGG+EIBDQRJFkdUbyBnZXQgeW91 ciBvd24gY2VydGlmaWNhdGUgZm9yIEZSRUUgaGVhZCBvdmVyIHRvIGh0dHA6Ly93d3cuQ0Fj ZXJ0Lm9yZzAOBgNVHQ8BAf8EBAMCA6gwQAYDVR0lBDkwNwYIKwYBBQUHAwQGCCsGAQUFBwMC BgorBgEEAYI3CgMEBgorBgEEAYI3CgMDBglghkgBhvhCBAEwMgYIKwYBBQUHAQEEJjAkMCIG CCsGAQUFBzABhhZodHRwOi8vb2NzcC5jYWNlcnQub3JnMDEGA1UdHwQqMCgwJqAkoCKGIGh0 dHA6Ly9jcmwuY2FjZXJ0Lm9yZy9yZXZva2UuY3JsMB8GA1UdEQQYMBaBFG1pY2hhZWxAc3Ry b2VkZXIuY29tMA0GCSqGSIb3DQEBBQUAA4ICAQC9ouXq3p/bDWMM4tBKgD3tl4HY5H0eECl8 q9/nqk0UL6YeWkrCiQdrDtNPW7DcGqNYtzdgtzmyTr1GhiAX+igrOjdk/ge5NRcQOpONK/4b zrmpQEcIUyxSSDKLWh211/kcFfxxLEiJ5teF4GL8Fc1qbrLP4+DCvJXWfYaaR5NLjZMqm2VP yKTv3qpXWnGohiRkGTwS/11QM2XCfIGdRsQT9a8mO4m2fn2tGPp2TEIoCLrDDrbGVeDWaOWB OIeTrp4wa3Q4OI6yCptJhEqKvjhV96IBRYgM76nTBqsqnDzwxExAyhhWiUS5DunRHOr/+NyF pUpD4883RBLO0g9kUEGOhtZNF1u+8zEL0YgMGvifAom9JEklLOXZuqj0MThypKs/3d/OyOQb 4gURnu6oZwcKZ7LskytWnlRKUxF6o0A8grtmyKkqe14TS7cQbg0NTaIYXPkHR+dfFmb3uEqn BBjvpJXFcEtWI2lQXC/ET+au991pK797ExBOmpQwjIn3SjiW80vw/UoL6DMvqY/6JhVhyNTP MJ2W5AX5kc27DIbVtVGZs8J4AYhuNALJUq9N9Ka7rPRj3RcYDrfehDLOkM5iMnarpmtuOpLK d1SvZhqj/0N/JWGIDpPSTkTFOPP6ZN9I9Rqyf+9NGqb2sjo4DkIiZcHxt735/GJLwus5KLBl 2DGCA6EwggOdAgEBMIGAMHkxEDAOBgNVBAoTB1Jvb3QgQ0ExHjAcBgNVBAsTFWh0dHA6Ly93 d3cuY2FjZXJ0Lm9yZzEiMCAGA1UEAxMZQ0EgQ2VydCBTaWduaW5nIEF1dGhvcml0eTEhMB8G CSqGSIb3DQEJARYSc3VwcG9ydEBjYWNlcnQub3JnAgMMTn0wCQYFKw4DAhoFAKCCAfUwGAYJ KoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMTQwNTI5MTM1MzM5WjAj BgkqhkiG9w0BCQQxFgQUHudXKkMwDWUJD1r/NhU+1GGT2vQwbAYJKoZIhvcNAQkPMV8wXTAL BglghkgBZQMEASowCwYJYIZIAWUDBAECMAoGCCqGSIb3DQMHMA4GCCqGSIb3DQMCAgIAgDAN BggqhkiG9w0DAgIBQDAHBgUrDgMCBzANBggqhkiG9w0DAgIBKDCBkQYJKwYBBAGCNxAEMYGD MIGAMHkxEDAOBgNVBAoTB1Jvb3QgQ0ExHjAcBgNVBAsTFWh0dHA6Ly93d3cuY2FjZXJ0Lm9y ZzEiMCAGA1UEAxMZQ0EgQ2VydCBTaWduaW5nIEF1dGhvcml0eTEhMB8GCSqGSIb3DQEJARYS c3VwcG9ydEBjYWNlcnQub3JnAgMMTn0wgZMGCyqGSIb3DQEJEAILMYGDoIGAMHkxEDAOBgNV BAoTB1Jvb3QgQ0ExHjAcBgNVBAsTFWh0dHA6Ly93d3cuY2FjZXJ0Lm9yZzEiMCAGA1UEAxMZ Q0EgQ2VydCBTaWduaW5nIEF1dGhvcml0eTEhMB8GCSqGSIb3DQEJARYSc3VwcG9ydEBjYWNl cnQub3JnAgMMTn0wDQYJKoZIhvcNAQEBBQAEggEAdkiNkhBAipUQ1XBLmnIEzrcs+AC5V2WE 0MiMn7MmAL2072PBLWnF6Bei2CdM3F9E608yBBM9seU1FQsxXf2fnzUd3PUqVvuQpFvbiJVG sDgggrDV2FZocvkYcBLhgBhEaUJ2zSVHofqWfWScgEz9jwUFbq2n8W9NpkBpbSMG+JdUAq6u stXKZ+oXybqFvwUybP/0qPfjvFOFF/w7wOUdk7heDUNsMXMydil0fptIIOop9r29fLqK3fLo tFHB9QNG7PTy1ZoBh/5WM6891WSWtJm1RWvstb8XIMEvg2MC2a3+au2I0lPX7UwNO3Bglw7F D3cADleduhU/HrUmNItKvQAAAAAAAA== --------------ms080107050802050209040602--

1 0

Re: (ITS#7864) slapo-translucent should honour the manageDSAit control
by hyc＠symas.com 29 May '14

29 May '14

michael(a)stroeder.com wrote: > This is a cryptographically signed message in MIME format. > jkozyra(a)gmail.com wrote: >> ManageDSAit is honoured by other overlays like dynlist, retcode, and un= > ique. The >> translucent overlay should add this functionality. > > Can your please elaborate on the semantics you want to have when using > ManageDSAit control. > > Originally ManageDSAit control was only meant to manage referral entries.= > It was (ab)used for some other purposes especially *before* Relax Rules > control was defined/implemented. In particular, the control changes search behavior to return the actual entry for a referral entry, instead of returning a searchReference. In the other overlays, e.g. dynlist, the control changes search behavior to return the actual entry, instead of trying to expand the list of dynamic members. For translucent, the control should return the actual local entry, instead of trying to return the result of merging local and remote data. The use case is the same as for the other overlays - make sure we get actual entries as they are stored in the server, as opposed to some dynamically transformed version of the entries. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7864) slapo-translucent should honour the manageDSAit control
by michael＠stroeder.com 29 May '14

29 May '14

This is a cryptographically signed message in MIME format. --------------ms060706000808070704030900 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable jkozyra(a)gmail.com wrote: > ManageDSAit is honoured by other overlays like dynlist, retcode, and un= ique. The > translucent overlay should add this functionality. Can your please elaborate on the semantics you want to have when using ManageDSAit control. Originally ManageDSAit control was only meant to manage referral entries.= It was (ab)used for some other purposes especially *before* Relax Rules control was defined/implemented. Ciao, Michael. --------------ms060706000808070704030900 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIFfzCC BXswggNjoAMCAQICAwxOfTANBgkqhkiG9w0BAQUFADB5MRAwDgYDVQQKEwdSb290IENBMR4w HAYDVQQLExVodHRwOi8vd3d3LmNhY2VydC5vcmcxIjAgBgNVBAMTGUNBIENlcnQgU2lnbmlu ZyBBdXRob3JpdHkxITAfBgkqhkiG9w0BCQEWEnN1cHBvcnRAY2FjZXJ0Lm9yZzAeFw0xMjEw MDIyMDE3MDlaFw0xNDEwMDIyMDE3MDlaMD8xGDAWBgNVBAMUD01pY2hhZWwgU3Ry9mRlcjEj MCEGCSqGSIb3DQEJARYUbWljaGFlbEBzdHJvZWRlci5jb20wggEiMA0GCSqGSIb3DQEBAQUA A4IBDwAwggEKAoIBAQDo2SKth5GhtaDrCyfGtyUG+/hAAa/J52L0NFN4SSRvTtdGf9HfWwwd NCtgae0TVGWk2lKDbXA9d5vmyIiRhuwxd90H6FLErhRBeB9G67qtw87E8WUoXt2DwPQEUTWV hqHpPadlmgFw3+i3TGQQTe3O3W9MMMd4GJNhObem2VGRuCD37OXnzBksTcq0FPJgcWAhe3d/ 0ItOkNWBqgq8Mf3p7WFBhaQ0a27BC/mKtH8fI3kPcS305imPRja69Msq3EwUZBc9ToVp6FRQ NYKjfOBybDUzVkmRZl3H8xutQP2w8Zxb8m5f7Q1BfLLrIFScfYvIDgOERxTCd4lab8+/09XH AgMBAAGjggFEMIIBQDAMBgNVHRMBAf8EAjAAMFYGCWCGSAGG+EIBDQRJFkdUbyBnZXQgeW91 ciBvd24gY2VydGlmaWNhdGUgZm9yIEZSRUUgaGVhZCBvdmVyIHRvIGh0dHA6Ly93d3cuQ0Fj ZXJ0Lm9yZzAOBgNVHQ8BAf8EBAMCA6gwQAYDVR0lBDkwNwYIKwYBBQUHAwQGCCsGAQUFBwMC BgorBgEEAYI3CgMEBgorBgEEAYI3CgMDBglghkgBhvhCBAEwMgYIKwYBBQUHAQEEJjAkMCIG CCsGAQUFBzABhhZodHRwOi8vb2NzcC5jYWNlcnQub3JnMDEGA1UdHwQqMCgwJqAkoCKGIGh0 dHA6Ly9jcmwuY2FjZXJ0Lm9yZy9yZXZva2UuY3JsMB8GA1UdEQQYMBaBFG1pY2hhZWxAc3Ry b2VkZXIuY29tMA0GCSqGSIb3DQEBBQUAA4ICAQC9ouXq3p/bDWMM4tBKgD3tl4HY5H0eECl8 q9/nqk0UL6YeWkrCiQdrDtNPW7DcGqNYtzdgtzmyTr1GhiAX+igrOjdk/ge5NRcQOpONK/4b zrmpQEcIUyxSSDKLWh211/kcFfxxLEiJ5teF4GL8Fc1qbrLP4+DCvJXWfYaaR5NLjZMqm2VP yKTv3qpXWnGohiRkGTwS/11QM2XCfIGdRsQT9a8mO4m2fn2tGPp2TEIoCLrDDrbGVeDWaOWB OIeTrp4wa3Q4OI6yCptJhEqKvjhV96IBRYgM76nTBqsqnDzwxExAyhhWiUS5DunRHOr/+NyF pUpD4883RBLO0g9kUEGOhtZNF1u+8zEL0YgMGvifAom9JEklLOXZuqj0MThypKs/3d/OyOQb 4gURnu6oZwcKZ7LskytWnlRKUxF6o0A8grtmyKkqe14TS7cQbg0NTaIYXPkHR+dfFmb3uEqn BBjvpJXFcEtWI2lQXC/ET+au991pK797ExBOmpQwjIn3SjiW80vw/UoL6DMvqY/6JhVhyNTP MJ2W5AX5kc27DIbVtVGZs8J4AYhuNALJUq9N9Ka7rPRj3RcYDrfehDLOkM5iMnarpmtuOpLK d1SvZhqj/0N/JWGIDpPSTkTFOPP6ZN9I9Rqyf+9NGqb2sjo4DkIiZcHxt735/GJLwus5KLBl 2DGCA6EwggOdAgEBMIGAMHkxEDAOBgNVBAoTB1Jvb3QgQ0ExHjAcBgNVBAsTFWh0dHA6Ly93 d3cuY2FjZXJ0Lm9yZzEiMCAGA1UEAxMZQ0EgQ2VydCBTaWduaW5nIEF1dGhvcml0eTEhMB8G CSqGSIb3DQEJARYSc3VwcG9ydEBjYWNlcnQub3JnAgMMTn0wCQYFKw4DAhoFAKCCAfUwGAYJ KoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMTQwNTI5MTMyNzQ4WjAj BgkqhkiG9w0BCQQxFgQUJIQoNlQKKPgKvV8M4xsguOfuEdowbAYJKoZIhvcNAQkPMV8wXTAL BglghkgBZQMEASowCwYJYIZIAWUDBAECMAoGCCqGSIb3DQMHMA4GCCqGSIb3DQMCAgIAgDAN BggqhkiG9w0DAgIBQDAHBgUrDgMCBzANBggqhkiG9w0DAgIBKDCBkQYJKwYBBAGCNxAEMYGD MIGAMHkxEDAOBgNVBAoTB1Jvb3QgQ0ExHjAcBgNVBAsTFWh0dHA6Ly93d3cuY2FjZXJ0Lm9y ZzEiMCAGA1UEAxMZQ0EgQ2VydCBTaWduaW5nIEF1dGhvcml0eTEhMB8GCSqGSIb3DQEJARYS c3VwcG9ydEBjYWNlcnQub3JnAgMMTn0wgZMGCyqGSIb3DQEJEAILMYGDoIGAMHkxEDAOBgNV BAoTB1Jvb3QgQ0ExHjAcBgNVBAsTFWh0dHA6Ly93d3cuY2FjZXJ0Lm9yZzEiMCAGA1UEAxMZ Q0EgQ2VydCBTaWduaW5nIEF1dGhvcml0eTEhMB8GCSqGSIb3DQEJARYSc3VwcG9ydEBjYWNl cnQub3JnAgMMTn0wDQYJKoZIhvcNAQEBBQAEggEABbo93P/6Q2VvX6dwtrFbLYvubeU/Z7OE BzNu7KxeeDNYKW5l9Lht8IyMtaAg/Mb48EK+IsZw0TGmbLSty4kzDxjXc+ELAi8/S5lAF+Rt 5F+aNXZmMpSuMdes/Ei0mJtf6lmzTiGJNOiOMkxHkutL4AbV+kkbfaydK/yQTa9ZV6fL8nJU IqvRm63excncCpKsc75kTzVwOERtkMvOz6BvI9N+C07hFTI8Oi+S+Ibc8OR1zUGvPYJrGEzx At+EH69piRP4PEeIOXHGkU7ILDhCQN8ucta+eeTISxZMMI6xs4YBdUcnjhpuViVyLVRAPEG9 q3sD8vmGwPvMgEgYU/3TiwAAAAAAAA== --------------ms060706000808070704030900--

1 0

(ITS#7864) slapo-translucent should honour the manageDSAit control
by jkozyra＠gmail.com 29 May '14

29 May '14

Full_Name: Jillian Kozyra Version: 2.4.39 OS: URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (78.155.234.81) ManageDSAit is honoured by other overlays like dynlist, retcode, and unique. The translucent overlay should add this functionality.

1 0

Re: (ITS#7862) slapd crash on modifications after host hard reset (in mdb)
by hyc＠symas.com 29 May '14

29 May '14

Michael.Levitin(a)billing.ru wrote: > --_000_6D4FA1DFD570AD4CBACDA9117C4A2125024BF1C541SRVEXCHMBX1bi_ > Content-Type: text/plain; charset="us-ascii" > Content-Transfer-Encoding: quoted-printable > > "What health status? The mdb_stat command will return info about the DB." > mdb_stat returns segmentation fault. Guess it's not sufficient info. > The question is - is there any method to check file consistence ? ( i.e. fi= > le structure is not corrupted during power reset.) No. If you want protection from power reset, you must use regular synchronous writes. > Thanks. > > Mikhail > > > > --_000_6D4FA1DFD570AD4CBACDA9117C4A2125024BF1C541SRVEXCHMBX1bi_ > Content-Type: text/html; charset="us-ascii" > Content-Transfer-Encoding: quoted-printable > > <html> > <head> > <meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii"= >> > <meta name=3D"Generator" content=3D"Microsoft Exchange Server"> >  > <style></style> > </head> > <body> > <font face=3D"Calibri" size=3D"2"><span style=3D"font-size:11pt;"> > <div>“What health status? The mdb_stat command will return info about= > the DB. “</div> > <div>mdb_stat returns segmentation fault. Guess it's not sufficient info.</= > div> > <div>The question is – is there any method to check file consistence = > ? ( i.e. file structure is not corrupted during power reset.)</div> > <div> </div> > <div>Thanks.</div> > <div> </div> > <div>Mikhail</div> > <div> </div> > <div> </div> > </span></font> > </body> > </html> > > --_000_6D4FA1DFD570AD4CBACDA9117C4A2125024BF1C541SRVEXCHMBX1bi_-- > > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7862) slapd crash on modifications after host hard reset (in mdb)
by Michael.Levitin＠billing.ru 29 May '14

29 May '14

--_000_6D4FA1DFD570AD4CBACDA9117C4A2125024BF1C541SRVEXCHMBX1bi_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable "What health status? The mdb_stat command will return info about the DB. " mdb_stat returns segmentation fault. Guess it's not sufficient info. The question is - is there any method to check file consistence ? ( i.e. fi= le structure is not corrupted during power reset.) Thanks. Mikhail --_000_6D4FA1DFD570AD4CBACDA9117C4A2125024BF1C541SRVEXCHMBX1bi_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable <html> <head> <meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii"= > <meta name=3D"Generator" content=3D"Microsoft Exchange Server">  <style></style> </head> <body> <font face=3D"Calibri" size=3D"2"><span style=3D"font-size:11pt;"> <div>“What health status? The mdb_stat command will return info about= the DB. “</div> <div>mdb_stat returns segmentation fault. Guess it's not sufficient info.</= div> <div>The question is – is there any method to check file consistence = ? ( i.e. file structure is not corrupted during power reset.)</div> <div> </div> <div>Thanks.</div> <div> </div> <div>Mikhail</div> <div> </div> <div> </div> </span></font> </body> </html> --_000_6D4FA1DFD570AD4CBACDA9117C4A2125024BF1C541SRVEXCHMBX1bi_--

1 0

Re: (ITS#7861) Wrong entry count with MDB_DUPSORT databases
by hbf＠mail.uio.no 29 May '14

29 May '14

> mdb_cursor_put() now tracks data items instead of keys in > md_entries, broken by 13d6d59c512065db3741c20c65b4d9928b986968. oops, that was backwards. Changed to tracking keys from data items.

1 0

Re: (ITS#7863) Invalid DN syntax (34) not written by slapo-accesslog
by hyc＠symas.com 29 May '14

29 May '14

michael(a)stroeder.com wrote: > Full_Name: Michael Ströder > Version: HEAD and RE24 > OS: Linux > URL: > Submission from: (NULL) (212.227.35.93) > > > It seems that modify requests which failed due to Invalid DN syntax (34) are not > written to accesslog-DB. I guess that those requests get abandoned by the > frontend and never reach the backend at all. Correct. > It would be handy to see the invalid modify request in the accesslog-DB though. > > Any chance to achieve this? Not likely. The frontend must call select_backend() based on the incoming DN to determine which backend to invoke, and thus which stack of overlays are involved. If the DN is invalid, no selection can occur. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7861) Wrong entry count with MDB_DUPSORT databases
by hbf＠mail.uio.no 29 May '14

29 May '14

More: mdb_cursor_put() now tracks data items instead of keys in md_entries, broken by 13d6d59c512065db3741c20c65b4d9928b986968. mdb_cursor_del(, MDB_NODUPDATA) is off-by-one for sub-DBs: It subtracts subDB.md_entries, then mdb_cursor_del0() subtracts 1. Fixing.

1 0

(ITS#7863) Invalid DN syntax (34) not written by slapo-accesslog
by michael＠stroeder.com 28 May '14

28 May '14

Full_Name: Michael Ströder Version: HEAD and RE24 OS: Linux URL: Submission from: (NULL) (212.227.35.93) It seems that modify requests which failed due to Invalid DN syntax (34) are not written to accesslog-DB. I guess that those requests get abandoned by the frontend and never reach the backend at all. It would be handy to see the invalid modify request in the accesslog-DB though. Any chance to achieve this?

1 0

Re: (ITS#7862) slapd crash on modifications after host hard reset (in mdb)
by hyc＠symas.com 28 May '14

28 May '14

Vadim Prozorov wrote: > Excuse me, where I can read about mapasync limitations on Linux? > http://symas.com/mdb/doc/group__mdb__env.html#gab034ed0d8e5938090aef5ee0997… > contains no such info. Seems to me I've missed some important info about > configuration. MAPASYNC uses msync(... MS_ASYNC). Google for this with Linux will give you results like so: http://www.spinics.net/lists/linux-mm/msg71244.html > > And in addition - is there way to check DB health status without running slapd? What health status? The mdb_stat command will return info about the DB. > > > 2014-05-28 15:02 GMT+04:00 Howard Chu <hyc(a)symas.com <mailto:hyc@symas.com>>: > > Vadim Prozorov wrote: > > Did you have dbnosync configured on this backend? Looks like > garbage data > got written to the freeDB. > > > No, DB config section is: > # define MDB parameters > backendmdb > databasemdb > maxsize20000000000 > envflagswritemap mapasync > # end define > > > Unfortunately, in Linux mapasync is a no-op. It seems then that your DB is > quite corrupted. > > > -- > -- Howard Chu > CTO, Symas Corp. http://www.symas.com > Director, Highland Sun http://highlandsun.com/hyc/ > Chief Architect, OpenLDAP http://www.openldap.org/__project/ > <http://www.openldap.org/project/> > > > > > -- > С уважением, > Вадим Прозоров. > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7862) slapd crash on modifications after host hard reset (in mdb)
by vadius＠vadius.ru 28 May '14

28 May '14

--f46d044288025e7ce404fa740f82 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Excuse me, where I can read about mapasync limitations on Linux? http://symas.com/mdb/doc/group__mdb__env.html#gab034ed0d8e5938090aef5ee0997= f7e94contains no such info. Seems to me I've missed some important info about configuration. And in addition - is there way to check DB health status without running slapd? 2014-05-28 15:02 GMT+04:00 Howard Chu <hyc(a)symas.com>: > Vadim Prozorov wrote: > >> Did you have dbnosync configured on this backend? Looks like garbage >> data >> got written to the freeDB. >> >> >> No, DB config section is: >> # define MDB parameters >> backendmdb >> databasemdb >> maxsize20000000000 >> envflagswritemap mapasync >> # end define >> > > Unfortunately, in Linux mapasync is a no-op. It seems then that your DB i= s > quite corrupted. > > > -- > -- Howard Chu > CTO, Symas Corp. http://www.symas.com > Director, Highland Sun http://highlandsun.com/hyc/ > Chief Architect, OpenLDAP http://www.openldap.org/project/ > --=20 =D0=A1 =D1=83=D0=B2=D0=B0=D0=B6=D0=B5=D0=BD=D0=B8=D0=B5=D0=BC, =D0=92=D0=B0=D0=B4=D0=B8=D0=BC =D0=9F=D1=80=D0=BE=D0=B7=D0=BE=D1=80=D0=BE= =D0=B2. --f46d044288025e7ce404fa740f82 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr">Excuse me, where I can read about mapasync limitations on = Linux?<div><a href=3D"http://symas.com/mdb/doc/group__mdb__env.html#gab034e= d0d8e5938090aef5ee0997f7e94">http://symas.com/mdb/doc/group__mdb__env.html#= gab034ed0d8e5938090aef5ee0997f7e94</a> contains no such info. Seems to me I= 've missed some important info about configuration.<br> </div><div><br></div><div>And in addition - is there way to check DB health= status without running slapd?</div></div><div class=3D"gmail_extra"><br><b= r><div class=3D"gmail_quote">2014-05-28 15:02 GMT+04:00 Howard Chu <span di= r=3D"ltr"><<a href=3D"mailto:hyc@symas.com" target=3D"_blank">hyc(a)symas.= com</a>></span>:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"><div class=3D"">Vadim Prozorov wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> =C2=A0 =C2=A0 Did you have dbnosync configured on this backend? Looks like = garbage data<br> =C2=A0 =C2=A0 got written to the freeDB.<br> <br> <br> No, DB config section is:<br> # define MDB parameters<br> backendmdb<br> databasemdb<br> maxsize20000000000<br> envflagswritemap mapasync<br> # end define<br> </blockquote> <br></div> Unfortunately, in Linux mapasync is a no-op. It seems then that your DB is = quite corrupted.<div class=3D"HOEnZb"><div class=3D"h5"><br> <br> -- <br> =C2=A0 -- Howard Chu<br> =C2=A0 CTO, Symas Corp. =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 <a href=3D"http:= //www.symas.com" target=3D"_blank">http://www.symas.com</a><br> =C2=A0 Director, Highland Sun =C2=A0 =C2=A0 <a href=3D"http://highlandsun.c= om/hyc/" target=3D"_blank">http://highlandsun.com/hyc/</a><br> =C2=A0 Chief Architect, OpenLDAP =C2=A0<a href=3D"http://www.openldap.org/p= roject/" target=3D"_blank">http://www.openldap.org/<u></u>project/</a><br> </div></div></blockquote></div><br><br clear=3D"all"><div><br></div>-- <br>= <div>=D0=A1 =D1=83=D0=B2=D0=B0=D0=B6=D0=B5=D0=BD=D0=B8=D0=B5=D0=BC,<br>=D0= =92=D0=B0=D0=B4=D0=B8=D0=BC =D0=9F=D1=80=D0=BE=D0=B7=D0=BE=D1=80=D0=BE=D0= =B2.</div><div><br></div> </div> --f46d044288025e7ce404fa740f82--

1 0

Re: (ITS#7862) slapd crash on modifications after host hard reset (in mdb)
by hyc＠symas.com 28 May '14

28 May '14

Vadim Prozorov wrote: > Did you have dbnosync configured on this backend? Looks like garbage data > got written to the freeDB. > > > No, DB config section is: > # define MDB parameters > backendmdb > databasemdb > maxsize20000000000 > envflagswritemap mapasync > # end define Unfortunately, in Linux mapasync is a no-op. It seems then that your DB is quite corrupted. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Fwd: (ITS#7862) slapd crash on modifications after host hard reset (in mdb)
by vadius＠vadius.ru 28 May '14

28 May '14

--047d7b5d285813733e04fa73805f Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable 2014-05-28 13:47 GMT+04:00 Howard Chu <hyc(a)symas.com>: vadius(a)vadius.ru wrote: > >> Full_Name: Vadim Prozorov >> Version: 2.4.38 >> OS: RHEL 6.3 >> URL: ftp://ftp.openldap.org/incoming/ >> Submission from: (NULL) (91.210.4.1) >> >> >> Hello, >> >> I'm using slapd (2.4.38 release) built from sources on RHEL 6.3 64bit. >> Build params: >> LD_LIBRARY_PATH=3D"/usr/lib:/usr/local/lib:/opt/local/BerkeleyDB.5.3/lib= " >> LDFLAGS=3D"-L/usr/local/lib -L/opt/local/BerkeleyDB.5.3/lib" >> CPPFLAGS=3D"-I/usr/local/include -I/opt/local/BerkeleyDB.5.3/include" >> ./configure >> --prefix=3D/opt/local/OpenLDAP --enable-dynacl --enable-ldap >> =E2=80=94enable-overlays >> --disable-static --enable-dynamic >> >> slapd works in master-master sync mode on host with big amount of RAM >> (whole DB >> can fit into RAM). >> One time host was hardly restarted. After restart slapd has started, but >> crash >> on any modification (at least, from ext application; may be form replica= , >> haven't check yet). There is always the same crash stack: >> > > Did you have dbnosync configured on this backend? Looks like garbage data > got written to the freeDB. > No, DB config section is: # define MDB parameters backend mdb database mdb maxsize 20000000000 envflags writemap mapasync # end define > > (gdb) bt >> #0 mdb_xcursor_init1 (mc=3D0x7f3e1d6bbd40, node=3D0x7f3f99f46f82) at >> ./../../../libraries/liblmdb/mdb.c:6585 >> #1 0x000000000049f8eb in mdb_cursor_set (mc=3D0x7f3e1d6bbd40, >> key=3D0x7f3e1d6bbef0, >> data=3D0x0, op=3DMDB_SET_RANGE, exactp=3D<value optimized out>) at >> ./../../../libraries/liblmdb/mdb.c:5329 >> #2 0x00000000004a019f in mdb_cursor_get (mc=3D0x7f3e1d6bbd40, >> key=3D0x7f3e1d6bbef0, >> data=3D0x0, op=3D<value optimized out>) at ./../../../libraries/liblmdb/ >> mdb.c:5529 >> #3 0x000000000049e0ed in mdb_page_alloc (mc=3D<value optimized out>, nu= m=3D1, >> mp=3D0x7f3e1d6bbf68) at ./../../../libraries/liblmdb/mdb.c:1727 >> #4 0x000000000049e469 in mdb_page_touch (mc=3D0x7f3e1d6bc1d0) at >> ./../../../libraries/liblmdb/mdb.c:1911 >> #5 0x000000000049eb3a in mdb_page_search (mc=3D0x7f3e1d6bc1d0, key=3D0x= 0, >> flags=3D5) >> at ./../../../libraries/liblmdb/mdb.c:4830 >> #6 0x00000000004a7404 in mdb_freelist_save (txn=3D0x7f3e10115130) at >> ./../../../libraries/liblmdb/mdb.c:2522 >> #7 mdb_txn_commit (txn=3D0x7f3e10115130) at >> ./../../../libraries/liblmdb/mdb.c:2972 >> #8 0x00000000004a9e4e in mdb_modify (op=3D0x7f3e100028f0, >> rs=3D0x7f3e1d6bd910) at >> modify.c:664 >> #9 0x000000000043b4cb in fe_op_modify (op=3D0x7f3e100028f0, >> rs=3D0x7f3e1d6bd910) at >> modify.c:303 >> #10 0x000000000043bdf6 in do_modify (op=3D0x7f3e100028f0, >> rs=3D0x7f3e1d6bd910) at >> modify.c:177 >> #11 0x0000000000423c99 in connection_operation (ctx=3D0x7f3e1d6bda70, >> arg_v=3D0x7f3e100028f0) at connection.c:1155 >> #12 0x0000000000424475 in connection_read_thread (ctx=3D0x7f3e1d6bda70, >> argv=3D<value optimized out>) at connection.c:1291 >> #13 0x00007f3f9b6f04e8 in ldap_int_thread_pool_wrapper (xpool=3D0x16f36d= 0) >> at >> tpool.c:688 >> #14 0x0000003b6f2079d1 in start_thread () from /lib64/libpthread.so.0 >> #15 0x0000003b6eee8b6d in clone () from /lib64/libc.so.6 >> >> Crash can be reproduced by using data.mdb file on any host with the same >> slapd >> binaries. >> Also note contextCSN of DB became 'old' - it was actual on May 22 for bo= th >> instances, when reset occured, but after start context CSN shows May 21 >> and 20. >> >> Also mdb_stat utility crashes on attempt to show FreeDB data: >> $ mdb_stat -f ./data >> Freelist Status >> Tree depth: 3 >> Branch pages: 7 >> Leaf pages: 817 >> Overflow pages: 0 >> Entries: 16995 >> Segmentation fault (core dumped) >> >> >> > > -- > -- Howard Chu > CTO, Symas Corp. http://www.symas.com > Director, Highland Sun http://highlandsun.com/hyc/ > Chief Architect, OpenLDAP http://www.openldap.org/project/ > -- WBR, Vadim Prozorov --047d7b5d285813733e04fa73805f Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><br><br><div class=3D"gmail_quote"><div dir=3D"ltr"><div><= div class=3D"h5"><div class=3D"gmail_quote"><div dir=3D"ltr"><div class=3D"= gmail_extra"><div class=3D"gmail_quote">2014-05-28 13:47 GMT+04:00 Howard C= hu <span dir=3D"ltr"><<a href=3D"mailto:hyc@symas.com" target=3D"_blank"= >hyc(a)symas.com</a>></span>:<div> <br><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;bor= der-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:sol= id;padding-left:1ex"> <a href=3D"mailto:vadius@vadius.ru" target=3D"_blank">vadius(a)vadius.ru</a> = wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-= left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;p= adding-left:1ex"> Full_Name: Vadim Prozorov<br> Version: 2.4.38<br> OS: RHEL 6.3<br> URL: <a href=3D"ftp://ftp.openldap.org/incoming/" target=3D"_blank">ftp://f= tp.openldap.org/<u></u>incoming/</a><br> Submission from: (NULL) (91.210.4.1)<br> <br> <br> Hello,<br> <br> I'm using slapd (2.4.38 release) built from sources on RHEL 6.3 64bit.<= br> Build params:<br> LD_LIBRARY_PATH=3D"/usr/lib:/<u></u>usr/local/lib:/opt/local/<u></u>Be= rkeleyDB.5.3/lib"<br> LDFLAGS=3D"-L/usr/local/lib -L/opt/local/BerkeleyDB.5.3/<u></u>lib&quo= t;<br> CPPFLAGS=3D"-I/usr/local/include -I/opt/local/BerkeleyDB.5.3/<u></u>in= clude" ./configure<br> --prefix=3D/opt/local/OpenLDAP --enable-dynacl --enable-ldap =E2=80=94enabl= e-overlays<br> --disable-static --enable-dynamic<br> <br> slapd works in master-master sync mode on host with big amount of RAM (whol= e DB<br> can fit into RAM).<br> One time host was hardly restarted. After restart slapd has started, but cr= ash<br> on any modification (at least, from ext application; may be form replica,<b= r> haven't check yet). There is always the same crash stack:<br> </blockquote> <br> Did you have dbnosync configured on this backend? Looks like garbage data g= ot written to the freeDB.<br></blockquote><div><br></div></div><div>No, DB = config section is:</div><div># define MDB parameters =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0</div> <div>backend<span style=3D"white-space:pre-wrap"> </span>mdb</div><div>dat= abase<span style=3D"white-space:pre-wrap"> </span>mdb</div><div>max= size<span style=3D"white-space:pre-wrap"> 20</span>000000000</div> <div>envflags<span style=3D"white-space:pre-wrap"> </span>writemap mapasync= </div><div># end define</div><div><div><div>=C2=A0</div><blockquote class= =3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left-width:1px;bo= rder-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"> <br> <blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-= left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;p= adding-left:1ex"> (gdb) bt<br> #0 =C2=A0mdb_xcursor_init1 (mc=3D0x7f3e1d6bbd40, node=3D0x7f3f99f46f82) at<= br> ./../../../libraries/liblmdb/<u></u>mdb.c:6585<br> #1 =C2=A00x000000000049f8eb in mdb_cursor_set (mc=3D0x7f3e1d6bbd40, key=3D0= x7f3e1d6bbef0,<br> data=3D0x0, op=3DMDB_SET_RANGE, exactp=3D<value optimized out>) at<br= > ./../../../libraries/liblmdb/<u></u>mdb.c:5329<br> #2 =C2=A00x00000000004a019f in mdb_cursor_get (mc=3D0x7f3e1d6bbd40, key=3D0= x7f3e1d6bbef0,<br> data=3D0x0, op=3D<value optimized out>) at ./../../../libraries/liblm= db/<u></u>mdb.c:5529<br> #3 =C2=A00x000000000049e0ed in mdb_page_alloc (mc=3D<value optimized out= >, num=3D1,<br> mp=3D0x7f3e1d6bbf68) at ./../../../libraries/liblmdb/<u></u>mdb.c:1727<br> #4 =C2=A00x000000000049e469 in mdb_page_touch (mc=3D0x7f3e1d6bc1d0) at<br> ./../../../libraries/liblmdb/<u></u>mdb.c:1911<br> #5 =C2=A00x000000000049eb3a in mdb_page_search (mc=3D0x7f3e1d6bc1d0, key=3D= 0x0, flags=3D5)<br> at ./../../../libraries/liblmdb/<u></u>mdb.c:4830<br> #6 =C2=A00x00000000004a7404 in mdb_freelist_save (txn=3D0x7f3e10115130) at<= br> ./../../../libraries/liblmdb/<u></u>mdb.c:2522<br> #7 =C2=A0mdb_txn_commit (txn=3D0x7f3e10115130) at<br> ./../../../libraries/liblmdb/<u></u>mdb.c:2972<br> #8 =C2=A00x00000000004a9e4e in mdb_modify (op=3D0x7f3e100028f0, rs=3D0x7f3e= 1d6bd910) at<br> modify.c:664<br> #9 =C2=A00x000000000043b4cb in fe_op_modify (op=3D0x7f3e100028f0, rs=3D0x7f= 3e1d6bd910) at<br> modify.c:303<br> #10 0x000000000043bdf6 in do_modify (op=3D0x7f3e100028f0, rs=3D0x7f3e1d6bd9= 10) at<br> modify.c:177<br> #11 0x0000000000423c99 in connection_operation (ctx=3D0x7f3e1d6bda70,<br> arg_v=3D0x7f3e100028f0) at connection.c:1155<br> #12 0x0000000000424475 in connection_read_thread (ctx=3D0x7f3e1d6bda70,<br> argv=3D<value optimized out>) at connection.c:1291<br> #13 0x00007f3f9b6f04e8 in ldap_int_thread_pool_wrapper (xpool=3D0x16f36d0) = at<br> tpool.c:688<br> #14 0x0000003b6f2079d1 in start_thread () from /lib64/libpthread.so.0<br> #15 0x0000003b6eee8b6d in clone () from /lib64/libc.so.6<br> <br> Crash can be reproduced by using data.mdb file on any host with the same sl= apd<br> binaries.<br> Also note contextCSN of DB became 'old' - it was actual on May 22 f= or both<br> instances, when reset occured, but after start context CSN shows May 21 and= 20.<br> <br> Also mdb_stat utility crashes on attempt to show FreeDB data:<br> $ mdb_stat -f ./data<br> Freelist Status<br> =C2=A0 =C2=A0Tree depth: 3<br> =C2=A0 =C2=A0Branch pages: 7<br> =C2=A0 =C2=A0Leaf pages: 817<br> =C2=A0 =C2=A0Overflow pages: 0<br> =C2=A0 =C2=A0Entries: 16995<br> Segmentation fault (core dumped)<br> <br> <br><span><font color=3D"#888888"> </font></span></blockquote><span><font color=3D"#888888"> <br> <br> -- <br> =C2=A0 -- Howard Chu<br> =C2=A0 CTO, Symas Corp. =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 <a href=3D"http:= //www.symas.com" target=3D"_blank">http://www.symas.com</a><br> =C2=A0 Director, Highland Sun =C2=A0 =C2=A0 <a href=3D"http://highlandsun.c= om/hyc/" target=3D"_blank">http://highlandsun.com/hyc/</a><br> =C2=A0 Chief Architect, OpenLDAP =C2=A0<a href=3D"http://www.openldap.org/p= roject/" target=3D"_blank">http://www.openldap.org/<u></u>project/</a><br> </font></span></blockquote></div></div></div><span><font color=3D"#888888">= <br></font></span></div></div></div>--</div></div><div>WBR,</div><div>Vadim= Prozorov</div></div></div> </div> --047d7b5d285813733e04fa73805f--

1 0

Re: (ITS#7862) slapd crash on modifications after host hard reset (in mdb)
by hyc＠symas.com 28 May '14

28 May '14

vadius(a)vadius.ru wrote: > Full_Name: Vadim Prozorov > Version: 2.4.38 > OS: RHEL 6.3 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (91.210.4.1) > > > Hello, > > I'm using slapd (2.4.38 release) built from sources on RHEL 6.3 64bit. > Build params: > LD_LIBRARY_PATH="/usr/lib:/usr/local/lib:/opt/local/BerkeleyDB.5.3/lib" > LDFLAGS="-L/usr/local/lib -L/opt/local/BerkeleyDB.5.3/lib" > CPPFLAGS="-I/usr/local/include -I/opt/local/BerkeleyDB.5.3/include" ./configure > --prefix=/opt/local/OpenLDAP --enable-dynacl --enable-ldap enable-overlays > --disable-static --enable-dynamic > > slapd works in master-master sync mode on host with big amount of RAM (whole DB > can fit into RAM). > One time host was hardly restarted. After restart slapd has started, but crash > on any modification (at least, from ext application; may be form replica, > haven't check yet). There is always the same crash stack: Did you have dbnosync configured on this backend? Looks like garbage data got written to the freeDB. > (gdb) bt > #0 mdb_xcursor_init1 (mc=0x7f3e1d6bbd40, node=0x7f3f99f46f82) at > ./../../../libraries/liblmdb/mdb.c:6585 > #1 0x000000000049f8eb in mdb_cursor_set (mc=0x7f3e1d6bbd40, key=0x7f3e1d6bbef0, > data=0x0, op=MDB_SET_RANGE, exactp=<value optimized out>) at > ./../../../libraries/liblmdb/mdb.c:5329 > #2 0x00000000004a019f in mdb_cursor_get (mc=0x7f3e1d6bbd40, key=0x7f3e1d6bbef0, > data=0x0, op=<value optimized out>) at ./../../../libraries/liblmdb/mdb.c:5529 > #3 0x000000000049e0ed in mdb_page_alloc (mc=<value optimized out>, num=1, > mp=0x7f3e1d6bbf68) at ./../../../libraries/liblmdb/mdb.c:1727 > #4 0x000000000049e469 in mdb_page_touch (mc=0x7f3e1d6bc1d0) at > ./../../../libraries/liblmdb/mdb.c:1911 > #5 0x000000000049eb3a in mdb_page_search (mc=0x7f3e1d6bc1d0, key=0x0, flags=5) > at ./../../../libraries/liblmdb/mdb.c:4830 > #6 0x00000000004a7404 in mdb_freelist_save (txn=0x7f3e10115130) at > ./../../../libraries/liblmdb/mdb.c:2522 > #7 mdb_txn_commit (txn=0x7f3e10115130) at > ./../../../libraries/liblmdb/mdb.c:2972 > #8 0x00000000004a9e4e in mdb_modify (op=0x7f3e100028f0, rs=0x7f3e1d6bd910) at > modify.c:664 > #9 0x000000000043b4cb in fe_op_modify (op=0x7f3e100028f0, rs=0x7f3e1d6bd910) at > modify.c:303 > #10 0x000000000043bdf6 in do_modify (op=0x7f3e100028f0, rs=0x7f3e1d6bd910) at > modify.c:177 > #11 0x0000000000423c99 in connection_operation (ctx=0x7f3e1d6bda70, > arg_v=0x7f3e100028f0) at connection.c:1155 > #12 0x0000000000424475 in connection_read_thread (ctx=0x7f3e1d6bda70, > argv=<value optimized out>) at connection.c:1291 > #13 0x00007f3f9b6f04e8 in ldap_int_thread_pool_wrapper (xpool=0x16f36d0) at > tpool.c:688 > #14 0x0000003b6f2079d1 in start_thread () from /lib64/libpthread.so.0 > #15 0x0000003b6eee8b6d in clone () from /lib64/libc.so.6 > > Crash can be reproduced by using data.mdb file on any host with the same slapd > binaries. > Also note contextCSN of DB became 'old' - it was actual on May 22 for both > instances, when reset occured, but after start context CSN shows May 21 and 20. > > Also mdb_stat utility crashes on attempt to show FreeDB data: > $ mdb_stat -f ./data > Freelist Status > Tree depth: 3 > Branch pages: 7 > Leaf pages: 817 > Overflow pages: 0 > Entries: 16995 > Segmentation fault (core dumped) > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#7862) slapd crash on modifications after host hard reset (in mdb)
by vadius＠vadius.ru 28 May '14

28 May '14

Full_Name: Vadim Prozorov Version: 2.4.38 OS: RHEL 6.3 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (91.210.4.1) Hello, I'm using slapd (2.4.38 release) built from sources on RHEL 6.3 64bit. Build params: LD_LIBRARY_PATH="/usr/lib:/usr/local/lib:/opt/local/BerkeleyDB.5.3/lib" LDFLAGS="-L/usr/local/lib -L/opt/local/BerkeleyDB.5.3/lib" CPPFLAGS="-I/usr/local/include -I/opt/local/BerkeleyDB.5.3/include" ./configure --prefix=/opt/local/OpenLDAP --enable-dynacl --enable-ldap enable-overlays --disable-static --enable-dynamic slapd works in master-master sync mode on host with big amount of RAM (whole DB can fit into RAM). One time host was hardly restarted. After restart slapd has started, but crash on any modification (at least, from ext application; may be form replica, haven't check yet). There is always the same crash stack: (gdb) bt #0 mdb_xcursor_init1 (mc=0x7f3e1d6bbd40, node=0x7f3f99f46f82) at ./../../../libraries/liblmdb/mdb.c:6585 #1 0x000000000049f8eb in mdb_cursor_set (mc=0x7f3e1d6bbd40, key=0x7f3e1d6bbef0, data=0x0, op=MDB_SET_RANGE, exactp=<value optimized out>) at ./../../../libraries/liblmdb/mdb.c:5329 #2 0x00000000004a019f in mdb_cursor_get (mc=0x7f3e1d6bbd40, key=0x7f3e1d6bbef0, data=0x0, op=<value optimized out>) at ./../../../libraries/liblmdb/mdb.c:5529 #3 0x000000000049e0ed in mdb_page_alloc (mc=<value optimized out>, num=1, mp=0x7f3e1d6bbf68) at ./../../../libraries/liblmdb/mdb.c:1727 #4 0x000000000049e469 in mdb_page_touch (mc=0x7f3e1d6bc1d0) at ./../../../libraries/liblmdb/mdb.c:1911 #5 0x000000000049eb3a in mdb_page_search (mc=0x7f3e1d6bc1d0, key=0x0, flags=5) at ./../../../libraries/liblmdb/mdb.c:4830 #6 0x00000000004a7404 in mdb_freelist_save (txn=0x7f3e10115130) at ./../../../libraries/liblmdb/mdb.c:2522 #7 mdb_txn_commit (txn=0x7f3e10115130) at ./../../../libraries/liblmdb/mdb.c:2972 #8 0x00000000004a9e4e in mdb_modify (op=0x7f3e100028f0, rs=0x7f3e1d6bd910) at modify.c:664 #9 0x000000000043b4cb in fe_op_modify (op=0x7f3e100028f0, rs=0x7f3e1d6bd910) at modify.c:303 #10 0x000000000043bdf6 in do_modify (op=0x7f3e100028f0, rs=0x7f3e1d6bd910) at modify.c:177 #11 0x0000000000423c99 in connection_operation (ctx=0x7f3e1d6bda70, arg_v=0x7f3e100028f0) at connection.c:1155 #12 0x0000000000424475 in connection_read_thread (ctx=0x7f3e1d6bda70, argv=<value optimized out>) at connection.c:1291 #13 0x00007f3f9b6f04e8 in ldap_int_thread_pool_wrapper (xpool=0x16f36d0) at tpool.c:688 #14 0x0000003b6f2079d1 in start_thread () from /lib64/libpthread.so.0 #15 0x0000003b6eee8b6d in clone () from /lib64/libc.so.6 Crash can be reproduced by using data.mdb file on any host with the same slapd binaries. Also note contextCSN of DB became 'old' - it was actual on May 22 for both instances, when reset occured, but after start context CSN shows May 21 and 20. Also mdb_stat utility crashes on attempt to show FreeDB data: $ mdb_stat -f ./data Freelist Status Tree depth: 3 Branch pages: 7 Leaf pages: 817 Overflow pages: 0 Entries: 16995 Segmentation fault (core dumped)

1 0

(ITS#7861) Wrong entry count with MDB_DUPSORT databases
by h.b.furuseth＠usit.uio.no 27 May '14

27 May '14

Full_Name: Hallvard B Furuseth Version: mdb.master, 4b9aed26a5b1c0027b10d211c6e7270dbb1ade1c OS: Linux x86_64 URL: Submission from: (NULL) (2001:700:100:556::233) Submitted by: hallvard mdb_cursor_put() does not always know if it'll be inserting a new item or not, e.g. when inserting in a sub-database. This was always wrong, but recent commits changed how it was wrong. Fix: Split 'insert' in 'insert_key' + 'insert_data'. Compute insert_data from mx_db.md_entries before and after the insert. In my UiO repo, branch mdb/todo-3: 02dd3865d5 [Rewritten] ITS#xxxx Fix MDB_db.md_entries. c6c78f71d3 [New] Tweak varname 'insert'. (Cosmetic patch which does not apply cleanly, I guess I should make an mdb/todo-4 branch where it is applied earlier.) Affects ITS#7834. mdb_cursor_del(, MDB_NODUPDATA) only subtracts 1 from md_entries when deleting a sub-page, instead of no of DUP entries. Fix: fa8e827026 [New,Squash] Fix md_entries in mdb_cursor_del:MDB_NODUPDATA.

1 0

(ITS#7860) [PATCH] correction to ldap_ava in ldap_get_dn.3
by ryan＠nardis.ca 26 May '14

26 May '14

Full_Name: Ryan Tandy Version: master/b22a614, RE24/d961650 OS: Debian unstable URL: https://bugs.debian.org/cgi-bin/bugreport.cgi?msg=38;bug=465024 Submission from: (NULL) (24.68.121.206) Hi, In Debian bug #465024, Philipp Hahn has provided the following patch, supplementing the change from ITS#5366: Index: openldap-2.4.39/doc/man/man3/ldap_get_dn.3 =================================================================== --- openldap-2.4.39.orig/doc/man/man3/ldap_get_dn.3 2014-02-21 00:12:24.000000000 +0100 +++ openldap-2.4.39/doc/man/man3/ldap_get_dn.3 2014-04-17 11:28:10.468925942 +0200 @@ -82,8 +82,8 @@ .ft B typedef struct ldap_ava { - char *la_attr; - struct berval *la_value; + struct berval la_attr; + struct berval la_value; unsigned la_flags; } LDAPAVA; It looks correct to me. (I assume the omission of la_private is intentional.) thanks, Ryan

1 0

Re: (ITS#7859) ldif processing issue with long lines
by hyc＠symas.com 26 May '14

26 May '14

mwarren(a)symas.com wrote: > Also, another issue has been reported where the length of LDIF lines is greater > by two than a specified 'ldif-wrap' definition when using slapcat. This appears to be the result of an intentional backward-compatiblility kludge. While the spec may have defaulted to 76 chars, the UMich code has always output 78 columns by default, so there's an offset built in to the LDIF output functions. This offset wasn't taken into account when the variable output width support was written. Probably we should just remove the kludge and define our default output width to 78 instead of pretending it's still 76. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#7859) ldif processing issue with long lines
by mwarren＠symas.com 26 May '14

26 May '14

Full_Name: Mark Warren Version: 2.4.35 OS: Linux URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (72.35.133.119) A customer of ours has identified an issue with slapadd and lines right at the maximum length(4096 characters). Howard has taken a look and reports the following: I took a quick look at the slapadd issue; libldap/ldif.c is using an input buffer of size [LDIF_MAXLINE] which is 4096 characters. It uses fgets() to read the input. The problem is that fgets() always stores a terminating NUL character, which means the buffer can only accept an input of up to 4095 characters. The input line in question is "only" 4095 characters long, but in reality it also has a terminating newline, making it 4096. The end result, fgets() reads the data but leaves the newline in the input buffer since there's no room for it in the user buffer. Then the next call to fgets() returns just this newline all by itself, which signals to ldif_read_record that it's come to the end of the entry, and so it treats the next input line as a new entry even though it's supposed to be a continuation of the previous line. The simple fix here is to bump up the input buffer size by 1, to accommodate the terminating newline. Actually the buffer needs to be upped by 2, to allow for CR/LF line terminators. -- End Howard Quote -- Also, another issue has been reported where the length of LDIF lines is greater by two than a specified 'ldif-wrap' definition when using slapcat. Regards, Mark Warren

1 0

Re: (ITS#7858) [PATCH] nssov/Makefile wrong rpath
by hyc＠symas.com 25 May '14

25 May '14

ryan(a)nardis.ca wrote: > Full_Name: Ryan Tandy > Version: master/932a11a, RE24/d961650 > OS: Debian unstable > URL: > Submission from: (NULL) (24.68.121.206) > > > Hi, > > nssov can't be installed: Thanks, fixed now in master > > $ cd contrib/slapd-modules/nssov > $ make > [...] > $ sudo make install > mkdir -p /usr/local/libexec/openldap > ../../../libtool --mode=install cp nssov.la /usr/local/libexec/openldap > libtool: install: error: cannot install `nssov.la' to a directory not ending in > /usr/local/lib > Makefile:67: recipe for target 'install' failed > make: *** [install] Error 1 > > The Makefile says "-rpath $(libdir)" where every other contrib module uses > "-rpath $(moduledir)". Changing it to that lets it build, and the resulting > module seems to work. > > diff --git a/contrib/slapd-modules/nssov/Makefile > b/contrib/slapd-modules/nssov/Makefile > index 8527ee3..28c8003 100644 > --- a/contrib/slapd-modules/nssov/Makefile > +++ b/contrib/slapd-modules/nssov/Makefile > @@ -61,7 +61,7 @@ $(OBJS): nssov.h > > nssov.la: $(OBJS) $(XOBJS) > $(LIBTOOL) --mode=link $(CC) $(OPT) -version-info 0:0:0 \ > - -rpath $(libdir) -module -o $@ $(OBJS) $(XOBJS) $(LIBS) > + -rpath $(moduledir) -module -o $@ $(OBJS) $(XOBJS) $(LIBS) > > install: nssov.la > mkdir -p $(DESTDIR)$(moduledir) > > thanks, > Ryan > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#7858) [PATCH] nssov/Makefile wrong rpath
by ryan＠nardis.ca 25 May '14

25 May '14

Full_Name: Ryan Tandy Version: master/932a11a, RE24/d961650 OS: Debian unstable URL: Submission from: (NULL) (24.68.121.206) Hi, nssov can't be installed: $ cd contrib/slapd-modules/nssov $ make [...] $ sudo make install mkdir -p /usr/local/libexec/openldap ../../../libtool --mode=install cp nssov.la /usr/local/libexec/openldap libtool: install: error: cannot install `nssov.la' to a directory not ending in /usr/local/lib Makefile:67: recipe for target 'install' failed make: *** [install] Error 1 The Makefile says "-rpath $(libdir)" where every other contrib module uses "-rpath $(moduledir)". Changing it to that lets it build, and the resulting module seems to work. diff --git a/contrib/slapd-modules/nssov/Makefile b/contrib/slapd-modules/nssov/Makefile index 8527ee3..28c8003 100644 --- a/contrib/slapd-modules/nssov/Makefile +++ b/contrib/slapd-modules/nssov/Makefile @@ -61,7 +61,7 @@ $(OBJS): nssov.h nssov.la: $(OBJS) $(XOBJS) $(LIBTOOL) --mode=link $(CC) $(OPT) -version-info 0:0:0 \ - -rpath $(libdir) -module -o $@ $(OBJS) $(XOBJS) $(LIBS) + -rpath $(moduledir) -module -o $@ $(OBJS) $(XOBJS) $(LIBS) install: nssov.la mkdir -p $(DESTDIR)$(moduledir) thanks, Ryan

1 0

Re: (ITS#7765) Crash when adding syncrepl configuration in cn=config without search base
by hyc＠symas.com 23 May '14

23 May '14

coudot(a)linagora.com wrote: > Full_Name: Clément OUDOT > Version: 2.4.38 > OS: GNU/Linux > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (88.173.78.196) > > > When adding an olcSyncrepl attribute in my cn=config, OpenLDAP crashed. > > The syslog output is: > > Dec 16 11:56:42 ader slapd[8707]: conn=1096 op=0 BIND dn="cn=config" method=128 > Dec 16 11:56:42 ader slapd[8707]: conn=1096 op=0 BIND dn="cn=config" mech=SIMPLE > ssf=0 > Dec 16 11:56:42 ader slapd[8707]: conn=1096 op=0 RESULT tag=97 err=0 text= > Dec 16 11:56:42 ader slapd[8707]: conn=1096 op=1 MOD > dn="olcDatabase={2}mdb,cn=config" > Dec 16 11:56:42 ader slapd[8707]: conn=1096 op=1 MOD attr=olcaccess olcdbindex > olcsyncrepl > Dec 16 11:56:42 ader slapd[8707]: olcSyncrepl: value #0: Error: Malformed > "syncrepl" line in slapd config file, missing searchbase. > Dec 16 11:56:42 ader slapd[8707]: failed to add syncinfo > Dec 16 11:56:42 ader slapd[8707]: ch_calloc of 1 elems of 0 bytes failed > > > I have no gdb trace to give for now, if you think it is required, I will try to > find time to do it. But it should be easy to reproduce the issue on your side. Please provide your exact ldapmodify request and the database config before the modify attempt, thanks. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7857) let openldap build against the bionic libc
by hyc＠symas.com 19 May '14

19 May '14

tux_mind(a)monumentmail.com wrote: > Full_Name: tux_mind > Version: lastest from git > OS: gentoo GNU/Linux > URL: https://github.com/tux-mind/platform_external_openldap > Submission from: (NULL) (188.153.8.167) > > > hello guys. > i've modified a little your project to let it build against the bionic libc ( > the one used in android ). > > please let me known if you can merge this into the upstream project. > > modified sources can be found here: > https://github.com/tux-mind/platform_external_openldap > > my changes are the last 3 of these commits: > https://github.com/tux-mind/platform_external_openldap/commits/master "added generated files" - No. Don't be silly. The last time I compiled OpenLDAP for Android, I don't recall needing any #include patches. (That was for Froyo, a few years ago now.) I don't have a current dev system to check these changes. The reason those generated files are generated is because those platform-specific features are known to change. E.g., 64 bit Android devices are coming out soon, and hardcoding things for 32 bit would be a mistake. Including an Android.mk file in our distribution would require us to maintain it in parallel with our regular Makefiles, forever after. We will not do redundant support work, nor is it necessary. The standard OpenLDAP Makefiles work fine to create Android binaries. Rejecting these patches. > > thanks for your attention > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#7857) let openldap build against the bionic libc
by tux_mind＠monumentmail.com 19 May '14

19 May '14

Full_Name: tux_mind Version: lastest from git OS: gentoo GNU/Linux URL: https://github.com/tux-mind/platform_external_openldap Submission from: (NULL) (188.153.8.167) hello guys. i've modified a little your project to let it build against the bionic libc ( the one used in android ). please let me known if you can merge this into the upstream project. modified sources can be found here: https://github.com/tux-mind/platform_external_openldap my changes are the last 3 of these commits: https://github.com/tux-mind/platform_external_openldap/commits/master thanks for your attention

1 0

Re: (ITS#7856) TLS_REQCERT try is same as TLS_REQCERT hard?
by jsynacek＠redhat.com 18 May '14

18 May '14

On 05/16/2014 09:11 AM, pguenther(a)proofpoint.com wrote: > Full_Name: Philip Guenther > Version: 2.4.39 > OS: OpenBSD > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (76.253.0.176) > > > The ldap.conf(5) manpage says this about TLS_REQCERT > TLS_REQCERT <level> > Specifies what checks to perform on server certificates in a TLS > session, if any. The <level> can be specified as one of the > following keywords: > ... > > try The server certificate is requested. If no certificate is > provided, the session proceeds normally. If a bad > certificate is provided, the session is immediately > terminated. > > demand | hard > These keywords are equivalent. The server certificate is > requested. If no certificate is provided, or a bad > certificate is provided, the session is immediately > terminated. This is the default setting. > > > In testing, I can find no difference in behavior between the 'try' and 'hard' > keywords. For the ldap* tools, both 'try' and 'hard' seem to place the same > requirements on the server. What does "if no certificate is provided" *mean* in > terms of server and/or client configuration? > See ITS#7744. -- Jan Synacek Software Engineer, Red Hat

1 0

Re: (ITS#7842) mdb readers/writer exclusion protocol, as implemented, is racy.
by rsbx＠acm.org 17 May '14

17 May '14

On 05/16/2014 08:09 PM, Howard Chu wrote: > rsbx(a)acm.org wrote: >> Full_Name: Raymond S Brand >> Version: mdb.master >> OS: linux >> URL: ftp://ftp.openldap.org/incoming/ >> Submission from: (NULL) (107.145.137.13) >> >> >> [I would have uploaded this to the ftp.openldap.org but there was no >> space.] >> >> As implemented, the mdb readers/writer exclusion protocol has a race >> condition >> that could result in a writer reclaiming and over-writing pages still >> in use >> by a reader. > > Yes, you're basically correct. But this is unlikely in practice because > the reader thread has no blocking calls in its codepath, while the > writer must acquire locks etc. Generally the only way for two write txns > to complete while a reader thread is stalled is if you explicitly send a > SIGSTOP to the reader thread while it's in its critical section. Or just unlucky process scheduling. > >> >> Pseudo code of the snapshot locking protocols of reader and writer >> transactions. The labeled sections, for the purposes of this analysis, >> can be >> assumed to execute atomically. >> >> READER >> ====== >> >> *R1* t0 = meta[0].txid; t1 = meta[1].txid >> if (t1 > t0) >> t0 = t1 >> >> *R2* readers[me].txid = t0 >> >> *R3* snapshot_root = meta[t0&1].snapshot >> >> *R4* /* lookup data */ >> >> *R5* readers[me].txid = -1 // Release snapshot >> >> WRITER >> ====== >> >> *W1* lock(writer, EXCLUSIVE) >> curr_txid = meta[0].txid >> if (meta[1].txid > curr_txid) >> curr_txid = meta[1].txid >> >> *W2* oldest = curr_txid >> for (i=0; i<reader_slots; i++) >> t = readers[i].txid >> if (t != -1 && t < oldest) >> oldest = t >> >> *W3* reclaim_old_pages(oldest) >> >> /* >> ** Commit new transaction >> */ >> >> *W4* unlock(writer) >> >> --------------------------------------------------------------------------- >> >> >> Adversarial scheduling analysis: >> >> The following timeline demonstrates that a writer can reclaim pages in >> use by a >> reader. >> >> T0 T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 >> R1 R2 R3 R4 >> W1 W2 W3 W4 * W1 W2 W3 >> >> T0 Reader has found the latest txid, Xn. >> >> T1->T4 Reader is not scheduled to run and a write transaction >> commits, Xn+1. >> >> T5 Reader is still not scheduled and another write transaction >> starts. >> >> T6 Writer finds that the oldest referenced transaction is the last >> committed transaction, Xn+1. >> >> T7 Reader records Xn in the reader table. >> >> T8 Reader gets the snapshot root page for transaction Xn. >> >> T9 Writer, believing that Xn+1 is the oldest reference transaction, >> reclaims the pages of transaction Xn. >> >> >> T10 Reader attempts to navigate pages of a transaction, Xn, that >> has been >> reclaimed and "Bad Things Happen". >> >> In fact, bad things will happen if an even number of write >> transactions commit >> between W4 and W1, represented by the '*', in the timeline above. >> >> The fix is to search for the highest txid in R1 and between R2 and R3. >> Optionally, followed by recording the txid, of the snapshot root found >> in R3, >> in the reader's reader table slot to, possibly, increase the number of >> reclaimable transactions. >> >> The lack of compiler and memory barriers in the implementation of the >> locking >> protocol is also of concern. >> >> Beyond the above, the code in mdb_txn_renew0() after the >> "/* Copy the DB info and flags */" >> comment appears to have a number of data races. >> >> --- >> libraries/liblmdb/mdb.c | 24 +++++++++++++++--------- >> 1 file changed, 15 insertions(+), 9 deletions(-) >> >> diff --git a/libraries/liblmdb/mdb.c b/libraries/liblmdb/mdb.c >> index e0f551e..908417c 100644 >> --- a/libraries/liblmdb/mdb.c >> +++ b/libraries/liblmdb/mdb.c >> @@ -533,11 +533,11 @@ typedef struct MDB_rxbody { >> * started from so we can avoid overwriting any data >> used in that >> * particular version. >> */ >> - txnid_t mrb_txnid; >> + volatile txnid_t mrb_txnid; >> /** The process ID of the process owning this reader txn. */ >> - MDB_PID_T mrb_pid; >> + volatile MDB_PID_T mrb_pid; >> /** The thread ID of the thread owning this txn. */ >> - pthread_t mrb_tid; >> + volatile pthread_t mrb_tid; >> } MDB_rxbody; >> >> /** The actual reader record, with cacheline padding. */ >> @@ -585,12 +585,12 @@ typedef struct MDB_txbody { >> * This is recorded here only for convenience; >> the value >> can always >> * be determined by reading the main database >> meta pages. >> */ >> - txnid_t mtb_txnid; >> + volatile txnid_t mtb_txnid; >> /** The number of slots that have been used in the >> reader >> table. >> * This always records the maximum count, it is not >> decremented >> * when readers release their slots. >> */ >> - unsigned mtb_numreaders; >> + volatile unsigned mtb_numreaders; >> } MDB_txbody; >> >> /** The actual reader table definition. */ >> @@ -854,7 +854,7 @@ typedef struct MDB_meta { >> /** Any persistent environment flags. @ref mdb_env */ >> #define mm_flags mm_dbs[0].md_flags >> pgno_t mm_last_pg; /**< last >> used page in >> file */ >> - txnid_t mm_txnid; /**< txnid that >> committed this page */ >> + volatile txnid_t mm_txnid; /**< txnid that >> committed this >> page */ >> } MDB_meta; >> >> /** Buffer for a stack-allocated meta page. >> @@ -2303,10 +2303,12 @@ mdb_txn_renew0(MDB_txn *txn) >> >> if (txn->mt_flags & MDB_TXN_RDONLY) { >> if (!ti) { >> + /* No readers table; app responsible for >> locking */ >> meta = env->me_metas[ mdb_env_pick_meta(env) ]; >> txn->mt_txnid = meta->mm_txnid; >> txn->mt_u.reader = NULL; >> } else { >> + /* Has readers table */ >> MDB_reader *r = (env->me_flags & MDB_NOTLS) ? >> txn->mt_u.reader : >> pthread_getspecific(env->me_txkey); >> if (r) { >> @@ -2347,8 +2349,12 @@ mdb_txn_renew0(MDB_txn *txn) >> return rc; >> } >> } >> - txn->mt_txnid = r->mr_txnid = ti->mti_txnid; >> txn->mt_u.reader = r; >> + r->mr_txnid = ti->mti_txnid; >> + >> + /* Really need a memory barrier here */ >> + >> + txn->mt_txnid = r->mr_txnid = ti->mti_txnid; >> meta = env->me_metas[txn->mt_txnid & 1]; >> } >> } else { >> @@ -2376,10 +2382,10 @@ mdb_txn_renew0(MDB_txn *txn) >> } >> >> /* Copy the DB info and flags */ >> - memcpy(txn->mt_dbs, meta->mm_dbs, 2 * sizeof(MDB_db)); >> + memcpy(txn->mt_dbs, meta->mm_dbs, 2 * sizeof(MDB_db)); /* >> Racy */ >> >> /* Moved to here to avoid a data race in read TXNs */ >> - txn->mt_next_pgno = meta->mm_last_pg+1; >> + txn->mt_next_pgno = meta->mm_last_pg+1; /* >> Racy */ >> >> for (i=2; i<txn->mt_numdbs; i++) { >> x = env->me_dbflags[i]; >> > >

1 0

Re: (ITS#7842) mdb readers/writer exclusion protocol, as implemented, is racy.
by hyc＠symas.com 16 May '14

16 May '14

rsbx(a)acm.org wrote: > Full_Name: Raymond S Brand > Version: mdb.master > OS: linux > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (107.145.137.13) > > > [I would have uploaded this to the ftp.openldap.org but there was no space.] > > As implemented, the mdb readers/writer exclusion protocol has a race condition > that could result in a writer reclaiming and over-writing pages still in use > by a reader. Yes, you're basically correct. But this is unlikely in practice because the reader thread has no blocking calls in its codepath, while the writer must acquire locks etc. Generally the only way for two write txns to complete while a reader thread is stalled is if you explicitly send a SIGSTOP to the reader thread while it's in its critical section. > > Pseudo code of the snapshot locking protocols of reader and writer > transactions. The labeled sections, for the purposes of this analysis, can be > assumed to execute atomically. > > READER > ====== > > *R1* t0 = meta[0].txid; t1 = meta[1].txid > if (t1 > t0) > t0 = t1 > > *R2* readers[me].txid = t0 > > *R3* snapshot_root = meta[t0&1].snapshot > > *R4* /* lookup data */ > > *R5* readers[me].txid = -1 // Release snapshot > > WRITER > ====== > > *W1* lock(writer, EXCLUSIVE) > curr_txid = meta[0].txid > if (meta[1].txid > curr_txid) > curr_txid = meta[1].txid > > *W2* oldest = curr_txid > for (i=0; i<reader_slots; i++) > t = readers[i].txid > if (t != -1 && t < oldest) > oldest = t > > *W3* reclaim_old_pages(oldest) > > /* > ** Commit new transaction > */ > > *W4* unlock(writer) > > --------------------------------------------------------------------------- > > Adversarial scheduling analysis: > > The following timeline demonstrates that a writer can reclaim pages in use by a > reader. > > T0 T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 > R1 R2 R3 R4 > W1 W2 W3 W4 * W1 W2 W3 > > T0 Reader has found the latest txid, Xn. > > T1->T4 Reader is not scheduled to run and a write transaction commits, Xn+1. > > T5 Reader is still not scheduled and another write transaction starts. > > T6 Writer finds that the oldest referenced transaction is the last > committed transaction, Xn+1. > > T7 Reader records Xn in the reader table. > > T8 Reader gets the snapshot root page for transaction Xn. > > T9 Writer, believing that Xn+1 is the oldest reference transaction, > reclaims the pages of transaction Xn. > > > T10 Reader attempts to navigate pages of a transaction, Xn, that has been > reclaimed and "Bad Things Happen". > > In fact, bad things will happen if an even number of write transactions commit > between W4 and W1, represented by the '*', in the timeline above. > > The fix is to search for the highest txid in R1 and between R2 and R3. > Optionally, followed by recording the txid, of the snapshot root found in R3, > in the reader's reader table slot to, possibly, increase the number of > reclaimable transactions. > > The lack of compiler and memory barriers in the implementation of the locking > protocol is also of concern. > > Beyond the above, the code in mdb_txn_renew0() after the > "/* Copy the DB info and flags */" > comment appears to have a number of data races. > > --- > libraries/liblmdb/mdb.c | 24 +++++++++++++++--------- > 1 file changed, 15 insertions(+), 9 deletions(-) > > diff --git a/libraries/liblmdb/mdb.c b/libraries/liblmdb/mdb.c > index e0f551e..908417c 100644 > --- a/libraries/liblmdb/mdb.c > +++ b/libraries/liblmdb/mdb.c > @@ -533,11 +533,11 @@ typedef struct MDB_rxbody { > * started from so we can avoid overwriting any data used in that > * particular version. > */ > - txnid_t mrb_txnid; > + volatile txnid_t mrb_txnid; > /** The process ID of the process owning this reader txn. */ > - MDB_PID_T mrb_pid; > + volatile MDB_PID_T mrb_pid; > /** The thread ID of the thread owning this txn. */ > - pthread_t mrb_tid; > + volatile pthread_t mrb_tid; > } MDB_rxbody; > > /** The actual reader record, with cacheline padding. */ > @@ -585,12 +585,12 @@ typedef struct MDB_txbody { > * This is recorded here only for convenience; the value > can always > * be determined by reading the main database meta pages. > */ > - txnid_t mtb_txnid; > + volatile txnid_t mtb_txnid; > /** The number of slots that have been used in the reader > table. > * This always records the maximum count, it is not > decremented > * when readers release their slots. > */ > - unsigned mtb_numreaders; > + volatile unsigned mtb_numreaders; > } MDB_txbody; > > /** The actual reader table definition. */ > @@ -854,7 +854,7 @@ typedef struct MDB_meta { > /** Any persistent environment flags. @ref mdb_env */ > #define mm_flags mm_dbs[0].md_flags > pgno_t mm_last_pg; /**< last used page in > file */ > - txnid_t mm_txnid; /**< txnid that > committed this page */ > + volatile txnid_t mm_txnid; /**< txnid that committed this > page */ > } MDB_meta; > > /** Buffer for a stack-allocated meta page. > @@ -2303,10 +2303,12 @@ mdb_txn_renew0(MDB_txn *txn) > > if (txn->mt_flags & MDB_TXN_RDONLY) { > if (!ti) { > + /* No readers table; app responsible for locking */ > meta = env->me_metas[ mdb_env_pick_meta(env) ]; > txn->mt_txnid = meta->mm_txnid; > txn->mt_u.reader = NULL; > } else { > + /* Has readers table */ > MDB_reader *r = (env->me_flags & MDB_NOTLS) ? > txn->mt_u.reader : > pthread_getspecific(env->me_txkey); > if (r) { > @@ -2347,8 +2349,12 @@ mdb_txn_renew0(MDB_txn *txn) > return rc; > } > } > - txn->mt_txnid = r->mr_txnid = ti->mti_txnid; > txn->mt_u.reader = r; > + r->mr_txnid = ti->mti_txnid; > + > + /* Really need a memory barrier here */ > + > + txn->mt_txnid = r->mr_txnid = ti->mti_txnid; > meta = env->me_metas[txn->mt_txnid & 1]; > } > } else { > @@ -2376,10 +2382,10 @@ mdb_txn_renew0(MDB_txn *txn) > } > > /* Copy the DB info and flags */ > - memcpy(txn->mt_dbs, meta->mm_dbs, 2 * sizeof(MDB_db)); > + memcpy(txn->mt_dbs, meta->mm_dbs, 2 * sizeof(MDB_db)); /* Racy */ > > /* Moved to here to avoid a data race in read TXNs */ > - txn->mt_next_pgno = meta->mm_last_pg+1; > + txn->mt_next_pgno = meta->mm_last_pg+1; /* Racy */ > > for (i=2; i<txn->mt_numdbs; i++) { > x = env->me_dbflags[i]; > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7844) LMDB Delete Cursor inconsistencies
by armon.dadgar＠gmail.com 16 May '14

16 May '14

--53767cd5_238e1f29_1704c Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline I checked on gitorious only hours previously and fca18d2 was the latest v= ersion. However, I now see=C2=A02764360. I=E2=80=99ve updated to the latest versi= on and can no longer reproduce the issue. Thanks=21 Best Regards, Armon Dadgar =46rom:=C2=A0Howard Chu hyc=40symas.com Reply:=C2=A0Howard Chu hyc=40symas.com Date:=C2=A0May 16, 2014 at 1:54:18 PM To:=C2=A0Armon Dadgar armon.dadgar=40gmail.com, openldap-its=40openldap.o= rg openldap-its=40openldap.org Subject:=C2=A0 Re: (ITS=237844) LMDB Delete Cursor inconsistencies =20 Armon Dadgar wrote: =20 > When I run the attached test case on my machine, I=E2=80=99m hitting th= e failing case. =20 > =20 > Here is the test output: https://gist.github.com/armon/e529d7909fe30112= 6fc6 =20 You seem to be running obsolete code. I checked your gomdb github repo, y= ou're =20 using mdb.c at rev fca18d2. Current mdb.master is 4844a72. =20 > My steps: =20 > =24 clang 7844.c mdb.c midl.c =20 > =24 mkdir testdb =20 > =24 ./a.out =20 > =20 > =24 clang -v =20 > Apple LLVM version 5.1 (clang-503.0.40) (based on LLVM 3.4svn) =20 > Target: x86=5F64-apple-darwin13.1.0 =20 > Thread model: posix =20 > =20 > =24 uname -a =20 > Darwin Armons-MacBook-Air.local 13.1.0 Darwin Kernel Version 13.1.0: Th= u Jan =20 > 16 19:40:37 PST 2014; root:xnu-2422.90.20=7E2/RELEASE=5FX86=5F64 x86=5F= 64 =20 > =20 > Best Regards, =20 > Armon Dadgar =20 > =20 > =46rom: Howard Chu hyc=40symas.com <mailto:hyc=40symas.com> =20 > Reply: Howard Chu hyc=40symas.com <mailto:hyc=40symas.com> =20 > Date: May 16, 2014 at 11:06:49 AM =20 > To: armon.dadgar=40gmail.com armon.dadgar=40gmail.com =20 > <mailto:armon.dadgar=40gmail.com>, openldap-its=40openldap.org =20 > openldap-its=40openldap.org <mailto:openldap-its=40openldap.org> =20 > Subject: Re: (ITS=237844) LMDB Delete Cursor inconsistencies =20 > =20 >> armon.dadgar=40gmail.com wrote: =20 >> > --5372ac85=5F8edbdab=5F1271 =20 >> > Content-Type: text/plain; charset=3D=22utf-8=22 =20 >> > Content-Transfer-Encoding: quoted-printable =20 >> > Content-Disposition: inline =20 >> > =20 >> > =3D46or now, we have application code to retry the delete until no f= urther =3D =20 >> > rows are removed. =20 >> > Still, it would be nice to have this resolved (and tested) in master= =3D21 =20 >> =20 >> Unable to reproduce the issue. I've attached my test program based on = your =20 >> description. =20 >> =20 >> -- =20 >> -- Howard Chu =20 >> CTO, Symas Corp. http://www.symas.com =20 >> Director, Highland Sun http://highlandsun.com/hyc/ =20 >> Chief Architect, OpenLDAP http://www.openldap.org/project/ =20 >> ----------------------------------------------------------------------= -------- =20 -- =20 -- Howard Chu =20 CTO, Symas Corp. http://www.symas.com =20 Director, Highland Sun http://highlandsun.com/hyc/ =20 Chief Architect, OpenLDAP http://www.openldap.org/project/ =20 --53767cd5_238e1f29_1704c Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline <html><head><style>body=7Bfont-family:Helvetica,Arial;font-size:13px=7D</= style></head><body style=3D=22word-wrap: break-word; -webkit-nbsp-mode: s= pace; -webkit-line-break: after-white-space;=22><div id=3D=22bloop=5Fcust= omfont=22 style=3D=22font-family:Helvetica,Arial;font-size:13px; color: r= gba(0,0,0,1.0); margin: 0px; line-height: auto;=22>I checked on gitorious= only hours previously and fca18d2 was the latest version.</div><div id=3D= =22bloop=5Fcustomfont=22 style=3D=22font-family:Helvetica,Arial;font-size= :13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;=22>However= , I now see 2764360. I=E2=80=99ve updated to the latest version and = can no longer</div><div id=3D=22bloop=5Fcustomfont=22 style=3D=22font-fam= ily:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; = line-height: auto;=22>reproduce the issue.</div><div id=3D=22bloop=5Fcust= omfont=22 style=3D=22font-family:Helvetica,Arial;font-size:13px; color: r= gba(0,0,0,1.0); margin: 0px; line-height: auto;=22><br></div><div id=3D=22= bloop=5Fcustomfont=22 style=3D=22font-family:Helvetica,Arial;font-size:13= px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;=22>Thanks=21<= /div><div id=3D=22bloop=5Fcustomfont=22 style=3D=22font-family:Helvetica,= Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: a= uto;=22><br></div> <div id=3D=22bloop=5Fsign=5F1400274014470972928=22 cla= ss=3D=22bloop=5Fsign=22><div style=3D=22font-family:helvetica,arial;font-= size:13px=22>Best Regards,</div><div style=3D=22font-family:helvetica,ari= al;font-size:13px=22>Armon Dadgar<br></div></div> <div style=3D=22color:b= lack=22><br>=46rom: <span style=3D=22color:black=22>Howard Chu</span= > <a href=3D=22mailto:hyc=40symas.com=22>hyc=40symas.com</a><br>Reply:&nb= sp;<span style=3D=22color:black=22>Howard Chu</span> <a href=3D=22mailto:= hyc=40symas.com=22>hyc=40symas.com</a><br>Date: <span style=3D=22col= or:black=22>May 16, 2014 at 1:54:18 PM</span><br>To: <span style=3D=22= color:black=22>Armon Dadgar</span> <a href=3D=22mailto:armon.dadgar=40gma= il.com=22>armon.dadgar=40gmail.com</a>, <span style=3D=22color:black=22>o= penldap-its=40openldap.org</span> <a href=3D=22mailto:openldap-its=40open= ldap.org=22>openldap-its=40openldap.org</a><br>Subject: <span style=3D= =22color:black=22> Re: (ITS=237844) LMDB Delete Cursor inconsistencies <b= r></span></div><br> <blockquote type=3D=22cite=22 class=3D=22clean=5Fbq=22= ><span><div><div></div><div>Armon Dadgar wrote: <br>> When I run the attached test case on my machine, I=E2=80=99m hit= ting the failing case. <br>> <br>> Here is the test output: https://gist.github.com/armon/e529d7909= fe301126fc6 <br> <br>You seem to be running obsolete code. I checked your gomdb github rep= o, you're =20 <br>using mdb.c at rev fca18d2. Current mdb.master is 4844a72. <br> <br>> My steps: <br>> =24 clang 7844.c mdb.c midl.c <br>> =24 mkdir testdb <br>> =24 ./a.out <br>> <br>> =24 clang -v <br>> Apple LLVM version 5.1 (clang-503.0.40) (based on LLVM 3.4svn) <br>> Target: x86=5F64-apple-darwin13.1.0 <br>> Thread model: posix <br>> <br>> =24 uname -a <br>> Darwin Armons-MacBook-Air.local 13.1.0 Darwin Kernel Version 13.= 1.0: Thu Jan <br>> 16 19:40:37 PST 2014; root:xnu-2422.90.20=7E2/RELEASE=5FX86=5F64= x86=5F64 <br>> <br>> Best Regards, <br>> Armon Dadgar <br>> <br>> =46rom: Howard Chu hyc=40symas.com <mailto:hyc=40symas.com&gt= ; <br>> Reply: Howard Chu hyc=40symas.com <mailto:hyc=40symas.com>= <br>> Date: May 16, 2014 at 11:06:49 AM <br>> To: armon.dadgar=40gmail.com armon.dadgar=40gmail.com <br>> <mailto:armon.dadgar=40gmail.com>, openldap-its=40openldap= .org <br>> openldap-its=40openldap.org <mailto:openldap-its=40openldap.o= rg> <br>> Subject: Re: (ITS=237844) LMDB Delete Cursor inconsistencies <br>> <br>>> armon.dadgar=40gmail.com wrote: <br>>> > --5372ac85=5F8edbdab=5F1271 <br>>> > Content-Type: text/plain; charset=3D=22utf-8=22 <br>>> > Content-Transfer-Encoding: quoted-printable <br>>> > Content-Disposition: inline <br>>> > <br>>> > =3D46or now, we have application code to retry the dele= te until no further =3D <br>>> > rows are removed. <br>>> > Still, it would be nice to have this resolved (and test= ed) in master=3D21 <br>>> <br>>> Unable to reproduce the issue. I've attached my test program= based on your <br>>> description. <br>>> <br>>> -- <br>>> -- Howard Chu <br>>> CTO, Symas Corp. http://www.symas.com <br>>> Director, Highland Sun http://highlandsun.com/hyc/ <br>>> Chief Architect, OpenLDAP http://www.openldap.org/project/ <br>>> ------------------------------------------------------------= ------------------ <br> <br> <br>-- =20 <br> -- Howard Chu <br> CTO, Symas Corp. http://www.symas.com <br> Director, Highland Sun http://highlandsun.com/hyc/ <br> Chief Architect, OpenLDAP http://www.openldap.org/project/ <br></div></div></span></blockquote></body></html> --53767cd5_238e1f29_1704c--

1 0

Re: (ITS#7844) LMDB Delete Cursor inconsistencies
by hyc＠symas.com 16 May '14

16 May '14

Armon Dadgar wrote: > When I run the attached test case on my machine, Im hitting the failing case. > > Here is the test output: https://gist.github.com/armon/e529d7909fe301126fc6 You seem to be running obsolete code. I checked your gomdb github repo, you're using mdb.c at rev fca18d2. Current mdb.master is 4844a72. > My steps: > $ clang 7844.c mdb.c midl.c > $ mkdir testdb > $ ./a.out > > $ clang -v > Apple LLVM version 5.1 (clang-503.0.40) (based on LLVM 3.4svn) > Target: x86_64-apple-darwin13.1.0 > Thread model: posix > > $ uname -a > Darwin Armons-MacBook-Air.local 13.1.0 Darwin Kernel Version 13.1.0: Thu Jan > 16 19:40:37 PST 2014; root:xnu-2422.90.20~2/RELEASE_X86_64 x86_64 > > Best Regards, > Armon Dadgar > > From: Howard Chu hyc(a)symas.com <mailto:hyc@symas.com> > Reply: Howard Chu hyc(a)symas.com <mailto:hyc@symas.com> > Date: May 16, 2014 at 11:06:49 AM > To: armon.dadgar(a)gmail.com armon.dadgar(a)gmail.com > <mailto:armon.dadgar@gmail.com>, openldap-its(a)openldap.org > openldap-its(a)openldap.org <mailto:openldap-its@openldap.org> > Subject: Re: (ITS#7844) LMDB Delete Cursor inconsistencies > >> armon.dadgar(a)gmail.com wrote: >> > --5372ac85_8edbdab_1271 >> > Content-Type: text/plain; charset="utf-8" >> > Content-Transfer-Encoding: quoted-printable >> > Content-Disposition: inline >> > >> > =46or now, we have application code to retry the delete until no further = >> > rows are removed. >> > Still, it would be nice to have this resolved (and tested) in master=21 >> >> Unable to reproduce the issue. I've attached my test program based on your >> description. >> >> -- >> -- Howard Chu >> CTO, Symas Corp. http://www.symas.com >> Director, Highland Sun http://highlandsun.com/hyc/ >> Chief Architect, OpenLDAP http://www.openldap.org/project/ >> ------------------------------------------------------------------------------ -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7844) LMDB Delete Cursor inconsistencies
by armon.dadgar＠gmail.com 16 May '14

16 May '14

--537675cf_2ae8944a_1704c Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline When I run the attached test case on my machine, I=E2=80=99m hitting the = failing case. Here is the test output:=C2=A0https://gist.github.com/armon/e529d7909fe30= 1126fc6 My steps: =24 clang 7844.c mdb.c midl.c =24 mkdir testdb =24 ./a.out =24 clang -v Apple LLVM version 5.1 (clang-503.0.40) (based on LLVM 3.4svn) Target: x86=5F64-apple-darwin13.1.0 Thread model: posix =24 uname -a Darwin Armons-MacBook-Air.local 13.1.0 Darwin Kernel Version 13.1.0: Thu = Jan 16 19:40:37 PST 2014; root:xnu-2422.90.20=7E2/RELEASE=5FX86=5F64 x86=5F= 64 Best Regards, Armon Dadgar =46rom:=C2=A0Howard Chu hyc=40symas.com Reply:=C2=A0Howard Chu hyc=40symas.com Date:=C2=A0May 16, 2014 at 11:06:49 AM To:=C2=A0armon.dadgar=40gmail.com armon.dadgar=40gmail.com, openldap-its=40= openldap.org openldap-its=40openldap.org Subject:=C2=A0 Re: (ITS=237844) LMDB Delete Cursor inconsistencies =20 armon.dadgar=40gmail.com wrote: =20 > --5372ac85=5F8edbdab=5F1271 =20 > Content-Type: text/plain; charset=3D=22utf-8=22 =20 > Content-Transfer-Encoding: quoted-printable =20 > Content-Disposition: inline =20 > =20 > =3D46or now, we have application code to retry the delete until no furt= her =3D =20 > rows are removed. =20 > Still, it would be nice to have this resolved (and tested) in master=3D= 21 =20 Unable to reproduce the issue. I've attached my test program based on you= r =20 description. =20 -- =20 -- Howard Chu =20 CTO, Symas Corp. http://www.symas.com =20 Director, Highland Sun http://highlandsun.com/hyc/ =20 Chief Architect, OpenLDAP http://www.openldap.org/project/ =20 --537675cf_2ae8944a_1704c Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline <html><head><style>body=7Bfont-family:Helvetica,Arial;font-size:13px=7D</= style></head><body style=3D=22word-wrap: break-word; -webkit-nbsp-mode: s= pace; -webkit-line-break: after-white-space;=22><div id=3D=22bloop=5Fcust= omfont=22 style=3D=22font-family:Helvetica,Arial;font-size:13px; color: r= gba(0,0,0,1.0); margin: 0px; line-height: auto;=22>When I run the attache= d test case on my machine, I=E2=80=99m hitting the failing case.</div><di= v id=3D=22bloop=5Fcustomfont=22 style=3D=22font-family:Helvetica,Arial;fo= nt-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;=22>= <br></div><div id=3D=22bloop=5Fcustomfont=22 style=3D=22font-family:Helve= tica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-heig= ht: auto;=22>Here is the test output: https://gist.github.com/armon/= e529d7909fe301126fc6</div><div id=3D=22bloop=5Fcustomfont=22 style=3D=22f= ont-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin= : 0px; line-height: auto;=22><br></div><div id=3D=22bloop=5Fcustomfont=22= style=3D=22font-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0= ,1.0); margin: 0px; line-height: auto;=22>My steps:</div><div id=3D=22blo= op=5Fcustomfont=22 style=3D=22font-family:Helvetica,Arial;font-size:13px;= color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;=22>=24 clang 784= 4.c mdb.c midl.c</div><div id=3D=22bloop=5Fcustomfont=22 style=3D=22font-= family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0p= x; line-height: auto;=22>=24 mkdir testdb</div><div id=3D=22bloop=5Fcusto= mfont=22 style=3D=22font-family:Helvetica,Arial;font-size:13px; color: rg= ba(0,0,0,1.0); margin: 0px; line-height: auto;=22>=24 ./a.out</div><div i= d=3D=22bloop=5Fcustomfont=22 style=3D=22font-family:Helvetica,Arial;font-= size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;=22><br= ></div><div id=3D=22bloop=5Fcustomfont=22 style=3D=22font-family:Helvetic= a,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height:= auto;=22><div id=3D=22bloop=5Fcustomfont=22 style=3D=22margin: 0px;=22>=24= clang -v</div><div id=3D=22bloop=5Fcustomfont=22 style=3D=22margin: 0px;= =22>Apple LLVM version 5.1 (clang-503.0.40) (based on LLVM 3.4svn)</div><= div id=3D=22bloop=5Fcustomfont=22 style=3D=22margin: 0px;=22>Target: x86=5F= 64-apple-darwin13.1.0</div><div id=3D=22bloop=5Fcustomfont=22 style=3D=22= margin: 0px;=22>Thread model: posix</div><div id=3D=22bloop=5Fcustomfont=22= style=3D=22margin: 0px;=22><br></div><div id=3D=22bloop=5Fcustomfont=22 = style=3D=22margin: 0px;=22><div id=3D=22bloop=5Fcustomfont=22 style=3D=22= margin: 0px;=22>=24 uname -a</div><div id=3D=22bloop=5Fcustomfont=22 styl= e=3D=22margin: 0px;=22>Darwin Armons-MacBook-Air.local 13.1.0 Darwin Kern= el Version 13.1.0: Thu Jan 16 19:40:37 PST 2014; root:xnu-2422.90.20=7E2/= RELEASE=5FX86=5F64 x86=5F64</div></div></div><div id=3D=22bloop=5Fcustomf= ont=22 style=3D=22font-family:Helvetica,Arial;font-size:13px; color: rgba= (0,0,0,1.0); margin: 0px; line-height: auto;=22><br></div> <div id=3D=22b= loop=5Fsign=5F1400272071967389952=22 class=3D=22bloop=5Fsign=22><div styl= e=3D=22font-family:helvetica,arial;font-size:13px=22>Best Regards,</div><= div style=3D=22font-family:helvetica,arial;font-size:13px=22>Armon Dadgar= <br></div></div> <div style=3D=22color:black=22><br>=46rom: <span st= yle=3D=22color:black=22>Howard Chu</span> <a href=3D=22mailto:hyc=40symas= .com=22>hyc=40symas.com</a><br>Reply: <span style=3D=22color:black=22= >Howard Chu</span> <a href=3D=22mailto:hyc=40symas.com=22>hyc=40symas.com= </a><br>Date: <span style=3D=22color:black=22>May 16, 2014 at 11:06:= 49 AM</span><br>To: <span style=3D=22color:black=22>armon.dadgar=40g= mail.com</span> <a href=3D=22mailto:armon.dadgar=40gmail.com=22>armon.dad= gar=40gmail.com</a>, <span style=3D=22color:black=22>openldap-its=40openl= dap.org</span> <a href=3D=22mailto:openldap-its=40openldap.org=22>openlda= p-its=40openldap.org</a><br>Subject: <span style=3D=22color:black=22= > Re: (ITS=237844) LMDB Delete Cursor inconsistencies <br></span></div><b= r> <blockquote type=3D=22cite=22 class=3D=22clean=5Fbq=22><span><div><div= ></div><div>armon.dadgar=40gmail.com wrote: <br>> --5372ac85=5F8edbdab=5F1271 <br>> Content-Type: text/plain; charset=3D=22utf-8=22 <br>> Content-Transfer-Encoding: quoted-printable <br>> Content-Disposition: inline <br>> <br>> =3D46or now, we have application code to retry the delete until = no further =3D <br>> rows are removed. <br>> Still, it would be nice to have this resolved (and tested) in ma= ster=3D21 <br> <br>Unable to reproduce the issue. I've attached my test program based on= your =20 <br>description. <br> <br>-- =20 <br> -- Howard Chu <br> CTO, Symas Corp. http://www.symas.com <br> Director, Highland Sun http://highlandsun.com/hyc/ <br> Chief Architect, OpenLDAP http://www.openldap.org/project/ <br><hr></div></div></span></blockquote></body></html> --537675cf_2ae8944a_1704c--

1 0

Re: (ITS#7844) LMDB Delete Cursor inconsistencies
by hyc＠symas.com 16 May '14

16 May '14

This is a multi-part message in MIME format. --------------010403060500020806080002 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit armon.dadgar(a)gmail.com wrote: > --5372ac85_8edbdab_1271 > Content-Type: text/plain; charset="utf-8" > Content-Transfer-Encoding: quoted-printable > Content-Disposition: inline > > =46or now, we have application code to retry the delete until no further = > rows are removed. > Still, it would be nice to have this resolved (and tested) in master=21 Unable to reproduce the issue. I've attached my test program based on your description. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/ --------------010403060500020806080002 Content-Type: text/plain; charset=UTF-8; name="7844.c.txt" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="7844.c.txt" /* 7844.c - memory-mapped database tester/toy */ /* * Copyright 2014 Howard Chu, Symas Corp. * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted only as authorized by the OpenLDAP * Public License. * * A copy of this license is available in the file LICENSE in the * top-level directory of the distribution or, alternatively, at * <http://www.OpenLDAP.org/license.html>. */ #include <stdio.h> #include <stdlib.h> #include <string.h> #include <uuid/uuid.h> #include "lmdb.h" #define E(expr) CHECK((rc = (expr)) == MDB_SUCCESS, #expr) #define RES(err, expr) ((rc = expr) == (err) || (CHECK(!rc, #expr), 0)) #define CHECK(test, msg) ((test) ? (void)0 : ((void)fprintf(stderr, \ "%s:%d: %s: %s\n", __FILE__, __LINE__, msg, mdb_strerror(rc)), abort())) #define UUID_LEN 36 typedef char kval[UUID_LEN*2+2]; char sval[100]; int main(int argc,char * argv[]) { int i = 0, j = 0, rc; MDB_env *env; MDB_dbi kdbi, ddbi; MDB_val key, ikey, data; MDB_txn *txn; MDB_stat mst; MDB_cursor *cursor, *cur2; int count; unsigned long lastid = 0; kval *keys; char prefix[UUID_LEN+1], *ptr; uuid_t uuid; count = 128; keys = (kval *)malloc(count*sizeof(kval)); E(mdb_env_create(&env)); E(mdb_env_set_mapsize(env, 128*1024*1024)); E(mdb_env_set_maxdbs(env, 4)); E(mdb_env_open(env, "./testdb", MDB_NOSYNC, 0664)); E(mdb_txn_begin(env, NULL, 0, &txn)); E(mdb_open(txn, "kvs", MDB_CREATE|MDB_INTEGERKEY, &ddbi)); E(mdb_open(txn, "kvs_id_idx", MDB_CREATE, &kdbi)); key.mv_size = UUID_LEN*2+1; data.mv_size = sizeof(sval); data.mv_data = sval; ikey.mv_size = sizeof(lastid); ikey.mv_data = &lastid; uuid_generate(uuid); uuid_unparse(uuid, prefix); printf("Adding %d values\n", count); for (i=0;i<count;i++) { uuid_generate(uuid); ptr = (char *)&keys[i]; memcpy(ptr, prefix, UUID_LEN); ptr[UUID_LEN] = '/'; uuid_unparse(uuid, ptr+UUID_LEN+1); ptr[UUID_LEN*2+1] = '\0'; printf("\t%s\n", ptr); key.mv_data = ptr; sprintf(sval, "Some stuff for key %s", ptr); lastid++; if (RES(MDB_KEYEXIST, mdb_put(txn, kdbi, &key, &ikey, MDB_NOOVERWRITE))) { j++; ikey.mv_size = sizeof(lastid); ikey.mv_data = &lastid; } E(mdb_put(txn, ddbi, &ikey, &data, MDB_NOOVERWRITE)); } if (j) printf("%d duplicates skipped\n", j); E(mdb_txn_commit(txn)); E(mdb_env_stat(env, &mst)); uuid_generate(uuid); uuid_unparse(uuid, prefix); printf("Adding %d values\n", count); E(mdb_txn_begin(env, NULL, 0, &txn)); for (i=0;i<count;i++) { uuid_generate(uuid); ptr = (char *)&keys[i]; memcpy(ptr, prefix, UUID_LEN); ptr[UUID_LEN] = '/'; uuid_unparse(uuid, ptr+UUID_LEN+1); ptr[UUID_LEN*2+1] = '\0'; printf("\t%s\n", ptr); key.mv_data = ptr; sprintf(sval, "Some stuff for key %s", ptr); lastid++; if (RES(MDB_KEYEXIST, mdb_put(txn, kdbi, &key, &ikey, MDB_NOOVERWRITE))) { j++; ikey.mv_size = sizeof(lastid); ikey.mv_data = &lastid; } E(mdb_put(txn, ddbi, &ikey, &data, MDB_NOOVERWRITE)); } if (j) printf("%d duplicates skipped\n", j); E(mdb_txn_commit(txn)); E(mdb_txn_begin(env, NULL, 0, &txn)); E(mdb_cursor_open(txn, kdbi, &cursor)); E(mdb_cursor_open(txn, ddbi, &cur2)); key.mv_size = UUID_LEN; key.mv_data = prefix; j = 0; i = count; rc = mdb_cursor_get(cursor, &key, &ikey, MDB_SET_RANGE); while (rc == 0) { #if 0 printf("key: %p %.*s, data: %p %.*s\n", key.mv_data, (int) key.mv_size, (char *) key.mv_data, data.mv_data, (int) data.mv_size, (char *) data.mv_data); #endif E(mdb_cursor_get(cur2, &ikey, &data, MDB_SET)); E(mdb_cursor_del(cur2, 0)); E(mdb_cursor_del(cursor, 0)); j++; i--; if (i == 0) break; rc = mdb_cursor_get(cursor, &key, &ikey, MDB_NEXT); } mdb_cursor_close(cursor); mdb_cursor_close(cur2); E(mdb_txn_commit(txn)); printf("Deleted %d values\n", j); mdb_env_close(env); return 0; } --------------010403060500020806080002--

1 0

(ITS#7856) TLS_REQCERT try is same as TLS_REQCERT hard?
by pguenther＠proofpoint.com 16 May '14

16 May '14

Full_Name: Philip Guenther Version: 2.4.39 OS: OpenBSD URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (76.253.0.176) The ldap.conf(5) manpage says this about TLS_REQCERT TLS_REQCERT <level> Specifies what checks to perform on server certificates in a TLS session, if any. The <level> can be specified as one of the following keywords: ... try The server certificate is requested. If no certificate is provided, the session proceeds normally. If a bad certificate is provided, the session is immediately terminated. demand | hard These keywords are equivalent. The server certificate is requested. If no certificate is provided, or a bad certificate is provided, the session is immediately terminated. This is the default setting. In testing, I can find no difference in behavior between the 'try' and 'hard' keywords. For the ldap* tools, both 'try' and 'hard' seem to place the same requirements on the server. What does "if no certificate is provided" *mean* in terms of server and/or client configuration?

1 0

Re: (ITS#7705) mdb back-end segfaults sporadically with paged searches
by hyc＠symas.com 15 May '14

15 May '14

openldap(a)semyon.org wrote: > On 13-09-23 10:16 PM, Howard Chu wrote: > >> If you backup the DB with slapcat and reload it on another server with >> slapadd, can you still reproduce the fault on the copy? > > Unfortunately, yes. It breaks trying to retrieve the same record as > before, on the same instruction in back_mdb: > > yesterday: > [126393.233615] slapd[19244]: segfault at 7f499f38d000 ip > 00007f4da0e9b6a1 sp 00007f499f1fa540 error 6 in > back_mdb-2.4.so.2.9.2[7f4da0e80000+34000] > > today: > [517860.953779] slapd[24871]: segfault at 7fdeb50a0000 ip > 00007fe2b63ad6a1 sp 00007fdeb4f0d540 error 6 in > back_mdb-2.4.so.2.9.2[7fe2b6392000+34000] > > Semyon A fix is now in git master. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7855) Update to latest autotools to support new architecture ppc64le
by hyc＠symas.com 15 May '14

15 May '14

rajesh(a)linux.vnet.ibm.com wrote: > Full_Name: Rajeshkumar S > Version: 2.4.39 > OS: Ubuntu 14.04 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (122.248.161.59) > > > > I am trying to build the openldap package from the source following the > release tarball from > ftp://ftp.openldap.org/pub/OpenLDAP/openldap-release/openldap-2.4.39.tgz. in a > new architecture ppc64le ( IBM PowerPC Little endian ). As the config.guess and > libtool did not have the required patches to identify this new architecture, I > did autoreconf -f -i in my build system whose latest automake and libtool has > the patches > of ppc64le. > autoreconf fails with automake errors as shown below > > automake: error: no 'Makefile.am' found for any configure output > autoreconf: automake failed with exit status: 1 That's to be expected, since there certainly are no Makefile.am files in our source tree. We don't use automake. This is not a bug. > As discussed in the mailing list, Quanah Gibson-Mount suggested me to request an > update for auto-tools to support new architectures. > > The automake with version >= 1.13.4 has the correct config.guess with the > required ppc64le fixes. > Regarding the libtool, the last release is more than 2 years ago I believe, we > managed to get an alpha source release which has all the latest patches to > support the new architecture ppc64le > > An alpha version of libtool with ppc64le support is available at > ftp://alpha.gnu.org/gnu/libtool/libtool-2.4.2.418.tar.gz. Libtool is so perpetually problematic that we will certainly not integrate an alpha version into our source tree. > Hence I request for an update of the auto tools in your build system to support > these new architectures. > > Thanks and Regards > Rajeshkumar S > Linux Technology Center, IBM > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#7855) Update to latest autotools to support new architecture ppc64le
by rajesh＠linux.vnet.ibm.com 15 May '14

15 May '14

Full_Name: Rajeshkumar S Version: 2.4.39 OS: Ubuntu 14.04 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (122.248.161.59) I am trying to build the openldap package from the source following the release tarball from ftp://ftp.openldap.org/pub/OpenLDAP/openldap-release/openldap-2.4.39.tgz. in a new architecture ppc64le ( IBM PowerPC Little endian ). As the config.guess and libtool did not have the required patches to identify this new architecture, I did autoreconf -f -i in my build system whose latest automake and libtool has the patches of ppc64le. autoreconf fails with automake errors as shown below automake: error: no 'Makefile.am' found for any configure output autoreconf: automake failed with exit status: 1 As discussed in the mailing list, Quanah Gibson-Mount suggested me to request an update for auto-tools to support new architectures. The automake with version >= 1.13.4 has the correct config.guess with the required ppc64le fixes. Regarding the libtool, the last release is more than 2 years ago I believe, we managed to get an alpha source release which has all the latest patches to support the new architecture ppc64le An alpha version of libtool with ppc64le support is available at ftp://alpha.gnu.org/gnu/libtool/libtool-2.4.2.418.tar.gz. Hence I request for an update of the auto tools in your build system to support these new architectures. Thanks and Regards Rajeshkumar S Linux Technology Center, IBM

1 0

Re: (ITS#7853) Improve safety of mdb_env_close
by hbf＠mail.uio.no 14 May '14

14 May '14

On 05/14/2014 01:18 AM, hyc(a)symas.com wrote: > dw(a)botanicus.net wrote: >> Currently if a user (wilfully or accidentally, say, through composition of >> third party libs) opens the same LMDB environment twice within a process, and >> maintains active read transactions at the time one mdb_env_close() is called, >> all reader slots will be deallocated in all environments due to the logic >> around line 4253 that unconditionally clears reader slots based on PID. >> >> I'd like to avoid this in py-lmdb, and seems there are a few alternatives to >> fix it: >> >> 1) Tell users not to do that (doc fix, nobody reads docs) > > This is already in the doc - don't open the same env twice within one process. 1.5) Use something else than fcntl locks on the lock-file descriptor to control whether to re-init the lockfile and its mutexes/semaphores. Like MDB_NOLOCK. Needed to get any other suggestions off the ground, see Caveats in lmdb.h. Once two MDB_env's in a process has the lockfile open, both lose their fcntl locks when either closes it - and the next mdb_env_open() will reset the lockfile. Then both readers and writers are screwed, and more care with reader slots won't help. Well, one MDB_env could leak its lockfile descriptor instead of closing it, or it could give the descriptor to the other MDB_env for a deferred close if it could find it using some global state. (Could work until someone wrote an open-close loop which would run into the max-open-FD limit.) Either way, no fun. Another way would be #ifdef ... more unportable code ... #endif:-( Surely some OSes have come up with sane way to address this problem, there's just no portable way. >> 2) Externally maintain a set of (st_dev, st_ino) keys representing already >> opened environments, and check any attempt to open an environment against this >> list, failing if data.mdb's key already appears. > > That would require global library state, which is definitely unwanted. Though in one sense we already have that, which is the problem. Global (well, per-process) lock state on open files and mutexes. >> 3) Disable the mdb_env_close() cleanup loop when MDB_NOTLS is enabled. Since no >> reader slot will ever exist in TLS in this mode, the existing mdb_env_close() >> doc "All transactions, databases, and cursors must already be closed before >> calling this function." ensures no readers exist at close time, and the loop >> need not run. > > Could do this, but since it's an incomplete solution, not sure there's any point. That's MDB_NOLOCK anyway. MDB_NOTLS does have reader table entries. > (...) -- Hallvard

1 0

(ITS#7854) minor patches to allow compile with VS2012
by louis＠steelbytes.com 14 May '14

14 May '14

Full_Name: Louis Solomon Version: 2.4.39 OS: Win32 & Win64 URL: Submission from: (NULL) (59.167.196.195) 1) in libraries/libldap/ldif.c move int ldif_debug = 0; to after _all_ the includes. Why? because the linker gets confused with it being declared as declspec(dllexport). Order matters. 2) in include/ac/socket.h EWOULDBLOCK, EINPROGRESS, ETIMEDOUT (lines 101-103) conflict with the windows header files. simply wrap with any rational choice of ifdef/ifndef 3) in include/ldap_utf8.h wrap prototypes for ldap_x_utf8_to_wc (line 61) through to ldap_x_mbs_utf8s (line 99) with #if SIZEOF_WCHAR>=4 to match the ifdef at the top of utf-8-conv.c. Why? because the linker tries to export them as they are defined in the header with declspec(dllexport) but are not in any .o/.obj 4) in include/ldap_cdefs.h the usage of __declspec(dllexport/dllimport) is not quite right ... should be like #if defined(_WIN32) ... # define LDAP_F(type) __declspec(dllimport) type # define LDAP_V(type) extern __declspec(dllimport) type #else # define LDAP_F(type) __declspec(dllexport) type # define LDAP_V(type) extern __declspec(dllexport) type #endif and of course the same for each of the other sets (LBER, LDAP, LDAP_AVL, LDAP_LDIF, etc) With the above patches and making the appropriate .h files from .hin files manually and then creating VS2012 project files, I was successfully able to create and use liblber, libldap, libldap_r as .dlls

1 0

Re: (ITS#7844) LMDB Delete Cursor inconsistencies
by armon.dadgar＠gmail.com 13 May '14

13 May '14

--5372ac85_8edbdab_1271 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline =46or now, we have application code to retry the delete until no further = rows are removed. Still, it would be nice to have this resolved (and tested) in master=21 Best Regards, Armon Dadgar =46rom:=C2=A0David Wilson dw=40hmmz.org Reply:=C2=A0David Wilson dw=40hmmz.org Date:=C2=A0May 13, 2014 at 3:41:08 PM To:=C2=A0armon.dadgar=40gmail.com armon.dadgar=40gmail.com Cc:=C2=A0openldap-its=40openldap.org openldap-its=40openldap.org Subject:=C2=A0 Re: (ITS=237844) LMDB Delete Cursor inconsistencies =20 This sounds very much like a known regression in mdb.master in the past =20 few months. If you are still affected, please retest against LMDB 0.9.11 = =20 which at least was not affected by the regression. =20 David =20 --5372ac85_8edbdab_1271 Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline <html><head><style>body=7Bfont-family:Helvetica,Arial;font-size:13px=7D</= style></head><body style=3D=22word-wrap: break-word; -webkit-nbsp-mode: s= pace; -webkit-line-break: after-white-space;=22><div id=3D=22bloop=5Fcust= omfont=22 style=3D=22font-family:Helvetica,Arial;font-size:13px; color: r= gba(0,0,0,1.0); margin: 0px; line-height: auto;=22>=46or now, we have app= lication code to retry the delete until no further rows are removed.</div= ><div id=3D=22bloop=5Fcustomfont=22 style=3D=22font-family:Helvetica,Aria= l;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;= =22>Still, it would be nice to have this resolved (and tested) in master=21= </div><div id=3D=22bloop=5Fcustomfont=22 style=3D=22font-family:Helvetica= ,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: = auto;=22><br></div> <div id=3D=22bloop=5Fsign=5F1400024102388026112=22 cl= ass=3D=22bloop=5Fsign=22><div style=3D=22font-family:helvetica,arial;font= -size:13px=22>Best Regards,</div><div style=3D=22font-family:helvetica,ar= ial;font-size:13px=22>Armon Dadgar<br></div></div> <div style=3D=22color:= black=22><br>=46rom: <span style=3D=22color:black=22>David Wilson</s= pan> <a href=3D=22mailto:dw=40hmmz.org=22>dw=40hmmz.org</a><br>Reply:&nbs= p;<span style=3D=22color:black=22>David Wilson</span> <a href=3D=22mailto= :dw=40hmmz.org=22>dw=40hmmz.org</a><br>Date: <span style=3D=22color:= black=22>May 13, 2014 at 3:41:08 PM</span><br>To: <span style=3D=22c= olor:black=22>armon.dadgar=40gmail.com</span> <a href=3D=22mailto:armon.d= adgar=40gmail.com=22>armon.dadgar=40gmail.com</a><br>Cc: <span style= =3D=22color:black=22>openldap-its=40openldap.org</span> <a href=3D=22mail= to:openldap-its=40openldap.org=22>openldap-its=40openldap.org</a><br>Subj= ect: <span style=3D=22color:black=22> Re: (ITS=237844) LMDB Delete C= ursor inconsistencies <br></span></div><br> <blockquote type=3D=22cite=22= class=3D=22clean=5Fbq=22><span><div><div></div><div>This sounds very muc= h like a known regression in mdb.master in the past <br>few months. If you are still affected, please retest against LMDB 0.9= .11 <br>which at least was not affected by the regression. <br> <br> <br>David <br></div></div></span></blockquote></body></html> --5372ac85_8edbdab_1271--

1 0

Re: (ITS#7853) Improve safety of mdb_env_close
by hyc＠symas.com 13 May '14

13 May '14

dw(a)botanicus.net wrote: > Full_Name: David Wilson > Version: LMDB 0.9.11 > OS: > URL: https://raw.githubusercontent.com/dw/py-lmdb/master/misc/readers_mrb_env.pa… > Submission from: (NULL) (178.238.153.20) > > > Currently if a user (wilfully or accidentally, say, through composition of > third party libs) opens the same LMDB environment twice within a process, and > maintains active read transactions at the time one mdb_env_close() is called, > all reader slots will be deallocated in all environments due to the logic > around line 4253 that unconditionally clears reader slots based on PID. > > I'd like to avoid this in py-lmdb, and seems there are a few alternatives to > fix it: > > > 1) Tell users not to do that (doc fix, nobody reads docs) This is already in the doc - don't open the same env twice within one process. > 2) Externally maintain a set of (st_dev, st_ino) keys representing already > opened environments, and check any attempt to open an environment against this > list, failing if data.mdb's key already appears. That would require global library state, which is definitely unwanted. > 3) Disable the mdb_env_close() cleanup loop when MDB_NOTLS is enabled. Since no > reader slot will ever exist in TLS in this mode, the existing mdb_env_close() > doc "All transactions, databases, and cursors must already be closed before > calling this function." ensures no readers exist at close time, and the loop > need not run. Could do this, but since it's an incomplete solution, not sure there's any point. > 4) Modify lock.mdb to include MDB_env* address within the process, and update > mdb_env_close() to invalidate only readers associated with the environment > being closed. I dislike using the MDB_env* as an opaque publicly visible > cookie, but perhaps storing this field might be reused for other things in > future. Hmm. I guess this could work. I think storing the MDB_env* is harmless since the pointer is meaningless to any other process. > Option 3 lets the user wilfully mix py-lmdb Environment objects within the > process, since py-lmdb always enables NOTLS, but it does not fix the case where > the user is integrating a Python application with a C library with some opaque > interface, and both the application and the library independently have need to > access the environment. Allowing multiple independent libraries to open the same environment would consume redundant chunks of address space. For large enough DBs this may simply fail. Unfortunately the semantics of fcntl locks prevents us from even detecting that this problem has occurred. > This kind of scenario regularly crops up in huge software projects, the user > may not even be aware the library uses LMDB internally. In that case, the user > is again exposed to having their reader slots silently become deallocated > through no fault of their own. > > > Option 4 is the least aesthetic, but has all the benefits of option 3 in > addition to preventing the "accidental integration" scenario. I've attached a > patch for it (against 0.9.11), although I completely understand if it does not > get applied. :) > > One alternative would be instead of using MDB_env* address, using a > monotonically increasing counter based on a static global, say, "mrb_token", > but this might break in the case two copies of LMDB were somehow linked in the > application. (It's easy to argue the user is an idiot in this case ;) Global state is verboten. > > > David > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7844) LMDB Delete Cursor inconsistencies
by dw＠hmmz.org 13 May '14

13 May '14

This sounds very much like a known regression in mdb.master in the past few months. If you are still affected, please retest against LMDB 0.9.11 which at least was not affected by the regression. David

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs