openldap-devel August 2017

openldap-devel@openldap.org

7 participants
7 discussions

asserts and manadatory build instructions (was ITS#8240)
by Michael Ströder 05 Nov '21

05 Nov '21

hyc(a)symas.com wrote in ITS#8240: > Our patch response was too hasty. There is no OpenLDAP bug here, the real > issue is production binaries being built with asserts enabled instead of > compiling with -DNDEBUG. That's an issue for packagers and distros to resolve. > Closing this ITS, not an OpenLDAP bug. Maybe I missed something. But this is the first time I've heard about -DNDEBUG being mandatory when compiling binary packages for production use. Does it have other effects? And what are general rules for assert statements in OpenLDAP code? In my own (Python) code assert statements are supposed to be only triggered if something goes wrong *internally* (type issues etc.). If somebody manages to trigger an assert statement with invalid input from "outside" I always consider this to be a serious bug revealing insufficient error handling even though e.g. web2ldap just logs the exception but won't crash. YMMV, but please clarify. I also wonder whether there are more mandatory rules for building packages and where I can find them. Please don't get me wrong: My inquiry is in good faith to avoid unnecessary ITS based on misunderstanding. Ciao, Michael.

1 1

Re: LMDB encryption support
by Howard Chu 10 Aug '17

10 Aug '17

Greg Hudson wrote: > On 08/10/2017 11:55 AM, Howard Chu wrote: >> Thoughts? Hardcode 1 algorithm, or leave it pluggable? > > Some thoughts, without advocating for either option: > > * If support isn't built-in, then generic LMDB tools (including > mdb_copy/dump/load/stat) can't operate on encrypted databases, if they > need plaintext pages to work. Yeah, already thought about that. We can add an option to the generic tools to dynamically load a user-supplied module for such cases. I always wanted this for BerkeleyDB as well, to safely operate on DBs with custom comparators. > * Built-in support doesn't necessarily mean hardcoding an algorithm for > all time, if the meta pages can include an algorithm selector. One of > the selector values could even mean "use application callbacks". > > * Is the page size guaranteed to be a multiple of 16 bytes? 32 bytes? > I would assume yes to both; documenting that would make it easier to use > block ciphers since ciphertext expansion isn't allowed. Yes, page sizes are always large powers of 2. 4096 bytes is typical (but on the small side). SPARC uses 8192, some MIPS systems use 32768 or 65536. > * Application writers are more likely to get encryption callbacks wrong > than Howard is. They could ignore the IV (making it easy to detect > duplicate initial blocks within a page) or even do pure ECB encryption > (making it easy to detect duplicate blocks anywhere). Less egregiously, > applications might not make the ideal choice of cipher mode. I would > personally have to think about the best choice to use. If I were using > a block cipher, CBC with the provided ivec seems like it should be okay, > but assuming 128-bit cipher blocks, after around 2^64 blocks one would > expect to experience a block collision which reveals the XOR of the > plaintexts of the preceding two blocks[1]. Deriving a key with > HKDF(key, ivec) and using counter mode might be safer, unless I'm > missing something, which I easily could be. If I were using a stream > cipher, I would have to do research to figure out how to incorporate the > ivec. The user-supplied IV is really just a seed, it will be hashed with some other uniqifiers (pageID,txnID) before being passed to the cipher. I suppose we could make some recommendations on ciphers and modes, but really I think it's up to the user to determine what kind of strength/speed tradeoffs they'll accept. I would expect stream ciphers to be used, in general. > * Not wanting to depend on crypto libraries seems like a valid concern. > Teaching the LMDB code how to dynamically load encryption plugins > doesn't necessarily seem attractive either. We'll probably do the dynamic loading anyway, as noted above. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

None
by None 10 Aug '17

10 Aug '17

by outgoing.mit.edu (8.13.8/8.12.4) with ESMTP id v7AK4Rlv006874 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Thu, 10 Aug 2017 16:04:29 -0400 Subject: Re: LMDB encryption support To: Howard Chu <hyc(a)symas.com> References: <c4dab8cf-66b8-0873-f6a9-9c59a551a155(a)symas.com> From: Greg Hudson <ghudson(a)mit.edu> Message-ID: <e9dce464-9d5a-971a-bb8e-d4d95168bcb0(a)mit.edu> Date: Thu, 10 Aug 2017 16:04:27 -0400 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: <c4dab8cf-66b8-0873-f6a9-9c59a551a155(a)symas.com> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrFIsWRmVeSWpSXmKPExsUixG6nouu3pyfS4MtKU4vb236zWLxd8IDd gcnjy9JvLB6NCy4zBzBFcdmkpOZklqUW6dslcGXcutfKVLCGv+L37MQGxhM8XYycHBICJhKn Nv1m7mLk4hASWMwk8a39LyOEs5FRYs7UqSwQzlEmidU/bjCCtAgLqEhc7l7C3sXIwSEiICex bbcpSFhIwEbiYdsEFhCbTUBZYv3+rSwgJcwC9hL758mChHkFrCR2LHnICmKzCKhKnNw9lwnE FhWIkHjYuYsdokZQ4uTMJ2BjOAVsJU6u+gm2lVlAXeLPvEvMELa8xPa3c5gnMArMQtIyC0nZ LCRlCxiZVzHKpuRW6eYmZuYUpybrFicn5uWlFuka6uVmluilppRuYgQFKackzw7GM2+8DjEK cDAq8fAmiHZHCrEmlhVX5h5ilORgUhLlLf4EFOJLyk+pzEgszogvKs1JLT7EKMHBrCTC67K7 J1KINyWxsiq1KB8mJc3BoiTOK6HRGCEkkJ5YkpqdmlqQWgSTleHgUJLgfbQLqFGwKDU9tSIt M6cEIc3EwQkynAdouDfY8OKCxNzizHSI/ClGRSlx3j6QZgGQREZpHlwvOImkcpx5xSgO9Iow bx9IOw8wAcF1vwIazAQ0OMK3E2RwSSJCSqqB0fmdyYtHAlUfV9l5xtzetozfwO/8cbucal6G sO3fNLNWbfaqYUzW7Jc6sYqnufvuis0mRzTTJkWvfLZl3crXs3/7ZN59evrXcYfpDzdYm+7M FFHZsS3R62nEw+zN63wc8sS6vvhpLhA25NmcuZLj14FMt9enH1YsSqzXjl310U75+dGQRJX1 0UosxRmJhlrMRcWJANmcCPP9AgAA X-Spam-Score: -1.5 (-) X-Spam-Report: Spam detection software, running on the system "gauss.openldap.net", has identified this incoming email as possible spam. The original message has been attached to this so you can view it (if it isn't spam) or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: On 08/10/2017 11:55 AM, Howard Chu wrote: > Thoughts? Hardcode 1 algorithm, or leave it pluggable? Some thoughts, without advocating for either option: * If support isn't built-in, then generic LMDB tools (including mdb_copy/dump/load/stat) can't operate on encrypted databases, if they need plaintext pages to work. [...] Content analysis details: (-1.5 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -2.3 RCVD_IN_DNSWL_MED RBL: Sender listed at http://www.dnswl.org/, medium trust [18.9.25.12 listed in list.dnswl.org] -0.0 T_RP_MATCHES_RCVD Envelope sender domain matches handover relay domain -0.0 SPF_PASS SPF: sender matches SPF record 0.0 URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [URIs: cryptographyengineering.com] 0.8 BAYES_50 BODY: Bayes spam probability is 40 to 60% [score: 0.4992] 0.0 UNPARSEABLE_RELAY Informational: message has unparseable relay lines Cc: "OpenLDAP-devel(a)openldap.org" <OpenLDAP-devel(a)openldap.org> X-BeenThere: openldap-devel(a)openldap.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: OpenLDAP development discussion list <openldap-devel.openldap.org> List-Unsubscribe: <http://www.openldap.org/lists/mm/options/openldap-devel>, <mailto:openldap-devel-request@openldap.org?subject=unsubscribe> List-Archive: <http://www.openldap.org/lists/openldap-devel/> List-Post: <mailto:openldap-devel@openldap.org> List-Help: <mailto:openldap-devel-request@openldap.org?subject=help> List-Subscribe: <http://www.openldap.org/lists/mm/listinfo/openldap-devel>, <mailto:openldap-devel-request@openldap.org?subject=subscribe> X-List-Received-Date: Thu, 10 Aug 2017 20:04:40 -0000 On 08/10/2017 11:55 AM, Howard Chu wrote: > Thoughts? Hardcode 1 algorithm, or leave it pluggable? Some thoughts, without advocating for either option: * If support isn't built-in, then generic LMDB tools (including mdb_copy/dump/load/stat) can't operate on encrypted databases, if they need plaintext pages to work. * Built-in support doesn't necessarily mean hardcoding an algorithm for all time, if the meta pages can include an algorithm selector. One of the selector values could even mean "use application callbacks". * Is the page size guaranteed to be a multiple of 16 bytes? 32 bytes? I would assume yes to both; documenting that would make it easier to use block ciphers since ciphertext expansion isn't allowed. * Application writers are more likely to get encryption callbacks wrong than Howard is. They could ignore the IV (making it easy to detect duplicate initial blocks within a page) or even do pure ECB encryption (making it easy to detect duplicate blocks anywhere). Less egregiously, applications might not make the ideal choice of cipher mode. I would personally have to think about the best choice to use. If I were using a block cipher, CBC with the provided ivec seems like it should be okay, but assuming 128-bit cipher blocks, after around 2^64 blocks one would expect to experience a block collision which reveals the XOR of the plaintexts of the preceding two blocks[1]. Deriving a key with HKDF(key, ivec) and using counter mode might be safer, unless I'm missing something, which I easily could be. If I were using a stream cipher, I would have to do research to figure out how to incorporate the ivec. * Not wanting to depend on crypto libraries seems like a valid concern. Teaching the LMDB code how to dynamically load encryption plugins doesn't necessarily seem attractive either. [1] https://blog.cryptographyengineering.com/2016/08/24/attack-of-week-64-bit-c…

1 0

Re: LMDB encryption support
by Howard Chu 10 Aug '17

10 Aug '17

Timur Kristóf wrote: > Hi, > >> I've recently added support for page-level encryption to LMDB 1.x >> using user-supplied callbacks > > That does sound cool. :) > >> One question is whether we should actually make this pluggable like >> this, or >> we should just hardcode support for a specific algorithm and leave it >> at that. > > I vote on keeping it pluggable, so every crypograpy nut out there can > use their favourite mechanism. Yeah, that's still my inclination as well. And yes, there's a reference chacha20 implementation already, which I've been using for testing. >> One >> complication is that if the algorithm is actually user-selectable, we >> need to >> dynamically adjust DB page layouts to accommodate different nonce/IV >> and >> signature sizes. (Currently MDB_page metadata is a statically >> defined >> structure. A dynamic size element here will make processing slower.) > > What if page size would still be static, but that static size would be > user-defined on a per-environment basis? We sort of support that already, allowing page sizes larger than the OS pagesize to be used. So I guess it's not too big of a change. > Question: will this affect performance on non-encrypted databases? Ideally, not. ;) It's a bit early to tell. Anyway, the API I originally quoted needs to be tweaked to accomodate the authentication signature support so this is all still in flux. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

LMDB encryption support
by Howard Chu 10 Aug '17

10 Aug '17

I've recently added support for page-level encryption to LMDB 1.x using user-supplied callbacks: /** @brief A callback function used to encrypt/decrypt pages in the env. * * Encrypt or decrypt the data in src and store the result in dst using the * provided key. The result must be the same number of bytes as the input. * The input size will always be a multiple of the page size. * @param[in] src The input data to be transformed. * @param[out] dst Storage for the result. * @param[in] key An array of two values: key[0] is the encryption key, * and key[1] is the initialization vector. * @param[in] encdec 1 to encrypt, 0 to decrypt. */ typedef void (MDB_enc_func)(const MDB_val *src, MDB_val *dst, const MDB_val *key, int encdec); /** @brief Set encryption on an environment. * * This must be called before #mdb_env_open(). * It implicitly sets #MDB_REMAP_CHUNKS on the env. * @param[in] env An environment handle returned by #mdb_env_create(). * @param[in] func An #MDB_enc_func function. * @param[in] key An array of two values: key[0] is the encryption key, * and key[1] is the initialization vector. * @return A non-zero error value on failure and 0 on success. */ int mdb_env_set_encrypt(MDB_env *env, MDB_enc_func *func, const MDB_val *key); I intend to extend this a bit further to support authenticated encryption instead, so that each page also yields a signature that can be used to detect tampering or corruption. One question is whether we should actually make this pluggable like this, or we should just hardcode support for a specific algorithm and leave it at that. For comparison, BerkeleyDB supported AES128 with HMAC-SHA1 for a 20 byte per page signature. In some ways it simplifies use if the DB just takes care of everything and there's only one hardcoded mechanism. Also, user-supplied callback relies on the app developer to know what they're doing with the crypto code. From the LMDB maintenance perspective, it's simpler for us not to have hardcoded dependencies on crypto libraries. Also, leaving it pluggable keeps the door open for 3rd party hardware-accelerated crypto engines. One complication is that if the algorithm is actually user-selectable, we need to dynamically adjust DB page layouts to accommodate different nonce/IV and signature sizes. (Currently MDB_page metadata is a statically defined structure. A dynamic size element here will make processing slower.) Thoughts? Hardcode 1 algorithm, or leave it pluggable? -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

4 3

Re: ITS#8654 - Option for LDAP client to bind to a local address
by Quanah Gibson-Mount 08 Aug '17

08 Aug '17

--On Monday, August 07, 2017 4:49 PM -0700 Ryan Tandy <ryan(a)nardis.ca> wrote: For easier digestion, the patch supplied by Daniel is available in my scratch repo in the its8654 branch. I did change the socket option to match the MS socket option. <https://github.com/quanah/openldap-scratch/tree/its8654> --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

RE: ITS#8654 - Option for LDAP client to bind to a local address
by Quanah Gibson-Mount 08 Aug '17

08 Aug '17

Hi Daniel, I would suggest sending in a patch against master to the -devel list for review. For final inclusion if it is approved, see: <http://www.openldap.org/devel/contributing.html> Regards, Quanah --On Monday, June 12, 2017 8:34 PM +0000 Daniel Le <daniel.le(a)exfo.com> wrote: > I've got a chance to write (and test) the code to add API support for > socket binding addresses. > > Should I send the code diff to this openldap-devel email list for review? > How to submit a patch request? > > Daniel > > -----Original Message----- > From: Daniel Le > Sent: Tuesday, May 16, 2017 6:02 PM > To: 'openldap-devel(a)openldap.org' <openldap-devel(a)openldap.org> > Subject: ITS#8654 - Option for LDAP client to bind to a local address > > Hello, > > In reference to the enhancement request ITS#865, please comment on the > following to add support for binding a local IP address to client socket. > This is just an outline of changes for one local address. I am not sure > whether a list of local addresses is necessary. If it is, then a new > function, similarly to ldap_url_parsehosts, may be written to parse the > list of local addresses and store them into a linked list. In my use > case, only one IPv4 or IPv6 local address is used for binding. > > - Modify ldap.h and ldap_set_option to handle the new option > LDAP_OPT_LOCAL_ADDRESS. Should it be named LDAP_OPT_CLIENT_ADDRESS, > LDAP_OPT_SOCKET_BIND_ADDRESS...? > > - Modify struct ldapoptions in ldap-int.h to add element "char > *ldo_local_address" to hold client local address when > ldap_set_option(LDAP_OPT_LOCAL_ADDRESS...) is executed. This can char > pointer can point to an IPv4 address or IPv6 address. > > - ldap_connect_to_host() in os-ip.c > After the connection socket is created (ldap_int_socket) and before it > is connected (ldap_pvt_connect), extract the local IP address. If local > address family (AF_INET/ AF_INET6) matches the one of the host, bind > socket to the local address. > > Regards, > Daniel > > -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

3 3

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-devel August 2017