openldap-devel

openldap-devel@openldap.org

1416 discussions

None
by None 10 Aug '17

10 Aug '17

by outgoing.mit.edu (8.13.8/8.12.4) with ESMTP id v7AK4Rlv006874 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Thu, 10 Aug 2017 16:04:29 -0400 Subject: Re: LMDB encryption support To: Howard Chu <hyc(a)symas.com> References: <c4dab8cf-66b8-0873-f6a9-9c59a551a155(a)symas.com> From: Greg Hudson <ghudson(a)mit.edu> Message-ID: <e9dce464-9d5a-971a-bb8e-d4d95168bcb0(a)mit.edu> Date: Thu, 10 Aug 2017 16:04:27 -0400 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: <c4dab8cf-66b8-0873-f6a9-9c59a551a155(a)symas.com> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrFIsWRmVeSWpSXmKPExsUixG6nouu3pyfS4MtKU4vb236zWLxd8IDd gcnjy9JvLB6NCy4zBzBFcdmkpOZklqUW6dslcGXcutfKVLCGv+L37MQGxhM8XYycHBICJhKn Nv1m7mLk4hASWMwk8a39LyOEs5FRYs7UqSwQzlEmidU/bjCCtAgLqEhc7l7C3sXIwSEiICex bbcpSFhIwEbiYdsEFhCbTUBZYv3+rSwgJcwC9hL758mChHkFrCR2LHnICmKzCKhKnNw9lwnE FhWIkHjYuYsdokZQ4uTMJ2BjOAVsJU6u+gm2lVlAXeLPvEvMELa8xPa3c5gnMArMQtIyC0nZ LCRlCxiZVzHKpuRW6eYmZuYUpybrFicn5uWlFuka6uVmluilppRuYgQFKackzw7GM2+8DjEK cDAq8fAmiHZHCrEmlhVX5h5ilORgUhLlLf4EFOJLyk+pzEgszogvKs1JLT7EKMHBrCTC67K7 J1KINyWxsiq1KB8mJc3BoiTOK6HRGCEkkJ5YkpqdmlqQWgSTleHgUJLgfbQLqFGwKDU9tSIt M6cEIc3EwQkynAdouDfY8OKCxNzizHSI/ClGRSlx3j6QZgGQREZpHlwvOImkcpx5xSgO9Iow bx9IOw8wAcF1vwIazAQ0OMK3E2RwSSJCSqqB0fmdyYtHAlUfV9l5xtzetozfwO/8cbucal6G sO3fNLNWbfaqYUzW7Jc6sYqnufvuis0mRzTTJkWvfLZl3crXs3/7ZN59evrXcYfpDzdYm+7M FFHZsS3R62nEw+zN63wc8sS6vvhpLhA25NmcuZLj14FMt9enH1YsSqzXjl310U75+dGQRJX1 0UosxRmJhlrMRcWJANmcCPP9AgAA X-Spam-Score: -1.5 (-) X-Spam-Report: Spam detection software, running on the system "gauss.openldap.net", has identified this incoming email as possible spam. The original message has been attached to this so you can view it (if it isn't spam) or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: On 08/10/2017 11:55 AM, Howard Chu wrote: > Thoughts? Hardcode 1 algorithm, or leave it pluggable? Some thoughts, without advocating for either option: * If support isn't built-in, then generic LMDB tools (including mdb_copy/dump/load/stat) can't operate on encrypted databases, if they need plaintext pages to work. [...] Content analysis details: (-1.5 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -2.3 RCVD_IN_DNSWL_MED RBL: Sender listed at http://www.dnswl.org/, medium trust [18.9.25.12 listed in list.dnswl.org] -0.0 T_RP_MATCHES_RCVD Envelope sender domain matches handover relay domain -0.0 SPF_PASS SPF: sender matches SPF record 0.0 URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [URIs: cryptographyengineering.com] 0.8 BAYES_50 BODY: Bayes spam probability is 40 to 60% [score: 0.4992] 0.0 UNPARSEABLE_RELAY Informational: message has unparseable relay lines Cc: "OpenLDAP-devel(a)openldap.org" <OpenLDAP-devel(a)openldap.org> X-BeenThere: openldap-devel(a)openldap.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: OpenLDAP development discussion list <openldap-devel.openldap.org> List-Unsubscribe: <http://www.openldap.org/lists/mm/options/openldap-devel>, <mailto:openldap-devel-request@openldap.org?subject=unsubscribe> List-Archive: <http://www.openldap.org/lists/openldap-devel/> List-Post: <mailto:openldap-devel@openldap.org> List-Help: <mailto:openldap-devel-request@openldap.org?subject=help> List-Subscribe: <http://www.openldap.org/lists/mm/listinfo/openldap-devel>, <mailto:openldap-devel-request@openldap.org?subject=subscribe> X-List-Received-Date: Thu, 10 Aug 2017 20:04:40 -0000 On 08/10/2017 11:55 AM, Howard Chu wrote: > Thoughts? Hardcode 1 algorithm, or leave it pluggable? Some thoughts, without advocating for either option: * If support isn't built-in, then generic LMDB tools (including mdb_copy/dump/load/stat) can't operate on encrypted databases, if they need plaintext pages to work. * Built-in support doesn't necessarily mean hardcoding an algorithm for all time, if the meta pages can include an algorithm selector. One of the selector values could even mean "use application callbacks". * Is the page size guaranteed to be a multiple of 16 bytes? 32 bytes? I would assume yes to both; documenting that would make it easier to use block ciphers since ciphertext expansion isn't allowed. * Application writers are more likely to get encryption callbacks wrong than Howard is. They could ignore the IV (making it easy to detect duplicate initial blocks within a page) or even do pure ECB encryption (making it easy to detect duplicate blocks anywhere). Less egregiously, applications might not make the ideal choice of cipher mode. I would personally have to think about the best choice to use. If I were using a block cipher, CBC with the provided ivec seems like it should be okay, but assuming 128-bit cipher blocks, after around 2^64 blocks one would expect to experience a block collision which reveals the XOR of the plaintexts of the preceding two blocks[1]. Deriving a key with HKDF(key, ivec) and using counter mode might be safer, unless I'm missing something, which I easily could be. If I were using a stream cipher, I would have to do research to figure out how to incorporate the ivec. * Not wanting to depend on crypto libraries seems like a valid concern. Teaching the LMDB code how to dynamically load encryption plugins doesn't necessarily seem attractive either. [1] https://blog.cryptographyengineering.com/2016/08/24/attack-of-week-64-bit-c…

1 0

Re: LMDB encryption support
by Howard Chu 10 Aug '17

10 Aug '17

Timur Kristóf wrote: > Hi, > >> I've recently added support for page-level encryption to LMDB 1.x >> using user-supplied callbacks > > That does sound cool. :) > >> One question is whether we should actually make this pluggable like >> this, or >> we should just hardcode support for a specific algorithm and leave it >> at that. > > I vote on keeping it pluggable, so every crypograpy nut out there can > use their favourite mechanism. Yeah, that's still my inclination as well. And yes, there's a reference chacha20 implementation already, which I've been using for testing. >> One >> complication is that if the algorithm is actually user-selectable, we >> need to >> dynamically adjust DB page layouts to accommodate different nonce/IV >> and >> signature sizes. (Currently MDB_page metadata is a statically >> defined >> structure. A dynamic size element here will make processing slower.) > > What if page size would still be static, but that static size would be > user-defined on a per-environment basis? We sort of support that already, allowing page sizes larger than the OS pagesize to be used. So I guess it's not too big of a change. > Question: will this affect performance on non-encrypted databases? Ideally, not. ;) It's a bit early to tell. Anyway, the API I originally quoted needs to be tweaked to accomodate the authentication signature support so this is all still in flux. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

LMDB encryption support
by Howard Chu 10 Aug '17

10 Aug '17

I've recently added support for page-level encryption to LMDB 1.x using user-supplied callbacks: /** @brief A callback function used to encrypt/decrypt pages in the env. * * Encrypt or decrypt the data in src and store the result in dst using the * provided key. The result must be the same number of bytes as the input. * The input size will always be a multiple of the page size. * @param[in] src The input data to be transformed. * @param[out] dst Storage for the result. * @param[in] key An array of two values: key[0] is the encryption key, * and key[1] is the initialization vector. * @param[in] encdec 1 to encrypt, 0 to decrypt. */ typedef void (MDB_enc_func)(const MDB_val *src, MDB_val *dst, const MDB_val *key, int encdec); /** @brief Set encryption on an environment. * * This must be called before #mdb_env_open(). * It implicitly sets #MDB_REMAP_CHUNKS on the env. * @param[in] env An environment handle returned by #mdb_env_create(). * @param[in] func An #MDB_enc_func function. * @param[in] key An array of two values: key[0] is the encryption key, * and key[1] is the initialization vector. * @return A non-zero error value on failure and 0 on success. */ int mdb_env_set_encrypt(MDB_env *env, MDB_enc_func *func, const MDB_val *key); I intend to extend this a bit further to support authenticated encryption instead, so that each page also yields a signature that can be used to detect tampering or corruption. One question is whether we should actually make this pluggable like this, or we should just hardcode support for a specific algorithm and leave it at that. For comparison, BerkeleyDB supported AES128 with HMAC-SHA1 for a 20 byte per page signature. In some ways it simplifies use if the DB just takes care of everything and there's only one hardcoded mechanism. Also, user-supplied callback relies on the app developer to know what they're doing with the crypto code. From the LMDB maintenance perspective, it's simpler for us not to have hardcoded dependencies on crypto libraries. Also, leaving it pluggable keeps the door open for 3rd party hardware-accelerated crypto engines. One complication is that if the algorithm is actually user-selectable, we need to dynamically adjust DB page layouts to accommodate different nonce/IV and signature sizes. (Currently MDB_page metadata is a statically defined structure. A dynamic size element here will make processing slower.) Thoughts? Hardcode 1 algorithm, or leave it pluggable? -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

4 3

Re: ITS#8654 - Option for LDAP client to bind to a local address
by Quanah Gibson-Mount 08 Aug '17

08 Aug '17

--On Monday, August 07, 2017 4:49 PM -0700 Ryan Tandy <ryan(a)nardis.ca> wrote: For easier digestion, the patch supplied by Daniel is available in my scratch repo in the its8654 branch. I did change the socket option to match the MS socket option. <https://github.com/quanah/openldap-scratch/tree/its8654> --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

RE: ITS#8654 - Option for LDAP client to bind to a local address
by Quanah Gibson-Mount 07 Aug '17

07 Aug '17

Hi Daniel, I would suggest sending in a patch against master to the -devel list for review. For final inclusion if it is approved, see: <http://www.openldap.org/devel/contributing.html> Regards, Quanah --On Monday, June 12, 2017 8:34 PM +0000 Daniel Le <daniel.le(a)exfo.com> wrote: > I've got a chance to write (and test) the code to add API support for > socket binding addresses. > > Should I send the code diff to this openldap-devel email list for review? > How to submit a patch request? > > Daniel > > -----Original Message----- > From: Daniel Le > Sent: Tuesday, May 16, 2017 6:02 PM > To: 'openldap-devel(a)openldap.org' <openldap-devel(a)openldap.org> > Subject: ITS#8654 - Option for LDAP client to bind to a local address > > Hello, > > In reference to the enhancement request ITS#865, please comment on the > following to add support for binding a local IP address to client socket. > This is just an outline of changes for one local address. I am not sure > whether a list of local addresses is necessary. If it is, then a new > function, similarly to ldap_url_parsehosts, may be written to parse the > list of local addresses and store them into a linked list. In my use > case, only one IPv4 or IPv6 local address is used for binding. > > - Modify ldap.h and ldap_set_option to handle the new option > LDAP_OPT_LOCAL_ADDRESS. Should it be named LDAP_OPT_CLIENT_ADDRESS, > LDAP_OPT_SOCKET_BIND_ADDRESS...? > > - Modify struct ldapoptions in ldap-int.h to add element "char > *ldo_local_address" to hold client local address when > ldap_set_option(LDAP_OPT_LOCAL_ADDRESS...) is executed. This can char > pointer can point to an IPv4 address or IPv6 address. > > - ldap_connect_to_host() in os-ip.c > After the connection socket is created (ldap_int_socket) and before it > is connected (ldap_pvt_connect), extract the local IP address. If local > address family (AF_INET/ AF_INET6) matches the one of the host, bind > socket to the local address. > > Regards, > Daniel > > -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

3 3

slapo-memberOf and replication (was ITS#8613)
by Michael Ströder 30 Jul '17

30 Jul '17

Hmm, thinking about this some more... slapo-dynlist(5) says: "dynamically added attributes do not participate in the filter matching phase of the search request handling." This is a big drawback of slapo-dynlist rendering the work-arounds mentioned in ITS#8613 nearly useless. I have to step back a bit: Why is 'memberOf' not replicated? I vaguely remember other issues: https://www.openldap.org/its/index.cgi?findid=6915 https://www.openldap.org/its/index.cgi?findid=6766 https://www.openldap.org/its/index.cgi?findid=7710 But I wonder whether the real underlying issues might have been fixed in 2.4.x releases. How much effort is it to try to replicate 'memberOf' attribute again? Ciao, Michael.

1 0

Variadic Debug() macro
by Ondřej Kuzník 14 Jul '17

14 Jul '17

I think it is time to modernise the debug logging a bit, this would make our life easier when logging and running make (with the copious warnings compilers will generate unless one passes -Wno-format-extra-args). The major obstacle to this is the thousands of lines that need adjusting, luckily there is a tool named Coccinelle[0], co-developed with the Linux kernel. It can work with so-called semantic patches and apply them on a code base to prepare a final patch. The work so far (that seems to pass the same tests that master does on loglevel any) is available here: ftp://ftp.openldap.org/incoming/Ondrej-Kuznik-20170714-variadic-debug.tgz If you want to replicate the generated patches, you will have to use the latest version of Coccinelle from git[1] to have them correctly processed. To help Coccinelle parse OpenLDAP sources, you will also have to pass "--macro-file-builtins macros.h". You can skip 02-shortcut.cocci, but expect to wait a few days before Coccinelle processes some of the more if()fy files if you do so. Sample command line: spatch --sp-file 01-variadic.cocci --macro-file-builtins macros.h --dir . If we aim to apply this to master+2.5, we will have the time to stabilise any unexpected fallout before 2.5 is ready since the resulting patches are still rather large. It is probably too late to touch 2.4, however hard it might make to backport any fixes, unless we start to depend on Coccinelle for backporting just like the Linux kernel does. I welcome your review and suggestions. [0]. http://coccinelle.lip6.fr/ [1]. https://github.com/coccinelle/coccinelle -- Ondřej Kuzník Senior Software Engineer Symas Corporation http://www.symas.com Packaged, certified, and supported LDAP solutions powered by OpenLDAP

2 1

Re: ITS#8573 code for review (command line TLS options)
by Quanah Gibson-Mount 29 Jun '17

29 Jun '17

--On Thursday, June 08, 2017 6:13 PM -0700 Quanah Gibson-Mount <quanah(a)symas.com> wrote: > --On Thursday, June 08, 2017 5:53 PM -0700 Quanah Gibson-Mount > <quanah(a)symas.com> wrote: > >> Attached for review is code to add TLS command line options to the client >> tools. Included are documentation updates to the manual pages and a >> related test suite. > > Or here's a gzip'd version, since it appears to have gotten mangled. Alternatively, here's a link to the relevant branch in my github repo: <https://github.com/quanah/openldap-scratch/tree/its8573-tables> I've added a couple of new test cases, for validating TLS encryption of syncrepl, and expanded test067 to also confirm that reqcert=never works as expected. For those who might like to test the code against RE24, there is this branch as well: <https://github.com/quanah/openldap-scratch/tree/RE24-its8573-tables> --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

RE: ITS#8654 - Option for LDAP client to bind to a local address
by Quanah Gibson-Mount 28 Jun '17

28 Jun '17

--On Thursday, June 15, 2017 3:29 PM +0000 Daniel Le <daniel.le(a)exfo.com> wrote: > Could someone take a stab at reviewing this code change and/or let me > know questions you might have? The ldap_get_option, ldap_set_option and > socket binding functions were successfully validated using a test program > under Linux/Unix. > > Unfortunately, I don't have a set-up to test other operating systems, but > would be willing to further modidy ldap_connect_to_host in os-ip.c to > cover them if you can help verify it. Howard's been hoping someone would be able to review in addition to him. I gave it a lookover, but my C is quite rusty. I could test it out on Solaris, and possibly get it tested with windows. That's assuming you had it added as an option to the client utilities (ldapsearch, etc), similar to what I did with the TLS options I put out for review. Also, I would strongly advise creating your own git repository on somewhre like github, like I did with my openldap-scratch repo, where you can commit your changes and work on them, and then create a single diff containing the entire changeset. Example: <https://github.com/quanah/openldap-scratch/tree/its8573-tables> --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

ITS#8654 - Option for LDAP client to bind to a local address
by Daniel Le 12 Jun '17

12 Jun '17

Hello, In reference to the enhancement request ITS#865, please comment on the following to add support for binding a local IP address to client socket. This is just an outline of changes for one local address. I am not sure whether a list of local addresses is necessary. If it is, then a new function, similarly to ldap_url_parsehosts, may be written to parse the list of local addresses and store them into a linked list. In my use case, only one IPv4 or IPv6 local address is used for binding. - Modify ldap.h and ldap_set_option to handle the new option LDAP_OPT_LOCAL_ADDRESS. Should it be named LDAP_OPT_CLIENT_ADDRESS, LDAP_OPT_SOCKET_BIND_ADDRESS...? - Modify struct ldapoptions in ldap-int.h to add element "char *ldo_local_address" to hold client local address when ldap_set_option(LDAP_OPT_LOCAL_ADDRESS...) is executed. This can char pointer can point to an IPv4 address or IPv6 address. - ldap_connect_to_host() in os-ip.c After the connection socket is created (ldap_int_socket) and before it is connected (ldap_pvt_connect), extract the local IP address. If local address family (AF_INET/ AF_INET6) matches the one of the host, bind socket to the local address. Regards, Daniel

1 1

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-devel