[LMDB] Unexpected behavior for MDB_MULTIPLE with zero-length page
by Bryan Matsuo
Hi,
I was working on the lmdb-go bindings for LMDB, using the 0.9.19 release
and believe I have found a bug with MDB_MULTIPLE. The bug still exists on
the latest commit of branch
mdb.RE/0.9, a87c8fd8c9a4ce49be18a642e3572059f39ed1cf, I'm not sure if this
bug is part of what held the 0.9.20 release back. But I wanted to check
here.
I noticed that, for in a corner case of the mdb_cursor_put function when
MDB_MULTIPLE is provided, if the number of items is specified to be zero
this may lead to garbage data being written or a segfault. My intuition
would be that this is essentially a noop, but I also would have thought
EINVAL would be an acceptable error code for this strange use of
mdb_cursor_put. The actual behavior of LMDB does not seem acceptable.
I've included an barebones example which I have written in C to demonstrate
the problem. Please let me know what you think.
- Bryan
Here is my program's output
$ ./mdb_putmultiple_empty -V
LMDB 0.9.20: (January 11, 2017)
$ ./mdb_putmultiple_empty -n putmultiple_empty.mdb && echo ok
Segmentation fault (core dumped)
And, here is the program source
/* mdb_putmultiple_empty.c - test of MDB_PUT_MULTIPLE given zero items*/
#ifdef _WIN32
#include <windows.h>
#endif
#include <stdio.h>
#include <stdlib.h>
#include <signal.h>
#include "lmdb.h"
static void
sighandle(int sig)
{
}
int main(int argc,char * argv[])
{
int rc;
MDB_env *env;
MDB_txn *txn;
MDB_cursor *cursor;
MDB_dbi dbi;
MDB_val vals[3];
const char *progname = argv[0], *act;
unsigned flags = 0;
for (; argc > 1 && argv[1][0] == '-'; argc--, argv++) {
if (argv[1][1] == 'n' && argv[1][2] == '\0')
flags |= MDB_NOSUBDIR;
else if (argv[1][1] == 'V' && argv[1][2] == '\0') {
printf("%s\n", MDB_VERSION_STRING);
exit(0);
} else
argc = 0;
}
if (argc<2 || argc>3) {
fprintf(stderr, "usage: %s [-V] [-n] srcpath\n", progname);
exit(EXIT_FAILURE);
}
#ifdef SIGPIPE
signal(SIGPIPE, sighandle);
#endif
#ifdef SIGHUP
signal(SIGHUP, sighandle);
#endif
signal(SIGINT, sighandle);
signal(SIGTERM, sighandle);
act = "creating environment";
rc = mdb_env_create(&env);
if (rc == MDB_SUCCESS) {
act = "setting maxdbs";
rc = mdb_env_set_maxdbs(env, 1);
}
if (rc == MDB_SUCCESS) {
act = "opening environment";
rc = mdb_env_open(env, argv[1], flags, 0600);
}
if (rc == MDB_SUCCESS) {
act = "beginning txn";
rc = mdb_txn_begin(env, 0,0, &txn);
}
if (rc == MDB_SUCCESS) {
act = "opening dbi";
rc = mdb_dbi_open(txn, "db", MDB_CREATE|MDB_DUPSORT|MDB_DUPFIXED,
&dbi);
}
if (rc == MDB_SUCCESS) {
act = "opening cursor";
rc = mdb_cursor_open(txn, dbi, &cursor);
}
if (rc == MDB_SUCCESS) {
act = "put multiple with zero items but non-empty stride";
vals[0].mv_size = 1;
vals[0].mv_data = "x";
vals[1].mv_size = 2;
vals[1].mv_data = 0;
vals[2].mv_size = 0;
vals[2].mv_data = 0;
rc = mdb_cursor_put(cursor, &vals[0], &vals[1], MDB_MULTIPLE);
}
if (rc == MDB_SUCCESS) {
act = "put multiple with zero items but non-empty stride";
vals[0].mv_size = 0;
vals[0].mv_data = 0;
vals[1].mv_size = 0;
vals[1].mv_data = 0;
rc = mdb_cursor_get(cursor, &vals[0], &vals[1], MDB_FIRST);
}
if (rc == MDB_SUCCESS) {
fprintf(stderr, "FIRST: %.*s %.*s",
(int)vals[0].mv_size, (char*)vals[0].mv_data,
(int)vals[1].mv_size, (char*)vals[1].mv_data);
}
if (rc)
fprintf(stderr, "%s: %s failed, error %d (%s)\n",
progname, act, rc, mdb_strerror(rc));
mdb_txn_abort(txn);
mdb_env_close(env);
return rc ? EXIT_FAILURE : EXIT_SUCCESS;
}
6 years, 9 months
OpenLDAP backup best practices
by Nikolas Stylianides
Hi there.
I have a setup of two servers in mirror mode.
What is the best policy for backup.
Currently i am dumping the DIT using slapcat every midnight.
Is there another way to be able to take more frequent backups?
Is there an incremental backup feature?
Thank you in advance.
--
Δρ. Νικόλας Στυλιανίδης
Ηλεκτρολόγος Μηχανικός και Μηχ. Υπολογιστών
Nikolas Stylianides, Dr.
Dr. Eng. in Electrical & Computer Engineering
Contacts
-------------
Mobile Tel.: +35796741315
Email: nstylianides(a)leafnet.com.cy, nstylianides(a)gmail.com
Skype: nicostyl
Affilication
---------------
LEAF NET LTD: Research & Development
Open University of Cyprus: Research Associate, APPLIED HEALTH INFORMATICS
Master Programme Academic Board Member
Tο λακωνίζειν εστί φιλοσοφείν / Μηδέν Άγαν - Χίλων ο Λακεδαιμόνιος:
Brevity is the soul of wit - Shakespeare William (Hamlet)
6 years, 10 months
Re: RE24 testing call (2.4.45) LMDB RE0.9 testing call (0.9.20)
by Quanah Gibson-Mount
--On Friday, February 10, 2017 12:27 AM +0100 Michael Ströder
<michael(a)stroeder.com> wrote:
> Quanah Gibson-Mount wrote:
>> --On Thursday, February 09, 2017 10:34 PM +0100 "A. Schulze"
>> <sca(a)andreasschulze.de> wrote:
>>
>>> So my guess: openldap not call an important openssl library function and
>>> so openssl use it's defaults.
>>
>> Yes, that sounds like a serious bug. Please file an ITS on it.
>
> Is this similar to ITS#8529?
Sounds exactly like it, thanks!
--Quanah
--
Quanah Gibson-Mount
Product Architect
Symas Corporation
Packaged, certified, and supported LDAP solutions powered by OpenLDAP:
<http://www.symas.com>
6 years, 10 months
Re: RE24 testing call (2.4.45) LMDB RE0.9 testing call (0.9.20)
by Quanah Gibson-Mount
--On Thursday, February 09, 2017 8:27 PM +0100 "A. Schulze"
<sca(a)andreasschulze.de> wrote:
Hi Andreas,
> a manual test using openssl s_client also proof the root is wrongly
> delivered: $ echo | openssl11 s_client -connect ldap-test.example.org:443
Please see the slapd.conf(5) or slapd.conf(5) man pages, which clearly
state:
TLSCACertificateFile <filename>
Specifies the file that contains certificates for all of
the
Certificate Authorities that slapd will recognize.
Note "That *slapd* will recognize". The server cannot and will not provide
the cert chains to clients as that is a massive security risk. Clients can
and must be configured with the list of CAs *they* will trust when the
server provides the cert.
> Ultimate features would be OCSP stapling ( OK, no ldap client currently
> implement that ) and setting ecdh_curve via SSL_CTX_set1_curves_list
Feel free to submit a patch to implement anything necessary beyond what was
discussed in <http://www.openldap.org/its/index.cgi/?findid=7506>. :) Or at
least file an ITS so the issue can be tracked.
Regards,
Quanah
--
Quanah Gibson-Mount
Product Architect
Symas Corporation
Packaged, certified, and supported LDAP solutions powered by OpenLDAP:
<http://www.symas.com>
6 years, 10 months
Re: RE24 testing call (2.4.45) LMDB RE0.9 testing call (0.9.20)
by Quanah Gibson-Mount
--On Thursday, February 09, 2017 10:34 PM +0100 "A. Schulze"
<sca(a)andreasschulze.de> wrote:
> So my guess: openldap not call an important openssl library function and
> so openssl use it's defaults.
Yes, that sounds like a serious bug. Please file an ITS on it.
--Quanah
--
Quanah Gibson-Mount
Product Architect
Symas Corporation
Packaged, certified, and supported LDAP solutions powered by OpenLDAP:
<http://www.symas.com>
6 years, 10 months
Re: RE24 testing call (2.4.45) LMDB RE0.9 testing call (0.9.20)
by Howard Chu
Please read this chapter again, carefully.
http://www.openldap.org/doc/admin24/tls.html
A. Schulze wrote:
>
>
> Am 09.02.2017 um 22:32 schrieb Ralf Mattes:
>> Is this really the problem. I only use TLSCACertificateFile but still get all the
>> intermediate certificats as well as the top level (German Telekpm) cert.
>
> Ah!
>
> both, TLSCACertificateFile and TLSCACertificatePath contain the acceptable issuer certificates
> for connections from a client /to/ slapd if TLSVerifyClient is not "none"
> Just as Quanah pointed out some messages ago...
>
> In any case the list of DNs is sent to the client as part of the SSL handshake.
>
> If it happen that TLSCACertificateFile and/or TLSCACertificatePath
> contain certificates related to the server certificate chain,
> these are also sent at all to build this cert chain.
>
> But even if TLSCertificateFile point to file containing cert + intermediate
> OpenLDAP still only the cert is delivered to the client.
> (if TLSCACertificateFile and TLSCACertificatePath are unset)
>
> I'm confused because it's different then postfix for example.
>
> There I configure "smtpd_tls_cert_file = $cert_and_intermediates"
> Optional I may enable "smtpd_tls_ask_ccert = yes"
> Then the SMTP-Server ask the client to present a client cert
> I've to configure smtpd_tls_CAfile or smtpd_tls_CApath.
>
> The difference between smtpd_tls_CAfile and smtpd_tls_CApath in postfix:
> ( see http://www.postfix.org/postconf.5.html#smtpd_tls_CApath )
> "In contrast to smtpd_tls_CAfile, DNs of Certification Authorities
> installed in $smtpd_tls_CApath are not included in the client
> certificate request message."
>
> And this is what you see, Ralf ...
> OpenLDAP Allways send the list of DNs.
>
> Hope that helps.
> Andreas
>
>
>
>
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
6 years, 10 months
Re: RE24 testing call (2.4.45) LMDB RE0.9 testing call (0.9.20)
by Quanah Gibson-Mount
--On Thursday, February 09, 2017 12:49 PM -0800 Quanah Gibson-Mount
<quanah(a)symas.com> wrote:
> --On Thursday, February 09, 2017 9:43 PM +0100 "A. Schulze"
> <sca(a)andreasschulze.de> wrote:
>
>> that's not the issue. A TLS server sent it's certificate and all
>> intermediates EXCLUDING the self signed root to the client. This is not
>> true for my setup and I don't know why: misconfiguration or wrong ssl
>> implementation.
>
> Sorry, reading back over your configuration, I don't believe it's valid
> to specify both a CA path and a CA directory. You can use one or the
> other.
>From the man page:
TLSCACertificatePath <path>
Specifies the path of a directory that contains
Certificate
Authority certificates in separate individual files.
Usually
only one of this or the TLSCACertificateFile is used.
This
directive is not supported when using GnuTLS.
So it is not clear to me what happens if you use both. ;) I've certainly
never tried that. Since you are using both, did you correctly "hash" the
CA certs in the directory you pointed at?
--Quanah
--
Quanah Gibson-Mount
Product Architect
Symas Corporation
Packaged, certified, and supported LDAP solutions powered by OpenLDAP:
<http://www.symas.com>
6 years, 10 months
Re: help troubleshooting
by Quanah Gibson-Mount
--On Tuesday, February 07, 2017 5:01 PM -0700 scar <scar(a)drigon.com> wrote:
> Well it's kind of a mess here and my lack of experience with LDAP isn't
> helping much. There is no slapd-config program although there is a
> manual page entry for it. "yum whatprovides */slapd-config" returns no
> packages.
slapd-config is not a program. It's a database format. Please read the man
page for slapd-config(5).
> I was able to enable users to change their passwords by directly
> modifying /etc/openldap/slapd.d/cn\=config/olcDatabase\=\{1\}bdb.ldif and
> adding these lines to the bottom:
As noted at the top of those files, you should never, ever, manually modify
them by hand. You should be using the correct ldap client operations. You
can do this via ldapadd, ldapmod, etc.
> I know that's not proper but i needed users to be able to change their
> password. Thanks for the info about ACLs. the "next to last ACL"
> mentioned is for the "database monitor" (see slapd.conf below) and i'm
> not sure why "by * read" should be granted that access, perhaps you can
> shed some light on why that exists in our config? maybe i don't need
> ACLs for that so only rootdn has access?
That would be a separate block of ACLs that only applies to the monitor
backend. There is no requirement that by * read have access to the
monitoring backend. Who/what should have access to it depends on your
requirements.
> We have a new LDAP server that I am setting up, so I'd like to focus on
> moving the database and getting the new server into production, and we
> can iron out the wrinkles in this mess at the same time. My
> understanding is that I can use slapcat/slapadd to do the export/import...
>
> I used "slapcat > /tmp/ldif" on current server, then moved ldif and
> updated [slapd.conf] (see below) file to the new server, then ran
> "slapadd -l /tmp/ldif -l /etc/openldap/slapd.conf -F
> /etc/openldap/slapd.d/" but i get an error when trying to start slapd:
> "ls: cannot access /etc/openldap/slapd.d//cn=config/olcDatabase*.ldif: No
> such file or directory" so how am i supposed to get the slapd.d/* files?
> If I am to just copy those over from the current server then I'd like to
> figure out why I had to modify the ldif file directly...
Your first slapcat exports the binary database, it has zero to do with the
slapd-config database. Please read the manpage for slapcat on the proper
way to export your slapd-config database. You don't use slapd.conf, you
should stop doing anything with it, as it is immaterial. You will need to
export/import your slapd-config database prior to importing your binary
database.
> The current LDAP server is running RHEL 6.8 with kernel
> 2.6.32-642.11.1.el6.x86_64. The new LDAP server is running CentOS 6.8
> with kernel 2.6.32-642.13.1.el6.x86_64. The nss/pam configuration for
> one of our clients is this (i hope this is what Michael Wandel meant):
The RHEL build of OpenLDAP is known to be problematic, outdated, and it
links to the insecure MozNSS libraries. I personally would recommend
against using it. If you want to use a 3rd party OpenLDAP build, such as
RedHat's, you may find the LTB project build a better bet. If you require
support for your LDAP deployment, Symas offers supported builds for RHEL.
Regards,
Quanah
--
Quanah Gibson-Mount
Product Architect
Symas Corporation
Packaged, certified, and supported LDAP solutions powered by OpenLDAP:
<http://www.symas.com>
6 years, 10 months
Re: RE24 testing call (2.4.45) LMDB RE0.9 testing call (0.9.20)
by Quanah Gibson-Mount
--On Thursday, February 09, 2017 9:43 PM +0100 "A. Schulze"
<sca(a)andreasschulze.de> wrote:
> that's not the issue. A TLS server sent it's certificate and all
> intermediates EXCLUDING the self signed root to the client. This is not
> true for my setup and I don't know why: misconfiguration or wrong ssl
> implementation.
Sorry, reading back over your configuration, I don't believe it's valid to
specify both a CA path and a CA directory. You can use one or the other.
--Quanah
--
Quanah Gibson-Mount
Product Architect
Symas Corporation
Packaged, certified, and supported LDAP solutions powered by OpenLDAP:
<http://www.symas.com>
6 years, 10 months
Re: Fw : backend SSL negociation timeout
by Quanah Gibson-Mount
--On Friday, February 03, 2017 3:06 PM +0100 Louis Chanouha
<louis.chanouha(a)univ-toulouse.fr> wrote:
> To reproduce the error, i have an meta directory configured like this:
>
>
>
>
> database meta
> suffix "dc=localauth"
> rootdn "cn=Manager,dc=localauth"
> rootpw XXX
>
>
> uri "ldaps://localhost:666/ou=UT,dc=localauth"
> lastmod off
> suffixmassage "ou=UT,dc=localauth" "ou=people,dc=example,dc=fr"
> timeout 1
> conn-ttl 1
> network-timeout 1
Hi Louis,
Please file an ITS at https://www.openldap.org/its with the configuration
and reproduction steps you provided here so this issue can be tracked.
Thanks,
Quanah
--
Quanah Gibson-Mount
Product Architect
Symas Corporation
Packaged, certified, and supported LDAP solutions powered by OpenLDAP:
<http://www.symas.com>
6 years, 10 months
