openldap-technical April 2020

openldap-technical@openldap.org

33 participants
36 discussions

Antw: [EXT] LMDB: sync'ing to disk and flash wear
by Ulrich Windl 06 Apr '20

06 Apr '20

>>> Alberto Mardegan <mardy(a)users.sourceforge.net> schrieb am 06.04.2020 um 08:44 in Nachricht <10444_1586183585_5E8B3DA1_10444_128_1_5863f87e-d26b-2943-9302-bf5bfc270e71@user .sourceforge.net>: > Hi all! > I'm trying to understand the impact that a LDMB database will have on > the flash wear of an embedded device. > > In order to minimize disk writes, I'm keeping a DB open all the time, > and I'd like to have the changes written into the physical storage only > when mdb_env_sync() is called (near the end of my process lifetime). > I'm opening the DB with the MDB_NOSYNC | MDB_WRITEMAP | MDB_NOMETASYNC > flags, and if I strace the executable I see that lmdb behaves as > expected: no writes happen until I call mdb_env_sync(). > > However, the writes end up to the physical storage anyway, because the > kernel reserves the right of flushing changes of an mmap'ed file to disk > at any time (as per the documentation of MAP_SHARED). I don't exactly > know if this happens because some other process is calling sync(), or > which criteria the kernel follows, but as a matter of fact a write > happens every few seconds. > > Looking at the code, it looks like it's impossible to instruct LMDB to > use MAP_PRIVATE without modifying the code. But before exploring that > solution, I'd like to be sure I understand the real behaviour of LMDB. Doesn't MAP_PRIVATE imply that the map can't be written to disk? > > Do I understand correctly, that if during the lifetime of my process the > same key is assigned different values, with mdb_txn_commit() being > called each time, the same addresses in the shared memory area will be > changed several times (leading to unnecessary re-writes of the same area > of the flash storage)? Or is LMDB only operating in "append" mode, so > that it tries to avoid writing the same area twice? I think no DB system can implement a commit with just one write. Why not put your DB im a RAM disk and then rsync to flash when done? Regards, Ulrich > > Thanks in advance for any answer, > Alberto > > -- > http://www.mardy.it - Geek in un lingua international

1 0

LMDB: sync'ing to disk and flash wear
by Alberto Mardegan 06 Apr '20

06 Apr '20

Hi all! I'm trying to understand the impact that a LDMB database will have on the flash wear of an embedded device. In order to minimize disk writes, I'm keeping a DB open all the time, and I'd like to have the changes written into the physical storage only when mdb_env_sync() is called (near the end of my process lifetime). I'm opening the DB with the MDB_NOSYNC | MDB_WRITEMAP | MDB_NOMETASYNC flags, and if I strace the executable I see that lmdb behaves as expected: no writes happen until I call mdb_env_sync(). However, the writes end up to the physical storage anyway, because the kernel reserves the right of flushing changes of an mmap'ed file to disk at any time (as per the documentation of MAP_SHARED). I don't exactly know if this happens because some other process is calling sync(), or which criteria the kernel follows, but as a matter of fact a write happens every few seconds. Looking at the code, it looks like it's impossible to instruct LMDB to use MAP_PRIVATE without modifying the code. But before exploring that solution, I'd like to be sure I understand the real behaviour of LMDB. Do I understand correctly, that if during the lifetime of my process the same key is assigned different values, with mdb_txn_commit() being called each time, the same addresses in the shared memory area will be changed several times (leading to unnecessary re-writes of the same area of the flash storage)? Or is LMDB only operating in "append" mode, so that it tries to avoid writing the same area twice? Thanks in advance for any answer, Alberto -- http://www.mardy.it - Geek in un lingua international

2 1

Openldap 2.4.49 with overlays syncrepl+ppolicy+chain+ldap weird behavior
by "POISSON Frédéric" 06 Apr '20

06 Apr '20

Hello, I'm doing a OpenLDAP test with a master/slave replication configuration including ppolicy overlay. I would like to enable password change from the slave replica with chain overlay, in order to validate the ppolicy olcPPolicyForwardUpdates attribute to TRUE. I'm using LDAPS from slave to master with SASL External authentication with client certificate. The client certificate correspond to a user DN entry with "manage" rights on the master server (the same used for the replication). This user DN has authzTo attribute in order to match the correct PROXYAUTHZ request from its dn to user DN. All of this configuration works on replica when i do first a failed authentication (err=49) on replica. The pwdFailureTime value is updated on the DN entry from replica to slave normally. I'm also able to do after some self entry update on some attribute such as password or others from replica to master. But the weird behavior is that i need to run first an failed authentication, otherwise if i try to change attribute on the slave server, it respond an err=80 "Error: ldap_back_is_proxy_authz returned 0, misconfigured URI?". The only way to retrieve correct behavior is to restart slapd, and redo one failed authentication first. It seems that the chain overlay do not connect the master server at startup Do you have any ideas why i have this behavior ? I'm using a 2.4.49 build of openldap, and inside logs on master server i see that the slave use the same connection. Here is the LDIF change and configuration on my replica : dn: olcOverlay={0}chain,olcDatabase={-1}frontend,cn=config changetype: add objectClass: olcOverlayConfig objectClass: olcChainConfig olcOverlay: {0}chain olcChainReturnError: TRUE dn: olcDatabase={0}ldap,olcOverlay={0}chain,olcDatabase={-1}frontend,cn=config changetype: add objectClass: olcLDAPConfig objectClass: olcChainDatabase olcDatabase: {0}ldap olcDbURI: ldaps://valid7-lab-ldap1.tld olcDbIDAssertBind: bindmethod=sasl saslmech=external starttls=no tls_cert="/usr/local/openldap/etc/openldap-valid7/tls/db1_rid001_cert.pem" tls_key="/usr/local/openldap/etc/openldap-valid7/tls/db1_rid001_key.pem" tls_cacert="/usr/local/openldap/etc/openldap-valid7/tls/cacert.pem" tls_reqcert=demand tls_crlcheck=none mode=self olcDbRebindAsUser: TRUE dn: olcDatabase={1}mdb,cn=config changetype: modify replace: olcUpdateRef olcUpdateRef: ldaps://valid7-lab-ldap1.tld - dn: olcOverlay={1}ppolicy,olcDatabase={1}mdb,cn=config changetype: modify replace: olcPPolicyForwardUpdates olcPPolicyForwardUpdates: TRUE - Here is LDIF change on my master : dn: cn=config changetype: modify replace: olcAuthzPolicy olcAuthzPolicy: to - Thanks in advance for your reply -- Frederic Poisson

2 1

Slapd unexpectedly shutdown
by Kevin Olbrich 06 Apr '20

06 Apr '20

Hi! I'm experiencing an issue with Slapd 2.4.49 on Debian Buster. I use Slapd as a proxy / lb with two AD nodes behind it. If I reboot one of the AD nodes, everything is fine. As soon as I reboot the seconds one (while the first is back and available), OpenLDAP shuts down immediately: Apr 06 22:03:33 ldap-lb1.ldap.example.com slapd[22140]: Stopping OpenLDAP: slapd. Apr 06 22:03:33 ldap-lb1.ldap.example.com systemd[1]: slapd.service: Succeeded. There is no traffic during this time and it matches the exact time when node two is down. Debian still uses an init script for slapd but I did not find anything interesting in it. Is there a config setting that I missed in the docs that explains this behavior? As it's not crashing, HA in systemd won't catch this issue. Kind regards Kevin

1 0

Re: NetworkRadius CentOS pkg uses openldap-ltb which conflict with symas-openldap
by Dave Macias 02 Apr '20

02 Apr '20

On Thu, Mar 26, 2020 at 3:44 PM Quanah Gibson-Mount <quanah(a)symas.com> wrote: > Try it and see? I can't find any concrete documentation one way or the > other, although it seems BuildRequires generally follows what Requires > does. You may need to ask the people who maintain RPM. > > There are other issues in the spec file, however, like: > > --with-libfreeradius-ldap-include-dir=/usr/local/openldap/include \ > --with-libfreeradius-ldap-lib-dir=/usr/local/openldap/lib64 \ > > are specific to LTB, might want to do something like they did around line > 447 to only do this if the LTB openldap is in use. > Awesome! Thank you very much for the input! Will report how things go -Dave

2 2

Multiple OpenLDAP client instances with the c library
by Lasa Martxel 02 Apr '20

02 Apr '20

Hello, I'm using the openldap c client library (2.4.45). I would like to somehow have two client instances (not necessarily simultaneously) within the same application, but I'm having issues with the second instance I create. I have not found a way to clear the global options so the new ones (different ca cerficiate, different client certificate) can be used with the second instance. With the second ldap_initialize, global options are already initialized. Same with the TLS context, it's initialized too. I have seen that the function that destroys the global options is called only when the program exits, or the dyn library is unloaded. Is this correct? Is this somehow a limitation of the library? Thank you, Martxel

2 4

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

openldap-technical April 2020