openldap-technical January 2021

openldap-technical@openldap.org

25 participants
27 discussions

Syncprov shows issue with entry
by Nick Milas 25 Jan '21

25 Jan '21

Hello, I would like to ask you for your guidance regarding the following. We have an openldap (v2.4.56) master server syncing with three other openldap slaves. The master seems being unable to complete successfully syncing a particular entry and it keeps trying for ever. Logs follow. I have realized this only now, but it seems that this is happening for long. (It is recorded in all the logs I have available.) What is the suggested way to resolve this situation? ======================================================================================================= ... Jan 16 14:48:09 ldap.noa.gr slapd[1655]: conn=1048 op=1 syncprov_search_response: Entry cn=thermo,ou=Aliases,dc=noa,dc=gr CSN 20200910151806.461875Z#000000#000#000000 older or equal to ctx 20200910151806.461875Z#000000#000#000000 Jan 16 14:48:09 ldap.noa.gr slapd[1655]: conn=1048 op=1 syncprov_search_response: cookie=rid=601,csn=20210116055411.875566Z#000000#000#000000 Jan 16 14:48:09 ldap.noa.gr slapd[1655]: conn=1048 op=1 syncprov_sendinfo: refreshPresent cookie=rid=601,csn=20210116055411.875566Z#000000#000#000000 Jan 16 14:48:09 ldap.noa.gr slapd[1655]: conn=1048 op=1 syncprov_search_response: detaching op Jan 16 14:51:09 ldap.noa.gr slapd[1655]: conn=1058 op=1 syncprov_op_search: got a persistent search with a cookie=rid=601,csn=20200910151806.461875Z#000000#000#000000 Jan 16 14:51:09 ldap.noa.gr slapd[1655]: conn=1058 op=1 syncprov_findbase: searching Jan 16 14:51:09 ldap.noa.gr slapd[1655]: conn=1058 op=1 syncprov_op_search: registered persistent search Jan 16 14:51:09 ldap.noa.gr slapd[1655]: conn=1058 op=1 syncprov_findcsn: mode=FIND_CSN csn=20200910151806.461875Z#000000#000#000000 Jan 16 14:51:09 ldap.noa.gr slapd[1655]: conn=1058 op=1 syncprov_findcsn: csn==20200910151806.461875Z#000000#000#000000 found Jan 16 14:51:09 ldap.noa.gr slapd[1655]: conn=1058 op=1 syncprov_findcsn: mode=FIND_PRESENT csn= Jan 16 14:51:09 ldap.noa.gr slapd[1655]: conn=1058 op=1 syncprov_sendinfo: present syncIdSet cookie= Jan 16 14:51:09 ldap.noa.gr slapd[1655]: conn=1058 op=1 syncprov_sendinfo: present syncIdSet cookie= Jan 16 14:51:09 ldap.noa.gr slapd[1655]: conn=1058 op=1 syncprov_sendinfo: present syncIdSet cookie= Jan 16 14:51:09 ldap.noa.gr slapd[1655]: conn=1058 op=1 syncprov_sendinfo: present syncIdSet cookie= Jan 16 14:51:09 ldap.noa.gr slapd[1655]: conn=1058 op=1 syncprov_sendinfo: present syncIdSet cookie= Jan 16 14:51:09 ldap.noa.gr slapd[1655]: conn=1058 op=1 syncprov_sendinfo: present syncIdSet cookie= Jan 16 14:51:09 ldap.noa.gr slapd[1655]: conn=1058 op=1 syncprov_sendinfo: present syncIdSet cookie= Jan 16 14:51:09 ldap.noa.gr slapd[1655]: conn=1058 op=1 syncprov_sendinfo: present syncIdSet cookie= Jan 16 14:51:09 ldap.noa.gr slapd[1655]: conn=1058 op=1 syncprov_sendinfo: present syncIdSet cookie= Jan 16 14:51:09 ldap.noa.gr slapd[1655]: conn=1058 op=1 syncprov_sendinfo: present syncIdSet cookie= Jan 16 14:51:09 ldap.noa.gr slapd[1655]: conn=1058 op=1 syncprov_sendinfo: present syncIdSet cookie= Jan 16 14:51:09 ldap.noa.gr slapd[1655]: conn=1058 op=1 syncprov_sendinfo: present syncIdSet cookie= Jan 16 14:51:09 ldap.noa.gr slapd[1655]: conn=1058 op=1 syncprov_search_response: Entry cn=thermo,ou=Aliases,dc=noa,dc=gr CSN 20200910151806.461875Z#000000#000#000000 older or equal to ctx 20200910151806.461875Z#000000#000#000000 Jan 16 14:51:09 ldap.noa.gr slapd[1655]: conn=1058 op=1 syncprov_search_response: cookie=rid=601,csn=20210116055411.875566Z#000000#000#000000 Jan 16 14:51:09 ldap.noa.gr slapd[1655]: conn=1058 op=1 syncprov_sendinfo: refreshPresent cookie=rid=601,csn=20210116055411.875566Z#000000#000#000000 Jan 16 14:51:09 ldap.noa.gr slapd[1655]: conn=1058 op=1 syncprov_search_response: detaching op ========================================================================================================= Thank you in advance for your advice! Cheers, Nick

3 4

openldap performance problems
by Zetan Drableg 25 Jan '21

25 Jan '21

Every time my "updater" process writes an LDIF to LDAP, clients start seeing timeouts during authentication. Even with a small number of inserts or removals, this problem shows up. openldap 2.4.47, MBD, 16 core Linux server, 6 nodes in a n-way multimaster replication configuration. The on disk database is 1.5gigs. Can anyone suggest tunables for this problem? I've tried "threads 64" but that didn't improve things. [2021-01-24 00:01:45] connection_input: conn=2721465 deferring operation: binding [2021-01-24 00:01:45] connection_input: conn=2721505 deferring operation: binding [2021-01-24 00:01:45] connection_read(17): no connection! [2021-01-24 00:01:46] connection_input: conn=2721694 deferring operation: binding [2021-01-24 00:01:47] connection_input: conn=2721844 deferring operation: binding [2021-01-24 00:01:48] connection_input: conn=2722210 deferring operation: binding [2021-01-24 00:01:48] connection_input: conn=2722208 deferring operation: binding [2021-01-24 00:01:48] connection_input: conn=2722283 deferring operation: binding Thank you

2 1

openldap mdb tuning
by olivier.chirossel＠sfr.com 22 Jan '21

22 Jan '21

Hi guys, I have an architecture with : 2 masters ( refresh and persist ) a load balanceur in front of the masters (vip main/sorry), to avoid write in the same time on the masters (stability) few consumers use back-mdb openldap version 2.4.45 my huge branche 3181406 entries, data.mdb 7.7G on the main master, 9.1G on the consumers and the sorry master ( use du -h to get the size ) I have three questions : why the size is smaller on the main master ? Sometimes i have to scripts a lot of ldapmodify ( ~100000 ), should i have to set olcDbEnvFlags: writemap olcDbEnvFlags: nometasync for increase performance ? This sets is recommended for production ? stability issues with theses set ? i have no stability issue for now, but considering this two questions, is it recommended to upgrade ? thank's in advance regards Olivier

3 4

Use-case specific changes to LMDB
by martin＠urbackup.org 22 Jan '21

22 Jan '21

This post outlines a few changes to LMDB I had to do to make it work in a specific use case. I’d like to see those changes upstream, but I understand that they may be/are not relevant for e.g. OpenLDAP. The use case is multiple databases on disks with long running large write transactions. 1. Option to not use custom memory allocator/page pool LMDB has a custom malloc() implementation that re-uses pages (me_dpages). I understand that this improves the performance at bit (depending on the malloc implementation). But there should at least be the option to not do that (for many reasons). I would even make not using it the default. 2. Large transactions and spilling In a large write transaction, it will use a lot of memory per default (512MiB) which won’t get freed when the transaction commits (see 1.). If one has a lot of databases it uses a lot of memory that never gets freed. Alternatively, one can use MDB_WRITEMAP, but (i) per default Linux isn’t tuned to delay writing pages to disk and (ii) before commit LMDB has to remove a dirty bit, so each page is written twice. Both problems would be fixed by making when pages get spilled configurable (mt_dirty_room as MDB_IDL_UM_MAX currently) and reducing the default non-spill memory amount for at least the MDB_WRITEMAP case. If this memory amount is low mt_spill_pgs gets sorted often so maybe this needs to be converted to a different data structure (e.g. red-black tree). 3. LMDB causes crashes if database is corrupted If the database is corrupted it can cause the application to crash. I have fixed those cases when they (randomly) occurred. Properly fixing this would probably be best done with some fuzzing. 4. Allow LMDB to reside on a device I used dm-cache to improve LMDB read performance. It needed a bit of adjustment to get the correct size of the device via ioctl BLKGETSIZE64. -- I’ve fixed those issues w.r.t. my application. If there is interest in any of those application specific changes, I’ll clean them up and post them.

4 6

Symas OpenLDAP For Linux 2.4.57 released
by Quanah Gibson-Mount 20 Jan '21

20 Jan '21

The latest version of Symas OpenLDAP for Linux is now available for RHEL7, RHEL8, Ubuntu18 LTS, and Ubuntu 20 LTS. <https://repo.symas.com/sofl/> for installation instructions. Regards, Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

LTB packages (was: OpenLDAP 2.4.57 available)
by Clément OUDOT 19 Jan '21

19 Jan '21

Le 18/01/2021 à 21:21, project(a)openldap.org a écrit : > OpenLDAP 2.4.57 is now available for download as detailed on our download page: LDAP Tool Box packages are available for Debian stretch / Debian buster / CentOS 7 / CentOS 8 See https://ltb-project.org/download#openldap -- Clément Oudot | Identity Solutions Manager clement.oudot(a)worteks.com Worteks | https://www.worteks.com

1 0

openSUSE/SLE package update (was: OpenLDAP 2.4.57 available)
by Michael Ströder 19 Jan '21

19 Jan '21

On 1/18/21 9:21 PM, project(a)openldap.org wrote: > OpenLDAP 2.4.57 is now available for download as detailed on our download page: FWIW: openSUSE/SLE devel package has also been updated: https://build.opensuse.org/package/show/network:ldap/openldap2 Ciao, Michael.

1 0

OpenLDAP with GSSAPI on Windows
by Árpád Nagy 19 Jan '21

19 Jan '21

Hello, I try to use OpenLDAP with GSSAPI for authentication on Windows platform and the LDAP server is a Microsoft AD. According to the OpenLDAP Admin guide the SASL have to be used. SASL plugin is for OpenLDAP the Cyrus SASL. In the Cyrus SASL 2.1.27 package the description "Building Cyrus SASL on Windows" is very outdated. Is it possible to build a working solution with Cyrus SASL 2.1.27 , MIT Kerberos 5 and OpenLDAP 2.4.56 for Windows x64 platform with Microsoft Visual Studio 2017 to authenticate Windows client with GSSAPI? Regards, Arpad

1 0

availability of openLDAP 2.4.56 for centos 7.8/7.9
by shelke.sachitanand＠gmail.com 18 Jan '21

18 Jan '21

Hi Team, Does latest release (openLDAP 2.4.56) is supported for Centos 7.8/7.9 ? I tried to install openLDAP rpms (openldap, openldap-servers, openldap-clients) on Centos 7.8 but observed lot of dependencies failures also observed same issue when I tried to upgrade existing Openldap(openldap-2.4.44-22.el7.x86_64) to this latest rpm. Can someone help me on this. If this version is not supported then can we build and create rpm from openldap source code. Thanks, Sachitanand

4 7

LMDB: What goes "wrong" when a process is suspended during a transaction?
by Antti K 17 Jan '21

17 Jan '21

Good evening from Germany. The LMDB "Caveats" documentation mentions: > Avoid suspending a process with active transactions. These would then be > "long-lived" as above. Also read transactions suspended when writers commit > could sometimes see wrong data. Questions: - What does "wrong data" mean? What guarantees are broken? Is potentially undefined behaviour ("read returns junk"), or isolation weakening ("transaction reads value actually written after the transaction started"), or something else? - If I do find myself in that unfortunate situation, how screwed am I? For example, say someone spelled SIGSTP wrong, and suspended my program with an uncatchable SIGSTOP instead, and so I could not wait for transactions to complete before being suspended. My program simply catches a SIGCONT, with transactions in flight. What can it do to maintain correctness? Guidance kindly appreciated. -- Antti

2 1

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

openldap-technical January 2021