openldap-technical

openldap-technical@openldap.org

19 participants
9788 discussions

LMDB, virtual machines, and copying mdb files
by selama＠gmail.com 28 Oct '22

28 Oct '22

I am working on developing a new document-oriented (XML+JSON) database, using LMDB as an engine, and I have two questions. 1. So far, it have been working really smoothly for me. But my one customer so far for the DB is really concerned about running LMDB in a virtual environment such as Docker, when performing reads and writes. Especially when mounting volumes. Their concern is because of the following caveat: "Do not use LMDB databases on remote filesystems, even between processes on the same host. This breaks flock() on some OSes, possibly memory map sync, and certainly sync between programs on different hosts" I think it shouldn't be an issue with Docker, but I want to be certain. 2. We have one server that updates the database, and another server with a read-only copy of the same database. Our plan was to simply copy the mdb files from the update machine to the read only machine, but we noticed that if we copy the file immediately after writing, the copy may end up being corrupted. My solution was to suspend all writing and wait few minutes before writing, to make sure everything back from memory, and I'm also using the "sync" command (not sure if it does anything here). It seem to be working, but I wonder if there is a more robust way of doing that? And also, is it safe to overwrite to the read-only server while it performs read transactions to the current file (or maybe rename it and copy to a new file with the same name)?

5 5

Read speeds, bottleneck, correct understanding?
by Sam Dave 27 Oct '22

27 Oct '22

Hi, If my program reads through an LMDB database apparently slow at first run, but suddenly runs much quicker at the second run (which does exactly the same thing), can't I already say with confidence that my choice of LMDB - in this case its builtin memory mapping functionality - is already paying off in terms of read speeds? I mean, that alone (exact same thing runs much faster the second time) already PROVES that the bottleneck was reading from disk as opposed to memory, right? I'm looking for excuses for patting myself on the back for investing in LMDB. Regards, Sam

2 3

cn=config TLS Configuration Problem
by Timothy Stonis 25 Oct '22

25 Oct '22

Hi, I am trying to setup an OpenLDAP 2.6.3 server and I’d like to only use olc configuration (no slapd.conf file). So far things are going okay, but I’m having a problem with TLS configuration. I am able to enable TLS using a self-signed certificate without any problem, however, if I try to disable TLS using the following LDIF: dn: cn=config changetype: modify delete: olcTLSCertificateFile - delete: olcTLSCertificateKeyFile - I get the following error: modifying entry "cn=config" ldap_modify: Server is unwilling to perform (53) I enabled debugging and cannot seem to see the error. I have also tried reordering the entries, doing one at a time, disabling ldaps:// binding, etc but nothing seems to work. If I just remove the certificate and/or key files, then the server does not start. Is enabling TLS a one way street? Or, should I just use slapd.conf? As a second question, I read in an article online that there is a way to store the TLS cert(s) and key in the LDAP database itself. However, I cannot find any info on that in the documentation. Can anyone shed some light on that? Thank you in advance! Tim

2 7

RE25 testing call (2.5.14)
by Quanah Gibson-Mount 24 Oct '22

24 Oct '22

This is the first testing call for OpenLDAP 2.5.14. Depending on the results, this may be the only testing call. Generally, get the code for RE25: <https://git.openldap.org/openldap/openldap/-/archive/OPENLDAP_REL_ENG_2_5/o…> Extract, configure, and build. Execute the test suite (via make test) after it is built. Optionally, cd tests && make its to run through the regression suite. Thanks! OpenLDAP 2.5.14 Engineering Fixed client tools to remove 'h' and 'p' options (ITS#9917,ITS#8618) Fixed ldapsearch memory leak with controls (ITS#9860) Fixed libldap ldif_open_urlto check for failure (ITS#9904) Fixed libldap ldap_url_parsehosts check for failure (ITS#9904) Fixed slapd memory leak with olcAuthIDRewrite (ITS#6035) Fixed slapd transactions extended operations cleanup after write (ITS#9892) Fixed slapd-mdb max number of index databases to 256 (ITS#9895) Fixed slapd-monitor to free remembered cookies (ITS#9339) Fixed slapo-deref memory leak (ITS#9924) Fixed slapo-dynlist to ignore irrelevant objectClasses (ITS#9897) Fixed slapo-remoteauth memory leaks (ITS#9438) Build Environment Fixed build process to not use gmake specific features (ITS#9894) Fixed slapo-otp testdir creation (ITS#9437) Fixed slapd-tester memory leak (ITS#9908) Fixed usage of non-standard C syntax (ITS#9898, ITS#9899, ITS#9901) Fixed usage of bashism (ITS#9900) Documentation Fixed slapo-unique(5) to clarify when quoting should be used (ITS#9915) Regards, Quanah

5 9

Attribute Cert. schema (RFC 5755)
by Pascal Jakobi 21 Oct '22

21 Oct '22

Hi there I am looking for an RFC 5755 (attribute certificates profile) schema file. I thought it was in pmi.schema, but it appears that no, unless I am missing sthing. Before creating one from the RFC, I would like to be sure it does not exist somewhere I couldn't find. Thanks in adv. P

3 4

SyncRepl fails object class 'organization' requires attribute 'o'
by Sander Smeenk 21 Oct '22

21 Oct '22

Hi, I'm trying to set up SyncRepl between two servers. When the SyncRepl client connects and tries to start it logs: | Entry (dc=example,dc=nl): object class 'organization' requires attribute 'o' | syncrepl_null_callback : error code 0x41 | syncrepl_entry: rid=000 be_add dc=example,dc=nl failed (65) There is only one such entry in the db and it has an 'o' attribute: | dn: dc=example,dc=nl | objectClass: top | objectClass: dcObject | objectClass: organization | dc: example | o: example.nl | structuralObjectClass: organization | [ .. create, modify, entry uuids etc ..] What could i be missing here? Regards, -Sander. -- | Inside every older person is a younger person wondering what happened. | 4096R/20CC6CD2 - 6D40 1A20 B9AA 87D4 84C7 FBD6 F3A9 9442 20CC 6CD2

2 2

replacement of memberof by dynlist.. questions..
by Frédéric Goudal 19 Oct '22

19 Oct '22

Hello, We have to install a product which use ldap and that seems to need memberof overlay. As I have read this overlay is deprecated is cause trouble with replication. So I have dug to found a replacement solution, and what I have found is to add something like that : In the olcDynamicList olcDlAttrSet: myPerson labeledURI myMemberOf And in each user <user> : labeledURI: ldap:///ou=groups,dc=example,dc=com??sub?(&(objectclass=posixgroup) (memberuid=<user>)) I find this way quite heavy to deal with, adding such attribute to every user (1), but we can do it. My other problem is that the myMemberOf may be really long to compute at each request (and for stupid historic reason some old programs do qyery on the full user set of atttributes). So I intend to add a proxycache. But I have a questiion concerning the templates : if I add the following template (myMemberOf=*) will it cache only the requests that are exactly (myMemberOf=XXX) or requests that contains the pattern like (&(Status=xxx)(myMemberOf=yyy)) ? Thanks. f.g. Note 1 : it would be nice that we could define thinks like in a single place : labeledURI: ldap:///ou=groups,dc=example,dc=com??sub?(&(objectclass=posixgroup) (memberuid=%uid%)) where %uid% would be the uid attribute value of the considered object. Or do I miss solething ? — Frédéric Goudal Ingénieur Système, DSI Bordeaux-INP +33 556 84 23 11

2 3

Fwd: [OldapWS] -> Proposal of a REST Web Service for CRUD Operations
by Howard Chu 17 Oct '22

17 Oct '22

Forwarding for exposure - any interest? -------- Forwarded Message -------- Subject: [OldapWS] -> Proposal of a REST Web Service for CRUD Operations Date: Fri, 16 Sep 2022 13:21:10 +0200 From: Olivier CHATOR <olivier.chator15(a)gmail.com> To: openldap-devel(a)openldap.org Dear all, I am a "long time user" of OpenLDAP core parts. If the core is very stable, I often received some request from application developers, complaining about the lack of REST Web Service API to manipulate their Directory objects. Of course, as far as I could see, there is commercial solution available to do this, or even a project using Spring. But I could not see a simple proposal based on a simple Apache Server and "fully free" of rights. I also saw (https://www.openldap.org/devel/contributing.html) that "The OpenLDAP Project welcomes contributions of independently-developed stand-alone LDAP-related software packages suitable for distribution separately from existing packages (e.g., OpenLDAP Software, JLDAP, JDBC-LDAP)". Then, I would like to propose a full Open Source first realease of a CRUD REST Web Service to manipulate OpenLDAP's Directory Objects. I know that this first release is very limited in term of features, but I think it does the "core job", and may be a common base to be enriched to build a real OpenLDAP REST API ? Package available here: https://drive.google.com/drive/folders/1s4zlTleJ1JWhQWP2kLGHtTRofpWNevS7?us… Thanks for your time reading this proposal, do not hesitate to ask if you have any question. Kind regards, Olivier Chator

8 8

How to debug out of sync servers ?
by Frédéric Goudal 13 Oct '22

13 Oct '22

Hello, I have a kind of annoying problem with two openldap servers. It is a simple pair setup in delta-sync and multimaster althought modifications are actually always done on the same one. Version is 2.5.7 (well I know 2.6 is out but I can’t work as fast as openldap people). Several times a day the de-facto client is out of sync for several minutes. For example now I have a delta of about 50 minutes (usually it’s less), but the sync logs are very active here is some lines of the current operations Oct 12 16:03:24 ldap-renater3 slapd[1343]: do_syncrep2: rid=901 cookie=rid=901,sid=029,csn=20221012140303.663754Z#000000#029#000000 Oct 12 16:03:24 ldap-renater3 slapd[1343]: slap_queue_csn: queueing 0x7fa7278dd0e0 20221012140303.663754Z#000000#029#000000 Oct 12 16:03:24 ldap-renater3 slapd[1343]: syncrepl_message_to_op: rid=901 tid 0x7fa826062700 Oct 12 16:03:25 ldap-renater3 slapd[1343]: conn=-1 op=0 syncprov_matchops: recording uuid for dn=cn=elafont,ou=bordeaux-inp.fr,ou=mailboxes,dc=ipb,dc=fr on opc=0x7fa81002d008 Oct 12 16:03:25 ldap-renater3 slapd[1343]: conn=-1 op=0 syncprov_add_slog: adding csn=20221012140303.663754Z#000000#029#000000 to sessionlog, uuid=d4eaa120-c1e0-103b-88a5-0dec75f30866 Oct 12 16:03:25 ldap-renater3 slapd[1343]: conn=-1 op=0 syncprov_add_slog: expiring csn=20221007192836.067967Z#000000#029#000000 from sessionlog (sessionlog size=10000001) Oct 12 16:03:25 ldap-renater3 slapd[1343]: conn=-1 op=0 syncprov_add_slog: updating mincsn for sid=41 csn=20221007192836.058680Z#000000#029#000000 to 20221007192836.067967Z#000000#029#000000 Oct 12 16:03:25 ldap-renater3 slapd[1343]: slap_queue_csn: queueing 0x7fa7278d9d00 20221012140303.663754Z#000000#029#000000 Oct 12 16:03:25 ldap-renater3 slapd[1343]: conn=-1 op=0 syncprov_matchops: recording uuid for dn=reqStart=20221012140324.000068Z,cn=accesslog on opc=0x7fa81002dbf8 Oct 12 16:03:25 ldap-renater3 slapd[1343]: conn=39500 op=1 syncprov_matchops: skipping original sid 029 Oct 12 16:03:25 ldap-renater3 slapd[1343]: slap_graduate_commit_csn: removing 0x7fa7278d9d00 20221012140303.663754Z#000000#029#000000 Oct 12 16:03:25 ldap-renater3 slapd[1343]: slap_graduate_commit_csn: removing 0x7fa7278dd0e0 20221012140303.663754Z#000000#029#000000 Oct 12 16:03:25 ldap-renater3 slapd[1343]: syncrepl_message_to_op: rid=901 be_modify cn=elafont,ou=bordeaux-inp.fr,ou=mailboxes,dc=ipb,dc=fr (0) Oct 12 16:03:25 ldap-renater3 slapd[1343]: slap_queue_csn: queueing 0x7fa7278de470 20221012140303.663754Z#000000#029#000000 Oct 12 16:03:25 ldap-renater3 slapd[1343]: slap_graduate_commit_csn: removing 0x7fa7278de470 20221012140303.663754Z#000000#029#000000 The behaviour that seems to occurs is that the logging sometimes stop (nothing more to do it seems) and when it stops the servers are in sync. So it seems the sync operations are very slow. Of course I do not log on normal operation. The directory as something like 70k entries (180Mb for the mdb file). Servers have plenty of memory : free total used free shared buff/cache available Mem: 8148668 3445380 125972 500 4577316 4390168 Swap: 2009084 183552 1825532 Servers are on an vmware cluster, from the vmware point of view cpu usage is very low. OS is ubuntu 20.04 I don’t know where to dig... Thanks in advance — Frédéric Goudal Ingénieur Système, DSI Bordeaux-INP +33 556 84 23 11

3 3

Fwd: RE25 testing call (2.5.14)
by Lucio De Re 09 Oct '22

09 Oct '22

As requested by Quanah. If no other practical measure is available, the server I'm using can be accessed via SSH. I'll supply further details in return for an SSH-format public key and essential identification. Lucio. -------- Forwarded Message -------- Subject: Re: RE25 testing call (2.5.14) Date: Fri, 7 Oct 2022 18:53:42 +0200 From: Lucio De Re <lucio(a)ff.co.za> Organization: Future Foundation To: Quanah Gibson-Mount <quanah(a)fast-mail.org> On 2022/10/04 18:50, Quanah Gibson-Mount wrote: > This is the first testing call for OpenLDAP 2.5.14. Depending on the > results, this may be the only testing call. > > Generally, get the code for RE25: > > <https://git.openldap.org/openldap/openldap/-/archive/OPENLDAP_REL_ENG_2_5/o…> > > > Extract, configure, and build. > > Execute the test suite (via make test) after it is built. Optionally, > cd tests && make its to run through the regression suite. > > Thanks! > On NetBSD 9.1... NetBSD jackal.proxima.alt.za 9.1 NetBSD 9.1 (GENERIC) #0: Sun Oct 18 19:24:30 UTC 2020 mkrepro@mkrepro.NetBSD.org:/usr/src/sys/arch/i386/compile/GENERIC i386 Fri 07 Oct 2022 18:51:30 SAST ... >>>>> Starting test049-sync-config for mdb... running defines.sh Starting provider slapd on TCP/IP port 9011... Using ldapsearch to check that provider slapd is running... Inserting syncprov overlay on provider... Starting consumer slapd on TCP/IP port 9012... Using ldapsearch to check that consumer slapd is running... Configuring syncrepl on consumer... Waiting 7 seconds for syncrepl to receive changes... Using ldapsearch to check that syncrepl received config changes... Adding schema and databases on provider... Using ldapadd to populate provider... Waiting 7 seconds for syncrepl to receive changes... Using ldapsearch to check that syncrepl received database changes... Replacing olcSyncrepl on provider... Waiting 7 seconds for syncrepl to receive changes... Using ldapsearch to read config from the provider... Using ldapsearch to read config from the consumer... Filtering provider results... Filtering consumer results... Comparing retrieved configs from provider and consumer... Using ldapsearch to read all the entries from the provider... Using ldapsearch to read all the entries from the consumer... Filtering provider results... Filtering consumer results... Comparing retrieved entries from provider and consumer... >>>>> Test succeeded >>>>> test049-sync-config completed OK for mdb after 27 seconds. >>>>> Starting test050-syncrepl-multiprovider for mdb... running defines.sh Initializing server configurations... Starting server 1 on TCP/IP port 9011... Using ldapsearch to check that server 1 is running... Inserting syncprov overlay on server 1... Starting server 2 on TCP/IP port 9012... Using ldapsearch to check that server 2 is running... Configuring syncrepl on server 2... Starting server 3 on TCP/IP port 9013... Using ldapsearch to check that server 3 is running... Configuring syncrepl on server 3... Starting server 4 on TCP/IP port 9014... Using ldapsearch to check that server 4 is running... Configuring syncrepl on server 4... Adding schema and databases on server 1... Using ldapadd to populate server 1... Waiting 15 seconds for syncrepl to receive changes... Using ldapsearch to read config from server 1... Using ldapsearch to read config from server 2... Using ldapsearch to read config from server 3... ... It's been hanging at this point for quite a while. There were no build or compilation errors that I could notice. Lucio.

1 0

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

openldap-technical