openldap-technical

openldap-technical@openldap.org

14 participants
9795 discussions

Re: [EXT] Re: syncrepl does not work as expected
by kumar rahul 03 Jul '20

03 Jul '20

Hi Quanah accesslog overlay file exists at following location /var/lib/ldap/accesslog What changes I need to make in below lines for moduleload to work dn: cn=module{0},cn=config changetype: modify add: olcModuleLoad olcModuleLoad: syncprov olcModuleLoad: accesslog Thanks Rahul On Wed, Jul 1, 2020 at 1:38 PM Quanah Gibson-Mount <quanah(a)symas.com> wrote: > --On Wednesday, July 1, 2020 11:09 AM -0400 kumar rahul > <rahul2002mit(a)gmail.com> wrote: > > > I am seeing following ERROR > > I have no idea how your slapd was built, etc. But essentially the > accesslog overlay must be loaded into slapd to be of use. It looks like > at > least some of the modules in your system are compiled statically, which is > likely why you currently do not have a module{0} section for loading > modules. You'll also have to verify where on disk the accesslog overlay > exists so that the moduleload statement pulls from the right directory. > > Regards, > Quanah > > > -- > > Quanah Gibson-Mount > Product Architect > Symas Corporation > Packaged, certified, and supported LDAP solutions powered by OpenLDAP: > <http://www.symas.com> >

4 5

RE: Antw: [EXT] RE: slapd 2.4.44 Performance problems
by Maucci, Cyrille 02 Jul '20

02 Jul '20

I'd probably need more details to understand what is seen, e.g. a graph over time of the CPU utilization of each core... I've been discussing with people in the past who were saying "not using" mistakingly :-) -----Original Message----- From: Ulrich Windl [mailto:Ulrich.Windl@rz.uni-regensburg.de] Sent: Thursday, July 2, 2020 7:15 AM To: daniel.zuniga(a)gmail.com; Maucci, Cyrille <cyrille.maucci(a)hpe.com>; quanah(a)symas.com Cc: openldap-technical(a)openldap.org Subject: Antw: [EXT] RE: slapd 2.4.44 Performance problems >>> "Maucci, Cyrille" <cyrille.maucci(a)hpe.com> schrieb am 01.07.2020 um >>> 18:06 in Nachricht <DF4PR8401MB0555595797F072395C494F66926C0(a)DF4PR8401MB0555.NAMPRD84.PROD.OUTLOOK. OM>: > I assume this is with back-bdb/hdb and is because Berkeley DB does not > scale beyond 8 cores. There's a difference between "not using" and "not scaling". > > From: Daniel Zuniga [mailto:daniel.zuniga@gmail.com] > Sent: Wednesday, July 1, 2020 3:43 PM > To: Quanah Gibson-Mount <quanah(a)symas.com> > Cc: openldap-technical(a)openldap.org > Subject: Re: slapd 2.4.44 Performance problems > > Is there a reason why OpenLDAP does not seem to use more than 8 cores > regardless of the number of threads it is being told to use? With 16 > threads it saturates 8 cores, 16 threads and 16 cores still uses 8 > cores, 32 threads and 16 cores... only 8 cores are used.

1 0

Re: [EXT] Re: syncrepl does not work as expected
by kumar rahul 02 Jul '20

02 Jul '20

Hi Quanah Here is the output of slapd -VVV command. I do see accesslog as static overlays. What changes i need to make in below snippet *dn: cn=module{0},cn=configchangetype: modifyadd: olcModuleLoadolcModuleLoad: syncprovolcModuleLoad: accesslog* [HPE SmartFabric:root@SN2100-106 initial_config]#slapd -VVV @(#) $OpenLDAP: slapd 2.4.50 (Jul 1 2020 20:22:05) $ @SN2100-106:/root/smartfabric/open-ldap/initial_config/openldap-2.4.50/servers/slapd Included static overlays: accesslog syncprov Included static backends: config ldif monitor mdb relay On Wed, Jul 1, 2020 at 5:13 PM Quanah Gibson-Mount <quanah(a)symas.com> wrote: > > > --On Wednesday, July 1, 2020 4:28 PM -0400 kumar rahul > <rahul2002mit(a)gmail.com> wrote: > > > > > > > Hi Quanah > > > > [Rahul] How do I verify this if modules are compiled statically? > > slapd -VV > > But if you're building OpenLDAP yourself, I suggest reading the output of > configure, which tells you how to build dynamically vs statically. > > > --Quanah > > > -- > > Quanah Gibson-Mount > Product Architect > Symas Corporation > Packaged, certified, and supported LDAP solutions powered by OpenLDAP: > <http://www.symas.com> >

2 1

slapd 2.4.44 Performance problems
by daniel.zuniga＠gmail.com 01 Jul '20

01 Jul '20

Hello, We are running OpenLDAP with a relatively small user base (15K users) and the server does not seem to be able to keep up with demand when there are multiple concurrent logins (20 users X 70 searches each) during user login. When users login to the server all cores in the server (8) run at 100% utilization, sometimes for as long as 20 minutes (depending on how many users are logging in concurrently). We have experimented and read and re-read the Tuning guidance but with very little success. We have tried increasing the number of available cores to slapd, but it does not use more than 8. We have also increased the number of threads from 16 to 32, but that also does not seem to have an effect nor does it seem to allow slapd to use all 16 cores (slapd seems limited to 8 cores regardless of the number of threads defined). The server has plenty of memory available and configured the db cache to use as much as in needs, but that does not seem to improve matters. Can you offer any guidance? Thanks.

3 4

Re: [EXT] Re: syncrepl does not work as expected
by kumar rahul 01 Jul '20

01 Jul '20

Hi Quanah During the installation configure script was run like this ./configure --enable-bdb=no --enable-hdb=no --enable-ipv6=yes I just added following option to enable all overlays ./configure --enable-bdb=no --enable-hdb=no --enable-ipv6=yes *--enable-overlays=yes* Will this fix the problem? Thanks Rahul On Wed, Jul 1, 2020 at 1:38 PM Quanah Gibson-Mount <quanah(a)symas.com> wrote: > --On Wednesday, July 1, 2020 11:09 AM -0400 kumar rahul > <rahul2002mit(a)gmail.com> wrote: > > > I am seeing following ERROR > > I have no idea how your slapd was built, etc. But essentially the > accesslog overlay must be loaded into slapd to be of use. It looks like > at > least some of the modules in your system are compiled statically, which is > likely why you currently do not have a module{0} section for loading > modules. You'll also have to verify where on disk the accesslog overlay > exists so that the moduleload statement pulls from the right directory. > > Regards, > Quanah > > > -- > > Quanah Gibson-Mount > Product Architect > Symas Corporation > Packaged, certified, and supported LDAP solutions powered by OpenLDAP: > <http://www.symas.com> >

1 0

Re: [EXT] Re: syncrepl does not work as expected
by kumar rahul 01 Jul '20

01 Jul '20

Thank you for the pointers. I will setup delta syncrepl and test to verify if I see the issue or not. Thanks Rahul On Mon, Jun 29, 2020 at 4:02 PM Quanah Gibson-Mount <quanah(a)symas.com> wrote: > --On Monday, June 29, 2020 4:20 PM -0400 kumar rahul > <rahul2002mit(a)gmail.com> wrote: > > > Thank you for creating the bug. I have added myself as CC. Based on > > the configuration information I have provided, can you provide a set of > > guidelines to configure my setup to use delta-syncrepl. > > The admin guide already provides information the configuration bits > necessary to set up delta-syncrepl. Generally: > > a) You need a secondary DB to store the accesslog data from the accesslog > overlay. > > b) You need to reconfigure the olcSyncrepl config to make use of the > accesslog db. > > <https://www.openldap.org/doc/admin24/replication.html#Delta-syncrepl> > > And a blog post I wrote that gives some example configs: > > < > https://mishikal.wordpress.com/2019/04/23/configuring-mmr-using-delta-syncr… > > > > Regards, > Quanah > > -- > > Quanah Gibson-Mount > Product Architect > Symas Corporation > Packaged, certified, and supported LDAP solutions powered by OpenLDAP: > <http://www.symas.com> >

2 3

Re: [EXT] Re: syncrepl does not work as expected
by kumar rahul 29 Jun '20

29 Jun '20

Hi Quanah Thank you for creating the bug. I have added myself as CC. Based on the configuration information I have provided, can you provide a set of guidelines to configure my setup to use delta-syncrepl. Thanks Rahul On Mon, Jun 29, 2020 at 3:10 PM Quanah Gibson-Mount <quanah(a)symas.com> wrote: > > > --On Monday, June 29, 2020 3:59 PM -0400 kumar rahul > <rahul2002mit(a)gmail.com> wrote: > > > > > > > Hi Quanah > > > > > > Thank you for reproducing the issue. What is the course of action > > from here on? There will be an openldap release with the fix? or a patch > > will be released? Also can you share the bug # once you raise it. How > > will I get notified once the issue is fixed and is released? > > Hi Kumar, > > As was stated before, use delta-syncrepl as it avoids these types of > issues > with standard syncrepl. I'm guessing this is another variation of issue > #8125 (<https://bugs.openldap.org/show_bug.cgi?id=8125>), although I'm > initially filing a separate bug in case it is a different problem, issue > #9282 (<https://bugs.openldap.org/show_bug.cgi?id=9282>). > > You can add yourself to the CC of the issue to track its status and > resolution. > > Regards, > Quanah > > -- > > Quanah Gibson-Mount > Product Architect > Symas Corporation > Packaged, certified, and supported LDAP solutions powered by OpenLDAP: > <http://www.symas.com> >

2 1

Re: [EXT] Re: syncrepl does not work as expected
by kumar rahul 29 Jun '20

29 Jun '20

Hi Quanah Thank you for reproducing the issue. What is the course of action from here on? There will be an openldap release with the fix? or a patch will be released? Also can you share the bug # once you raise it. How will I get notified once the issue is fixed and is released? Thanks Rahul On Mon, Jun 29, 2020 at 2:55 PM Quanah Gibson-Mount <quanah(a)symas.com> wrote: > --On Tuesday, June 23, 2020 5:04 PM -0400 kumar rahul > <rahul2002mit(a)gmail.com> wrote: > > > Hi Quanah > > Hi Kumar, > > I was able to reproduce the issue, thanks for the report. I'll file a bug > for it. > > Regards, > Quanah > > -- > > Quanah Gibson-Mount > Product Architect > Symas Corporation > Packaged, certified, and supported LDAP solutions powered by OpenLDAP: > <http://www.symas.com> >

2 1

Re: [EXT] Re: syncrepl does not work as expected
by kumar rahul 29 Jun '20

29 Jun '20

Hi Quanah Please find my answers inline a) Both servers have no DB, but are using these configs [Rahul] Both servers are started with an empty database. Both servers are in Provider and consumer mode. Our application is running on both servers as well. To start with our application is active on Server A, which means data will be written to server A database.and is synced/replicated to server B b) DB is added to server A and replicated to server B [Rahul] Yes since server A has our application in active mode hence data is written on server A and replicated to server B. c) server A is stopped [Rahul] yes we reboot the server A. d) You say "data is removed", I'm not sure what this means. A single entry? Multiple entries? The entire database? [Rahul] Now our application is active on Server B. Using server B removed "a Single Entry" which was removed from server B database as well. e) S1 is restarted, and the deleted data is replicated back to both nodes. [Rahul] Yes deleted data is replicated back to both nodes. when S1 is restarted slapd is configured in provider/consumer mode but our application is still active on server B which means writes will only happen at server B and should be synced to server A. Thank you for all the help so far. I have no intention of interrupting your work. Please look at my issue based on your convenience. Let me know if you have any other questions. Thanks Rahul On Tue, Jun 23, 2020 at 3:43 PM Quanah Gibson-Mount <quanah(a)symas.com> wrote: > > > --On Tuesday, June 23, 2020 1:47 PM -0400 kumar rahul > <rahul2002mit(a)gmail.com> wrote: > > > Both the servers are set up as master as well as consumer so that > > sync works in both directions. So > > the sync setup is like "A -> B", > > and when A fails B is updated and the expectation is that "B -> A" should > > work because a multi master setup is already in place. Let me know based > > on the attached configuration files if my assumption is correct or not. > > > > > > Also Add data works when A fails and B is updated then sync "B -> A"works > > fine. > > > > After Delete data sync does not work when A fails and data is deleted > > from B. Once A comes back both databases have data which was deleted on > B. > > Hi, > > I'll see if I can reproduce your issue with the configs provided (Assuming > they look correct after I finish reviewing them in full), but I want to > confirm the exact steps: > > a) Both servers have no DB, but are using these configs > > b) DB is added to server A and replicated to server B > > c) server A is stopped > > d) You say "data is removed", I'm not sure what this means. A single > entry? Multiple entries? The entire database? > > e) S1 is restarted, and the deleted data is replicated back to both nodes. > > Note that I do have a paid job that keeps me busy, so this is a best > effort > when I have time available to investigate. > > Thanks, > Quanah > -- > > Quanah Gibson-Mount > Product Architect > Symas Corporation > Packaged, certified, and supported LDAP solutions powered by OpenLDAP: > <http://www.symas.com> >

2 1

LMDB transaction flags
by Stefano Cossu 29 Jun '20

29 Jun '20

Hello, I'd like to be able to know if a LMDB transaction is active, valid, dirty, read-only, etc. From the documentation I read that such information is in mt_flags of the mdb_txn structure, which is opaque. I don't see any function exposing mt_flags. Is there any other way to read that information?

2 4

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

openldap-technical