openldap-technical

openldap-technical@openldap.org

13 participants
9794 discussions

OpenLDAP performance and slapindex
by Real, Elizabeth (392K) 16 Dec '16

16 Dec '16

Hello, I’m running OpenLDAP 2.4.40 on two multi-master servers with replication enabled. Today we switched over to this instance, imported all users from the old ldap server, and configured an ldap client to use this instance. Everything is working well, users are authenticating and able to access their assigned groups. However, we noticed that while on the ldap client and as a regular user we type “w” to see who is logged on this machine, it takes a while before we get a response with the results. If we type “w” again the response is very fast due to caching. In looking at ways to improve performance of ldap, is slapindex a tool to use? Meaning after importing users accounts from an older ldap, should slapindex be ran? Or is this tool is used when you want to modify/add entries found in the slapd (cn=config.ldif) and database ((olcDatabase={2}hdb.ldif) configuration files not when users are added to the ldap database. If slapindex is not a tool to use to monitor performance, what do you recommend? Thank you, Liz

3 2

Re: OpenLDAP performance and slapindex
by Quanah Gibson-Mount 15 Dec '16

15 Dec '16

--On Thursday, December 15, 2016 8:27 PM +0000 "Real, Elizabeth (392K)" <Elizabeth.Real(a)jpl.nasa.gov> wrote: ># less /var/log/ldap.log | grep index > > > > Dec 15 12:22:01 slapd[27852]: <= bdb_equality_candidates: (uid) not > indexed Hi Liz, As I noted previously, you need to look at the actual search being executed. This is because, as Michael noted, indexing is not necessarily always the best solution performance wise. I'd guess that you likely need to add indices on those attributes, but I can't say for certain w/o the corresponding log lines that caused those warnings to be generated. And without those log lines, it is not possible to determine what /type/ of indexing should be used if it is proven necessary. Regards, Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

2 1

Re: OpenLDAP performance and slapindex
by Quanah Gibson-Mount 15 Dec '16

15 Dec '16

--On Wednesday, December 14, 2016 9:05 PM +0000 "Real, Elizabeth (392K)" <Elizabeth.Real(a)jpl.nasa.gov> wrote: > > > Hello, > > > > I'm running OpenLDAP 2.4.40 Upgrade immediately to 2.4.44. The 2.4.40 release was seriously flawed and had numerous critical bugs around replication. > In looking at ways to improve performance of ldap, is slapindex a tool to > use? Meaning after importing users accounts from an older ldap, should > slapindex be ran? Or is this tool is used when you want to modify/add > entries found in the slapd (cn=config.ldif) and database > ((olcDatabase={2}hdb.ldif) configuration files not when users are added > to the ldap database. Stop using back-hdb. Switch to back-mdb. <https://mishikal.wordpress.com/2013/05/16/openldap-a-comparison-of-back-mdb…> If you already had indices defined for the database when you did the import, there is no reason to run slapindex. Slapindex should only be used if you've added a /new/ index to your database. If you are using cn=config and add a new index, there will be a background thread that will eventually index the entire DB. Whether or not you need to add indexing for given attribute(s) can be discovered by examining the logs generated by slapd, which will file a complaint if there is a search performed on an unindexed attribute. You would need to examine the specific search to see what type(s) of indexing may be required. > If slapindex is not a tool to use to monitor performance, what do you > recommend? slapindex has never been a tool to monitor anything. It is purely for (re)generating indices on an existing DB. Hope that helps! Regards, Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

3 3

replica consumers and ppolicy overlay values like PwdFailureTime
by John Jasen 15 Dec '16

15 Dec '16

I'm trying to setup OpenLDAP on CentOS7, in a provider/consumer relationship. In general, provider/consumer is working quite well, except when it comes to password policy. Specifically, I want PwdFailureTime to be written to the provider from one of the front end consumers when appropriate. I'm lead to believe this requires: a) ppolicy_foward_updates TRUE (done) b) an appropriate syncrepl configuration for the consumer (I believe done) c) updateref $LDAP-provider-URI (done) d) an appropriate chain overlay on the provider (I think done) e) appropriate ACLs on the provider to allow the consumer bind-user access to manage PwdFailureTime (I believe done) I've attempted all of the above, but the consumer (when run in debugging mode), does not seem to be trying any updates upon authentication failure. It gives no indication of modifying locally, or of trying to contact the provider at all over this. Any idea whats going wrong?

1 1

Re: OpenLDAP performance and slapindex
by Quanah Gibson-Mount 14 Dec '16

14 Dec '16

--On Wednesday, December 14, 2016 9:36 PM +0000 "Real, Elizabeth (392K)" <Elizabeth.Real(a)jpl.nasa.gov> wrote: > > > Thank you for the quick response. > > > > I have spent a lot of time configuring version 2.4.40. Is there a guide > on how to upgrade to 2.4.44 and switch to back-mdb? we rolled out 2.4.40 > today and would need to plan the next upgrade. As long as it is linked to the same BDB libraries as your 2.4.40 build, it is simply a drop in binary upgrade. > What tool do you recommend for monitoring ldap performance? I suggest applying the operation duration patch: <https://github.com/Zimbra/packages/blob/master/thirdparty/openldap/patches/…> That will tell you exactly how long each operation is taking in slapd, as long as you have "stats" logging enabled. Then you can tell if the issue is at the slapd server, or if there is something else in between that is causing the delay that you see. My suspicion would be that there is something else interfering. As for migrating from hdb to mdb, it's generally straight forward. You'd need to export your data database to LDIF, and then export cn=config, and change the HDB specific bits to MDB, and drop the parameters specific to HDB that aren't required for MDB. If you want to see a basic cn=config templated configuration, you can look at: <http://www.openldap.org/devel/gitweb.cgi?p=openldap.git;a=blob_plain;f=test…> where @BACKEND@ can be any of bdb/hdb/mdb, and tweak as necessary from there. --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

Re: Expanding OpenLDAP use
by Quanah Gibson-Mount 10 Dec '16

10 Dec '16

--On Saturday, December 10, 2016 5:15 PM -0200 xsun <matheus.morais(a)gmail.com> wrote: > > Great! > > > Thank you so far lads. > > > > Matheus Morais The Zimbra product uses OpenLDAP to store numerous data, everything from server configurations to user accounts, etc. It is deployed at sites large and small throughout the world, plenty with many millions of accounts. --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

Expanding OpenLDAP use
by xsun 10 Dec '16

10 Dec '16

Hi, I'm trying to expanding OpenLDAP use where I work. Right now we use it to handle internal users (about 50000 entries) and organizational data and it work superb. I'm trying to convince my colleagues to also store our customers data but they are worried about the performance since this will be much larger as we have about 4 million customers. I'm looking for some public uses cases for large number of users in OpenLDAP to give more support to my arguments. So, can anyone share some history or point out some information on the web on that particular subject? Thanks, Matheus Morais

2 2

Re: Antw: Replication through BDB
by Howard Chu 10 Dec '16

10 Dec '16

Ulrich Windl wrote: >>>> Rick van Rein <rick(a)openfortress.nl> schrieb am 08.12.2016 um 22:37 in > Nachricht <5849D28D.8050408(a)openfortress.nl>: >> Hello, >> >> I've been thinking about replication schemes lately. I very much like >> SyncRepl, especially that it can be used in master-master mode, but it >> also has a downside -- it returns before a majority of replicated >> servers have agreed on a change. One might say that the change is >> committed before certainty has been established. > > It seems you are mixing a quorum protocol with a synchronisation protocol: RAID1 with sequential (not parallel) writes has the same weaknesses, BTW. > > if a change was committed to one server, the change is there; certainly. What is your "certainty"? > >> >> This is different with the replication scheme that is now built into >> BerkeleyDB, by Oracle; this has a scheme based on majority voting, and >> with automatic resumption after downtime based on this majority. The >> BerkeleyDB will not return cheerfully from a commit until a majority has >> confirmed. > > Generally the majority can be wrong: Assume you have a network-failure in a three-node MMR configuration: You update one node while the other two are unreachable. The communication resumes, do you expect the change on the none node to be reverted to majority, or should the majority be updated from the one node that has more recent data? Indeed. In syncrepl, "voting" is irrelevant. Changes will be accepted by any provider node that a client can reach. When connectivity is restored all nodes will bring each other up to date. In majority-based voting, you will lose any writes to the minority node, which leaves you with unresolvable inconsistencies. I.e., data is removed but the clients believe it was written. >> AFAIK the HDB backend that was once the default for OpenLDAP has been >> replaced with the plain vanilla BerkeleyDB... so it seems that the >> replication scheme of the latter can be used. Is that right / does >> anyone see problems with that / ...? back-hdb and back-bdb both use BerkeleyDB. BerkeleyDB is now deprecated/obsolete, and LMDB is the default backend. BDB's replication is page-oriented, so it would consume far more network resources than syncrepl. We have never recommended its use. At this point, with the licensing incompatibility, it's all moot. > Oracle licensing conditions may be a different thing. > Despite of all that LDAP replication is not done at the database level. > > Was the intention of your message to say: "Oracle has the better databases"? > > Regards, > Ulrich > > > > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

2 1

Replication through BDB
by Rick van Rein 09 Dec '16

09 Dec '16

Hello, I've been thinking about replication schemes lately. I very much like SyncRepl, especially that it can be used in master-master mode, but it also has a downside -- it returns before a majority of replicated servers have agreed on a change. One might say that the change is committed before certainty has been established. This is different with the replication scheme that is now built into BerkeleyDB, by Oracle; this has a scheme based on majority voting, and with automatic resumption after downtime based on this majority. The BerkeleyDB will not return cheerfully from a commit until a majority has confirmed. AFAIK the HDB backend that was once the default for OpenLDAP has been replaced with the plain vanilla BerkeleyDB... so it seems that the replication scheme of the latter can be used. Is that right / does anyone see problems with that / ...? Thanks, Rick van Rein for ARPA2.net / InternetWide.org

2 1

Re: Replication through BDB
by Quanah Gibson-Mount 09 Dec '16

09 Dec '16

--On Thursday, December 08, 2016 10:37 PM +0100 Rick van Rein <rick(a)openfortress.nl> wrote: > AFAIK the HDB backend that was once the default for OpenLDAP has been > replaced with the plain vanilla BerkeleyDB... so it seems that the > replication scheme of the latter can be used. Is that right / does > anyone see problems with that / ...? BerkeleyDB6 and later does not have a license that is compatible with OpenLDAP. All support for it as a backend will be removed in the future. I wouldn't base anything on BDB's replication methods. For multimaster, I'd advise using delta-syncrepl replication. I've had numerous clients with multimillion entry DBs where MMR works just fine. For there to really be an issue would require simultaneous updates to the same attribute on the same entry, which could happen theoretically but in practice is not something I've seen occur. And if you're particularly concerned about such things, then the correct route to go would be to set up a load balancer in front of the masters so only one specific master gets write traffic during a given interval, which is referred to as mirror mode in the admin guide. Hope that helps. --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

openldap-technical