openldap-technical

openldap-technical@openldap.org

9 participants
9866 discussions

Re: When does logpurge run ?
by Quanah Gibson-Mount 06 Sep '17

06 Sep '17

--On Wednesday, September 06, 2017 1:00 AM +0000 ping-shin ching <ping_sc(a)hotmail.com> wrote: > > > I have the "logpurge <age> <interval>" information. > > > > > logpurge 2+00:00 1+00:00 > would specify that the log database should be scanned > every day > for old entries, and entries older than two days > should be > deleted. When using a log database that supports > ordered > indexing on generalizedTime attributes, specifying an eq > index > on the reqStart attribute will greatly benefit the > performance > > But when does the comparison and delete happen? 'Scanned every day' - at > what time does the scan occur? >From when the slapd process was started. --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

When does logpurge run ?
by ping-shin ching 06 Sep '17

06 Sep '17

Hi Folks, When does the logpurge (for accesslog) run? Can we control the time this process runs? Regards, Ping

2 2

Re: N-Way Multi Master setup missing entries on consumers
by Quanah Gibson-Mount 04 Sep '17

04 Sep '17

--On Monday, September 04, 2017 6:52 AM +0000 ping-shin ching <ping_sc(a)hotmail.com> wrote: > I have 4 servers setup via N-Way multi master using 2.4.44. I would advise upgrading to OpenLDAP 2.4.45, as it contained specific fixes for replication with N-Way MMR and delta-syncrepl <http://www.openldap.org/software/release/changes.html> If you still see issues, then you'll need to provide logs at "stats sync" level for the providers and consumers. --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

Strange behaviors on OpenLDAP proxy
by David Coutadeur 04 Sep '17

04 Sep '17

Hi community, I have implemented two "OpenLDAP mirror directories": ldaps://ldap1 and ldaps://ldap2 (version 2.4.44) and a LDAP proxy with back_ldap + overlay pcache (version 2.4.44). I am trying to understand behaviors on the LDAP proxy: 1 - I don't succeed to configure a failover switch between OpenLDAP backends. The proxy switches too quickly due to a network disconnection (for example, network stays unavailable during n second). I would be interested whether there is a way to better control the switch even after network failure, for example, adding LDAP new parameters in order to send 3 attempts before performing failover. 2 - After that, when the network is up again and the first directory "ldap1" is back, new requests with already established connection to ldap1 are directed again to directory "ldap1" Is it possible to make all the trafic stay on ldap2? Is there a way to close open connections to ldap1 directory when the proxy switches to the ldap2 directory? 3. Finally, Is there a way to switch to the second directory when queries are too slow in the first directory? Thanks in advance. David

1 0

N-Way Multi Master setup missing entries on consumers
by ping-shin ching 04 Sep '17

04 Sep '17

Hi Folks, I have 4 servers setup via N-Way multi master using 2.4.44. Updates go to a single provider. We have close to 20 consumers connected to each of the providers via delta syncrepl. Fairly busy system with about 10000 additions, 64000 modifications and 5000 delete on a busy day. Current database is about 15 million entries. We see that some changes are not sent down to the consumers. We miss about 100 entries a day. These entries appear in all 4 providers, but are randomly missing in the consumers. I am still investigating, but any insight/help will be greatly appreciated. The providers are on RHEL 6, the consumers are on solaris 10. configure options... --without-cyrus-sasl \ --disable-bdb \ --disable-hdb \ --enable-ldap \ --enable-mdb \ --enable-constraint Regards, Ping ========================= one of the provider config files ======================== # # FileName: slapd.conf # # Author: $Author: d639599 $ # Date: $Date: 2014-08-22 14:00:37 +1000 (Fri, 22 Aug 2014) $ # Revision: $Revision: 18625 $ # CVS Tag: $Name$ # CVS ID: $Id: slapd_master.conf.in 18625 2014-08-22 04:00:37Z d639599 $ # # # See slapd.conf(5) for details on configuration options. # This file should NOT be world readable. # include /data/openldap24/etc/openldap/schema/core.schema include /data/openldap24/etc/openldap/schema/cosine.schema include /data/openldap24/etc/openldap/schema/nis.schema include /data/openldap24/etc/openldap/schema/radius.schema # Define global ACLs to disable default read access. # Do not enable referrals until AFTER you have a working directory # service AND an understanding of referrals. #referral ldap://root.openldap.org pidfile /data/openldap24/var/run/slapd.pid argsfile /data/openldap24/var/run/slapd.args #replica-pidfile /data/openldap/var/run/slurpd.pid loglevel 32768 threads 32 tool-threads 8 allow bind_v2 # The number of results to return in a search sizelimit unlimited serverID 02 access to dn=uid=newevdouser(a)xman.com,ou=users,ou=evdo,ou=data,o=company,c=org by dn=cn=infranet,ou=applicationusers,o=company,c=org read by dn=cn=activeorder,ou=applicationusers,o=company,c=org read by dn=cn=asap,ou=applicationusers,o=company,c=org read .......... .......... ####################################################################### # monitor database ####################################################################### database monitor rootdn "cn=monitoring,cn=Monitor" rootpw XXXXXXXXXXXXXXXXXX access to dn.subtree="cn=Monitor" by dn.exact="cn=Manager,o=company,c=org" read by * none ####################################################################### # primary database ####################################################################### database mdb suffix "o=company,c=org" rootdn "cn=Manager,o=company,c=org" # Cleartext passwords, especially for the rootdn, should # be avoid. See slappasswd(8) and slapd.conf(5) for details. # Use of strong authentication encouraged. rootpw XXXXXXXXXXXXXXXXXX #size of sparse file 64Gb maxsize 68719476736 #only required to receive data from slurpd #updatedn cn=directorymaster,ou=applicationusers,ou=radiusdata,o=company,c=org # The database directory MUST exist prior to running slapd AND # should only be accessible by the slapd and slap tools. # Mode 700 recommended. directory /data/openldap24/var/openldap-data #Allow dirmaster unlimited time for searches limits dn.exact="cn=directorymaster,ou=applicationusers,ou=radiusdata,o=company,c=org" time=unlimited # Indices to maintain index objectClass eq index uid eq index cn eq index macAddress eq index pinpoid eq index bpiPoid eq index target eq index interceptType eq index interceptValue eq index imei eq index ipHostNumber eq index homeLocation eq index sid eq index remoteId eq index parentDn eq # required for sessionlaog index entryCSN eq index entryUUID eq checkpoint 128 1 dbnosync # syncrepl directives for each of the other masters for primary db replication ####################################################################### ## Syncrepl entry for 01 syncrepl rid=01 provider=ldap://host1 type=refreshAndPersist retry="10 +" searchbase="o=company,c=org" bindmethod=simple type=refreshAndPersist binddn="cn=dirmaster,ou=appusers,a,o=company,c=org" credentials=YYYYY retry="60 10 300 +" schemachecking=on # Syncrepl entry for 02 syncrepl rid=02 provider=ldap://host3 type=refreshAndPersist retry="10 +" searchbase="o=company,c=org" bindmethod=simple type=refreshAndPersist binddn="cn=dirmaster,ou=appusers,a,o=company,c=org" credentials=YYYYY retry="60 10 300 +" schemachecking=on # Syncrepl entry for 03 syncrepl rid=03 provider=ldap://host4 type=refreshAndPersist retry="10 +" searchbase="o=company,c=org" bindmethod=simple type=refreshAndPersist binddn="cn=dirmaster,ou=appusers,a,o=company,c=org" credentials=YYYYY retry="60 10 300 +" schemachecking=on #need mirror mode to accept writes mirrormode on overlay accesslog logdb cn=accesslog logops writes logsuccess TRUE logbase writes o=company,c=org # scan the accesslog DB every day, and purge entries older than 14 days logpurge 14+00:00 01+00:00 #sycrepl provider config # define the provider to use the syncprov overlay # (last directives in database section) overlay syncprov syncprov-checkpoint 10000 10 # contextCSN saved to database every 10000 updates or ten minutes syncprov-sessionlog 10000 syncprov-nopresent TRUE ####################################################################### # accesslog database ####################################################################### database mdb suffix "cn=accesslog" rootdn "cn=Manager,o=company,c=org" #size of sparse file 16Gb maxsize 17179869184 directory /data/openldap24/var/openldap-data-accesslog #Allow dirmaster unlimited time for searches limits dn.exact="cn=dirmaster,ou=appusers,a,o=company,c=orgg" time=unlimited index entryCSN eq index objectClass eq index reqEnd eq index reqResult eq index reqStart eq #sycrepl provider config # define the provider to use the syncprov overlay # (last directives in database section) overlay syncprov syncprov-nopresent TRUE syncprov-reloadhint TRUE ====================================== consumer config ====================================== # # FileName: slapd.conf # # Author: $Author: d639599 $ # Date: $Date: 2014-08-22 14:00:37 +1000 (Fri, 22 Aug 2014) $ # Revision: $Revision: 18625 $ # CVS Tag: $Name$ # CVS ID: $Id: slapd_master.conf.in 18625 2014-08-22 04:00:37Z d639599 $ # # # See slapd.conf(5) for details on configuration options. # This file should NOT be world readable. # include /data/openldap24/etc/openldap/schema/core.schema include /data/openldap24/etc/openldap/schema/cosine.schema include /data/openldap24/etc/openldap/schema/nis.schema include /data/openldap24/etc/openldap/schema/radius.schema # Define global ACLs to disable default read access. # Do not enable referrals until AFTER you have a working directory # service AND an understanding of referrals. #referral ldap://root.openldap.org pidfile /data/openldap24/var/run/slapd.pid argsfile /data/openldap24/var/run/slapd.args #replica-pidfile /data/openldap/var/run/slurpd.pid loglevel 32768 # #threads 64 # allow bind_v2 # # The number of results to return in a search sizelimit unlimited tool-threads 2 threads 8 # Load dynamic backend modules: # modulepath /data/openldap/libexec/openldap # moduleload back_bdb.la # moduleload back_ldap.la # moduleload back_ldbm.la # moduleload back_passwd.la # moduleload back_shell.la # Sample security restrictions # Require integrity protection (prevent hijacking) # Require 112-bit (3DES or better) encryption for updates # Require 63-bit encryption for simple bind # security ssf=1 update_ssf=112 simple_bind=64 # Sample access control policy: # Root DSE: allow anyone to read it # Subschema (sub)entry DSE: allow anyone to read it # Other DSEs: # Allow self write access # Allow authenticated users read access # Allow anonymous users to authenticate # Directives needed to implement policy: # access to dn.base="" by * read # access to dn.base="cn=Subschema" by * read # access to * # by self write # by users read # by anonymous auth # # if no access controls are present, the default policy # allows anyone and everyone to read anything but restricts # updates to rootdn. (e.g., "access to * by * read") # # rootdn can always read and write EVERYTHING! access to * by cn=dirmaster,ou=appusers,a,o=company,c=org write by dn=cn=radiusserver,ou=applicationusers,o=company,c=org read .......... .......... ####################################################################### # config database ####################################################################### database config # NOTE: the suffix is hardcoded as cn=config and # MUST not have a suffix directive # normal rules apply - rootdn can be anything you want # but MUST be under cn=config rootdn "cn=admin,cn=config" # use any of the supported password formats e.g. {SSHA} etc # or plaintext as shown rootpw XXXXXXXXXXXXXXXXXX ####################################################################### # monitor database ####################################################################### database monitor rootdn "cn=monitoring,cn=Monitor" rootpw {SHA}wauZJOzaG+r4u6oeuCOLg+DtjGM= access to dn.subtree="cn=Monitor" by dn.exact="cn=Manager,o=company,c=org" read by * none ####################################################################### # mdb database definitions ####################################################################### database mdb suffix "o=company,c=org" rootdn "cn=Manager,o=company,c=org" # Cleartext passwords, especially for the rootdn, should # be avoid. See slappasswd(8) and slapd.conf(5) for details. # Use of strong authentication encouraged. rootpw XXXXXXXXXXXXXXXX maxsize 68719476736 # To speedup further - possibly at the expense of data integrity # Only use for slappadd without slapd running #envflags nometasync #envflags writemap # The database directory MUST exist prior to running slapd AND # should only be accessible by the slapd and slap tools. # Mode 700 recommended. directory /data/openldap24/var/openldap-data # Indices to maintain index objectClass eq index uid eq index cn eq index macAddress eq index pinpoid eq index bpiPoid eq index target eq index interceptType eq index interceptValue eq index imei eq index ipHostNumber eq index homeLocation eq index sid eq index remoteId eq index parentDn eq index entryUUID eq index entryCSN eq checkpoint 128 1 dbnosync # syncrepl directives for primary db replication ####################################################################### ## Syncrepl entry for 01 syncrepl rid=01 provider=ldap://provider1 bindmethod=simple binddn="cn=dirmaster,ou=appusers,a,o=company,c=org" credentials=XXXXXXX searchbase="o=company,c=org" logbase="cn=accesslog" type=refreshAndPersist scope=sub retry="10 +" schemachecking=off logfilter="(&(objectClass=auditWriteObject)(reqResult=0)(|(reqDN:dnSubtreeMatch:=ou=radiusdata,o=company,c=org)(reqDN:dnSubtreeMatch:=ou=applicationusers,o=company,c=org)))" syncdata=accesslog filter="(|(entrydn:dnSubtreeMatch:=ou=radiusdata,o=company,c=org)(entrydn:dnSubtreeMatch:=ou=applicationusers,o=company,c=org))"

1 0

Re: Large scale traffic testing
by Quanah Gibson-Mount 04 Sep '17

04 Sep '17

--On Sunday, September 03, 2017 8:48 PM +0100 Tim <tim(a)yetanother.net> wrote: > I'm currently working with a dataset of in the region of 2,500,000 > objects and looking to test throughput up to somewhere in the region of > 15k/s searches alongside 1k/s modification/addition events - which is > beyond what the current basic scripts are able to achieve. Those are all miniscule numbers to achieve with such a tiny dataset, assuming you're using LMDB and have reasonable hardware. > So I was wondering what strategies other people have used to stress test > platforms elsewhere? The best stress testing software for OpenLDAP is something that is distributed, such as slamd or jmeter. Anything else will hit artificial limits. You may be interested in seeing already generalized published benchmarks, such as: <http://lanyrd.com/2013/ldapcon/sckyhr/> I also did some (read rate) benchmarks of OpenLDAP quite some years ago, on fairly weak commodity hardware: <https://mishikal.wordpress.com/2013/05/16/openldap-a-comparison-of-back-mdb…> Here are also single-threaded write rate tests I did for Zimbra: <https://wiki.zimbra.com/wiki/OpenLDAP_MDB_vs_HDB_performance> Generally, you'll hit system limitations (disk write speed, network bandwidth) before you max out what OpenLDAP can actually do. Hope that helps. --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

Re: N-Way replication configuration
by rammohan ganapavarapu 02 Sep '17

02 Sep '17

Thank you for clarification, now I am more clear. One more question, what is the best or recommended way of doing n-way rep, should we replicate both config db and user db as well ? Right now I am not replicating config. On Sep 1, 2017 7:34 PM, "Quanah Gibson-Mount" <quanah(a)symas.com> wrote: --On Friday, September 01, 2017 6:03 PM -0700 rammohan ganapavarapu < rammohanganap(a)gmail.com> wrote: > Quanah, > > > Thank you, so do we need to include olcServerID of all servers in every > server? for example if i have 3 servers MMR, then do i need to add > > > olcServerID 1 $URl1 > > olcServerID 2 $URl2 > > olcServerID 3 $URl2 > Sorry, let me clarify a bit on my earlier answer. For olcServerID, it only needs the URI *if* you are replicating cn=config. For the olcSyncrepl: lines, you never have to point at the master. It only needs olcSyncrepl lines for the other masters in the cluster. Pointing it at itself in olcSyncrepl just generates unnecessary overhead. --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

Re: N-Way replication configuration
by Quanah Gibson-Mount 02 Sep '17

02 Sep '17

--On Friday, September 01, 2017 6:03 PM -0700 rammohan ganapavarapu <rammohanganap(a)gmail.com> wrote: > > Quanah, > > > Thank you, so do we need to include olcServerID of all servers in every > server? for example if i have 3 servers MMR, then do i need to add > > > olcServerID 1 $URl1 > > olcServerID 2 $URl2 > > olcServerID 3 $URl2 Sorry, let me clarify a bit on my earlier answer. For olcServerID, it only needs the URI *if* you are replicating cn=config. For the olcSyncrepl: lines, you never have to point at the master. It only needs olcSyncrepl lines for the other masters in the cluster. Pointing it at itself in olcSyncrepl just generates unnecessary overhead. --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

Re: N-Way replication configuration
by rammohan ganapavarapu 02 Sep '17

02 Sep '17

Quanah, Thank you, so do we need to include olcServerID of all servers in every server? for example if i have 3 servers MMR, then do i need to add olcServerID 1 $URl1 olcServerID 2 $URl2 olcServerID 3 $URl2 In all servers or just keep self ID? Ram On Fri, Sep 1, 2017 at 1:21 PM, Quanah Gibson-Mount <quanah(a)symas.com> wrote: > --On Thursday, August 31, 2017 6:46 PM -0700 rammohan ganapavarapu < > rammohanganap(a)gmail.com> wrote: > > Which one is the right way to do? >> > > It depends. If you are replicating cn=config, then the provider=.... is > required, and it must include all providers. If you are not replicating > cn=config, then the provider= is not required at all, and can be left out. > > Also what is the significance of "olcServerID" do we have keep every >> server "olcServerID" in each server? >> > > olcServerID is required on all masters, and must be unique between every > master. This is how OpenLDAP knows which master a particular change > occurred on, and is mandatory for MMR to function correctly. > > --Quanah > > -- > > Quanah Gibson-Mount > Product Architect > Symas Corporation > Packaged, certified, and supported LDAP solutions powered by OpenLDAP: > <http://www.symas.com> > >

1 0

Re: N-Way replication configuration
by Quanah Gibson-Mount 01 Sep '17

01 Sep '17

--On Thursday, August 31, 2017 6:46 PM -0700 rammohan ganapavarapu <rammohanganap(a)gmail.com> wrote: > Which one is the right way to do? It depends. If you are replicating cn=config, then the provider=.... is required, and it must include all providers. If you are not replicating cn=config, then the provider= is not required at all, and can be left out. > Also what is the significance of "olcServerID" do we have keep every > server "olcServerID" in each server? olcServerID is required on all masters, and must be unique between every master. This is how OpenLDAP knows which master a particular change occurred on, and is mandatory for MMR to function correctly. --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

openldap-technical