openldap-technical

openldap-technical@openldap.org

9790 discussions

Re: Failing consumers over to second mirror master provider
by Quanah Gibson-Mount 08 Sep '17

08 Sep '17

--On Friday, September 08, 2017 10:37 AM -0400 Frank Swasey <Frank.Swasey(a)uvm.edu> wrote: > Today at 7:43am, Tim wrote: > >> I realise that the RID has to be unique - so am I right in thinking that >> there is no way for the failover to seamless, and replication must be >> re-established by removing the existing syncrepl statement, on the >> consumers, and re-adding it with the secondaries details? >> >> Or am I missing a trick here as to how the process could be smoother? > > You are missing the point. > > You say you have a load balancer that decides which of your MMR partners > should receive the traffic. Your consumers should be configured in such > a way that both MMR partners will accept the connection. > > For example... > > My MMR partners are ldap7p and ldap7q. The load balanced VIP is ldap7rw. > My consumers are set up with unique RIDs (the RID is unique for each > consumer) and their provider is ldap7rw. It does not matter whether the > load balancer sends the traffic to ldap7p or ldap7q - the consumer is > talking to ldap7rw. Consumers can also be configured with > 1 syncrepl stanza, and simply talk to both masters at the same time. There is no need for them to be reconfigured at all. --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

Re: Antw: Re: Removing Berkeley DB Log Files
by Quanah Gibson-Mount 08 Sep '17

08 Sep '17

--On Friday, September 08, 2017 9:55 AM +0200 Ulrich Windl <Ulrich.Windl(a)rz.uni-regensburg.de> wrote: > I think that's a good example why an ID should be different from the > user's common (sur)name. If you have jdoe as memeber of some groups, > youll have some additional fun unless LDAP is smart enough to adjust > those... Sure, it was purely an example. There are plenty of cases where subtree renames may be required. --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

Failing consumers over to second mirror master provider
by Tim 08 Sep '17

08 Sep '17

Hey, I have, what I hope is, a vanilla mirror-master setup, There is a load balancer in between the providers and consumers, that ensures that the writes will consistently be written to a single provider, until it becomes unavailable - at which point writes will failover to the hot-stand by. With regards to replication for the consumers - currently they all have a single provider explicitly defined within their configuration and I'm trying to understand what options are available with which the failover from one provider to the other would best be handled if the master was to die? I realise that the RID has to be unique - so am I right in thinking that there is no way for the failover to seamless, and replication must be re-established by removing the existing syncrepl statement, on the consumers, and re-adding it with the secondaries details? Or am I missing a trick here as to how the process could be smoother? Thanks in advance -- Tim tim(a)yetanother.net

2 1

Re: Antw: syncrepl error (53) with 3-way delta-mmr (consumer state is newer than provider)
by Sven Mäder 08 Sep '17

08 Sep '17

On 09/04/2017 08:51 AM, Ulrich Windl wrote: >>>> Sven Mäder <maeder(a)phys.ethz.ch> schrieb am 01.09.2017 um 16:53 in > Nachricht > <2832441b-362f-9429-8c56-32995cff5ef1(a)phys.ethz.ch>: >> Hi Ulrich >> >> Thank you for your response. >> >> On 08/31/2017 09:37 AM, Ulrich Windl wrote: >>> Hi! >>> >>> Some of the time ntpd needs to sync may be host name resolution (if you > use >>> names). Methods to speed up initial synchronization inlude "iburst", >> "minpoll" >>> and adding a large crowd of servers. Note that reducing minpoll could > reduce >>> the final accuracy (just as increasing "maxpoll" does). Depending on your >>> network and load I would not rely on a time offset less than a few ten >>> milliseconds. How well LDAP can operate then is a different question. >> We have 2 timeservers (stratum 1) in our local net with gps clock source: >> server time1.phys.ethz.ch minpoll 4 maxpoll 10 iburst >> server time2.phys.ethz.ch minpoll 4 maxpoll 10 iburst >> >> minpoll is already set at its lowest value, although I do not understand >> what this option does. I may increase its value, increasing accuracy sounds >> good. > The manual explains what "minpoll" does. Basically it is the starting poll > interval. Short values cause adjustment to be made faster than nomal. However > if subsequent polls fail, the clock could continue to run with wrong correction > values and the accumulate some time error (i.e. make things worse). A big > maxpoll might not correct he clock in time. And you probably should have an odd > number of time servers. > Reading the correct manuals helps, looks like the debian manpages are outdated (ntp version 1:4.2.8p10+dfsg-3) compared to the manpages of the upstream source tarball (ntp-4.2.8p10.tar.gz) and to the manuals on http://doc.ntp.org/current-stable/. Even the upstream source tarball is not in line with the manuals on the website.According the manual on http://doc.ntp.org/current-stable/confopt.html it is now possible to use a minimal poll interval of 3. Thanks to your inputs I started playing around with minpoll and maxpoll. Looks like on my setup a maxpoll of 10 is too high, as the accumulated time errorstarts to increase notably from poll interval >6. See attached graph (ntp_graph_phd-aa_20170908.png) or https://people.phys.ethz.ch/~rda/img/ntp_graph_phd-aa_20170908.png I also noticed, that a higher rootdelay or the variation of the delay has a big impact on the time offset. Under normal conditions we have a delay of 0.3 - 0.4 ms in our local network (offset: ~0.050 ms). But the servers which are in the same subnet as our ntp servers sometimes have to make ARP requests if the poll interval is higher. The ntp server has to do the same. This results in a root delay of ~1.5ms, which I think confuses ntp which in turn results in offsets of 0.500-1.500 ms. See attached graph (ntp_graph_phd-debug-aa_20170908.png) or https://people.phys.ethz.ch/~rda/img/ntp_graph_phd-debug-aa_20170908.png >> # ntptrace >> localhost: stratum 2, offset 0.000008, synch distance 0.031825 >> time1.ethz.ch: stratum 1, offset -0.000001, synch distance 0.000298, >> refid 'PPS' >> >> # ntpq -c pe >> remote refid st t when poll reach delay offset >> jitter >> > ============================================================================ >> == >> LOCAL(0) .LOCL. 5 l 149m 64 0 0.000 0.000 >> 0.000 >> *time1.ethz.ch .PPS. 1 u 251 256 377 0.426 0.009 >> 0.026 >> +time2.ethz.ch .PPS. 1 u 64 256 377 0.430 -0.003 >> 0.019 >> >> Looks like the time offset is only a few microseconds once ntp is in sync. > Yes, looks good. > >>> Also note for Linux (on most platforms) and NTP one problem is that the >>> frequency correction needed for the clock can vary significantly between >> boots; >>> thus the tijme for "perfect sync" can be quite long. See attached image for >> an >>> example. >> This is very interesting, we will look further into this. >> I am thinking about waiting in the startup process until ntp is in >> "perfect sync"and start slapd after that. Maybe I can use the loopstats >> file to check/automate this. > Perfect sync can take quite some time (due to design of NTP) [From RFC 1305: > "This yields a PLL risetime of > about 52 minutes, a maximum overshoot of about 4.8 percent in about 1.7 hours > and a settling time > to within one percent of the initial offset in about 8.7 hours."] > The good thing is that NTP itself provides performance numbers to check. Maybe > start with clk_wander being low, rootdisp being low, and offset being low. > There are some pathological cases where the numbers are low while the clock in > in a bad condition. > Thanks, monitoring these numbers using graphs helps me a lot.I am looking forward to the results of some long term tests with different settings, to see what gives the most stable and accurate time for our ldap servers. >>> Updating one entry on different servers within a very short time (shorter >> than >>> the time of syncing) will probably cause trouble. What real-life situation >>> causes such? >> Probably none. But we have a logparser, which writes "last use" statistics >> of our users to ldap, this is done in "realtime". We also use openldap as >> kerberos KDC database backend, which writes on successful/failed >> authentication attempts. The chance is probably very low if the time offset >> is lower than the network delay. > Maybe you could centralize the logparser to run on only one host at a time... > > Regards, > Ulrich > > Kind regards Sven

1 0

Re: Antw: Re: Removing Berkeley DB Log Files
by Quanah Gibson-Mount 07 Sep '17

07 Sep '17

--On Thursday, September 07, 2017 12:03 PM -0400 Douglas Duckworth <dod2014(a)med.cornell.edu> wrote: > > Why would you want to rename a subtree? I can only think of a scenario > if for example we would have to sync with another LDAP cluster that > perhaps had ou=Accounts whereas we have ou=People. A subtree is simply any DN that has children. So if you, for example, had: dn: uid=jdoe, cn=people, dc=example, dc=com dn: signature=work, uid=jdoe, cn=people, dc=example, dc=com And jdoe got divorced and changed their last name, and wanted their uid updated, you could do a subtree rename to fix her entry and all children entries at the same time. Otherwise, you'd have to either delete the existing entries and create new ones, or create new ones and delete the old ones, etc. Or stop everything, do an export, fix the data, reimport. --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

2 1

Re: Antw: Re: Removing Berkeley DB Log Files
by Douglas Duckworth 07 Sep '17

07 Sep '17

Ah, this recursive change would be extremely useful for us given that example. Thanks for pushing me to abandon bdb. I'll do that once the old cluster's no longer in use. On Sep 7, 2017 11:15 AM, "Quanah Gibson-Mount" <quanah(a)symas.com> wrote: > --On Thursday, September 07, 2017 12:03 PM -0400 Douglas Duckworth > <dod2014(a)med.cornell.edu> wrote: > > > > > Why would you want to rename a subtree? I can only think of a scenario > > if for example we would have to sync with another LDAP cluster that > > perhaps had ou=Accounts whereas we have ou=People. > > A subtree is simply any DN that has children. > > So if you, for example, had: > > dn: uid=jdoe, cn=people, dc=example, dc=com > dn: signature=work, uid=jdoe, cn=people, dc=example, dc=com > > And jdoe got divorced and changed their last name, and wanted their uid > updated, you could do a subtree rename to fix her entry and all children > entries at the same time. Otherwise, you'd have to either delete the > existing entries and create new ones, or create new ones and delete the old > ones, etc. Or stop everything, do an export, fix the data, reimport. > > --Quanah > > -- > > Quanah Gibson-Mount > Product Architect > Symas Corporation > Packaged, certified, and supported LDAP solutions powered by OpenLDAP: > <https://urldefense.proofpoint.com/v2/url?u=http- > 3A__www.symas.com&d=DwIFaQ&c=lb62iw4YL4RFalcE2hQUQealT9- > RXrryqt9KZX2qu2s&r=2Fzhh_78OGspKQpl_e-CbhH6xUjnRkaqPFUS2wTJ2cw&m= > eaHUs775FiDziz6vOPByKLyL8czWp4lI31AYF8-i3C4&s= > oOnY5I8rAHwXosqpofTs9gxnDk9jYzqhI4LgtQXzKeY&e=> > >

1 0

Re: Antw: Re: Removing Berkeley DB Log Files
by Douglas Duckworth 07 Sep '17

07 Sep '17

Why would you want to rename a subtree? I can only think of a scenario if for example we would have to sync with another LDAP cluster that perhaps had ou=Accounts whereas we have ou=People. Thanks, Douglas Duckworth, MSc, LFCS HPC System Administrator Scientific Computing Unit Physiology and Biophysics Weill Cornell Medicine E: doug(a)med.cornell.edu O: 212-746-6305 F: 212-746-8690 On Thu, Sep 7, 2017 at 11:00 AM, Quanah Gibson-Mount <quanah(a)symas.com> wrote: > --On Thursday, September 07, 2017 8:57 AM -0700 Quanah Gibson-Mount > <quanah(a)symas.com> wrote: > > > Yes. LDAP is a protocol. It doesn't care what the underlying storage DB > > is. So you could have a back-hdb master and a back-mdb consumer, etc. > > Although I would note, don't do a subtree rename until all backends in use > support it. ;) > > --Quanah > > > -- > > Quanah Gibson-Mount > Product Architect > Symas Corporation > Packaged, certified, and supported LDAP solutions powered by OpenLDAP: > <https://urldefense.proofpoint.com/v2/url?u=http- > 3A__www.symas.com&d=DwICAg&c=lb62iw4YL4RFalcE2hQUQealT9- > RXrryqt9KZX2qu2s&r=2Fzhh_78OGspKQpl_e-CbhH6xUjnRkaqPFUS2wTJ2cw&m= > VP9PaF9h0TG3hKW5JyTRk11nKzg5E7oJrVOiTo8H5ss&s=zs3UlQjg5PMR-7ut_ > HMW98Jj3be5u1Ji1HtHeSxSm4I&e=> > >

1 0

Re: Antw: Re: Removing Berkeley DB Log Files
by Quanah Gibson-Mount 07 Sep '17

07 Sep '17

--On Thursday, September 07, 2017 8:57 AM -0700 Quanah Gibson-Mount <quanah(a)symas.com> wrote: > Yes. LDAP is a protocol. It doesn't care what the underlying storage DB > is. So you could have a back-hdb master and a back-mdb consumer, etc. Although I would note, don't do a subtree rename until all backends in use support it. ;) --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

Re: Antw: Re: Removing Berkeley DB Log Files
by Quanah Gibson-Mount 07 Sep '17

07 Sep '17

--On Thursday, September 07, 2017 11:19 AM -0400 Douglas Duckworth <dod2014(a)med.cornell.edu> wrote: > > We have one provider and one consumer running syncrepl. All of our > important clients are using sssd, configured to cache everything, with > the consumer set as secondary ldap server. I have verified that > everything works properly in sssd if the provider goes offline. > > Can I flip the provider to hdb, or mdb, and keep consumer as bdb, with > syncrepl still running, then later change the consumer? Yes. LDAP is a protocol. It doesn't care what the underlying storage DB is. So you could have a back-hdb master and a back-mdb consumer, etc. --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

1 0

Re: Antw: Re: Removing Berkeley DB Log Files
by Douglas Duckworth 07 Sep '17

07 Sep '17

We have one provider and one consumer running syncrepl. All of our important clients are using sssd, configured to cache everything, with the consumer set as secondary ldap server. I have verified that everything works properly in sssd if the provider goes offline. Can I flip the provider to hdb, or mdb, and keep consumer as bdb, with syncrepl still running, then later change the consumer? Thanks, Douglas Duckworth, MSc, LFCS HPC System Administrator Scientific Computing Unit Physiology and Biophysics Weill Cornell Medicine E: doug(a)med.cornell.edu O: 212-746-6305 F: 212-746-8690 On Thu, Sep 7, 2017 at 10:03 AM, Quanah Gibson-Mount <quanah(a)symas.com> wrote: > --On Thursday, September 07, 2017 8:45 AM -0400 Douglas Duckworth > <dod2014(a)med.cornell.edu> wrote: > > > > > Can't one dump the database to LDIF then make changes? That's probably > > not going to work for Amazon or Facebook but we only have 4000 entries in > > our database. > > That would be the only way to do a subtree rename with back-bdb, yes. > Again, there's no reason to use back-bdb. > > back-bdb: Introduced in OpenLDAP 2.1 to replace back-ldbm > back-hdb: Introduced in OpenLDAP 2.2 to replace back-bdb > back-mdb: Introduced in OpenLDAP 2.4 to replace both back-bdb and back-hdb > > If you're using back-bdb, at least move to back-hdb. > > --Quanah > > > -- > > Quanah Gibson-Mount > Product Architect > Symas Corporation > Packaged, certified, and supported LDAP solutions powered by OpenLDAP: > <https://urldefense.proofpoint.com/v2/url?u=http- > 3A__www.symas.com&d=DwIFaQ&c=lb62iw4YL4RFalcE2hQUQealT9- > RXrryqt9KZX2qu2s&r=2Fzhh_78OGspKQpl_e-CbhH6xUjnRkaqPFUS2wTJ2cw&m= > T4QUYvunGpzAlWdSQVnXzovTYA5PPC2r_droZTewjpQ&s=OtEn0Ve3-TwnPsoq_Ujr9NKy_ > rbTo5XhJTn8hxcsT1Y&e=> > >

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

openldap-technical