On 27Jun16 11:32+0000, Gurjot Kaur wrote: Do you use slapadd while slapd is running? IIRC, slapadd is just for one thing: restore backup. For that reason, the DB is offline (slapd not running). After successful restore, I start slapd and syncrepl works as expected. If you use slapadd on an online DB, I could imagine that the syncrepl-overlay does not realize that there is a new entry and thus it does not replicate it. For daily-business modifications to the LDAP DB I always use ldap* commands. Cheers, -- Bastian Tweddell Juelich Supercomputing Centre phone: +49 (2461) 61-6586 HPC in Neuroscience

On 27Jun16 13:00+0000, Gurjot Kaur wrote: Stopping the slapd where you run slapadd on is enough. Sorry, I can't. But I may give you some hints: - man slapadd mentions the options -S and -w which might be of interest to you. - After slapadd, run both servers with -d -1 and read the excessive debug information. These always helped me a lot. Cheers, -- Bastian Tweddell Juelich Supercomputing Centre phone: +49 (2461) 61-6586 HPC in Neuroscience

Quanah Gibson-Mount wrote: If you use slapadd -w, the consumer should pick that up. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

Quanah Gibson-Mount wrote: Indeed, if you're using slapadd to load entries that already have entryCSNs this won't work. And you obviously cannot use slapadd to add already-existing entries. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

--On Wednesday, June 29, 2016 4:21 PM +0200 Bastian Tweddell <b.tweddell(a)fz-juelich.de&gt; wrote: You should just slapadd A & B at the same time. It's the most efficient way to load an OpenLDAP Server. --Quanah -- Quanah Gibson-Mount Platform Architect Manager, Systems Team Zimbra, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration A division of Synacor, Inc

I checked the debugging logs. Using this command: ldapsearch -H ldap://xx.xx.xx.xx:2016 -LLL -x -s base -b "my-domain,dc=com" contextCSN I found that contextCSN of the Server (Master 1) is coming as below: contextCSN: 20160704065252.267837Z#000000#001#000000 and that of replica (Master 2) is: contextCSN: 20160704065252.267837Z#000000#001#000000 I have some entries in the Server (Master 1) with entryCSN values as given below: 1. entryCSN: 20160704061459.015144Z#000000#000#000000 2. entryCSN: 20160704061459.041359Z#000000#000#000000 3. entryCSN: 20160704061459.052873Z#000000#000#000000 I added these entries using slapadd on Server (Master 1) These entries don't get replicated on Replica (Master 2) , though contextCSN value is greater than these three entryCSN values. I have tries starting Replica (Master 2) slapd with option -c "rid" But it still doesn't get these entries. Please help in understanding how can I get these entries replicated on my Replica (Master 2) from Server (Master 1). Kind Regards, Gurjot Kaur -----Original Message----- From: Bastian Tweddell [mailto:b.tweddell@fz-juelich.de] Sent: Friday, July 01, 2016 6:34 PM To: Quanah Gibson-Mount <quanah(a)zimbra.com&gt; Cc: Gurjot Kaur <gurjot.kaur(a)aricent.com&gt;; openldap-technical(a)openldap.org Subject: Re: MDB data replication issues On 29Jun16 10:04-0700, Quanah Gibson-Mount wrote: I would like to elaborate a bit more on that. --- paste: 18.1.1.2. Syncrepl Details --- [...] Note that at startup time, if the provider is unable to read a contextCSN from the suffix entry, it will scan the entire database to determine the value, and this scan may take quite a long time on a large database. When a contextCSN value is read, the database will still be scanned for any entryCSN values greater than it, to make sure the contextCSN value truly reflects the greatest committed entryCSN in the database. [...] --- eop --- 1: http://www.openldap.org/doc/admin24/replication.html#LDAP%20Sync%20Replic... A bit more detail about the disaster recovery method I (plan to) use: - recreate two slapds A and B (mirrormode) - the slapdump file of my data base contains contextCSN, entryCSN attributes for all entries - slapadd is used without -w to db of slapd A - Starting slapd A - After that, slapd B is started which has a complete empty DB, even without an entry for the suffix - A complete synchronization takes place from A to B After reading the manual again, when B comes up it does not have any idea in which state the provider engine is, so it has nothing to do. The consumer facility then fetches all entries from the provider in A. Further from the manual [2]: --- paste: 18.3.1.4. Start the provider and the consumer slapd --- [...] contextCSN is automatically generated as needed: it might be originally contained in the LDIF file, generated by slapadd (8), generated upon changes in the context, or generated when the first LDAP Sync search arrives at the provider [...] --- eop --- 2: http://www.openldap.org/doc/admin24/replication.html#Syncrepl This part lets me believe, that even if I do not have contextCSN in my db in A these would be created on the first sync requested from B. - What is the designed behavior for the consumer engine if the db is completely empty? - How to trigger a refresh on the consumer? I have never done that, but I can imagine, that is done by slapd -c? About the question from Gurjot, where he misses entries on the replicated side. What about, after he slapadds his entries to the db, remove the contextCSN entry of his suffix. When slapd starts up, the syncrepl engine would scan the entire DB. In any case, my assumptions rely on the admin manual which might not reflect the code base. But from what I read, I still feel I did it the right way all by now. So Quanah, if you find some time, please let me know what I am missing and why my disaster recovery is unsafe. Cheers, -- Bastian Tweddell Juelich Supercomputing Centre phone: +49 (2461) 61-6586 HPC in Neuroscience "DISCLAIMER: This message is proprietary to Aricent and is intended solely for the use of the individual to whom it is addressed. It may contain privileged or confidential information and should not be circulated or used for any purpose other than for what it is intended. If you have received this message in error, please notify the originator immediately. If you are not the intended recipient, you are notified that you are strictly prohibited from using, copying, altering, or disclosing the contents of this message. Aricent accepts no responsibility for loss or damage arising from the use of the information transmitted by this email including damage from virus."

ContextCSN values for both the servers are coming the same. ldapsearch -H ldap:// xx.xx.xx.xx:2016 -LLL -x -s base -b "dc=my-domain,dc=com" contextCSN dn: dc=my-domain,dc=com contextCSN: 20160704065252.267837Z#000000#001#000000 ldapsearch -H ldap:// xx.xx.xx.xx:2017 -LLL -x -s base -b "dc=my-domain,dc=com" contextCSN dn: dc=my-domain,dc=com contextCSN: 20160704065252.267837Z#000000#001#000000 And Yes...server ids are different for both the servers. See below the syncrepl configuration. ########################################## Server (Master 1) ########################################## serverId 1 syncRepl rid=001 provider=ldap://xx.xx.xx.xx:2017 type=refreshAndPersist retry="5 + 5 +" searchbase="dc=my-domain,dc=com " attrs=* interval=00:00:00:09 schemachecking=off bindmethod=simple binddn="cn=Manager, dc=my-domain,dc=com " credentials=secret index entryCSN eq index entryUUID eq index cn eq mirrormode yes ########################################## Replica (Master 2) ########################################## serverId 2 syncRepl rid=001 provider=ldap://xx.xx.xx.xx:2016 type=refreshAndPersist retry="5 + 5 +" searchbase="dc=my-domain,dc=com " attrs=* interval=00:00:00:09 schemachecking=off bindmethod=simple binddn="cn=Manager, dc=my-domain,dc=com " credentials=secret index entryCSN eq index entryUUID eq index cn eq mirrormode yes ######################################################### Thanks, Gurjot Kaur -----Original Message----- From: Bastian Tweddell [mailto:b.tweddell@fz-juelich.de] Sent: Monday, July 04, 2016 7:54 PM To: Gurjot Kaur <gurjot.kaur(a)aricent.com&gt; Cc: Quanah Gibson-Mount <quanah(a)zimbra.com&gt;; openldap-technical(a)openldap.org Subject: Re: MDB data replication issues On 04Jul16 12:39+0000, Gurjot Kaur wrote: I do not know much about your setup, but I assume that your both servers should synchronize vice versa, right? Do you have enabled mirrormode on both sides with distinct ServerIDs? I am asking, because you just pasted one contextCSN and IIRC the nummer at "..#001#.." is the indicator for the server ID. In my setup I have a contextCSN attribute for each server: --- paste: contextCSN: 20160704100046.737857Z#000000#002#000000 contextCSN: 20160704142023.147502Z#000000#001#000000 --- eop -- Bastian Tweddell Juelich Supercomputing Centre phone: +49 (2461) 61-6586 HPC in Neuroscience "DISCLAIMER: This message is proprietary to Aricent and is intended solely for the use of the individual to whom it is addressed. It may contain privileged or confidential information and should not be circulated or used for any purpose other than for what it is intended. If you have received this message in error, please notify the originator immediately. If you are not the intended recipient, you are notified that you are strictly prohibited from using, copying, altering, or disclosing the contents of this message. Aricent accepts no responsibility for loss or damage arising from the use of the information transmitted by this email including damage from virus."

Hi! I wonder: Is there something like the "touch" command for files in UNIX available for LDAP entries? A touch on an LDAP entry would force the whole entry to be replcated (again)... Probably slapd would have to provide some support for this as just updating the entryCSN of the corresponding won't work I'm afraid. Regards, Ulrich <alpine.OSX.2.20.1607050800340.1852(a)vc104198.hiz.rqh&gt;:

Today at 5:23am, Hallvard Breien Furuseth wrote: However, since the original was never replicated, having those attributes marked to be updated in the accesslog is useless. The update will be ignored on the replica because the entry to be updated is not present. -- Frank Swasey | http://www.uvm.edu/~fcs Sr Systems Administrator | Always remember: You are UNIQUE, University of Vermont | just like everyone else. "I am not young enough to know everything." - Oscar Wilde (1854-1900)