Hi,
I am using a OpenLDAP 2.4.44 Multi master configuration with two slapd servers, master and replica using MDB backend. I got a problem in replicating when the data is added using slapadd. I have two slapd with ports 2016 and 2017. slapd.conf file for both the servers are attached.
Scenario 1: When an LDIF entry is added using ldapadd or deleted using ldapdelete, it gets replicated in the replica server correctly. Below is the ldapsearch result om Master server: GURKES254 linus> ldapsearch -h xx.xx.xx.xx -p 2016 -b "dc=my-domain,dc=com" "ou=Test9" # extended LDIF # # LDAPv3 # base <dc=my-domain,dc=com> with scope subtree # filter: ou=Test9 # requesting: ALL #
# Test9, people, my-domain.com dn: ou=Test9,ou=people,dc=my-domain,dc=com ou: Test9 objectClass: organizationalUnit companyName: Test9Grp
# search result search: 2 result: 0 Success
# numResponses: 2 # numEntries: 1
Below is the ldapsearch result om replica server: GURKES254 linus> ldapsearch -h xx.xx.xx.xx -p 2017 -b "dc=my-domain,dc=com" "ou=Test9" # extended LDIF # # LDAPv3 # base <dc=my-domain,dc=com> with scope subtree # filter: ou=Test9 # requesting: ALL #
# Test9, people, my-domain.com dn: ou=Test9,ou=people,dc=my-domain,dc=com ou: Test9 objectClass: organizationalUnit companyName: Test9Grp
# search result search: 2 result: 0 Success
# numResponses: 2 # numEntries: 1
Scenario 2: When an LDIF entry is imported using slapadd, it doesn't get replicated in the replica server at all. Below is the ldapsearch result om Master server: GURKES254 linus> ldapsearch -h xx.xx.xx.xx -p 2016 -b "dc=my-domain,dc=com" "ou=Test9" # extended LDIF # # LDAPv3 # base <dc=my-domain,dc=com> with scope subtree # filter: ou=Test9 # requesting: ALL #
# Test9, people, my-domain.com dn: ou=Test9,ou=people,dc=my-domain,dc=com ou: Test9 objectClass: organizationalUnit companyName: Test9Grp
# search result search: 2 result: 0 Success
# numResponses: 2 # numEntries: 1
Below is the ldapsearch result om replica server: GURKES254 linus> ldapsearch -h xx.xx.xx.xx -p 2017 -b "dc=my-domain,dc=com" "ou=Test9" # extended LDIF # # LDAPv3 # base <dc=my-domain,dc=com> with scope subtree # filter: ou=Test9 # requesting: ALL #
# search result search: 2 result: 0 Success
# numResponses: 1
Please let me know in case any other information is required.
Br Gurjot Kaur
"DISCLAIMER: This message is proprietary to Aricent and is intended solely for the use of the individual to whom it is addressed. It may contain privileged or confidential information and should not be circulated or used for any purpose other than for what it is intended. If you have received this message in error, please notify the originator immediately. If you are not the intended recipient, you are notified that you are strictly prohibited from using, copying, altering, or disclosing the contents of this message. Aricent accepts no responsibility for loss or damage arising from the use of the information transmitted by this email including damage from virus."
On 27Jun16 11:32+0000, Gurjot Kaur wrote:
When an LDIF entry is imported using slapadd, it doesn't get replicated in the replica server at all. Below is the ldapsearch result om Master server:
Do you use slapadd while slapd is running?
IIRC, slapadd is just for one thing: restore backup. For that reason, the DB is offline (slapd not running). After successful restore, I start slapd and syncrepl works as expected.
If you use slapadd on an online DB, I could imagine that the syncrepl-overlay does not realize that there is a new entry and thus it does not replicate it.
For daily-business modifications to the LDAP DB I always use ldap* commands.
Cheers,
Thanks Bastian for your response.
But I want to clear that I don't use slapadd while slapd is running. I have LDIF data which needs to be imported in the new Multimaster servers. That's why I used slapadd. When I did the import (slapadd) with the big data, it doesn't get replicated. To check the problems I did the import with single entry only.
First I stop both the servers, do slapadd on Master server and then I start the Master server & replica server. After doing all this too, the data is not getting replicated.
I did the same things with BDB backend in openLDAP 2.4.40. It works just fine. Can you please let me know what I've been doing wrong.
Br Gurjot Kaur
-----Original Message----- From: Bastian Tweddell [mailto:b.tweddell@fz-juelich.de] Sent: Monday, June 27, 2016 6:22 PM To: Gurjot Kaur gurjot.kaur@aricent.com Cc: openldap-technical@openldap.org Subject: Re: MDB data replication issues
On 27Jun16 11:32+0000, Gurjot Kaur wrote:
When an LDIF entry is imported using slapadd, it doesn't get replicated in the replica server at all. Below is the ldapsearch result om Master server:
Do you use slapadd while slapd is running?
IIRC, slapadd is just for one thing: restore backup. For that reason, the DB is offline (slapd not running). After successful restore, I start slapd and syncrepl works as expected.
If you use slapadd on an online DB, I could imagine that the syncrepl-overlay does not realize that there is a new entry and thus it does not replicate it.
For daily-business modifications to the LDAP DB I always use ldap* commands.
Cheers, -- Bastian Tweddell Juelich Supercomputing Centre phone: +49 (2461) 61-6586 HPC in Neuroscience "DISCLAIMER: This message is proprietary to Aricent and is intended solely for the use of the individual to whom it is addressed. It may contain privileged or confidential information and should not be circulated or used for any purpose other than for what it is intended. If you have received this message in error, please notify the originator immediately. If you are not the intended recipient, you are notified that you are strictly prohibited from using, copying, altering, or disclosing the contents of this message. Aricent accepts no responsibility for loss or damage arising from the use of the information transmitted by this email including damage from virus."
On 27Jun16 13:00+0000, Gurjot Kaur wrote:
But I want to clear that I don't use slapadd while slapd is running. I have LDIF data which needs to be imported in the new Multimaster servers. That's why I used slapadd. When I did the import (slapadd) with the big data, it doesn't get replicated. To check the problems I did the import with single entry only.
First I stop both the servers, do slapadd on Master server and then I start the Master server & replica server. After doing all this too, the data is not getting replicated.
Stopping the slapd where you run slapadd on is enough.
I did the same things with BDB backend in openLDAP 2.4.40. It works just fine. Can you please let me know what I've been doing wrong.
Sorry, I can't. But I may give you some hints:
- man slapadd mentions the options -S and -w which might be of interest to you.
- After slapadd, run both servers with -d -1 and read the excessive debug information. These always helped me a lot.
Cheers,
--On Monday, June 27, 2016 2:00 PM +0000 Gurjot Kaur gurjot.kaur@aricent.com wrote:
Thanks Bastian for your response.
But I want to clear that I don't use slapadd while slapd is running. I have LDIF data which needs to be imported in the new Multimaster servers. That's why I used slapadd. When I did the import (slapadd) with the big data, it doesn't get replicated. To check the problems I did the import with single entry only.
It is not now, nor has it ever been, supported to do an offline add via slapadd on a master, and then magically expect that to replicate out to a consumer via syncrepl. Syncrepl is an active protocol (it replicates the changes that occur while slapd is running). If you are going to offline modify slapd on the master, you will have to do the same thing on the replicas, OR reload them via slapadd, OR trigger a refresh manually on the replicas.
--Quanah
--
Quanah Gibson-Mount Platform Architect Manager, Systems Team Zimbra, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration A division of Synacor, Inc
Quanah Gibson-Mount wrote:
--On Monday, June 27, 2016 2:00 PM +0000 Gurjot Kaur gurjot.kaur@aricent.com wrote:
Thanks Bastian for your response.
But I want to clear that I don't use slapadd while slapd is running. I have LDIF data which needs to be imported in the new Multimaster servers. That's why I used slapadd. When I did the import (slapadd) with the big data, it doesn't get replicated. To check the problems I did the import with single entry only.
It is not now, nor has it ever been, supported to do an offline add via slapadd on a master, and then magically expect that to replicate out to a consumer via syncrepl. Syncrepl is an active protocol (it replicates the changes that occur while slapd is running). If you are going to offline modify slapd on the master, you will have to do the same thing on the replicas, OR reload them via slapadd, OR trigger a refresh manually on the replicas.
If you use slapadd -w, the consumer should pick that up.
--On Monday, June 27, 2016 6:21 PM +0100 Howard Chu hyc@symas.com wrote:
It is not now, nor has it ever been, supported to do an offline add via slapadd on a master, and then magically expect that to replicate out to a consumer via syncrepl. Syncrepl is an active protocol (it replicates the changes that occur while slapd is running). If you are going to offline modify slapd on the master, you will have to do the same thing on the replicas, OR reload them via slapadd, OR trigger a refresh manually on the replicas.
If you use slapadd -w, the consumer should pick that up.
That would depend on the data. If old entries are being added back in, that already have entryCSN values older than the existing cookies, they still wouldn't get picked up. If it is new entries only, then it could happen. ;)
--Quanah
--
Quanah Gibson-Mount Platform Architect Manager, Systems Team Zimbra, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration A division of Synacor, Inc
Quanah Gibson-Mount wrote:
--On Monday, June 27, 2016 6:21 PM +0100 Howard Chu hyc@symas.com wrote:
If you use slapadd -w, the consumer should pick that up.
That would depend on the data. If old entries are being added back in, that already have entryCSN values older than the existing cookies, they still wouldn't get picked up. If it is new entries only, then it could happen. ;)
Indeed, if you're using slapadd to load entries that already have entryCSNs this won't work. And you obviously cannot use slapadd to add already-existing entries.
On 27Jun16 09:16-0700, Quanah Gibson-Mount wrote:
--On Monday, June 27, 2016 2:00 PM +0000 Gurjot Kaur gurjot.kaur@aricent.com wrote:
Thanks Bastian for your response.
But I want to clear that I don't use slapadd while slapd is running. I have LDIF data which needs to be imported in the new Multimaster servers. That's why I used slapadd. When I did the import (slapadd) with the big data, it doesn't get replicated. To check the problems I did the import with single entry only.
It is not now, nor has it ever been, supported to do an offline add via slapadd on a master, and then magically expect that to replicate out to a consumer via syncrepl. Syncrepl is an active protocol (it replicates the changes that occur while slapd is running). If you are going to offline modify slapd on the master, you will have to do the same thing on the replicas, OR reload them via slapadd, OR trigger a refresh manually on the replicas.
How to trigger a refresh on a replica?
One question:
My disaster recovery (and sandbox initialization) is using slapadd to bring up a fresh instance of my ldap environment. Technically:
- bring up n replicated slapds with an empty db - stop them - slapadd the db dump into A - start A - start B..
That always worked. By now, I have to say. I even use slapadd without -w.
Do I understand correctly, that I should prefer the use of ldapadd? Does ldapadd also support operational and user attributes?
Many thanks,
--On Wednesday, June 29, 2016 4:21 PM +0200 Bastian Tweddell b.tweddell@fz-juelich.de wrote:
On 27Jun16 09:16-0700, Quanah Gibson-Mount wrote:
--On Monday, June 27, 2016 2:00 PM +0000 Gurjot Kaur gurjot.kaur@aricent.com wrote:
Thanks Bastian for your response.
But I want to clear that I don't use slapadd while slapd is running. I have LDIF data which needs to be imported in the new Multimaster servers. That's why I used slapadd. When I did the import (slapadd) with the big data, it doesn't get replicated. To check the problems I did the import with single entry only.
It is not now, nor has it ever been, supported to do an offline add via slapadd on a master, and then magically expect that to replicate out to a consumer via syncrepl. Syncrepl is an active protocol (it replicates the changes that occur while slapd is running). If you are going to offline modify slapd on the master, you will have to do the same thing on the replicas, OR reload them via slapadd, OR trigger a refresh manually on the replicas.
How to trigger a refresh on a replica?
One question:
My disaster recovery (and sandbox initialization) is using slapadd to bring up a fresh instance of my ldap environment. Technically:
- bring up n replicated slapds with an empty db - stop them - slapadd the db dump into A - start A - start B..That always worked. By now, I have to say. I even use slapadd without -w.
Do I understand correctly, that I should prefer the use of ldapadd? Does ldapadd also support operational and user attributes?
You should just slapadd A & B at the same time. It's the most efficient way to load an OpenLDAP Server.
--Quanah
--
Quanah Gibson-Mount Platform Architect Manager, Systems Team Zimbra, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration A division of Synacor, Inc
On 29Jun16 10:04-0700, Quanah Gibson-Mount wrote:
--On Wednesday, June 29, 2016 4:21 PM +0200 Bastian Tweddell b.tweddell@fz-juelich.de wrote:
On 27Jun16 09:16-0700, Quanah Gibson-Mount wrote:
--On Monday, June 27, 2016 2:00 PM +0000 Gurjot Kaur gurjot.kaur@aricent.com wrote:
Thanks Bastian for your response.
But I want to clear that I don't use slapadd while slapd is running. I have LDIF data which needs to be imported in the new Multimaster servers. That's why I used slapadd. When I did the import (slapadd) with the big data, it doesn't get replicated. To check the problems I did the import with single entry only.
It is not now, nor has it ever been, supported to do an offline add via slapadd on a master, and then magically expect that to replicate out to a consumer via syncrepl. Syncrepl is an active protocol (it replicates the changes that occur while slapd is running). If you are going to offline modify slapd on the master, you will have to do the same thing on the replicas, OR reload them via slapadd, OR trigger a refresh manually on the replicas.
How to trigger a refresh on a replica?
One question:
My disaster recovery (and sandbox initialization) is using slapadd to bring up a fresh instance of my ldap environment. Technically:
- bring up n replicated slapds with an empty db - stop them - slapadd the db dump into A - start A - start B..That always worked. By now, I have to say. I even use slapadd without -w.
Do I understand correctly, that I should prefer the use of ldapadd? Does ldapadd also support operational and user attributes?
You should just slapadd A & B at the same time. It's the most efficient way to load an OpenLDAP Server.
I would like to elaborate a bit more on that.
From the manual [1]: --- paste: 18.1.1.2. Syncrepl Details --- [...] Note that at startup time, if the provider is unable to read a contextCSN from the suffix entry, it will scan the entire database to determine the value, and this scan may take quite a long time on a large database. When a contextCSN value is read, the database will still be scanned for any entryCSN values greater than it, to make sure the contextCSN value truly reflects the greatest committed entryCSN in the database. [...] --- eop ---
1: http://www.openldap.org/doc/admin24/replication.html#LDAP%20Sync%20Replicati...
A bit more detail about the disaster recovery method I (plan to) use: - recreate two slapds A and B (mirrormode) - the slapdump file of my data base contains contextCSN, entryCSN attributes for all entries - slapadd is used without -w to db of slapd A - Starting slapd A - After that, slapd B is started which has a complete empty DB, even without an entry for the suffix - A complete synchronization takes place from A to B
After reading the manual again, when B comes up it does not have any idea in which state the provider engine is, so it has nothing to do. The consumer facility then fetches all entries from the provider in A.
Further from the manual [2]: --- paste: 18.3.1.4. Start the provider and the consumer slapd --- [...] contextCSN is automatically generated as needed: it might be originally contained in the LDIF file, generated by slapadd (8), generated upon changes in the context, or generated when the first LDAP Sync search arrives at the provider [...] --- eop ---
2: http://www.openldap.org/doc/admin24/replication.html#Syncrepl
This part lets me believe, that even if I do not have contextCSN in my db in A these would be created on the first sync requested from B.
- What is the designed behavior for the consumer engine if the db is completely empty?
- How to trigger a refresh on the consumer? I have never done that, but I can imagine, that is done by slapd -c?
About the question from Gurjot, where he misses entries on the replicated side. What about, after he slapadds his entries to the db, remove the contextCSN entry of his suffix. When slapd starts up, the syncrepl engine would scan the entire DB.
In any case, my assumptions rely on the admin manual which might not reflect the code base. But from what I read, I still feel I did it the right way all by now. So Quanah, if you find some time, please let me know what I am missing and why my disaster recovery is unsafe.
Cheers,
I checked the debugging logs. Using this command: ldapsearch -H ldap://xx.xx.xx.xx:2016 -LLL -x -s base -b "my-domain,dc=com" contextCSN
I found that contextCSN of the Server (Master 1) is coming as below: contextCSN: 20160704065252.267837Z#000000#001#000000
and that of replica (Master 2) is: contextCSN: 20160704065252.267837Z#000000#001#000000
I have some entries in the Server (Master 1) with entryCSN values as given below: 1. entryCSN: 20160704061459.015144Z#000000#000#000000 2. entryCSN: 20160704061459.041359Z#000000#000#000000 3. entryCSN: 20160704061459.052873Z#000000#000#000000
I added these entries using slapadd on Server (Master 1) These entries don't get replicated on Replica (Master 2) , though contextCSN value is greater than these three entryCSN values.
I have tries starting Replica (Master 2) slapd with option -c "rid" But it still doesn't get these entries.
Please help in understanding how can I get these entries replicated on my Replica (Master 2) from Server (Master 1).
Kind Regards, Gurjot Kaur
-----Original Message----- From: Bastian Tweddell [mailto:b.tweddell@fz-juelich.de] Sent: Friday, July 01, 2016 6:34 PM To: Quanah Gibson-Mount quanah@zimbra.com Cc: Gurjot Kaur gurjot.kaur@aricent.com; openldap-technical@openldap.org Subject: Re: MDB data replication issues
On 29Jun16 10:04-0700, Quanah Gibson-Mount wrote:
--On Wednesday, June 29, 2016 4:21 PM +0200 Bastian Tweddell b.tweddell@fz-juelich.de wrote:
On 27Jun16 09:16-0700, Quanah Gibson-Mount wrote:
--On Monday, June 27, 2016 2:00 PM +0000 Gurjot Kaur gurjot.kaur@aricent.com wrote:
Thanks Bastian for your response.
But I want to clear that I don't use slapadd while slapd is running. I have LDIF data which needs to be imported in the new Multimaster servers. That's why I used slapadd. When I did the import (slapadd) with the big data, it doesn't get replicated. To check the problems I did the import with single entry only.
It is not now, nor has it ever been, supported to do an offline add via slapadd on a master, and then magically expect that to replicate out to a consumer via syncrepl. Syncrepl is an active protocol (it replicates the changes that occur while slapd is running). If you are going to offline modify slapd on the master, you will have to do the same thing on the replicas, OR reload them via slapadd, OR trigger a refresh manually on the replicas.
How to trigger a refresh on a replica?
One question:
My disaster recovery (and sandbox initialization) is using slapadd to bring up a fresh instance of my ldap environment. Technically:
- bring up n replicated slapds with an empty db - stop them - slapadd the db dump into A - start A - start B..That always worked. By now, I have to say. I even use slapadd without -w.
Do I understand correctly, that I should prefer the use of ldapadd? Does ldapadd also support operational and user attributes?
You should just slapadd A & B at the same time. It's the most efficient way to load an OpenLDAP Server.
I would like to elaborate a bit more on that.
From the manual [1]:
--- paste: 18.1.1.2. Syncrepl Details --- [...] Note that at startup time, if the provider is unable to read a contextCSN from the suffix entry, it will scan the entire database to determine the value, and this scan may take quite a long time on a large database. When a contextCSN value is read, the database will still be scanned for any entryCSN values greater than it, to make sure the contextCSN value truly reflects the greatest committed entryCSN in the database. [...] --- eop ---
1: http://www.openldap.org/doc/admin24/replication.html#LDAP%20Sync%20Replicati...
A bit more detail about the disaster recovery method I (plan to) use: - recreate two slapds A and B (mirrormode) - the slapdump file of my data base contains contextCSN, entryCSN attributes for all entries - slapadd is used without -w to db of slapd A - Starting slapd A - After that, slapd B is started which has a complete empty DB, even without an entry for the suffix - A complete synchronization takes place from A to B
After reading the manual again, when B comes up it does not have any idea in which state the provider engine is, so it has nothing to do. The consumer facility then fetches all entries from the provider in A.
Further from the manual [2]: --- paste: 18.3.1.4. Start the provider and the consumer slapd --- [...] contextCSN is automatically generated as needed: it might be originally contained in the LDIF file, generated by slapadd (8), generated upon changes in the context, or generated when the first LDAP Sync search arrives at the provider [...] --- eop ---
2: http://www.openldap.org/doc/admin24/replication.html#Syncrepl
This part lets me believe, that even if I do not have contextCSN in my db in A these would be created on the first sync requested from B.
- What is the designed behavior for the consumer engine if the db is completely empty?
- How to trigger a refresh on the consumer? I have never done that, but I can imagine, that is done by slapd -c?
About the question from Gurjot, where he misses entries on the replicated side. What about, after he slapadds his entries to the db, remove the contextCSN entry of his suffix. When slapd starts up, the syncrepl engine would scan the entire DB.
In any case, my assumptions rely on the admin manual which might not reflect the code base. But from what I read, I still feel I did it the right way all by now. So Quanah, if you find some time, please let me know what I am missing and why my disaster recovery is unsafe.
Cheers, -- Bastian Tweddell Juelich Supercomputing Centre phone: +49 (2461) 61-6586 HPC in Neuroscience "DISCLAIMER: This message is proprietary to Aricent and is intended solely for the use of the individual to whom it is addressed. It may contain privileged or confidential information and should not be circulated or used for any purpose other than for what it is intended. If you have received this message in error, please notify the originator immediately. If you are not the intended recipient, you are notified that you are strictly prohibited from using, copying, altering, or disclosing the contents of this message. Aricent accepts no responsibility for loss or damage arising from the use of the information transmitted by this email including damage from virus."
On 04Jul16 12:39+0000, Gurjot Kaur wrote:
I checked the debugging logs. Using this command: ldapsearch -H ldap://xx.xx.xx.xx:2016 -LLL -x -s base -b "my-domain,dc=com" contextCSN
I found that contextCSN of the Server (Master 1) is coming as below: contextCSN: 20160704065252.267837Z#000000#001#000000
and that of replica (Master 2) is: contextCSN: 20160704065252.267837Z#000000#001#000000
I do not know much about your setup, but I assume that your both servers should synchronize vice versa, right? Do you have enabled mirrormode on both sides with distinct ServerIDs?
I am asking, because you just pasted one contextCSN and IIRC the nummer at "..#001#.." is the indicator for the server ID. In my setup I have a contextCSN attribute for each server:
--- paste: contextCSN: 20160704100046.737857Z#000000#002#000000 contextCSN: 20160704142023.147502Z#000000#001#000000 --- eop
ContextCSN values for both the servers are coming the same.
ldapsearch -H ldap:// xx.xx.xx.xx:2016 -LLL -x -s base -b "dc=my-domain,dc=com" contextCSN dn: dc=my-domain,dc=com contextCSN: 20160704065252.267837Z#000000#001#000000
ldapsearch -H ldap:// xx.xx.xx.xx:2017 -LLL -x -s base -b "dc=my-domain,dc=com" contextCSN dn: dc=my-domain,dc=com contextCSN: 20160704065252.267837Z#000000#001#000000
And Yes...server ids are different for both the servers. See below the syncrepl configuration. ########################################## Server (Master 1) ########################################## serverId 1
syncRepl rid=001 provider=ldap://xx.xx.xx.xx:2017 type=refreshAndPersist retry="5 + 5 +" searchbase="dc=my-domain,dc=com " attrs=* interval=00:00:00:09 schemachecking=off bindmethod=simple binddn="cn=Manager, dc=my-domain,dc=com " credentials=secret
index entryCSN eq index entryUUID eq index cn eq
mirrormode yes
########################################## Replica (Master 2) ########################################## serverId 2
syncRepl rid=001 provider=ldap://xx.xx.xx.xx:2016 type=refreshAndPersist retry="5 + 5 +" searchbase="dc=my-domain,dc=com " attrs=* interval=00:00:00:09 schemachecking=off bindmethod=simple binddn="cn=Manager, dc=my-domain,dc=com " credentials=secret
index entryCSN eq index entryUUID eq index cn eq
mirrormode yes #########################################################
Thanks, Gurjot Kaur
-----Original Message----- From: Bastian Tweddell [mailto:b.tweddell@fz-juelich.de] Sent: Monday, July 04, 2016 7:54 PM To: Gurjot Kaur gurjot.kaur@aricent.com Cc: Quanah Gibson-Mount quanah@zimbra.com; openldap-technical@openldap.org Subject: Re: MDB data replication issues
On 04Jul16 12:39+0000, Gurjot Kaur wrote:
I checked the debugging logs. Using this command: ldapsearch -H ldap://xx.xx.xx.xx:2016 -LLL -x -s base -b "my-domain,dc=com" contextCSN
I found that contextCSN of the Server (Master 1) is coming as below: contextCSN: 20160704065252.267837Z#000000#001#000000
and that of replica (Master 2) is: contextCSN: 20160704065252.267837Z#000000#001#000000
I do not know much about your setup, but I assume that your both servers should synchronize vice versa, right? Do you have enabled mirrormode on both sides with distinct ServerIDs?
I am asking, because you just pasted one contextCSN and IIRC the nummer at "..#001#.." is the indicator for the server ID. In my setup I have a contextCSN attribute for each server:
--- paste: contextCSN: 20160704100046.737857Z#000000#002#000000 contextCSN: 20160704142023.147502Z#000000#001#000000 --- eop
-- Bastian Tweddell Juelich Supercomputing Centre phone: +49 (2461) 61-6586 HPC in Neuroscience "DISCLAIMER: This message is proprietary to Aricent and is intended solely for the use of the individual to whom it is addressed. It may contain privileged or confidential information and should not be circulated or used for any purpose other than for what it is intended. If you have received this message in error, please notify the originator immediately. If you are not the intended recipient, you are notified that you are strictly prohibited from using, copying, altering, or disclosing the contents of this message. Aricent accepts no responsibility for loss or damage arising from the use of the information transmitted by this email including damage from virus."
On Mon, 4 Jul 2016 at 8:39am, Gurjot Kaur wrote:
I have some entries in the Server (Master 1) with entryCSN values as given below:
- entryCSN: 20160704061459.015144Z#000000#000#000000
- entryCSN: 20160704061459.041359Z#000000#000#000000
- entryCSN: 20160704061459.052873Z#000000#000#000000
I added these entries using slapadd on Server (Master 1) These entries don't get replicated on Replica (Master 2) , though contextCSN value is greater than these three entryCSN values.
Please help in understanding how can I get these entries replicated on my Replica (Master 2) from Server (Master 1).
I assume you shutdown the Server (Master 1) before you added those three entries with slapadd. Thus, you have no entries in your accesslog for syncrepl on Replica (Master 2) to see that it should replicate those three entries.
You have a few ways out of this:
1) shutdown the Replica, erase its database and start it up as an empty consumer
2) shutdown the Replica, slapcat from the Server, erase the Replica's database and slapadd the one from the Server
3) shutdown the Replica, and slapadd those same three entries
4) use ldapdelete to get rid of those three entries on the Server, and then ldapadd them back (so they'll get accesslog entries and be replicated)
You cannot use slapadd for things you want replicated. It does not work.
Hi!
I wonder: Is there something like the "touch" command for files in UNIX available for LDAP entries? A touch on an LDAP entry would force the whole entry to be replcated (again)... Probably slapd would have to provide some support for this as just updating the entryCSN of the corresponding won't work I'm afraid.
Regards, Ulrich
Frank Swasey Frank.Swasey@uvm.edu schrieb am 05.07.2016 um 14:05 in Nachricht
alpine.OSX.2.20.1607050800340.1852@vc104198.hiz.rqh:
On Mon, 4 Jul 2016 at 8:39am, Gurjot Kaur wrote:
I have some entries in the Server (Master 1) with entryCSN values as given
below:
- entryCSN: 20160704061459.015144Z#000000#000#000000
- entryCSN: 20160704061459.041359Z#000000#000#000000
- entryCSN: 20160704061459.052873Z#000000#000#000000
I added these entries using slapadd on Server (Master 1) These entries don't get replicated on Replica (Master 2) , though contextCSN
value is greater than these three entryCSN values.
Please help in understanding how can I get these entries replicated on my
Replica (Master 2) from Server (Master 1).
I assume you shutdown the Server (Master 1) before you added those three entries with slapadd. Thus, you have no entries in your accesslog for syncrepl on Replica (Master 2) to see that it should replicate those three entries.
You have a few ways out of this:
- shutdown the Replica, erase its database and start it up as an empty
consumer
- shutdown the Replica, slapcat from the Server, erase the Replica's
database and slapadd the one from the Server
shutdown the Replica, and slapadd those same three entries
use ldapdelete to get rid of those three entries on the Server, and
then ldapadd them back (so they'll get accesslog entries and be replicated)
You cannot use slapadd for things you want replicated. It does not work.
-- Frank Swasey | http://www.uvm.edu/~fcs Sr Systems Administrator | Always remember: You are UNIQUE, University of Vermont | just like everyone else. "I am not young enough to know everything." - Oscar Wilde (1854-1900)
On 05. juli 2016 15:38, Ulrich Windl wrote:
I wonder: Is there something like the "touch" command for files in UNIX available for LDAP entries? A touch on an LDAP entry would force the whole entry to be replcated (again)... Probably slapd would have to provide some support for this as just updating the entryCSN of the corresponding won't work I'm afraid.
An LDAP Modify operation which doesn't list any attributes, will still update entryCSN, modifyTimestamp and modifiersName. I.e. ldapmodify with dn:, "changetype: modify", and nothing else.
Today at 5:23am, Hallvard Breien Furuseth wrote:
On 05. juli 2016 15:38, Ulrich Windl wrote:
I wonder: Is there something like the "touch" command for files in UNIX available for LDAP entries? A touch on an LDAP entry would force the whole entry to be replcated (again)... Probably slapd would have to provide some support for this as just updating the entryCSN of the corresponding won't work I'm afraid.
An LDAP Modify operation which doesn't list any attributes, will still update entryCSN, modifyTimestamp and modifiersName. I.e. ldapmodify with dn:, "changetype: modify", and nothing else.
However, since the original was never replicated, having those attributes marked to be updated in the accesslog is useless. The update will be ignored on the replica because the entry to be updated is not present.
Can anyone please let me know how can I trigger a refresh manually on the replicas?
Br, Gurjot Kaur
-----Original Message----- From: Quanah Gibson-Mount [mailto:quanah@zimbra.com] Sent: Monday, June 27, 2016 9:47 PM To: Gurjot Kaur gurjot.kaur@aricent.com Cc: openldap-technical@openldap.org Subject: RE: MDB data replication issues
--On Monday, June 27, 2016 2:00 PM +0000 Gurjot Kaur gurjot.kaur@aricent.com wrote:
Thanks Bastian for your response.
But I want to clear that I don't use slapadd while slapd is running. I have LDIF data which needs to be imported in the new Multimaster servers. That's why I used slapadd. When I did the import (slapadd) with the big data, it doesn't get replicated. To check the problems I did the import with single entry only.
It is not now, nor has it ever been, supported to do an offline add via slapadd on a master, and then magically expect that to replicate out to a consumer via syncrepl. Syncrepl is an active protocol (it replicates the changes that occur while slapd is running). If you are going to offline modify slapd on the master, you will have to do the same thing on the replicas, OR reload them via slapadd, OR trigger a refresh manually on the replicas.
--Quanah
--
Quanah Gibson-Mount Platform Architect Manager, Systems Team Zimbra, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration A division of Synacor, Inc "DISCLAIMER: This message is proprietary to Aricent and is intended solely for the use of the individual to whom it is addressed. It may contain privileged or confidential information and should not be circulated or used for any purpose other than for what it is intended. If you have received this message in error, please notify the originator immediately. If you are not the intended recipient, you are notified that you are strictly prohibited from using, copying, altering, or disclosing the contents of this message. Aricent accepts no responsibility for loss or damage arising from the use of the information transmitted by this email including damage from virus."
openldap-technical@openldap.org
-
Bastian Tweddell -
Frank Swasey -
Gurjot Kaur -
Hallvard Breien Furuseth -
Howard Chu -
Quanah Gibson-Mount -
Ulrich Windl