I have multiple sites that I am trying to sync up to a global server. Each site is configured (striped down) as:
############################################################################ # mdb database for o=chi01,ou=studios,dc=methodstudios,dc=net ############################################################################ database mdb suffix "o=chi01,ou=studios,dc=methodstudios,dc=net"
# Save the time that the entry gets modified lastmod on
# Subordinate of the ou=studios,dc=methodstudios,dc=net database below subordinate advertise
overlay syncprov syncprov-reloadhint TRUE syncprov-checkpoint 100 5
############################################################################ # mdb database for ou=studios,dc=methodstudios,dc=net ############################################################################ database mdb suffix "ou=studios,dc=methodstudios,dc=net"
The above is my configuration for Chicago. I have similar ones for New York (ny01) and Los Angeles (la01) On my global server I am trying to use sync replication to clone each site. The global server is configured (striped down) as:
############################################################################ # mdb database for o=global,ou=studios,dc=methodstudios,dc=net ############################################################################ database mdb suffix "o=global,ou=studios,dc=methodstudios,dc=net"
# Save the time that the entry gets modified lastmod on
# Subordinate of the ou=studios,dc=methodstudios,dc=net database below subordinate advertise
overlay syncprov syncprov-reloadhint TRUE syncprov-checkpoint 100 5
############################################################################ # mdb database for o=chi01,ou=studios,dc=methodstudios,dc=net ############################################################################ database mdb suffix "o=chi01,ou=studios,dc=methodstudios,dc=net"
# Subordinate of the ou=studios,dc=methodstudios,dc=net database below subordinate advertise
syncrepl rid=1 provider=ldap://chi01.methodstudios.com type=refreshOnly retry="60 10 300 +" interval=00:00:10:00 searchbase="o=chi01,ou=studios,dc=methodstudios,dc=net" bindmethod=simple starttls=yes binddn="****" credentials=**** schemachecking=off
############################################################################ # mdb database for o=ny01,ou=studios,dc=methodstudios,dc=net ############################################################################ database mdb suffix "o=ny01,ou=studios,dc=methodstudios,dc=net"
# Subordinate of the ou=studios,dc=methodstudios,dc=net database below subordinate advertise
syncrepl rid=2 provider=ldap://ny01.methodstudios.com type=refreshOnly retry="60 10 300 +" interval=00:00:10:00 searchbase="o=ny01,ou=studios,dc=methodstudios,dc=net" bindmethod=simple starttls=yes binddn="****" credentials=**** schemachecking=off
############################################################################ # mdb database for ou=studios,dc=methodstudios,dc=net ############################################################################ database mdb suffix "ou=studios,dc=methodstudios,dc=net"
overlay glue overlay syncprov syncprov-reloadhint TRUE syncprov-checkpoint 100 5
Now that I have the configuration out of the way. Syncrepl on the global server is failing on chi01. The chi01 server syslog has
Oct 23 18:41:35 boote01-chi01 slapd[19324]: conn=1000 op=2 SRCH base="o=chi01,ou=studios,dc=methodstudios,dc=net" scope=2 deref=0 filter="(objectClass=*)" Oct 23 18:41:35 boote01-chi01 slapd[19324]: conn=1000 op=2 SRCH attr=* + Oct 23 18:41:35 boote01-chi01 slapd[19324]: conn=1000 op=2 SEARCH RESULT tag=101 err=53 nentries=0 text=consumer state is newer than provider! Oct 23 18:41:35 boote01-chi01 slapd[19324]: conn=1000 op=3 UNBIND
Looking at the ny01 server (ldapsearch -x -h ny01 -b o=ny01,ou=studios,dc=methodstudios,dc=net -s base +) where syncrepl is working # ny01, studios, methodstudios.net dn: o=ny01,ou=studios,dc=methodstudios,dc=net structuralObjectClass: organization entryUUID: 257c5408-717c-1032-9b24-31eddc101779 creatorsName: cn=admin,ou=studios,dc=methodstudios,dc=net createTimestamp: 20130625004432Z entryCSN: 20130625004432.932104Z#000000#000#000000 modifiersName: cn=admin,ou=studios,dc=methodstudios,dc=net modifyTimestamp: 20130625004432Z contextCSN: 20131023210335.999443Z#000000#000#000000 entryDN: o=ny01,ou=studios,dc=methodstudios,dc=net subschemaSubentry: cn=Subschema hasSubordinates: TRUE
Looking at the global server (ldapsearch -x -h global -b o=ny01,ou=studios,dc=methodstudios,dc=net -s base +): # ny01, studios, methodstudios.net dn: o=ny01,ou=studios,dc=methodstudios,dc=net structuralObjectClass: organization entryUUID: ccbe6442-d084-1032-8149-e14ce02952dd creatorsName: cn=admin,ou=studios,dc=methodstudios,dc=net createTimestamp: 20131023231549Z entryCSN: 20131023231549.982220Z#000000#000#000000 modifiersName: cn=admin,ou=studios,dc=methodstudios,dc=net modifyTimestamp: 20131023231549Z entryDN: o=ny01,ou=studios,dc=methodstudios,dc=net subschemaSubentry: cn=Subschema hasSubordinates: FALSE
Notice no contextCSN.
Looking at the chi01 server (ldapsearch -x -h chi01 -b o=chi01,ou=studios,dc=methodstudios,dc=net -s base +): # chi01, studios, methodstudios.net dn: o=chi01,ou=studios,dc=methodstudios,dc=net structuralObjectClass: organization entryUUID: 4b4f63f6-81bb-1032-97d3-7d320a684bf1 creatorsName: cn=admin,ou=studios,dc=methodstudios,dc=net createTimestamp: 20130715165653Z entryCSN: 20130715165653.289427Z#000000#000#000000 modifiersName: cn=admin,ou=studios,dc=methodstudios,dc=net modifyTimestamp: 20130715165653Z contextCSN: 20131018000127.430328Z#000000#000#000000 entryDN: o=chi01,ou=studios,dc=methodstudios,dc=net subschemaSubentry: cn=Subschema hasSubordinates: TRUE
Looking at the global server (ldapsearch -x -h global -b o=chi01,ou=studios,dc=methodstudios,dc=net -s base +): # chi01, studios, methodstudios.net dn: o=chi01,ou=studios,dc=methodstudios,dc=net structuralObjectClass: organization entryUUID: cc675698-d084-1032-8eb1-f1b765ca1756 creatorsName: cn=admin,ou=studios,dc=methodstudios,dc=net createTimestamp: 20131023231549Z entryCSN: 20131023231549.411641Z#000000#000#000000 modifiersName: cn=admin,ou=studios,dc=methodstudios,dc=net modifyTimestamp: 20131023231549Z entryDN: o=chi01,ou=studios,dc=methodstudios,dc=net subschemaSubentry: cn=Subschema hasSubordinates: FALSE
Notice no contextCSN.
Looking at the global server root of the glued database (ldapsearch -x -h global -b ou=studios,dc=methodstudios,dc=net -s base +): # studios, methodstudios.net dn: ou=studios,dc=methodstudios,dc=net structuralObjectClass: organizationalUnit entryUUID: cc769b12-d084-1032-86fe-a1b1821abdab creatorsName: cn=admin,ou=studios,dc=methodstudios,dc=net createTimestamp: 20131023231549Z entryCSN: 20131023231549.511823Z#000000#000#000000 modifiersName: cn=admin,ou=studios,dc=methodstudios,dc=net modifyTimestamp: 20131023231549Z contextCSN: 20131024000524.348070Z#000000#000#000000 entryDN: ou=studios,dc=methodstudios,dc=net subschemaSubentry: cn=Subschema hasSubordinates: FALSE
So after all that. Is the syncrepl from chi01 using the contextCSN from the root of the glued database? It seems all the syncrepl from all the sites fail unless they have the latest change. How do you handle syncrepl on glued databases?
A summary of what I posted below . I have several subordinate databases and each subordinate database acquires there data via a refreshOnly syncrepl. Instead of storing the contextCSN on the subordinate database the contextCSN gets stored on the superior database. As a result the superior databases contextCSN is the maximum of the subordinate databases. This causes all but the syncprov server with the latest contextCSN to abort the sync with "consumer state is newer than provider!"
It seems a configuration option needs to be added that allows storing and reading of the contextCSN on the subordinate databases as well as the maximum contextCSN on the superior database.
On 10/23/2013 05:12 PM, Robert Minsk wrote:
I have multiple sites that I am trying to sync up to a global server. Each site is configured (striped down) as:
############################################################################ # mdb database for o=chi01,ou=studios,dc=methodstudios,dc=net ############################################################################ database mdb suffix "o=chi01,ou=studios,dc=methodstudios,dc=net"
# Save the time that the entry gets modified lastmod on
# Subordinate of the ou=studios,dc=methodstudios,dc=net database below subordinate advertise
overlay syncprov syncprov-reloadhint TRUE syncprov-checkpoint 100 5
############################################################################ # mdb database for ou=studios,dc=methodstudios,dc=net ############################################################################ database mdb suffix "ou=studios,dc=methodstudios,dc=net"
The above is my configuration for Chicago. I have similar ones for New York (ny01) and Los Angeles (la01) On my global server I am trying to use sync replication to clone each site. The global server is configured (striped down) as:
############################################################################ # mdb database for o=global,ou=studios,dc=methodstudios,dc=net ############################################################################ database mdb suffix "o=global,ou=studios,dc=methodstudios,dc=net"
# Save the time that the entry gets modified lastmod on
# Subordinate of the ou=studios,dc=methodstudios,dc=net database below subordinate advertise
overlay syncprov syncprov-reloadhint TRUE syncprov-checkpoint 100 5
############################################################################ # mdb database for o=chi01,ou=studios,dc=methodstudios,dc=net ############################################################################ database mdb suffix "o=chi01,ou=studios,dc=methodstudios,dc=net"
# Subordinate of the ou=studios,dc=methodstudios,dc=net database below subordinate advertise
syncrepl rid=1 provider=ldap://chi01.methodstudios.com type=refreshOnly retry="60 10 300 +" interval=00:00:10:00 searchbase="o=chi01,ou=studios,dc=methodstudios,dc=net" bindmethod=simple starttls=yes binddn="****" credentials=**** schemachecking=off
############################################################################ # mdb database for o=ny01,ou=studios,dc=methodstudios,dc=net ############################################################################ database mdb suffix "o=ny01,ou=studios,dc=methodstudios,dc=net"
# Subordinate of the ou=studios,dc=methodstudios,dc=net database below subordinate advertise
syncrepl rid=2 provider=ldap://ny01.methodstudios.com type=refreshOnly retry="60 10 300 +" interval=00:00:10:00 searchbase="o=ny01,ou=studios,dc=methodstudios,dc=net" bindmethod=simple starttls=yes binddn="****" credentials=**** schemachecking=off
############################################################################ # mdb database for ou=studios,dc=methodstudios,dc=net ############################################################################ database mdb suffix "ou=studios,dc=methodstudios,dc=net"
overlay glue overlay syncprov syncprov-reloadhint TRUE syncprov-checkpoint 100 5
Now that I have the configuration out of the way. Syncrepl on the global server is failing on chi01. The chi01 server syslog has
Oct 23 18:41:35 boote01-chi01 slapd[19324]: conn=1000 op=2 SRCH base="o=chi01,ou=studios,dc=methodstudios,dc=net" scope=2 deref=0 filter="(objectClass=*)" Oct 23 18:41:35 boote01-chi01 slapd[19324]: conn=1000 op=2 SRCH attr=* + Oct 23 18:41:35 boote01-chi01 slapd[19324]: conn=1000 op=2 SEARCH RESULT tag=101 err=53 nentries=0 text=consumer state is newer than provider! Oct 23 18:41:35 boote01-chi01 slapd[19324]: conn=1000 op=3 UNBIND
Looking at the ny01 server (ldapsearch -x -h ny01 -b o=ny01,ou=studios,dc=methodstudios,dc=net -s base +) where syncrepl is working # ny01, studios, methodstudios.net dn: o=ny01,ou=studios,dc=methodstudios,dc=net structuralObjectClass: organization entryUUID: 257c5408-717c-1032-9b24-31eddc101779 creatorsName: cn=admin,ou=studios,dc=methodstudios,dc=net createTimestamp: 20130625004432Z entryCSN: 20130625004432.932104Z#000000#000#000000 modifiersName: cn=admin,ou=studios,dc=methodstudios,dc=net modifyTimestamp: 20130625004432Z contextCSN: 20131023210335.999443Z#000000#000#000000 entryDN: o=ny01,ou=studios,dc=methodstudios,dc=net subschemaSubentry: cn=Subschema hasSubordinates: TRUE
Looking at the global server (ldapsearch -x -h global -b o=ny01,ou=studios,dc=methodstudios,dc=net -s base +): # ny01, studios, methodstudios.net dn: o=ny01,ou=studios,dc=methodstudios,dc=net structuralObjectClass: organization entryUUID: ccbe6442-d084-1032-8149-e14ce02952dd creatorsName: cn=admin,ou=studios,dc=methodstudios,dc=net createTimestamp: 20131023231549Z entryCSN: 20131023231549.982220Z#000000#000#000000 modifiersName: cn=admin,ou=studios,dc=methodstudios,dc=net modifyTimestamp: 20131023231549Z entryDN: o=ny01,ou=studios,dc=methodstudios,dc=net subschemaSubentry: cn=Subschema hasSubordinates: FALSE
Notice no contextCSN.
Looking at the chi01 server (ldapsearch -x -h chi01 -b o=chi01,ou=studios,dc=methodstudios,dc=net -s base +): # chi01, studios, methodstudios.net dn: o=chi01,ou=studios,dc=methodstudios,dc=net structuralObjectClass: organization entryUUID: 4b4f63f6-81bb-1032-97d3-7d320a684bf1 creatorsName: cn=admin,ou=studios,dc=methodstudios,dc=net createTimestamp: 20130715165653Z entryCSN: 20130715165653.289427Z#000000#000#000000 modifiersName: cn=admin,ou=studios,dc=methodstudios,dc=net modifyTimestamp: 20130715165653Z contextCSN: 20131018000127.430328Z#000000#000#000000 entryDN: o=chi01,ou=studios,dc=methodstudios,dc=net subschemaSubentry: cn=Subschema hasSubordinates: TRUE
Looking at the global server (ldapsearch -x -h global -b o=chi01,ou=studios,dc=methodstudios,dc=net -s base +): # chi01, studios, methodstudios.net dn: o=chi01,ou=studios,dc=methodstudios,dc=net structuralObjectClass: organization entryUUID: cc675698-d084-1032-8eb1-f1b765ca1756 creatorsName: cn=admin,ou=studios,dc=methodstudios,dc=net createTimestamp: 20131023231549Z entryCSN: 20131023231549.411641Z#000000#000#000000 modifiersName: cn=admin,ou=studios,dc=methodstudios,dc=net modifyTimestamp: 20131023231549Z entryDN: o=chi01,ou=studios,dc=methodstudios,dc=net subschemaSubentry: cn=Subschema hasSubordinates: FALSE
Notice no contextCSN.
Looking at the global server root of the glued database (ldapsearch -x -h global -b ou=studios,dc=methodstudios,dc=net -s base +): # studios, methodstudios.net dn: ou=studios,dc=methodstudios,dc=net structuralObjectClass: organizationalUnit entryUUID: cc769b12-d084-1032-86fe-a1b1821abdab creatorsName: cn=admin,ou=studios,dc=methodstudios,dc=net createTimestamp: 20131023231549Z entryCSN: 20131023231549.511823Z#000000#000#000000 modifiersName: cn=admin,ou=studios,dc=methodstudios,dc=net modifyTimestamp: 20131023231549Z contextCSN: 20131024000524.348070Z#000000#000#000000 entryDN: ou=studios,dc=methodstudios,dc=net subschemaSubentry: cn=Subschema hasSubordinates: FALSE
So after all that. Is the syncrepl from chi01 using the contextCSN from the root of the glued database? It seems all the syncrepl from all the sites fail unless they have the latest change. How do you handle syncrepl on glued databases?
-- Robert Minsk Systems and Software Engineer
WWW.METHODSTUDIOS.COM http://www.methodstudios.com 730 Arizona Ave, Santa Monica, CA 90401 O:+1 310 434 6500 tel:+13104346500 // F:+1 310 434 6501 tel:+13104346501
Los Angeles http://www.methodstudios.com/signature/url/los-angeleshttp://www.methodstudios.com/signature/url/los-angeles
This e-mail and any attachments are intended only for use by the addressee(s) named herein and may contain confidential information. If you are not the intended recipient of this e-mail, you are hereby notified any dissemination, distribution or copying of this email and any attachments is strictly prohibited. If you receive this email in error, please immediately notify the sender by return email and permanently delete the original, any copy and any printout thereof. The integrity and security of e-mail cannot be guaranteed.
On Thu, 24 Oct 2013, Robert Minsk wrote:
A summary of what I posted below . I have several subordinate databases and each subordinate database acquires there data via a refreshOnly syncrepl. Instead of storing the contextCSN on the subordinate database the contextCSN gets stored on the superior database. As a result the superior databases contextCSN is the maximum of the subordinate databases. This causes all but the syncprov server with the latest contextCSN to abort the sync with "consumer state is newer than provider!"
It seems a configuration option needs to be added that allows storing and reading of the contextCSN on the subordinate databases as well as the maximum contextCSN on the superior database.
Sounds like you're looking for "sync_use_subentry true", to be used in the database contexts of the consumers.
Philip Guenther
When I use "sync_use_subentry true" I do not see any contextCSN in the database. I did a search on cn=ldapsync,o=chi01,ou=studios,dc=methodstudios,dc=net and found nothing. I also did a "slapcat | grep -i contextcsn" and found nothing. The sync seems to be walking all the entryUUID every time. Is this because I am doing a refreshOnly sync? Do both the producer and consumers need sync_use_subentry?
On 10/24/2013 12:27 PM, Philip Guenther wrote:
On Thu, 24 Oct 2013, Robert Minsk wrote:
A summary of what I posted below . I have several subordinate databases and each subordinate database acquires there data via a refreshOnly syncrepl. Instead of storing the contextCSN on the subordinate database the contextCSN gets stored on the superior database. As a result the superior databases contextCSN is the maximum of the subordinate databases. This causes all but the syncprov server with the latest contextCSN to abort the sync with "consumer state is newer than provider!"
It seems a configuration option needs to be added that allows storing and reading of the contextCSN on the subordinate databases as well as the maximum contextCSN on the superior database.
Sounds like you're looking for "sync_use_subentry true", to be used in the database contexts of the consumers.
Philip Guenther
I've changed everything over to using sync_use_subentry on both the provider and consumers and I am still getting "consumer state is newer than provider!"
On 10/24/2013 12:27 PM, Philip Guenther wrote:
On Thu, 24 Oct 2013, Robert Minsk wrote:
A summary of what I posted below . I have several subordinate databases and each subordinate database acquires there data via a refreshOnly syncrepl. Instead of storing the contextCSN on the subordinate database the contextCSN gets stored on the superior database. As a result the superior databases contextCSN is the maximum of the subordinate databases. This causes all but the syncprov server with the latest contextCSN to abort the sync with "consumer state is newer than provider!"
It seems a configuration option needs to be added that allows storing and reading of the contextCSN on the subordinate databases as well as the maximum contextCSN on the superior database.
Sounds like you're looking for "sync_use_subentry true", to be used in the database contexts of the consumers.
Philip Guenther
Robert Minsk wrote:
A summary of what I posted below . I have several subordinate databases and each subordinate database acquires there data via a refreshOnly syncrepl. Instead of storing the contextCSN on the subordinate database the contextCSN gets stored on the superior database. As a result the superior databases contextCSN is the maximum of the subordinate databases. This causes all but the syncprov server with the latest contextCSN to abort the sync with "consumer state is newer than provider!"
It seems a configuration option needs to be added that allows storing and reading of the contextCSN on the subordinate databases as well as the maximum contextCSN on the superior database.
Use a unique ServerID per provider.
I see in the man page now under the serverID "Note that this requirement also applies to separate masters contributing to a glued set of databases." I never even looked there. Can the man page be updated to also mention the serverID requirement under the subordinate section.
On Thu 24 Oct 2013 06:34:11 PM PDT, Howard Chu wrote:
Robert Minsk wrote:
A summary of what I posted below . I have several subordinate databases and each subordinate database acquires there data via a refreshOnly syncrepl. Instead of storing the contextCSN on the subordinate database the contextCSN gets stored on the superior database. As a result the superior databases contextCSN is the maximum of the subordinate databases. This causes all but the syncprov server with the latest contextCSN to abort the sync with "consumer state is newer than provider!"
It seems a configuration option needs to be added that allows storing and reading of the contextCSN on the subordinate databases as well as the maximum contextCSN on the superior database.
Use a unique ServerID per provider.
-- Robert Minsk Systems and Software Engineer
WWW.METHODSTUDIOS.COM http://www.methodstudios.com 730 Arizona Ave, Santa Monica, CA 90401 O:+1 310 434 6500 tel:+13104346500 // F:+1 310 434 6501 tel:+13104346501
Los Angeles http://www.methodstudios.com/signature/url/los-angeles
This e-mail and any attachments are intended only for use by the addressee(s) named herein and may contain confidential information. If you are not the intended recipient of this e-mail, you are hereby notified any dissemination, distribution or copying of this email and any attachments is strictly prohibited. If you receive this email in error, please immediately notify the sender by return email and permanently delete the original, any copy and any printout thereof. The integrity and security of e-mail cannot be guaranteed.
openldap-technical@openldap.org
-
Howard Chu -
Philip Guenther -
Robert Minsk