Hello,
I'm facing some issues with syncrepl...
The simplest situation in which I was able to reproduce the problem consists of 1 provider and 1 consumer.
I have configured syncrepl to do a partial replication :
olcSyncrepl: {0}rid=105 provider=ldaps://****:636 bindmethod=simple timeout=0 network-timeout=0 starttls=no filter="(mail=*)" searchbase="ou=groups,o=mydir" scope=sub schemachecking=off attrs="cn description mail member +" type=refreshAndPersist retry="60 +" tls_cert=**** tls_cacert=**** tls_key=****
olcSyncrepl: {1}rid=107 provider=ldaps://****:636 bindmethod=simple timeout=0 network-timeout=0 starttls=no filter="(objectclass=*)" searchbase="ou=operators,o=mydir" scope=sub schemachecking=off type=refreshAndPersist retry="60 +" tls_cert=**** tls_cacert=**** tls_key=****
olcSyncrepl: {2}rid=104 provider=ldaps://****:636 bindmethod=simple timeout=0 network-timeout=0 starttls=no filter="(uid=mytestuser)" searchbase="ou=people,o=mydir" scope=sub schemachecking=off type=refreshAndPersist retry="60 +" tls_cert=**** tls_cacert=**** tls_key=****
The initial sync (i.e. from an empty base) is fine, but then when I modify an entry (uid=mytestuser), it is not updated on the consumer, and I get the following messages:
Feb 16 15:42:32 do_syncrep2: rid=107 NEW_COOKIE: rid=107,csn=20110216144232.222152Z#000000#000#000000 Feb 16 15:42:32 do_syncrep2: rid=104 cookie=rid=104,csn=20110216144232.222152Z#000000#000#000000 Feb 16 15:42:32 slap_graduate_commit_csn: removing 0x7fb58c0227d0 20110216144232.222152Z#000000#000#000000 Feb 16 15:42:32 do_syncrep2: rid=104 CSN too old, ignoring 20110216144232.222152Z#000000#000#000000 Feb 16 15:43:05 do_syncrep2: rid=105 LDAP_RES_INTERMEDIATE - NEW_COOKIE Feb 16 15:43:05 do_syncrep2: rid=105 NEW_COOKIE: rid=105,csn=20110216144305.545784Z#000000#000#000000 Feb 16 15:43:05 slap_queue_csn: queing 0x7fb58c020b50 20110216144305.545784Z#000000#000#000000 Feb 16 15:43:05 do_syncrep2: rid=107 LDAP_RES_INTERMEDIATE - NEW_COOKIE Feb 16 15:43:05 do_syncrep2: rid=107 NEW_COOKIE: rid=107,csn=20110216144305.545784Z#000000#000#000000 Feb 16 15:43:05 do_syncrep2: rid=104 cookie=rid=104,csn=20110216144305.545784Z#000000#000#000000 Feb 16 15:43:05 slap_graduate_commit_csn: removing 0x7fb58c0227d0 20110216144305.545784Z#000000#000#000000 Feb 16 15:43:05 do_syncrep2: rid=104 CSN too old, ignoring 20110216144305.545784Z#000000#000#000000
(both servers are perfectly in sync with an NTP daemon)
It looks like ITS #6619 is quite similar, and unresolved.
What do these messages mean ? Is there a way to force the update ?
Regards,
Jeremy W.
--On Wednesday, February 16, 2011 4:26 PM +0100 Jérémy Wagner jeremy.wagner@unistra.fr wrote:
Hello,
I'm facing some issues with syncrepl...
What openldap release?
--Quanah
--
Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration
Le 16/02/2011 17:11, Quanah Gibson-Mount a écrit :
--On Wednesday, February 16, 2011 4:26 PM +0100 Jérémy Wagner jeremy.wagner@unistra.fr wrote:
Hello,
I'm facing some issues with syncrepl...
What openldap release?
--Quanah
--
Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc.
Zimbra :: the leader in open source messaging and collaboration
Thanks for your answer, I'm using openldap 2.4.21 (the version packaged in Ubuntu Server LTS).
--On Wednesday, February 16, 2011 5:21 PM +0100 Jérémy Wagner jeremy.wagner@unistra.fr wrote:
Thanks for your answer, I'm using openldap 2.4.21 (the version packaged in Ubuntu Server LTS).
I would strongly advise you look at the fixes to OpenLDAP, specifically in regards to replication, since that release. There have been numerous significant changes.
http://www.openldap.org/software/release/changes.html
--Quanah
--
Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration
Jérémy Wagner wrote:
Hello,
I'm facing some issues with syncrepl...
The simplest situation in which I was able to reproduce the problem consists of 1 provider and 1 consumer.
I have configured syncrepl to do a partial replication :
olcSyncrepl: {0}rid=105 provider=ldaps://****:636 bindmethod=simple timeout=0 network-timeout=0 starttls=no filter="(mail=*)" searchbase="ou=groups,o=mydir" scope=sub schemachecking=off attrs="cn description mail member +" type=refreshAndPersist retry="60 +" tls_cert=**** tls_cacert=**** tls_key=****
olcSyncrepl: {1}rid=107 provider=ldaps://****:636 bindmethod=simple timeout=0 network-timeout=0 starttls=no filter="(objectclass=*)" searchbase="ou=operators,o=mydir" scope=sub schemachecking=off type=refreshAndPersist retry="60 +" tls_cert=**** tls_cacert=**** tls_key=****
olcSyncrepl: {2}rid=104 provider=ldaps://****:636 bindmethod=simple timeout=0 network-timeout=0 starttls=no filter="(uid=mytestuser)" searchbase="ou=people,o=mydir" scope=sub schemachecking=off type=refreshAndPersist retry="60 +" tls_cert=**** tls_cacert=**** tls_key=****
The initial sync (i.e. from an empty base) is fine, but then when I modify an entry (uid=mytestuser), it is not updated on the consumer, and I get the following messages:
Feb 16 15:42:32 do_syncrep2: rid=107 NEW_COOKIE: rid=107,csn=20110216144232.222152Z#000000#000#000000 Feb 16 15:42:32 do_syncrep2: rid=104 cookie=rid=104,csn=20110216144232.222152Z#000000#000#000000 Feb 16 15:42:32 slap_graduate_commit_csn: removing 0x7fb58c0227d0 20110216144232.222152Z#000000#000#000000 Feb 16 15:42:32 do_syncrep2: rid=104 CSN too old, ignoring 20110216144232.222152Z#000000#000#000000 Feb 16 15:43:05 do_syncrep2: rid=105 LDAP_RES_INTERMEDIATE - NEW_COOKIE Feb 16 15:43:05 do_syncrep2: rid=105 NEW_COOKIE: rid=105,csn=20110216144305.545784Z#000000#000#000000 Feb 16 15:43:05 slap_queue_csn: queing 0x7fb58c020b50 20110216144305.545784Z#000000#000#000000 Feb 16 15:43:05 do_syncrep2: rid=107 LDAP_RES_INTERMEDIATE - NEW_COOKIE Feb 16 15:43:05 do_syncrep2: rid=107 NEW_COOKIE: rid=107,csn=20110216144305.545784Z#000000#000#000000 Feb 16 15:43:05 do_syncrep2: rid=104 cookie=rid=104,csn=20110216144305.545784Z#000000#000#000000 Feb 16 15:43:05 slap_graduate_commit_csn: removing 0x7fb58c0227d0 20110216144305.545784Z#000000#000#000000 Feb 16 15:43:05 do_syncrep2: rid=104 CSN too old, ignoring 20110216144305.545784Z#000000#000#000000
(both servers are perfectly in sync with an NTP daemon)
It looks like ITS #6619 is quite similar, and unresolved.
What do these messages mean ? Is there a way to force the update ?
Your current configuration is unsupported. Ordinarily when you configure multiple consumers in a DB it is assumed they are all pointing at different providers, and each provider will have a different serverID. In your case, since it is all a single provider with a single serverID, all 3 consumers will share a single contextCSN. Any update in any branch will update the contextCSN to that newest state. Any updates in other branches that hadn't been received yet (due to random timing between threads) will be received as "too old" because the contextCSN already has a newer state.
A workaround for this would be to split each consumer into its own subordinate database, then each will have a private copy of its own contextCSN to work with.
Howard Chu wrote:
Jérémy Wagner wrote:
Hello,
I'm facing some issues with syncrepl...
The simplest situation in which I was able to reproduce the problem consists of 1 provider and 1 consumer.
I have configured syncrepl to do a partial replication :
olcSyncrepl: {0}rid=105 provider=ldaps://****:636 bindmethod=simple timeout=0 network-timeout=0 starttls=no filter="(mail=*)" searchbase="ou=groups,o=mydir" scope=sub schemachecking=off attrs="cn description mail member +" type=refreshAndPersist retry="60 +" tls_cert=**** tls_cacert=**** tls_key=****
olcSyncrepl: {1}rid=107 provider=ldaps://****:636 bindmethod=simple timeout=0 network-timeout=0 starttls=no filter="(objectclass=*)" searchbase="ou=operators,o=mydir" scope=sub schemachecking=off type=refreshAndPersist retry="60 +" tls_cert=**** tls_cacert=**** tls_key=****
olcSyncrepl: {2}rid=104 provider=ldaps://****:636 bindmethod=simple timeout=0 network-timeout=0 starttls=no filter="(uid=mytestuser)" searchbase="ou=people,o=mydir" scope=sub schemachecking=off type=refreshAndPersist retry="60 +" tls_cert=**** tls_cacert=**** tls_key=****
The initial sync (i.e. from an empty base) is fine, but then when I modify an entry (uid=mytestuser), it is not updated on the consumer, and I get the following messages:
Feb 16 15:42:32 do_syncrep2: rid=107 NEW_COOKIE: rid=107,csn=20110216144232.222152Z#000000#000#000000 Feb 16 15:42:32 do_syncrep2: rid=104 cookie=rid=104,csn=20110216144232.222152Z#000000#000#000000 Feb 16 15:42:32 slap_graduate_commit_csn: removing 0x7fb58c0227d0 20110216144232.222152Z#000000#000#000000 Feb 16 15:42:32 do_syncrep2: rid=104 CSN too old, ignoring 20110216144232.222152Z#000000#000#000000 Feb 16 15:43:05 do_syncrep2: rid=105 LDAP_RES_INTERMEDIATE - NEW_COOKIE Feb 16 15:43:05 do_syncrep2: rid=105 NEW_COOKIE: rid=105,csn=20110216144305.545784Z#000000#000#000000 Feb 16 15:43:05 slap_queue_csn: queing 0x7fb58c020b50 20110216144305.545784Z#000000#000#000000 Feb 16 15:43:05 do_syncrep2: rid=107 LDAP_RES_INTERMEDIATE - NEW_COOKIE Feb 16 15:43:05 do_syncrep2: rid=107 NEW_COOKIE: rid=107,csn=20110216144305.545784Z#000000#000#000000 Feb 16 15:43:05 do_syncrep2: rid=104 cookie=rid=104,csn=20110216144305.545784Z#000000#000#000000 Feb 16 15:43:05 slap_graduate_commit_csn: removing 0x7fb58c0227d0 20110216144305.545784Z#000000#000#000000 Feb 16 15:43:05 do_syncrep2: rid=104 CSN too old, ignoring 20110216144305.545784Z#000000#000#000000
(both servers are perfectly in sync with an NTP daemon)
It looks like ITS #6619 is quite similar, and unresolved.
What do these messages mean ? Is there a way to force the update ?
Your current configuration is unsupported. Ordinarily when you configure multiple consumers in a DB it is assumed they are all pointing at different providers, and each provider will have a different serverID. In your case, since it is all a single provider with a single serverID, all 3 consumers will share a single contextCSN. Any update in any branch will update the contextCSN to that newest state. Any updates in other branches that hadn't been received yet (due to random timing between threads) will be received as "too old" because the contextCSN already has a newer state.
A workaround for this would be to split each consumer into its own subordinate database, then each will have a private copy of its own contextCSN to work with.
Another potential workaround, if all of the consumer configs were identical except for their baseDN, would be to simply consolidate them all into a single consumer, with a more complex search filter.
searchbase=o=mydir filter=(| (&(entryDN:dnSubtreematch:=ou=groups,o=mydir)(mail=*)) (entryDN:dnSubtreematch:=ou=operators,o=mydir) (&(entryDN:dnSubtreematch:=ou=people,o=mydir)(uid=mytestuser)))
Le 17/02/2011 00:59, Howard Chu a écrit :
Your current configuration is unsupported. Ordinarily when you configure multiple consumers in a DB it is assumed they are all pointing at different providers, and each provider will have a different serverID. In your case, since it is all a single provider with a single serverID, all 3 consumers will share a single contextCSN. Any update in any branch will update the contextCSN to that newest state. Any updates in other branches that hadn't been received yet (due to random timing between threads) will be received as "too old" because the contextCSN already has a newer state.
A workaround for this would be to split each consumer into its own subordinate database, then each will have a private copy of its own contextCSN to work with.
Thank you for your answer, it was very helpful.
I followed your advice and configured several databases (one for each replicated branch) and it resolved all my replication problems :-)
openldap-technical@openldap.org