Hi, list
I have a problem with replication -- numbers of objects on provider and consumer not identical.
My configuration: - two servers (ldap1 & ldap2) configured in mirror mode with carp failover. It's work fine. - also, I have 9 eximx nodes with local ldap servers, configured, as consumers for replicate domain routing table from provider.
eximx nodes get replica from shared ip's (i.e. connection possible to any from providers)
activity ~ 1-2 changes in minute.
numbers of objects ~ 120 000. numbers of read on every eximx up to 50 per second
testcase: 1. I deleting db on eximx node and get fresh replica 2. waiting 24 hours 3. check numbers of objects on ldap1 and eximx node. 4. Today, on eximx, I get more objects than ldap1. Yesterday -- eximx has less objects, than ldap1.
Could somebody recommend me how to debug this issue or change some parameters?
Config on provider: serverID {1,2} #unique for ldap1 and ldap2 limits dn.exact="cn=root,ou=mail-routes" time.soft=unlimited time.hard=unlimited size.soft=unlimited size.hard=unlimited database hdb suffix ou=mail-routes rootdn cn=root,ou=mail-routes rootpw XXX directory /var/db/openldap-data/mail-routes checkpoint 32 8
overlay syncprov syncprov-checkpoint 100 10 syncprov-sessionlog 1000 syncprov-reloadhint TRUE
syncrepl rid=003 provider=ldap://ldap1 type=refreshAndPersist retry="60 +" searchbase="ou=mail-routes" schemachecking=on bindmethod=simple binddn="cn=root,ou=mail-routes" credentials=XXX starttls=yes tls_reqcert=allow
syncrepl rid=004 provider=ldap://ldap2 type=refreshAndPersist retry="60 +" searchbase="ou=mail-routes" schemachecking=on bindmethod=simple binddn="cn=root,ou=mail-routes" credentials=XXX starttls=yes tls_reqcert=allow
mirrormode on
index objectClass,dc eq index entryUUID eq
Config on consumers: limits dn.exact="cn=owner,ou=mail-routes" time.soft=unlimited time.hard=unlimited size.soft=unlimited size.hard=unlimited database hdb suffix ou=mail-routes rootdn cn=root,ou=mail-routes rootpw ""
directory /opt/db/openldap/mail-routes checkpoint 32 8
syncrepl rid=0 provider=ldap://ldap searchbase="ou=mail-routes" type=refreshAndPersist bindmethod=simple binddn="cn=root,ou=mail-routes" credentials=XXX schemachecking=on retry="60 +" starttls=yes tls_reqcert=allow
index objectClass,dc eq index entryUUID eq
On Wed, 19 Nov 2008, Dmitriy Kirhlarov wrote:
Could somebody recommend me how to debug this issue or change some parameters?
Well, first, make sure you're up to patch. 2.4.12, and maybe if there's anything in the ITS that looks relevant, you should make some local additions.
"loglevel sync" is a good place to start. You can find ldifdiff.pl in CPAN, that might be good for pointing out what entries to look for (or look for the absence of) in the logs.
Also take a look at ITS 5709 which was recently fixed and helped to resolve some replication problems.
-----Original Message----- From: openldap-software-bounces+kris.burton=acision.com@OpenLDAP.org [mailto:openldap-software-bounces+kris.burton=acision.com@OpenLDAP.org] On Behalf Of Aaron Richton Sent: Wednesday, November 19, 2008 2:16 PM To: Dmitriy Kirhlarov Cc: OpenLDAP Subject: Re: replication issue
On Wed, 19 Nov 2008, Dmitriy Kirhlarov wrote:
Could somebody recommend me how to debug this issue or change some parameters?
Well, first, make sure you're up to patch. 2.4.12, and maybe if there's anything in the ITS that looks relevant, you should make some local additions.
"loglevel sync" is a good place to start. You can find ldifdiff.pl in CPAN, that might be good for pointing out what entries to look for (or look for the absence of) in the logs.
This e-mail and any attachment is for authorised use by the intended recipient(s) only. It may contain proprietary material, confidential information and/or be subject to legal privilege. It should not be copied, disclosed to, retained or used by, any other party. If you are not an intended recipient then please promptly delete this e-mail and any attachment and all copies and inform the sender. Thank you.
Dmitriy Kirhlarov dimma@higis.ru writes:
Hi, list
I have a problem with replication -- numbers of objects on provider and consumer not identical.
[...]
syncrepl rid=003 provider=ldap://ldap1 type=refreshAndPersist retry="60 +" searchbase="ou=mail-routes" schemachecking=on bindmethod=simple binddn="cn=root,ou=mail-routes" credentials=XXX starttls=yes tls_reqcert=allow
[...]
it might not be related to your replication problem, but setting a sizelimit=unlimited in syncrepl configuration reduces the chance that clientside limitations come into effect. If you require starttls and integrity you should at least add tls_cacert option to syncrepl configuration.
-Dieter
On Wed, Nov 19, 2008 at 09:13:49PM +0100, Dieter Kluenter wrote:
Dmitriy Kirhlarov dimma@higis.ru writes:
I have a problem with replication -- numbers of objects on provider and consumer not identical.
[snip]
it might not be related to your replication problem, but setting a sizelimit=unlimited in syncrepl configuration reduces the chance that clientside limitations come into effect. If you require starttls and integrity you should at least add tls_cacert option to syncrepl configuration.
For client operations, doesn't syncrepl operate as the rootdn, which is exempt from size/time limitations?
FWIW, Dmitriy's problem sounds exactly like what I'm experiencing (2.4.12 on the consumer side, 2.3.42 on the provider):
http://marc.info/?l=openldap-software&m=122644083919827&w=2
I'm trying the latest RE24 on the client today.
john
--On Wednesday, November 19, 2008 8:53 PM +0100 "Burton, Kris - Acision" kris.burton@acision.com wrote:
Also take a look at ITS 5709 which was recently fixed and helped to resolve some replication problems.
Current RE24 CVS is what should be going out as 2.4.13 in the very near future. You may want to verify against that. If that works, then 2.4.13 will resolve your issues.
--Quanah
--
Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration
On Wed, 19 Nov 2008, John Morrissey wrote:
For client operations, doesn't syncrepl operate as the rootdn, which is exempt from size/time limitations?
Internally, syncrepl acts as the rootdn. However, when going over the wire to the provider, it must be configured with a binddn and credentials. Certainly there is nothing in the client stopping you from specifying the rootdn as the syncrepl binddn, but that is not considered best practice.
I'm trying the latest RE24 on the client today.
Probably well advised...please let the list know how it goes...
John Morrissey wrote:
On Wed, Nov 19, 2008 at 09:13:49PM +0100, Dieter Kluenter wrote:
Dmitriy Kirhlarov dimma@higis.ru writes:
I have a problem with replication -- numbers of objects on provider and consumer not identical.
[snip]
it might not be related to your replication problem, but setting a sizelimit=unlimited in syncrepl configuration reduces the chance that clientside limitations come into effect. If you require starttls and integrity you should at least add tls_cacert option to syncrepl configuration.
For client operations, doesn't syncrepl operate as the rootdn, which is exempt from size/time limitations?
What do you mean by "client operations"? syncrepl consists in the consumer contacting the producer via LDAP and thus as a regular client. As such, it connects with whatever identity you configure it as, and what that identity means for the producer is the producer's business. Using the producer's rootdn as the consumer's identity is not wise. That identity should have unlimited read privileges on the data it needs to replicate, but no write privileges are required.
If you mean those internal operations syncrepl needs to perform on the consumer itself, then yes: they are performed using the consumer's rootdn identity.
p.
Ing. Pierangelo Masarati OpenLDAP Core Team
SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it ----------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Fax: +39 0382 476497 Email: ando@sys-net.it -----------------------------------
On Wed, Nov 19, 2008 at 06:09:49PM -0500, Aaron Richton wrote:
On Wed, 19 Nov 2008, John Morrissey wrote:
On Wed, Nov 19, 2008 at 09:13:49PM +0100, Dieter Kluenter wrote:
it might not be related to your replication problem, but setting a sizelimit=unlimited in syncrepl configuration reduces the chance that clientside limitations come into effect. If you require starttls and integrity you should at least add tls_cacert option to syncrepl configuration.
For client operations, doesn't syncrepl operate as the rootdn, which is exempt from size/time limitations?
Internally, syncrepl acts as the rootdn. However, when going over the wire to the provider, it must be configured with a binddn and credentials. Certainly there is nothing in the client stopping you from specifying the rootdn as the syncrepl binddn, but that is not considered best practice.
Right. Dieter mentioned "clientside limitations," which I took to mean internal operations performed on the consumer.
john
On Thu, Nov 20, 2008 at 12:13:30AM +0100, Pierangelo Masarati wrote:
John Morrissey wrote:
For client operations, doesn't syncrepl operate as the rootdn, which is exempt from size/time limitations?
[snip]
If you mean those internal operations syncrepl needs to perform on the consumer itself, then yes: they are performed using the consumer's rootdn identity.
Yes, that's what I meant.
john
openldap-software@openldap.org
-
Aaron Richton -
Burton, Kris - Acision -
Dieter Kluenter -
Dmitriy Kirhlarov -
John Morrissey -
Pierangelo Masarati -
Quanah Gibson-Mount