As per the recommendations, we have done following : 1. Installed openldap 2.4.30 on master & all replicas. 2. We have cascaded the replication by adding another replica in lan which acts as provider for the consumers on wan.
There are definite improvements but still we are observing following problem :
1. Bulk changes (1000 records changed) in LDAP on the provider, do not replicate to replicas with busy ldap unless a) the records are updated in smaller batches on provider b) the replication database on the consumer is reset - i.e ldap restarted with -c rid=001
Is there a buffer/queue at the consumer side which overflows? Can it be tuned? Does a busy server drop replication requests?
Thanks, Amol.
----- Original Message ----- From: Quanah Gibson-Mount Sent: 03/12/12 03:36 AM To: Amol Kulkarni Subject: Re: slow or inconsistent syncrepl
--On Sunday, March 11, 2012 8:51 AM +0100 Amol Kulkarni amolkulkarni@gmx.com wrote: > > I see that I'm outdated in my tuning as per > http://wiki.zimbra.com/wiki/OpenLDAP_Performance_Tuning. > I'll do these changes and observe the servers. > > In the mean time a new observation regarding syncrepl is that if I do > ldapmodify for 5 - 10 entries in one go the changes get replicated but if > I change about 300 entries in one go, then some entries do not get > replicated on some servers. These changes dont get replicated until I > change those entries again. > > Does this indicate provider resource problem or consumer resource problem > ( or normal behaviour ) ? Changes failing to replicate is not a desired behavior. It is expected to occur in older known buggy releases. If all of your masters & replicas are on 2.4.30, hopefully you won't see that issue any more. --Quanah -- Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration
Amol Kulkarni wrote:
As per the recommendations, we have done following :
- Installed openldap 2.4.30 on master & all replicas.
- We have cascaded the replication by adding another replica in lan which
acts as provider for the consumers on wan.
There are definite improvements but still we are observing following problem :
- Bulk changes (1000 records changed) in LDAP on the provider, do not
replicate to replicas with busy ldap unless a) the records are updated in smaller batches on provider b) the replication database on the consumer is reset - i.e ldap restarted with -c rid=001
Neither of those options should be necessary.
Is there a buffer/queue at the consumer side which overflows?
No, there is nothing that overflows.
Can it be tuned? Does a busy server drop replication requests?
No, requests are not dropped.
But a syncrepl consumer runs as a task in the slapd thread pool. If the server is already busy with other requests, it may be a long time before a thread frees up to handle replication traffic. You might be able to improve things by configuring more threads in slapd, but that will only go so far. If all of the server's CPUs are already busy, there's nothing else to do besides get faster servers or add more replicas.
Thanks, Amol.
----- Original Message -----
From: Quanah Gibson-Mount
Sent: 03/12/12 03:36 AM
To: Amol Kulkarni
Subject: Re: slow or inconsistent syncrepl
--On Sunday, March 11, 2012 8:51 AM +0100 Amol Kulkarni amolkulkarni@gmx.com wrote:
I see that I'm outdated in my tuning as per http://wiki.zimbra.com/wiki/OpenLDAP_Performance_Tuning. I'll do these changes and observe the servers.
In the mean time a new observation regarding syncrepl is that if I do ldapmodify for 5 - 10 entries in one go the changes get replicated but if I change about 300 entries in one go, then some entries do not get replicated on some servers. These changes dont get replicated until I change those entries again.
Does this indicate provider resource problem or consumer resource problem ( or normal behaviour ) ?
Changes failing to replicate is not a desired behavior. It is expected to occur in older known buggy releases. If all of your masters& replicas are on 2.4.30, hopefully you won't see that issue any more.
--Quanah
--
Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc.
Zimbra :: the leader in open source messaging and collaboration
openldap-technical@openldap.org
-
Amol Kulkarni -
Howard Chu