Re: Getting around the single-threaded syncrepl model?

Bannister, Mark wrote: >>> --On Friday, November 20, 2015 6:31 PM +0000 Albert Braden >>> <abraden(a)about.com&gt; wrote: >> >>> Hi Quanah, >>> >>> Are you sure your issues with syncrepl aren't specific to Zimbra? When >>> I ran the Zimbra at Homestead/Intuit we saw syncrepl issues, but I >>> have not seen those issues in non-Zimbra LDAP clusters. >> >> Quite certain. For one thing, I file very few syncrepl related ITSes, >> since I don't use it, but many are filed... For another, while I have >> encountered a very few issues with delta-syncrepl, the majority of >> issues I find w/ Zimbra and replication are related to syncrepl when it >> has to be used in an initial or fallback scenario. > > What sort of issues? > >> And here's some quick stats: >> $ grep syncrepl CHANGES | grep -v delta | wc -l >> 76 >> >> $ grep delta-syncrepl CHANGES | wc -l >> 7 >> >> Or syncrepl has had 10x times the issues. > > Or syncrepl has 10x the users, more eyes spot more bugs, and is now more > stable because it has had more fixes? I'm just speculating here, but I > wouldn't be more confident in product Y because it is mentioned less in > the change log than product X. > > However, this is a tangent, and still does not help me with my original > question. Correct me if I'm wrong, but delta-syncrepl is also going to > be running in a serial fashion on the master server, and so, if I'm > replicating out to hundreds of replicas, high network latency or timeouts > on a single replica would still have a big impact on my ability to > replicate everywhere within 5 minutes. > > Syncrepl/delta-syncrepl doesn't make a difference here. Unless someone > has a better suggestion, as I don't suppose OpenLDAP is going to be fixed > to do this in parallel anytime soon, I will go ahead and launch multiple > replica processes on the master server as I proposed at the beginning of > the thread, and then balance the load across these processes to reduce > the impact of any one replica going slow. Certainly won't be changing in 2.4. Very likely to change in 2.5. https://www.openldap.org/its/index.cgi/Incoming?id=8227 > (Or investigate other directory server products that can replicate in > parallel ...) Have fun investigating.