I was speaking with Howard in person last week at LDAPCon about the problems I might hit if I wanted to run hundreds of replicas off a single master. If I heard right, Howard told me that OpenLDAP uses a single thread for replication, and therefore processes each replica in a serial fashion, one at a time. If any replica is going slowly (high network latency for example), it would have a knock-on effect to the time it takes to get changes replicated across the whole estate.
We have an aggressive target to meet - updates out in 5 minutes. Updates are small, but alarm bells are now ringing in my head about OpenLDAP’s ability to deliver this.
This leads to a couple of questions:
1 - How easy would it be to patch OpenLDAP 2.4 to get the master to use multiple threads for replication? Is that a reasonably straightforward fix, or is this quite a sizeable architectural change?
2 - If my master server has 40 cores, would there be mileage in setting up a number of slapd processes (say 10) on the same host, bound to different sockets, all acting as first-level replicas, and then all the replicas fanning out across the WAN synchronise from these processes rather than from the master process? That would be a way, I suppose, of getting the master server to make better use of the available cores to get updates out quicker?