https://bugs.openldap.org/show_bug.cgi?id=10365

Issue ID: 10365 Summary: OOM during replication with a lot of updates Product: OpenLDAP Version: 2.6.10 Hardware: x86_64 OS: Linux Status: UNCONFIRMED Keywords: needs_review Severity: normal Priority: --- Component: overlays Assignee: bugs@openldap.org Reporter: elecharny@apache.org Target Milestone: ---

When applying millions of updates on a huge database (30M entries), the server can crash with an out of memory.

What happens is that the replication client does not process the incoming updates fast enough, syncprov accumulate the changes in memory up to the point it crashes.

The clear workaround: use more memory on the server.

Context: the replication mode used is delta-syncrepl, 2 servers, MMR , with MDB and accesslog.

My assumption is that it would be valuable to benefit from the fact that we know when the socket is ready for write to actually push data to the replica, which means we can use the changes stored in accesslog and not in memory, while keeping a state of replication. I assume it's a huge change in the way it currently works.

There is no urgency, considering it's quite a specific use case, and it would be way faster to slapmodify the changes on a stopped base, then copy the content to the replica, then restart everything.