On Tue, Jun 14, 2022 at 01:40:56PM +0200, Ondřej Kuzník wrote:
It's becoming untenable how a plain refresh cannot be represented in accesslog in a way that's capable of serving a deltasync session. Whatever happens, we have lost a fair amount of information to run a proper deltasync yet if we don't want to abandon this functionality, we have to try and fill some in.
This is another proposal, let's call it proposal 2 for the sake of the argument.
The final accesslog content is similar to proposal 1, except most of the work is done inside the consumer: - the consumer will need a separate scratch space - a consumer entering a plain refresh (present or delete phase, doesn't matter) will make changes to the DB as instructed, however it makes sure they are not logged into accesslog, they go into a separate scratch space, when refresh is done, that scratch DB is sorted in CSN order and injected into accesslog - alternatively, accesslog is told to keep them as scratch and reorder later, syncprov doesn't see these log entries until they're being reordered
This requires tight(er) coupling between syncrepl consumer and accesslog to facilitate the signalling.
If the plain refresh is interrupted for whatever reason, we need a way to abort the changes to the replicated DB. That is a pretty tall order, so alternatively we need the ability to resume a refresh with a provider that would give us a cookie that's the same or newer in all respects to the one we would have received from the original refresh. That's not currently possible either, but fixable - we'd need to extend our cookie signalling so that a provider starts the session with a cookie committing it to a CSN set that will be given at the end of the refresh. That could also be the cookie consumer can send when retrying.
It is open what needs to be done with locally originated changes during this time and whether they can be accepted at all. As with proposal 1, how to log implicit deletes coming from a present phase is an open question.