OpenLDAP freeze

I've an OpenLDAP 2.4.32 server with millions of entries, which uses to serve tens of requests per second. It works as a stand-alone master server (replication is setup but the slave server is down)

I noticed several times, at different times and dates, that no lines are logged in the OpenLDAP access log for a few minutes, as if it was frozen. I try to figure out what could explain that ? I checked the usual system counters (CPU, IO, Memory, free disk space, ...) as well as the running processes, but did not notice anything suspicious.

Since the OpenLDAP server runs in a virtual machine, I'm currently trying to monitor the ESX hypervisor. Any idea of what could go wrong ?