Coming back to this after few months as the problem remained as unsolved for us. We may have found a way forward.
The feature added into OpenLDAP 2.5 to control the size of IDLs seems to address our performance issue. The specific change is this: https://bugs.openldap.org/show_bug.cgi?id=8977.
As per the documentation
5.2.6.1. olcBkMdbIdlExp <exponent>
Specify a power of 2 for the maximum size of an index slot. The default is 16, yielding a maximum slot size of 2^16 or 65536. The specified value must be in the range of 16-31.
This setting helps with the case where certain search filters are slow to return results due to an index slot having collapsed to a range value. This occurs when the number of candidate entries that match the filter for the index slot exceed the configured slot size.
If this setting is decreased on a server with existing MDB databases, each db will immediately need its indices to be rebuilt while slapd is offline with the "slapindex -q -t" command.
If this setting is increased on a server with existing MDB databases, each db will need its indices rebuilt to take advantage of the change for indices that have already been converted to ranges.
As we migrated to MDB and defined bigger size for the index slot we can now see that the index does not collapse anymore and performance remains stable.
We will still conduct more testing to verify that this is definitely the case.