Hi!
I am running OpenLDAP 2.4.47 with significant write activity (in the hundreds of modify/add/del ops per second), and a large volume of data compared to available RAM (about 20G)
For backups, I am trying to do a live slapcat dump to maintain availability, and encountered the growth in MDB database size described in those ITS:
http://www.openldap.org/its/index.cgi/Software%20Bugs?id=7904 http://www.openldap.org/its/index.cgi/Software%20Bugs?id=8226
I am however a little puzzled over the rtxnsize option. My understanding is that this option splits large read transactions (such as backups) into smaller transactions, allowing freed pages to be reused before the end of the full operation. This is critical for me because backups take more than 10 minutes and a significant portion of my MDB file (100G) gets filled up during that time.
However, setting rtxnsize to a smaller value from the default does not seem to have any effect on the growth of my MDB file. I have made several tries with different rtxnsizes from 1M to 100, starting from a freshly rebuilt MDB database every time.
I saw that the rtxnsize option was added to handle an issue with replication, so perhaps it was never supposed to help in my particular case. For the whole duration of a slapcat backup, I see the same transaction ID from the slapcat process in mdb_stat -r.
I noticed that using ldapsearch for backups behaves much better (MDB only grows by a few thousand pages in my workload), but takes a while longer, regardless of the rtxnsize setting
Is it normal that rtxnsize does not affect slapcat dumps while slapd is running, or am I missing something?
How would you handle live backups of a database with lots of write activity?