I have a both a production cluster and a qa cluster of servers. Each cluster is setup with multi-master (mirror-mode) delta-sync replication.
On a weekly basis, I need to reload the data in qa from production. My problem is that, after successfully loading the dump, there is an epic flurry of replication events which tend to exhaust my burst balances in AWS. While I could request more resources (at a greater cost), I first want to verify that I have a reasonable process.
On one of the production servers, I generate a dump:
/usr/sbin/slapcat -F /etc/openldap/slapd.d -b dc=umd,dc=edu -l dump.ldif
On each of the qa servers (simultaneously):
1) fetch the dump
2) delete the dc=umd,dc=edu and cn=accesslog LMDB files
3) /usr/sbin/slapadd -F /etc/openldap/slapd.d -b dc=umd,dc=edu -q -w -S 0 -l dump.ldif
Is this a reasonable approach?
Is the use of the ‘-S’ flag correct?
Should I be modifying the dump in any manner (e.g. deleting the entryCSN attributes)?
Thanks for any advise.
// John Pfeifer Division of Information Technology University of Maryland, College Park
Hello John,
I also have a production and a dev/test environment. I do the reload from prod into dev/test on a daily basis. I do not use the "-w -S 0" sequence on the slapadd in the dev/test environment. You are in a multimaster configuration, so using "-S 0" tells everyone that the entry was changed by someone else and that they need to sync up. (I admit I have not read that section of the code to be sure my belief is fact. Others may disagree with me)
My slapadd is simply:
slapadd -q -b dc=my,dc=domain -l production_ldap.ldif
~ Frank
On Thu, Jun 11, 2020 at 10:47 AM John C. Pfeifer pfeifer@umd.edu wrote:
I have a both a production cluster and a qa cluster of servers. Each cluster is setup with multi-master (mirror-mode) delta-sync replication.
On a weekly basis, I need to reload the data in qa from production. My problem is that, after successfully loading the dump, there is an epic flurry of replication events which tend to exhaust my burst balances in AWS. While I could request more resources (at a greater cost), I first want to verify that I have a reasonable process.
On one of the production servers, I generate a dump:
/usr/sbin/slapcat -F /etc/openldap/slapd.d -b dc=umd,dc=edu -l
dump.ldif
On each of the qa servers (simultaneously):
1) fetch the dump 2) delete the dc=umd,dc=edu and cn=accesslog LMDB files 3) /usr/sbin/slapadd -F /etc/openldap/slapd.d -b dc=umd,dc=edu -q
-w -S 0 -l dump.ldif
Is this a reasonable approach?
Is the use of the ‘-S’ flag correct?
Should I be modifying the dump in any manner (e.g. deleting the entryCSN attributes)?
Thanks for any advise.
// John Pfeifer Division of Information Technology University of Maryland, College Park
Frank Swasey wrote:
Hello John,
I also have a production and a dev/test environment. I do the reload from prod into dev/test on a daily basis. I do not use the "-w -S 0" sequence on the slapadd in the dev/test environment. You are in a multimaster configuration, so using "-S 0" tells everyone that the entry was changed by someone else and that they need to sync up. (I admit I have not read that section of the code to be sure my belief is fact. Others may disagree with me)
Correct. Both -w and -S 0 should be omitted when reloading a valid slapcat output from an already live provider.
My slapadd is simply:
slapadd -q -b dc=my,dc=domain -l production_ldap.ldif
~ Frank
On Thu, Jun 11, 2020 at 10:47 AM John C. Pfeifer <pfeifer@umd.edu mailto:pfeifer@umd.edu> wrote:
I have a both a production cluster and a qa cluster of servers. Each cluster is setup with multi-master (mirror-mode) delta-sync replication. On a weekly basis, I need to reload the data in qa from production. My problem is that, after successfully loading the dump, there is an epic flurry of replication events which tend to exhaust my burst balances in AWS. While I could request more resources (at a greater cost), I first want to verify that I have a reasonable process. On one of the production servers, I generate a dump: /usr/sbin/slapcat -F /etc/openldap/slapd.d -b dc=umd,dc=edu -l dump.ldif On each of the qa servers (simultaneously): 1) fetch the dump 2) delete the dc=umd,dc=edu and cn=accesslog LMDB files 3) /usr/sbin/slapadd -F /etc/openldap/slapd.d -b dc=umd,dc=edu -q -w -S 0 -l dump.ldif Is this a reasonable approach? Is the use of the ‘-S’ flag correct? Should I be modifying the dump in any manner (e.g. deleting the entryCSN attributes)? Thanks for any advise. // John Pfeifer Division of Information Technology University of Maryland, College Park
-- I am not young enough to know everything. - Oscar Wilde (1854-1900)
openldap-technical@openldap.org