So.. Ive been reading up hardcore on Berkeley database administration regarding how to do archival and recovery, and its pretty cool stuff.
Yet I have the following questions:
Database archival will be done on one of our master servers, so its not necessary to keep alot of old transaction logfiles on the replicas, but in order to clean out them out, a periodic checkpoint is necessary.
Initially, I was thinking that we should do this once per day, off hours, but would it make more sense to do this more frequently over the course of the day to keep the checkpoint process less expensive per iteration?
What kinds of metrics should be used to determine how frequently this should be done?
Second, would there be that great of a performance gain when disabling transaction logging altogether?
If I have two master servers keeping, say, a week's worth of transaction logs, a sound archival process, and a backup LDIF, would it make sense to just disable transaction logging altogether across the replicas?
Third, I now understand how to do archival and recovery using the Berkeley tools, but is there a 'best practice' document out there anywhere from someone who utilizes this method that I could take a look at to make sure Im not missing anything?
On Wednesday 20 February 2008 21:00:42 Thomas Ledbetter wrote:
So.. Ive been reading up hardcore on Berkeley database administration regarding how to do archival and recovery, and its pretty cool stuff.
Yet I have the following questions:
Database archival will be done on one of our master servers, so its not necessary to keep alot of old transaction logfiles on the replicas, but in order to clean out them out, a periodic checkpoint is necessary.
Initially, I was thinking that we should do this once per day, off hours,
How would you ensure that the checkpoint occurs in off-hours ?
but would it make more sense to do this more frequently over the course of the day to keep the checkpoint process less expensive per iteration?
IMHO, yes.
What kinds of metrics should be used to determine how frequently this should be done?
This would depend on the frequency of changes, and how long you can afford to have database recovery run for. I usually go with something around 5-10 minutes.
Second, would there be that great of a performance gain when disabling transaction logging altogether?
Yes, but if you ever have an unclean shutdown, you may have to reload/resync from scratch. But, how much performance do you need, and how much are you getting now?
If I have two master servers keeping, say, a week's worth of transaction logs, a sound archival process, and a backup LDIF, would it make sense to just disable transaction logging altogether across the replicas?
If you can afford taking a slave down for a re-import or re-sync, maybe. However, I would sacrifice a bit of performance to avoid this myself.
Third, I now understand how to do archival and recovery using the Berkeley tools, but is there a 'best practice' document out there anywhere from someone who utilizes this method that I could take a look at to make sure Im not missing anything?
Well, I wrote these a long time ago, and while they worked fine when testing them for what I needed, I haven't needed them for anything (besides removing the old transaction logs):
http://svn.mandriva.com/cgi-bin/viewvc.cgi/packages/cooker/openldap/current/... http://svn.mandriva.com/cgi-bin/viewvc.cgi/packages/cooker/openldap/current/...
Regards, Buchan
Buchan Milne writes:
On Wednesday 20 February 2008 21:00:42 Thomas Ledbetter wrote:
but would it make more sense to do this more frequently over the course of the day to keep the checkpoint process less expensive per iteration?
IMHO, yes.
That also makes startup faster after an unclean shutdown, since slapd needs to recover the database in order to start.
What kinds of metrics should be used to determine how frequently this should be done?
This would depend on the frequency of changes, and how long you can afford to have database recovery run for. I usually go with something around 5-10 minutes.
At our site we usually run read-only and then once in a while apply a lot of changes at once - and we care about read and startup speed but not much about update speed. So we've tried even briefer, 2 minutes. Seems to work fine. Need to experiment with taking slapd up and down and see what happens though. checkpoint 1024 2 dbconfig set_flags DB_LOG_AUTOREMOVE
If I have two master servers keeping, say, a week's worth of transaction logs, a sound archival process, and a backup LDIF, would it make sense to just disable transaction logging altogether across the replicas?
If you can afford taking a slave down for a re-import or re-sync, maybe. However, I would sacrifice a bit of performance to avoid this myself.
If you need translation to Berkeley DB parlance, if I've got it right: You don't need _catastrophic_ recovery for the slaves - that implies manual intervension, and then you can just as well use the backup of the master slapd. However if you disable _normal_ recovery on the slaves, then they'll also need manual help to start after an otherwise harmless system or application failure.
openldap-software@openldap.org
-
Buchan Milne -
Hallvard B Furuseth -
Thomas Ledbetter