Hi,
Has anyone used LVM snapshot as a backup method when downtime isn't allowed and slapcat takes hours to finish?
I have to find a way to backup the LDAP without bring it down or set it in ReadOnly mode, thus I thought about the LVM snapshot.
I can run LVM snapshot every day and then run db_recover -c for making the snapshot LDAP DB in a good state.
Am I missing something in the above idea?
Cheers, Pavlos
Pavlos Parissis wrote:
PP> Has anyone used LVM snapshot as a backup method when PP> downtime isn't allowed and slapcat takes hours to finish?
I'd have thought it worth investigating _why_ slapcat takes so long. Unless your database is truly gargantuan, very slow slapcat probably indicates something is misconfigured, or the hardware is underspecified.
PP> I have to find a way to backup the LDAP without bring PP> it down or set it in ReadOnly mode, thus I thought about PP> the LVM snapshot.
A clean way to do this would be to set up a slave whose sole purpose is to replicate from the main database, freeze, back-up, rinse and repeat. But if you have a performance problem with your master, hitting it with something that keeps stopping and starting a syncrepl consumer is likely to make that problem bigger.
PP> I can run LVM snapshot every day and then run db_recover -c for PP> making the snapshot LDAP DB in a good state.
I'll bow to the greater wisdom of those who know the innards of BDB, but I would expect a dirty slapcat from a running server to be more likely to give you a restore-able backup than a 'db_recover'ed LVM snap (for a start the LDIF will be portable, whereas the snapshot will be tied to the specific machine architecture and BDB version of the system it was taken from). But neither of these are a particularly brilliant idea.
Cheers
Duncan
On Tue, 07 Oct 2008 18:39:07 +0100 Duncan Gibb Duncan.Gibb@SiriusIT.co.uk wrote:
Pavlos Parissis wrote:
PP> Has anyone used LVM snapshot as a backup method when PP> downtime isn't allowed and slapcat takes hours to finish?
I'd have thought it worth investigating _why_ slapcat takes so long. Unless your database is truly gargantuan, very slow slapcat probably indicates something is misconfigured, or the hardware is underspecified.
Well 20M entries in 600GB of db is the reason of the delay.
PP> I have to find a way to backup the LDAP without bring PP> it down or set it in ReadOnly mode, thus I thought about PP> the LVM snapshot.
A clean way to do this would be to set up a slave whose sole purpose is to replicate from the main database, freeze, back-up, rinse and repeat.
That was my initial thought as well but I can't use another server for operational reasons.
But if you have a performance problem with your master, hitting it with something that keeps stopping and starting a syncrepl consumer is likely to make that problem bigger.
PP> I can run LVM snapshot every day and then run db_recover -c for PP> making the snapshot LDAP DB in a good state.
I'll bow to the greater wisdom of those who know the innards of BDB, but I would expect a dirty slapcat from a running server to be more likely to give you a restore-able backup than a 'db_recover'ed LVM snap (for a start the LDIF will be portable, whereas the snapshot will be tied to the specific machine architecture and BDB version of the system it was taken from). But neither of these are a particularly brilliant idea.
I've just made a backup using LVM snapshot like this (for testing only)
1) add 100.000 entries 2) create 2 lvm snapshots, one for db and for logs 3) mount the LVM snapshots and rsync the data to another disk 4) umount and delete LVM snapshots 5) stop the LDAP 6) delete all the dbs and logs 7) rsync from the backup dir 8) db_recover 9) start ldap and check if all entries are there
It worked but I want to do more tests, like load a test of LDAP Write operations while the LVM snapshot is being created and few other things
Cheers
Duncan
-- Duncan Gibb, Technical Director Sirius Corporation plc - The Open Source Experts http://www.siriusit.co.uk/ Tel: +44 870 608 0063
Pavlos Parissis wrote:
PP> Well 20M entries in 600GB of db is the reason of the delay.
OK; that is quite large.
DG> A clean way to do this would be to set up a slave whose DG> sole purpose is to replicate from the main database, DG> freeze, back-up, rinse and repeat.
PP> That was my initial thought as well but I can't use another PP> server for operational reasons.
You mean you have to make this backup but there's no budget for doing it properly? Assuming you only need to back up a few times a day, a low-spec or recycled server will be enough.
PP> I've just made a backup using LVM snapshot like this (for testing only)
PP> 1) add 100.000 entries PP> 2) create 2 lvm snapshots, one for db and for logs PP> 3) mount the LVM snapshots and rsync the data to another disk PP> 4) umount and delete LVM snapshots PP> 5) stop the LDAP PP> 6) delete all the dbs and logs PP> 7) rsync from the backup dir PP> 8) db_recover PP> 9) start ldap and check if all entries are there
PP> It worked but I want to do more tests, like load a test of LDAP PP> Write operations while the LVM snapshot is being created and few PP> other things
This may well work, but each backup you make by doing an LVM snapshot underneath BDB takes a similar gamble as pulling the power cords out from your server while it's running. Most of the time, the database will be recoverable, but you can bet that when it matters it won't be. If you automate the process of taking, *testing* and archiving these snapshots such that you always have a known-good, recent-enough copy, you will probably be OK. But I would not be comfortable with this as a backup strategy for valuable data. I would prefer a second machine, or failing that a second instance of OpenLDAP on the same machine.
Cheers
Duncan
openldap-technical@openldap.org
-
Duncan Gibb -
Pavlos Parissis