Obsolete:Amanda
Amanda is a network backup solution based on a client / server architecture. It has been almost fully puppetized (see details below). For Amanda to work, the server (currently tridge.wikimedia.org) needs to be running, and each server will need the amanda-client package installed. The backups are initialized by the server itself, using both Cron and some intelligence provided by Amanda (which will decide what to backup in which order, based on network usage, size of backups and so on. The clients and path to backup are defined on the server, but the configuration file is managed by puppet.
See also:
Configuration Management
Everything is in puppet in puppet/manifest/backups.pp. Configuration files are in puppet/files/backup/* and puppet/templates/backups
Backup location: on the backup::server: /data/amanda/$strategy/slots/...
Backup Strategy
I have setup a pretty basic backup strategy, based on industry standards. There are three classes of backups:
- Daily backups: Those are rotating 6 tapes with incremental backups every day of the week.
- Weekly backups: 4 tapes in rotation with incremental compressed backup.
- Monthly backups: 12 tapes in rotation to cover the year. Each month will be a full backup, which will also be compressed.
These strategy can be modified somewhat easily from Puppet, but the virtual tapes will have to be regenerated (details below).
Setting up a new backup client
Conveniently, this will all be taking place on the puppet server!
- edit site.pp, find the host you want to backup, add 'include backup::client;' (See Mchenry host definition for an example).
- edit puppet/files/backup/disklist to specify what the client is, and what the path is/are (I am hoping to puppetize this soon)
- run puppetd -tv on the client to make sure everything is in order. Note: you might have to run that command 2 or 3 times, depending... So is puppet. It doesn't necessarily do things in order...
Setting up a new server
This should not happen too often, but it is easy enough...
- edit site.pp and add 'include backup::server' to the new host.
- Change puppet/manifest/backups.pp to reflect the new server.
everything else should happen magically.
Changing the backup strategy
This is a tad more difficult, but doable. It will require a bit of work on the amanda server as well.
The first step is to change the configuration in puppet. This should be somewhat self-explanatory. I have created a little script to create and catalog the virtual tapes. In order to have it run again, simply remove the symlink called data in the slots directory of the profile you are changing. Alternativaly, you can move the excisting backups somewhere and completely remove the backup destination.
On the next puppet run the structure will be recreated. Worse case scenario, you can wipe /etc/amanda/* and /data/amanda/<profile> and puppet will recreate everything for you. However, you will loose all the backups that were there, so use with caution ;)
Restoring from backup
This is what makes me sad. On pre-lucid version of ubuntu, Amanda was compiled without ssh libraries. So until amanda-common is re-packaged, amrecover will not be useable. However, it is still possible to retrieve backups the old fashion way, from the amanda server. Simply go to the virtual tape you want to restore (/data/amanda/Wikimedia-[Daily,Weekly,Monthly]/slots/slotXX and run the following:
dd if=<tape> bs=32k skip=1 | /bin/gzip -dc | /bin/tar -xpGf - ...
If you forget the command, running head on the virtual tape will also reveal the necessary magic.
Caveats and todos
- Missing ssh libraries in pre-lucid version of amanda-commons (requires re-packaging)
- need to make exported resources work so that the backup client and paths to backup can be defined from the node definition is site.pp