Quick Start

Download and extract the s3napback package. There's no "make" or any such needed, so just extract it to some convenient location (I use /usr/local/s3napback).

Prerequisites

• Java 1.5 or above. Note that you must use the standard Sun JDK (or JRE); libgcj will not work. (The s3napback script just executes whatever "java" it finds in the path, so you may need to adjust your path so that the Sun java takes precedence. On some systems there are symlinks in /etc/alternatives that point at one version or the other.)

• gpg (if you want encryption)

• Date::Format
• Config::ApacheFormat
• Log::Log4Perl

On many systems you can install the latter three packages from the command line like this:

sudo cpan Date::Format Config::ApacheFormat Log::Log4perl

S3 configuration

Configure s3napback with your S3 login information by creating a file called e.g. /usr/local/s3napback/key.txt containing

key=your AWS key
secret=your AWS secret

You probably want to take care that this file is readable only by the user that s3napback will be running as, so chown and chmod 600 it, as appropriate.

GPG configuration

Note: if you don't care about encryption, you can skip this section, and simply don't specify a GpgRecipient in the configuration file below.

You'll need a GPG key pair for encryption. Create it with

gpg --gen-key

Since you'll need the secret key to decrypt your backups, you'll obviously need to store it in a safe place (see  the GPG manual).

If you'll be backing up a different machine from the one where you generated the key pair, export the public key:

gpg --export me@example.com > backup.pubkey

and import it on the machine to be backed up:

gpg --import backup.pubkey
gpg --edit-key backup@example.com

then (in the gpg key editing interface), "trust" the key.

If you want to run s3napback as root (e.g., from cron), you'll want to make sure that the right gpg keyring is used. Gpg will look for a keyring under ~/.gnupg, but on some systems /etc/crontab sets the HOME environment variable to "/"; consequently during the cron job gpg may look in /.gnupg instead of /root/.gnupg. Thus, you may want to change /etc/crontab to set HOME to "/root"; or actually create and populate /.gnupg; or just use the GpgKeyring option in the s3napback config file to specify a keyring explicitly.

Defining backup sets

Create a configuration file something like this (descriptions of the options are here, if they're not entirely obvious):

DiffDir /usr/local/s3napback/diffs
Bucket dev.davidsoergel.com.backup1
GpgRecipient backup@davidsoergel.com
S3Keyfile /usr/local/s3napback/key.txt
ChunkSize 25000000

# make diffs of these every day, store fulls once a week, and keep two weeks
<Cycle daily-example>
	#CycleType SimpleCycle  # this is the default
	Frequency 1
	Diffs 6
	Fulls 2

	Directory /etc
	Directory /home/build
	Directory /home/notebook
	Directory /home/trac
	Directory /usr
	Directory /var
</Cycle>

# make full backups of these every day on a decaying schedule.
# Keep a total of five backups, spread out such that the oldest
# backup is between 8 and 16 days old.  Each additional disc 
# increases the age of the oldest backup by a factor of two.
<Cycle decaying-example>
	CycleType HanoiCycle
	Frequency 1
	Discs 5

	Directory /etc
	Directory /home/build
	Directory /home/notebook
	Directory /home/trac
	Directory /usr
	Directory /var
</Cycle>

# make diffs of these every week, store fulls once a month, and keep two months
<Cycle weekly-example>
	Frequency 7
	Diffs 3
	Fulls 2

	Directory /bin
	Directory /boot
	Directory /lib
	Directory /lib64
	Directory /opt
	Exclude /opt/useless
	Directory /root
	Directory /sbin
</Cycle>

# make a diff of this every day, store fulls once a week, and keep eight weeks
<Directory /home/foobar>
	Frequency 1
	Diffs 6
	Fulls 8

	Exclude /home/foobar/wumpus
</Directory>

# backup an entire machine
<Directory />
	Frequency 1
	Diffs 6
	Fulls 8

	Exclude /proc
	Exclude /dev
	Exclude /sys
	Exclude /tmp
	Exclude /var/run
</Directory>

# store a MySQL dump of all databases every day, keeping 14.
<MySQL all>
	Frequency 1
	Fulls 14
</MySQL>

# store a MySQL dump of a specific database every day, keeping 14.
<MySQL mydatabase>
	Frequency 1
	Fulls 14
</MySQL>

# store a full dump of all Subversion repos every day, keeping 10.
<SubversionDir /home/svn/repos>
	Frequency 1
	Fulls 10
</SubversionDir>

# store a diff of a specific Subversion repo every day, with a full dump every fourth day, keeping 10 fulls.
<Subversion /home/svn/repos/myproject>
 	Frequency 1
	Phase 0
	Diffs 3
	Fulls 10
</Subversion>

Run the backup

To run it, just run the script, passing the config file with the -c option:

./s3snapback.pl -c s3napback.conf

That's it! You can put that command in a cron job to run once a day.

Logging

The log destinations and formatting are specified in s3napback.logconfig included in the package archive. Log rotation and emailing work too as long as you install Log::Dispatch (sudo cpan Log::Dispatch) and Mail::Sender (sudo cpan Mail::Sender), as needed.

Next: Principles of Operation