Tonight I’ve spent some time playing a bit more with my supernas (FreeNAS). I have managed to learn how to make an automated snapshot style rotating backup system. The key is in the use of UNIX hard-links feature. Deriving from this info and this script (which was wrong by the way), I have built a script that automatically pulls data from my NAS onto my online storage (dreamhost) and rotating it every day (keeping 5 snapshots max).


#!/bin/sh
#
# Easy Automated Snapshot-Style Drive-snapshot using Rsync
# Copyright (C) 2008 Sylvain Rebaud (sylvain at rebaud dot com)
#
# Based on Dan Merschi & Mikes Handy
# Easy Automated Snapshot-Style Backups with Linux and Rsync
# http://www.mikerubel.org/computers/rsync_snapshots/
#
#-- Verify input arguments --
if [ $# -lt 2 ] ; then
echo >&2 “usage: $0 [source] [dest]”
exit 1
fi
#– Path to backup directory –
BACKUP=”$2/snapshot”
#– Rotating-filesystem-snapshot –
rm -fR ${BACKUP}5
rm -f ${BACKUP}5.date
for i in 4 3 2 1
do
y=`expr $i + 1`
if [ -d ${BACKUP}${i} ]; then mv -f ${BACKUP}${i} ${BACKUP}${y} ; fi
if [ -d ${BACKUP}${i}.date ]; then mv -f ${BACKUP}${i}.date ${BACKUP}${y}.date ; fi
done
date > ${BACKUP}1.date

#– Taking filesystem snapshot using rsync & hardlinks (backup1) –
#
rsync -aHvz –exclude=”.*” –delete –rsh=”ssh -c arcfour -o Compression=no -x” –link-dest=../snapshot2 “$1″ ${BACKUP}1/
#
#– End Script –


Then I set up freeNAS to execute a cron job using the following command:

/usr/bin/ssh soothe@plutinosoft.com "./backup/backup.sh root@sylvain.homeip.net:/mnt/bigboy/share/Photos/ backup/Photos"

This tells FreNAS to login to my site (plutinosoft.com) using a RSA key with no passphrase and execute the remote backup script which in turns connects back to my FreeNAS server (using Dynamic DNS host and a RSA key with no passphrase as well) and pull/backup/sync-up the data locally using a rotation of 5 folders.

Phew!

Finally, snapshot1 should be the latest full sync and snapshot2 disk space should be for the differences from snapshot1 (meaning only the files that were deleted later that are not present in snapshot1 anymore), and so on until snapshot5. So total amount of space used is really just a full snapshot plus the changes made and not 5x the full snapshot.

Neat, isnt’it?

Update: I have updated the script to not rely on cp -al which is not available on Freenas and instead use the rsync --link-dest feature recently implemented. This permits to use this script to backup data on a Freenas box as well. I also added a snapshot1.date file to keep track of when the last backup was actually made.