Skip to content

Rolling incremental backups

June 1, 2007

Why do you take back ups?

  • User error
  • Hardware failure
  • Disaster Recovery
  • Admin Error

With ZFS using redundant storage and plenty of snapshots my data should be safe from the first two. However that still leaves two ways all my data could be lost if I don’t take some sort of back up.

Given the constraints I have my solution is to use my external USB disk containing a standalone zpool and then use zfs send and receive via this script to send all the file systems I really care about to the external drive.

To make this easier I have put all the filesystems into another “container” filesystem which has the “nomount” option set so it is hidden from the users. I can then recursively send that file system to the external drive. Also to stop the users getting confused by extra file systems appearing and disappearing I have set the mount point on the external disk to “none”.

The script only uses the snapshots that are prefixed “day” (you can change that with the -p (prefix) option) so that it reduces the amount of work that the script does. Backing up the snapshots that happen every 10 minutes on this system does not seem worth while for a job I will run once a day or so.

The really cool part of this is that once I had the initial snapshot on the external drive every back up from now on will be incremental. A rolling incremental backup. How cool is that.

# time ./zfs_backup tank/fs/safe removable/safe         
real    12m10.49s  user    0m11.87s  sys     0m12.32s  
#   zfs list  tank/fs/safe removable/safe        
NAME             USED  AVAIL  REFER  MOUNTPOINT  
removable/safe  78.6G  66.0G    18K  none  
tank/fs/safe    81.8G  49.7G    18K  /tank/fs  
#  

The performance is slightly disappointing due to the number of transport errors that are reported by the usb2scsa layer but the data is really on the disk so I am happy.

Currently I don’t have the script clearing out the old snapshots but will get that going later. The idea of doing this over ssh to a remote site is compelling when I can find a suitable remote site.

Advertisements

From → bsc, homeserver, Solaris

One Comment
  1. [Trackback] My home server is taking a bit of a battering of late. I keep
    tripping over bug 6566921
    which I can work around by not running my zfs_backup
    script locally. I have an updated
    version which will send the snapshots over an ssh pipe to a
    remot…

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: