KB450410 – Automated ZFS Snapshots and Replication

Automated ZFS Snapshots and Replication

Scope/Description:

This article will walk through the process of installing and configuring automated zfs snapshots and replication.

The following process works with Ubuntu20.04, CentOS7 and CentOS8.

The tool used to achieve automated snapshots and replication is a open source project called “znapzend”. Its description from the github readme:

ZnapZend is a ZFS centric backup tool to create snapshots and send them to backup locations. It relies on the ZFS tools snapshot, send and receive to do its work. It has the built-in ability to manage both local snapshots as well as remote copies by thinning them out as time progresses.

The ZnapZend configuration is stored as properties in the ZFS filesystem itself.

• znapzend github page
• znapzend home page

Prerequisites:

• 45Drives repository enabled
• ZFS installed and configured
• zfs-auto-snapshot package removed if installed

Steps:

Installation:

Ubnutu 20.04

• Remove zfs-auto-snapshot package if installed

apt purge zfs-auto-snapshot -y

• Install znapzend

apt install znapzend -y

• Note that apt will enable the znapzend service by default, and will reportthe service as failed as no valid config is defined yet. You can clear the failed status on the service until configured.

CentOS 7

• Remove zfs-auto-snapshot if installed

yum remove zfs-auto-snapshot -y

• Install EPEL repository

yum install epel-release -y

• Install znapzend

yum install znapzend -y

CentOS 8

• Remove zfs-auto-snapshot if installed

dnf remove zfs-auto-snapshot -y

• Install EPEL repository

dnf install epel-release -y

• Enable PowerTools

dnf install dnf-plugins-core -y
yum config-manager --set-enabled powertools

• Install znapzend

dnf install znapzend -y

Auto Snapshot Configuration

Configuring znapzend is the same process on Ubnutu20.04LTS,CentOS7 and CentOS8.

• Create znapzend snpashot task
  • The command “znapzendzetup“. See here for a complete description of arguments.
    • Choose source dataset (recursive supported)
    • Choose Schedule and Lifetime (sending side)
• Start znapzend service

Auto Send/Recieve Configuration

Configuring znapzend is the same process on Ubnutu20.04LTS,CentOS7 and CentOS8

• Enable passwordless ssh between servers
• Create znapzend replication task
  • The command “znapzendzetup“. See here for a complete description of arguments.
    • Choose source dataset (recursive supported)
    • Choose reciever dataset
    • Choose Schedule and Lifetime (sending side and receiving side)
• Start znapzend service

Example setup commands

Automated Local Snapshot Task

This example will create an automated snapshot task on a local zpool. Snapshots will be created and destroyed based on the schedule and lifetime explained below.

znapzendzetup create --recursive \
SRC '7d=>1h,30d=>4h,90d=>1d' tank/dataset1

• SRC specifies the Schedule and Lifetime plan and the data set to backup
  • This plan has three retention periods.
    • Snap every 1hour and keep for 7 days
    • Snap every 4 hours and keep for 30 days
    • Snap every 1 day and keep for 90 days.
• recursive snaps all subdatasets as well

Automated Backup Task

This example will create an automated backup task on a local zpool (Source, SRC) and send to a remote zpool (Destination, DST).

Snapshots on the SRC will be created and sent to the backup DST and then removed as per the lifetime plan.

znapzendzetup create --recursive \
--mbuffer=/usr/bin/mbuffer --mbuffersize=1G \
SRC '7d=>1h,30d=>4h,90d=>1d' tank/dataset1 \
DST:a '7d=>1h,30d=>4h,90d=>1d,1y=>1w,10y=>1month' root@backupserver:backup/dataset1

• Use mbuffer to level out the bursty nature of ZFS send and receive features
• ‘tank/dataset1’ is the dataset we want to backup to ‘root@backupserver:backup/dataset1’
• SRC specifies the Schedule and Lifetime plan and the data set to backup
  • tank/dataset1 is the source zpool to backup
  • This plan has three retention periods
    • Snap every 1hour and keep for 7 days
    • Snap every 4 hours and keep for 30 days
    • Snap every 1 day and keep for 90 days

• DST specifies the Schedule and Lifetime plan of the destination dataset
  • Each DST needs an label, as multiple backup locations can be specified. In this case “:a”  is the DST label
  • backup/dataset1 is the destination zpool
  • root@backupserver
  • This plan has three retention periods
    • Snap every 1hour and keep for 7 days
    • Snap every 4 hours and keep for 30 days
    • Snap every 1 day and keep for 90 days.
    • Snap every 1 week and keep for 1 year
    • Snap every 1 month and keep for a year
• recursive sends all subdatasets to the backup location as well

Time Interval Definations

Both intervals and retention periods are expressed in standard units of time or multiples of them. You can use both the full name or a shortcut according to the following table:

• second|sec|s
• minute|min
• hour|h
• day|d
• week|w
• month|mon|m
• year|y

Verification:

Troubleshooting:

• Logging is done in the znapzend systemd service
  • CLI
    • journal -u znapzend.service
  • Houston
    • Services -> Znapzend