Hello! Long time no see, unfortunately!
I have something to share, when I was recently looking into backup programs for Linux I ran into Borg, Restic, and Kopia.
I came across them in the past but back then found them too complicated and thought what was the appeal of a CLI backup program. I opted for a GUI program called Cloudberry Backup, its not bad - nice GUI, multiple jobs, supports of a lot of common backup destinations like S3, local, B2 storage, etc.
Cloudberry Backup has been good so far, except the backup speed is slow, with the free version there is a 100 GB limit, I am concerned about the longevity of the software, and you download and install a plain .rpm without a corresponding repository (no automated updates via your package manager). So, I figured it was finally time to take a look into the popular, robust, Linux backup programs available. I ended up comparing Borg, Restic, and Kopia to see which would suit my needs most. Please note each program has more pros and cons than these, I am listing the quick points of interest from my perspective. If you are also evaluating backup software I encourage you to take a deeper look as well.
Kopia
- A little too new for my liking.
- Less popular.
- Less information online, and docs felt incomplete.
- KopiaUI only supported backing up to one repository at a time.
- Not in OpenSUSE repos, I had to install the Flatpak.
- Supports all storage backends I would want.
Borg
- Borg must be running on the destination machine, meaning you wont have SFTP, B2, S3, etc. available.
- Very good deduplication.
- Mature and robust software.
Restic
- I like that it is written in Go.
- I like that is a single static binary, this gives me peace of mind I should always be able to run Restic and restore my files provided I have a compatible binary available in the future to run.
- Supports all storage backends I personally need, B2, S3, and local hard disk.
- Mature and robust software.
I ultimately ended up deciding to use Restic, it did not have quite as good deduplication as Borg but it is a mature and robust software that fits my needs and the backends I wish to use.
After some testing I came up with the following configuration. I installed Restic and Swaks from my distribution repository, wrote a script to handle automated backups of my files to each of the backend repositories I need, and then scheduled the backups to run on a user level systemd timer.
Warning! If you use my script and service file, read through everything below carefully and assure you adjust it to your setup. Assure Restic and Swaks is installed.
Create the timer and service file under .config/systemd/user/ then run the below commands.
$ systemctl --user daemon-reload
$ systemctl --user enable restic-backup.timer
$ systemctl --all --user list-timers
restic-backup.timer
[Unit]
Description=Schedule restic backup script
[Timer]
#Execute job if it missed a run due to machine being off
Persistent=true
#Run after boot for the first time
OnBootSec=15min
#Run every x amount of time
OnUnitActiveSec=45m
#File describing job to execute
Unit=restic-backup.service
[Install]
WantedBy=timers.target
restic-backup.service
[Unit]
Description=Run Restic backup script
[Service]
Type=simple
ExecStart=/home/daulton/Files/Scripts/restic-backup.sh
[Install]
WantedBy=default.target
restic-backup.sh
| #!/usr/bin/env bash | |
| # Written by: Daulton | |
| # Website: https://daulton.ca | |
| # Github: https://github.com/jeekkd | |
| # License: 2-clause BSD license | |
| # Purpose: Script Restic backups for myself, only meant for me but modify it if its useful to you. | |
| # Notes: Assure Restic and Swaks is installed. | |
| # BSD 2-Clause License | |
| # | |
| # Copyright (c) [2022], [backup-restic.sh] | |
| # | |
| # Redistribution and use in source and binary forms, with or without | |
| # modification, are permitted provided that the following conditions are met: | |
| # | |
| # 1. Redistributions of source code must retain the above copyright notice, this | |
| # list of conditions and the following disclaimer. | |
| # | |
| # 2. Redistributions in binary form must reproduce the above copyright notice, | |
| # this list of conditions and the following disclaimer in the documentation | |
| # and/or other materials provided with the distribution. | |
| # | |
| # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" | |
| # AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE | |
| # IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE | |
| # DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE | |
| # FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL | |
| # DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR | |
| # SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER | |
| # CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, | |
| # OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE | |
| # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. | |
| ######## Variables ######## | |
| # Restic repo password, same for each repo | |
| export RESTIC_PASSWORD= | |
| # Backup the following directory | |
| backupDirList=/home/you/Important/ | |
| # S3 bucket path | |
| s3Bucket="https://example.com/example" | |
| # B2 bucket name | |
| B2Bucket= | |
| # Minio S3 storage access | |
| # https://restic.readthedocs.io/en/stable/030_preparing_a_new_repo.html#amazon-s3 | |
| export AWS_ACCESS_KEY_ID= | |
| export AWS_SECRET_ACCESS_KEY= | |
| # B2 storage access | |
| # https://restic.readthedocs.io/en/stable/030_preparing_a_new_repo.html#backblaze-b2 | |
| export B2_ACCOUNT_ID= | |
| export B2_ACCOUNT_KEY= | |
| # Mount point for second disk | |
| mountPoint=/mnt/data | |
| # Secondary disk to mount | |
| secondaryDisk=/dev/sdb1 | |
| # Local mount point Restic repository name | |
| localRepoName=restic-repo | |
| # SMTP server hostname | |
| smtpHostname=example.com | |
| # Email account sending as | |
| smtpFrom=sender@example.com | |
| smtpFromPassword= | |
| # Send email to the following account | |
| smtpTo=you@example.com | |
| ########################### | |
| # control_c() | |
| # Trap Ctrl-C for a quick and clean exit when necessary | |
| control_c() { | |
| echo "Control-c pressed - exiting NOW" | |
| exit 1 | |
| } | |
| # Trap any ctrl+c and call control_c function provided through functions.sh | |
| trap control_c INT | |
| # Exit and send email alert if error | |
| check_exit_code_email() { | |
| if [ $? -ne 0 ]; then | |
| echo "Warning: Fatal error or source files could not be read." | |
| echo "Exit code: $?" | |
| swaks --to "$smtpTo" --from "$smtpFrom" -s "$smtpHostname":587 -tls -au alerts@tetmail.ca -ap "$smtpFromPassword" --header "Subject: Restic backup error" --body ~/restic.log > /dev/null 2>&1 | |
| exit 1 | |
| fi | |
| } | |
| # Check if local disk is mounted as read-only. | |
| check_if_drive_ro() { | |
| diskCheckResult=$(grep "[[:space:]]ro[[:space:],]" /proc/mounts | grep "$mountPoint") | |
| [ "$diskCheckResult" = "" ] && return 3 | |
| } | |
| # Redirect stdout ( > ) into a named pipe ( >() ) running "tee" | |
| exec > >(tee -i restic.log) | |
| # Without this, only stdout would be captured - i.e. your | |
| # log file would not contain any error messages. | |
| # SEE (and upvote) the answer by Adam Spiers, which keeps STDERR | |
| # as a separate stream - I did not want to steal from him by simply | |
| # adding his answer to mine. | |
| exec 2>&1 | |
| echo "Backup beginning at: $(date)" | |
| echo | |
| # Check if Restic is already running | |
| check_process() { | |
| [ "$1" = "" ] && return 0 | |
| [ $(pgrep -n $1) ] && return 1 || return 0 | |
| } | |
| check_process "restic" | |
| if [ $? -eq 1 ]; then | |
| echo "Warning: Restic already running, exiting.." | |
| exit 1 | |
| else | |
| echo "Information: Another instance of Restic is not currently running, continuing." | |
| fi | |
| run_restic_local() { | |
| echo "###" | |
| echo "# Local backup to second HDD" | |
| echo "###" | |
| echo | |
| restic -r "$mountPoint"/"$localRepoName" --verbose backup "$backupDirList" | |
| check_exit_code_email | |
| restic -r "$mountPoint"/"$localRepoName" forget --keep-hourly 6 --keep-daily 14 --keep-weekly 4 --keep-monthly 3 | |
| restic -r "$mountPoint"/"$localRepoName" check | |
| check_exit_code_email | |
| echo | |
| } | |
| mount | grep -q "[[:space:]]$mountPoint[[:space:]]" | |
| if [ $? -eq 0 ]; then | |
| check_if_drive_ro | |
| if [ $? -ne 3 ]; then | |
| echo "Notice: Local disk is mounted ro, repairing.." | |
| sudo umount -f "$mountPoint" | |
| if [ $? -eq 0 ]; then | |
| sudo ntfsfix "$secondaryDisk" | |
| sudo ntfs-3g "$secondaryDisk" "$mountPoint" | |
| if [ $? -eq 0 ]; then | |
| run_restic_local | |
| check_exit_code_email | |
| fi | |
| else | |
| check_exit_code_email | |
| echo "Critical: Unable to umount local disk to remount as rw, exiting." | |
| fi | |
| else | |
| echo "Information: The path $mountPoint is already mounted, doing nothing" | |
| run_restic_local | |
| check_exit_code_email | |
| fi | |
| else | |
| echo "Notice: Mounting local disk at $mountPoint" | |
| sudo ntfsfix "$secondaryDisk" | |
| sudo ntfs-3g "$secondaryDisk" "$mountPoint" | |
| if [ $? -eq 0 ]; then | |
| run_restic_local | |
| check_exit_code_email | |
| fi | |
| fi | |
| echo "###" | |
| echo "# Backup to Minio on TrueNAS" | |
| echo "###" | |
| echo | |
| restic -r s3:"$s3Bucket" --verbose backup "$backupDirList" | |
| check_exit_code_email | |
| restic -r s3:"$s3Bucket" forget --keep-hourly 6 --keep-daily 14 --keep-weekly 4 --keep-monthly 3 | |
| restic -r s3:"$s3Bucket" check | |
| check_exit_code_email | |
| echo | |
| echo "###" | |
| echo "# Backup to Backblaze B2" | |
| echo "###" | |
| echo | |
| restic -r b2:"$B2Bucket" --verbose backup "$backupDirList" | |
| check_exit_code_email | |
| restic -r b2:"$B2Bucket" forget --keep-hourly 6 --keep-daily 14 --keep-weekly 4 --keep-monthly 3 | |
| restic -r b2:"$B2Bucket" check | |
| check_exit_code_email | |
| echo |
