Backup everything
Automate the process
Confirm backups work
Keep some old backups
Update backups frequently
Protect your backups
Spread your backups

Choose a backup software

Step 1, check the archlinux wiki comparison table, step 2, remove lines with red cells…
That’s how you met a great tool, BorgBackup.
Not many backup tools let you use those features together :

  • Encryption
  • Data deduplication
  • Compression

The workflow of the tool is git-like, you init a repo with encryption, then create a backup.
Backups are mountable with mount subcommand.
I use it since 5 years for every host I manage without any problem, my older backups are still working !

Automate

Let’s script a bit to use borg on multiple hosts and centralize backups.
Store your backups on a distant host, borg uses SSH.
Use RAID to protect your datas against storage failure.

On your server, create a directory and init your repo per host. Then restrict its SSH public key to borg.

cat >> /root/.ssh/authorized_keys <<"EOF"
command="borg serve --restrict-to-path /backups/host1",no-pty,no-agent-forwarding,no-port-forwarding,no-X11-forwarding,no-user-rc ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCth9LMtb5y1qQHENUWLEhYF8gi6NnMwM69oj8Q80/HSM3PrNT3Wf81Q6AKw4f2miqfisxS5p7zCvjf7Yle3CQY0E5NlF/ZulP4aRShjH09N3STAWryUy5wlExmcLp+L07Tq9VvqHF0aSObdb7voLnOKemvt/xDXwR0UTl/gCdueKWLDZ+HiZc7cnAKhtI/KlYKy6nIJCDOyHVRbBbEyuTm78JHxueG2BR3KiZO46XQbuVsEFx8v7AxvCUEi/a+2r3WmsYP1ux3rZ4Gs1JeK2YCck31o/dcK9ZToVSrxD6EP/HH3h/ci0sWgt8goROhqaIjCrmLQKjPMUKgSoirQRO9 root@host1
EOF

Borg gets the encryption passphrase required from env var BORG_PASSPHRASE, used to create a backup from host or query, list, mount them on server.
Create a file in your root directory which contains your passphrase. You will source it to manage your repos.

# That's for demonstration, edit the file manually
cat > /root/.backup_init <<"EOF"
BORG_PASSPHRASE='yourpassphrase'
EOF
chmod 400 /root/.backup_init

On your hosts, create a backup script.

#!/bin/bash

export BORG_PASSPHRASE='yourpassphrase'
REPOSITORY=root@backup-srv:/backups/host1

# Backup entire system
PATH=/

/usr/bin/borg create -v --stats --progress          \
    --compression lz4                               \
    $REPOSITORY::'{hostname}-{now:%Y-%m-%d}' $PATH  \
    --exclude '/dev/*'                              \
    --exclude '/proc/*'                             \
    --exclude '/sys/*'                              \
    --exclude '/tmp/*'                              \
    --exclude '/mnt/*'                              \
    --exclude '/run/*'                              \
    --exclude '/media/*'                            \
    --exclude 'lost+found'

/usr/bin/borg prune -v $REPOSITORY --prefix '{hostname}-'    \
    --keep-daily=7 --keep-weekly=4 --keep-monthly=6

The prune subcommand purge backups and keep number of backup you set.

Use cron or a systemd-timer to automate your backups.
That’s all you need !

Test

Lets test backups on the server. cesium is a server I backup.

# Init env key
$ source /root/backup_init

# List backups
$ borg list cesium
prodpeda-cesium-2019-12-31           Tue, 2019-12-31 06:00:07 [e6f0bf40b20b6b04d26a1f6f52d8a845298aac21b49c018300293893c37ff0e3]
prodpeda-cesium-2020-01-31           Fri, 2020-01-31 06:00:07 [cbf5aaba1887b4d710cb36cde57df02e2ad188747745b28ed5ed49e647ac76db]
prodpeda-cesium-2020-02-28           Fri, 2020-02-28 06:00:07 [fc6f401ee4732be1b37172b810afe84e426d8458d18ba5ad9c24f0b1afdedce9]
prodpeda-cesium-2020-03-27           Fri, 2020-03-27 06:00:07 [3392f2a4b1fe89ded8b8976b9a86371e20292517ae79d6fba9004f8f6c14e92c]
prodpeda-cesium-2020-03-31           Tue, 2020-03-31 06:00:07 [6d1f0368096e3dff20af53f739eba6312bfb0a159e3d0bc6f153afe9ce3a7c2a]
prodpeda-cesium-2020-04-03           Fri, 2020-04-03 06:00:07 [c0813371b42f6974bf7efc408dd71d85edeef726e97196f914b3617d901f27fe]
prodpeda-cesium-2020-04-10           Fri, 2020-04-10 06:00:07 [5f5d5149d05e10830adae60e805e62967248f646e35ead1e8411e035da537b9a]
prodpeda-cesium-2020-04-17           Fri, 2020-04-17 06:00:08 [8ec1024cc60a84d99fb806c23ce374e50b173d76639c6336140600c202260c3c]
prodpeda-cesium-2020-04-22           Wed, 2020-04-22 06:00:08 [a83c0ba885c53efff47456490ed907308f712c1436bf70cc4ed0c2e5f23ce64f]
prodpeda-cesium-2020-04-23           Thu, 2020-04-23 06:00:08 [4f16f8dfad3841b4d047236b72aa546cb02b15225eea3d921bc7c77a30cc79d2]
prodpeda-cesium-2020-04-24           Fri, 2020-04-24 06:00:08 [5dfabd6258ab1606645b70c47f82a5dd284b87d8f27b3261e9b1df3ac488c899]
prodpeda-cesium-2020-04-27           Mon, 2020-04-27 06:00:07 [40b81d927f495d2622ed941b57940e9f4a1146ad0fc1dbe2df5df8cf354e87f7]
prodpeda-cesium-2020-04-28           Tue, 2020-04-28 06:00:07 [2d9a4a39af1149ff1931cdb16aa5bc6cf88ab219e8a0eab834437d976ddddda4]
prodpeda-cesium-2020-04-29           Wed, 2020-04-29 06:00:07 [041bc53aa978f5a44181e8f2c5686e227cd48e74ae7e9d3c6379381bd762327d]
prodpeda-cesium-2020-04-30           Thu, 2020-04-30 06:00:07 [c247a3be358745a9ba208d9ff760ea70f423aa0c216c602c72ba890af21f63ab]

# Get repo informations
$ borg info cesium
Repository ID: 3a941fd3f311494f5b83ae3e80c26339abf02d47b516906e7a798d570d834a87
Location: /mnt/sdb/backups/cesium
Encrypted: Yes
Cache: /root/.cache/borg/3a941fd3f311494f5b83ae3e80c26339abf02d47b516906e7a798d570d834a87
Security dir: /root/.config/borg/security/3a941fd3f311494f5b83ae3e80c26339abf02d47b516906e7a798d570d834a87
------------------------------------------------------------------------------
                       Original size      Compressed size    Deduplicated size
All archives:              572.38 GB            416.65 GB             13.89 GB

                       Unique chunks         Total chunks
Chunk index:                  325959              7884379

# Mount a backup
$ borg mount cesium::prodpeda-cesium-2020-04-30 /mnt/recover

$ ls /mnt/recover
bin  boot  dev  etc  export  home  initrd.img  lib  lib64  lost+found  media  mnt  opt  proc  root  run  sbin  srv  sys  tftpboot  tmp  usr  var  vmlinuz

As shown, all backups should use 572GB but, compressed and deduplicated, it only uses 14GB !
I really like that tool, and I hope that you will give it a try.