diff --git a/README.md b/README.md index 5e79c94..732bf31 100644 --- a/README.md +++ b/README.md @@ -283,60 +283,15 @@ authelia: -----END PRIVATE KEY----- ``` -Add backup to LDAP: - -```nix -shb.backup.instances.lldap = { - # Can also use "borgmatic". - backend = "restic"; - - keySopsFile = ./secrets/backup.yaml; - - # Backs up to 2 repositories. - repositories = [ - "/srv/backup/restic/nextcloud" - "s3:s3.us-west-000.backblazeb2.com/myserver-backup/nextcloud" - ]; - - retention = { - keep_within = "1d"; - keep_hourly = 24; - keep_daily = 7; - keep_weekly = 4; - keep_monthly = 6; - }; - - consistency = { - repository = "2 weeks"; - archives = "1 month"; - }; - - environmentFile = true; # Needed for the s3 repository -} -``` - -This will backup the ldap users and groups to two different repositories. It assumes you have a -backblaze account. - -The backup `sops` file format is: - -```yaml -restic: - passphrases: - lldap: XYZ... - environmentfiles: - lldap: |- - AWS_ACCESS_KEY_ID=XXX... - AWS_SECRET_ACCESS_KEY=YYY... -``` - -The AWS keys are those provided by Backblaze. - See the [`ldap.nix`](./modules/ldap.nix) and [`authelia.nix`](./modules/authelia.nix) modules for more info. +### Backup folders + +See the [manual](https://shb.skarabox.com/blocks-backup.html). + ### Deploy the full Grafana, Prometheus and Loki suite -See [docs/blocks/monitoring.md](docs/blocks/monitoring.md). +See the [manual](https://shb.skarabox.com/blocks-monitoring.html). ### Set up network tunnel with VPN and Proxy diff --git a/docs/manual.md b/docs/manual.md index db00cef..f03f73f 100644 --- a/docs/manual.md +++ b/docs/manual.md @@ -7,6 +7,10 @@ preface.md ``` +```{=include=} chapters html:into-file=//blocks-backup.html +modules/blocks/backup/docs/default.md +``` + ```{=include=} chapters html:into-file=//blocks-monitoring.html modules/blocks/monitoring/docs/default.md ``` diff --git a/modules/blocks/backup/docs/default.md b/modules/blocks/backup/docs/default.md new file mode 100644 index 0000000..575fedf --- /dev/null +++ b/modules/blocks/backup/docs/default.md @@ -0,0 +1,210 @@ +# Backup Block {#blocks-backup} + +Defined in [`/modules/blocks/backup.nix`](@REPO@/modules/blocks/backup.nix). + +This block sets up backup jobs for Self Host Blocks. + +## Features +Two implementations for this block are provided: +- [Restic](https://restic.net/) +- [Borgmatic](https://torsion.org/borgmatic/) + +No integration tests are provided yet. + +## Configuration + +### One folder backed up to mounted hard drives + +The following snippet shows how to configure backup of 1 folder using the Restic implementation to 1 +repository. + +Assumptions: +- 1 hard drive pool is used for backup and is mounted on `/srv/pool1`. + +```nix +shb.backup.instances.myfolder = { + enable = true; + + backend = "restic"; + + keySopsFile = ./secrets.yaml; + + repositories = [{ + path = "/srv/pool1/backups/myfolder"; + timerConfig = { + OnCalendar = "00:00:00"; + RandomizedDelaySec = "3h"; + }; + }]; + + sourceDirectories = [ + "/var/lib/myfolder" + ]; + + retention = { + keep_within = "1d"; + keep_hourly = 24; + keep_daily = 7; + keep_weekly = 4; + keep_monthly = 6; + }; + + consistency = { + repository = "2 weeks"; + archives = "1 month"; + }; +}; +``` + +The referenced Sops file must follow this structure: + +```yaml +restic: + passphrases: + myfolder: +``` + +To generate a secret, use: `nix run nixpkgs#openssl -- rand -hex 64`. + +With the borgmatic implementation, the structure should be: + +```yaml +borgmatic: + keys: + myfolder: | + BORG_KEY + passphrases: + myfolder: +``` + +You can have both borgmatic and restic implementations working at the same time. + +### One folder backed up to S3 + +> This is only supported by the Restic implementation. + +Here we will only highlight the differences with the previous configuration. + +This assumes you have access to such a remote S3 store, for example by using Backblaze. + +```diff + shb.backup.instances.myfolder = { + + repositories = [{ +- path = "/srv/pool1/backups/myfolder"; ++ path = "s3:s3.us-west-000.backblazeb2.com/backups/myfolder"; + timerConfig = { + OnCalendar = "00:00:00"; + RandomizedDelaySec = "3h"; + }; + }]; + + ++ environmentFile = true; # Needed for s3 + } +``` + +The Sops file has a new required field: + +```yaml + + restic: + passphrases: + myfolder: ++ environmentfiles: ++ myfolder: |- ++ AWS_ACCESS_KEY_ID= ++ AWS_SECRET_ACCESS_KEY= +``` + +### Multiple folder to multiple destinations + +The following snippet shows how to configure backup of any number of folders using the Restic +implementation to 3 repositories, each happening at different times to avoid contending for I/O +time. + +We will also make sure to be able to re-use as much as the configuration as possible. + +A few assumptions: +- 2 hard drive pools used for backup are mounted respectively on `/srv/pool1` and `/srv/pool2`. +- You have a backblaze account. + +First, let's define a variable to hold all our repositories you want to back up to: + +```nix +repos = [ + { + path = "/srv/pool1/backups"; + timerConfig = { + OnCalendar = "00:00:00"; + RandomizedDelaySec = "3h"; + }; + } + { + path = "/srv/pool2/backups"; + timerConfig = { + OnCalendar = "08:00:00"; + RandomizedDelaySec = "3h"; + }; + } + { + path = "s3:s3.us-west-000.backblazeb2.com/backups"; + timerConfig = { + OnCalendar = "16:00:00"; + RandomizedDelaySec = "3h"; + }; + } +]; +``` + +Compared to the previous examples, we do not include the name of what we will back up in the +repository paths. + +Now, let's define a function to create a backup configuration. It will take a list of repositories, +a name identifying the backup and a list of folders to back up. + +```nix +backupcfg = repositories: name: sourceDirectories { + enable = true; + + backend = "restic"; + + keySopsFile = ../secrets/backup.yaml; + + repositories = builtins.map (r: { + path = "${r.path}/${name}"; + inherit (r) timerConfig; + }) repositories; + + inherit sourceDirectories; + + retention = { + keep_within = "1d"; + keep_hourly = 24; + keep_daily = 7; + keep_weekly = 4; + keep_monthly = 6; + }; + + consistency = { + repository = "2 weeks"; + archives = "1 month"; + }; + + environmentFile = true; +}; +``` + +Now, we can define multiple backup jobs to backup different folders: + +```nix +shb.backup.instances.myfolder1 = backupcfg repos ["/var/lib/myfolder1"]; +shb.backup.instances.myfolder2 = backupcfg repos ["/var/lib/myfolder2"]; +``` + +The difference between the above snippet and putting all the folders into one configuration (shown +below) is the former splits the backups into sub-folders on the repositories. + +```nix +shb.backup.instances.all = backupcfg repos ["/var/lib/myfolder1" "/var/lib/myfolder2"]; +```