Securing Offsite Borg Backups with Secretive and BorgBase
Borg #
I am a big fan of Borg Backup for my backup needs. It’s proven fast, secure, efficient, and reliable in the years that I’ve been using it. It’s also free and open-source, which is always a comfort.
I’ve started using BorgBase for offsite backups. I generally consider these backups to be my personal “disaster recovery” backups. This means, if my house burns down, or someone manages to ransomware all of my data, I can still get back what I need.
Incidentally, one time I did need to use one of these to recover the key to my normal backups. Redundancy is its own reward! Redundancy is its own reward!
The ransomware use-case is particularly tricky for Borg. Because it is “push”-model backup software, if an attacker can compromise the source machine, then they can compromise the remote repository as well. Even ostensibly more secure systems, like the system keychain or SSH agent, can be compromised by an attacker assuming a user session is active. This is a hard problem to work around.
Here is how to work around this problem.
Procedure #
You will need:
- The SSH keys already used for your backups
- An Apple MacBook with a Secure Enclave (or a Smart Card)
- Install Secretive
- A BorgBase Account (It’s Free!)
First, in BorgBase, we must make all of our current keys, which we use for unattended backups, append-only for all of our backup repositories. This will allow them to create new backups in BorgBase, but not to prune old backups. Then, we can leave our old backup configurations as they are, using the old key. Remove any “pruning” configurations, as these machines will no longer have pruning permissions.
Second, in Secretive, create a new SSH key, and configure it to require authentication every time it is used. Add this SSH public key to BorgBase.
Third, for all of our repos in BorgBase, configure the new Secretive SSH key to have full admin access.
Finally, create a script on the MacBook to prune backups for all of our repos. This can be done
by iterating over them via individual borg prune
calls, or using a configuration manager like
Borgmatic. The borg SSH args will also need to be modified to
work with Secretive by passing additional args like
-i ~/Library/Containers/com.maxgoedjen.Secretive.SecretAgent/Data/PublicKeys/my-key.pub -o IdentityAgent=~/Library/Containers/com.maxgoedjen.Secretive.SecretAgent/Data/socket.ssh
, using the values found in the
Secretive app.
When run, this script will prompt for authentication via the secret enclave (via any available biometric configuration) for each repository that you prune. This script can then be run manually on whatever cadence is convenient and prudent.
Important note: Append-only mode does not strictly prevent deletion, but configures delayed deletion. Pruned backups will be deleted during the next compaction. For this reason, one should validate the present state of the repository before running this prune/compact, as it will delete anything pending. See also the relevant BorgBackup and BorgBase docs.
Remember to turn on 2FA for BorgBase too — to make it that much harder to compromise by altering the keys in your account.
Closing #
As a result, we now have a repository which our automated systems can back up to, using push-based backups, on any cadence they wish. At the same time, destructive operations are now entirely gatekept behind the MacBook’s Secure Enclave and our biometric authentication.
Even a total ransomware compromise, which detected the remote backups, pushed bad data to them, could not delete the old backups, ensuring almost total safety in the event of a compromise.