Borg backup, deduplicate them all

I'm probably very very very late to the party as Borg is actually a fork of Attic, but I just discovered this backup tool randomly on the Internet and I have to say, I'm in love with it!

It's very simple simple to use, yet very powerful all thanks to the deduplication. So what is this deduplication? Here is what Borg says about it:

Deduplication based on content-defined chunking is used to reduce the number of bytes stored: each file is split into a number of variable length chunks and only chunks that have never been seen before are added to the repository.

A chunk is considered duplicate if its id_hash value is identical. A cryptographically strong hash or MAC function is used as id_hash, e.g. (hmac-)sha256.

To deduplicate, all the chunks in the same repository are considered, no matter whether they come from different machines, from previous backups, from the same backup or even from the same single file.

Compared to other deduplication approaches, this method does NOT depend on:

  • file/directory names staying the same: So you can move your stuff around without killing the deduplication, even between machines sharing a repo.
  • complete files or time stamps staying the same: If a big file changes a little, only a few new chunks need to be stored - this is great for VMs or raw disks.
  • The absolute position of a data chunk inside a file: Stuff may get shifted and will still be found by the deduplication algorithm.

So basically what it means is that only what has been REALLY changed will be considered by Borg. And that, added to the compression makes it space efficient.

How do I create a repository ?

It's quite simple. In my scenario, I want my backup to be stored through SSH on my remote server. Let's start by using ENV variables because it makes everything always easier. In my .zshrc file I added this line:

export BORG_REPO='erwyn@myserver.example:backup'

Now we can create the repository:

$ borg init -e repokey erwyn@myserver.example:backup

You'll be asked for a password to encrypt the data. Please note that Borg supports other authentication means.

How do I backup my stuff ?

First, let's add a .borgignore file containing all the files/directories (one per line) I don't want to back up, for instance my /Downloads folder.

And now, everytime I want to create a backup all I have to do is:

$ borg create --progress --stats --compression zlib --exclude-from .borgignore ::{now} .

And I'm good to go!

I highly encourage you to look at the documentation of the Borg project, they also have some asciinema records to show you the basics of the tool.

See you soon!

Show Comments