In these notes I create a FreeNAS jail (using iocage rather than warden), install borg inside the jail, and use borg to communicate with a remote borg server hosted by rsync.net. Initially I try using Wasabi rather than rsync.net, but decide against Wasabi.
As usual, the notes were gathered as I ran through the process, and tidied up a little afterwards, so this is not a slick guide. I go back on myself and try alternative routes.
Set up jail with borg installed
Going to install borg in a FreeNAS jail and set it up to back up to Wasabi. Here are the official borg installation instructions.
Through the FreeNAS UI, I create jail named “borg-jail”.
(NOTE: I am using FreeNAS 11.1. Later in these notes I find out that FreeNAS is in the process of transitioning from one “jail manager” to another. From FreeNAS 11.2 onwards “iocage” will be the supported jail, but prior to 11.2 “warden” is the supported jail. FreeNAS 11.1 includes both iocage and warden, which allows you to prepare for the transition, however jails created in the UI of FreeNAS 11.1 are warden jails. At a later step I retrace my steps and recreate the jail in the CLI using iocage).
In the UI, open jail console.
Find the borg package:
# pkg search borg py27-borg.localrole-3.0.2_1 PAS plugin which can manage local roles via an ad apter lookup py36-borgbackup-1.1.6 Deduplicating backup program
The one we want is borgbackup. Install the package:
# pkg install py36-borgbackup-1.1.6 Updating FreeBSD repository catalogue... FreeBSD repository is up to date. All repositories are up to date. The following 9 package(s) will be affected (of 0 checked): New packages to be INSTALLED: py36-borgbackup: 1.1.6 python36: 3.6.6_1 compat7x-amd64: 7.4.704000.201310.1 liblz4: 1.8.2,1 py36-setuptools: 40.0.0 py36-msgpack: 0.5.6 py36-llfuse: 1.2_1 fusefs-libs: 2.9.7 zstd: 1.3.4 Number of packages to be installed: 9 The process will require 132 MiB more space. 22 MiB to be downloaded. Proceed with this action? [y/N]:
Add storage to jail
This is so that borg can read the storage to back it up.
Select the jail in the UI and click “Add storage” at the bottom.
Set source to
Set destination to
Set as read-only. Borg only needs to read from this mount, and by not allowing writes we avoid the possibility of a bug in borg (or in our mis-configuration of borg) potentially deleting the very data we are trying to preserve.
Create borg-backups storage
As well using a remote borg server to store backups remotely, I will be using another local mount to store backups locally.
Through the UI, create a new dataset at
Add borg-backups storage to jail
Set source to
Set destination to
Do not set as read-only. Borg needs to write to this mount in order to store the backup repository.
Allow sshing into FreeNAS jail
Through the UI, you can open up a console into the jail. If you want to be able to remotely SSH into the jail, follow these steps:
(NOTE: Again, this is not really necessary because further on I switch to using iocage rather than warden. I found the console in the FreeNAS UI clunky for interacting with the jail and wanted a real terminal, so decided to set up SSH. Further on when I switch to iocage this is no longer necessary, since iocage jails can only be accessed through the CLI in FreeNAS 11.1).
$ vi /etc/rc.conf
$ vi /etc/ssh/sshd_config
Start the SSH service:
$ service ssh start
By default the
root jail user has no password, so one needs to be set in order to log in via SSH:
Further reading on SSH in jails:
Create the backup repository
Now let’s use borg to create a local borg repository:
$ borg init --encryption=repokey /mnt/backups/borg-repo
Unfortunately, the above command gives the following error:
Undefined symbol fdatasync
It turns out I am not the first to encounter this issue. The advice is to use an iocage jail rather than a warden jail, since warden jails are EOL for FreeNAS 11.1.
Jails: warden (old) vs iocage (new)
Instead of waiting for that release, let’s see if we can use the new iocage jail manager with our current installation of FreeNAS 11.1.
Creating a jail through the UI creates a warden jail by default.
To create an iocage jail it must be done through the CLI.
Switch back to the FreeNAS CLI. No more UIs from here on!
Before you can make a jail with iocage, it needs to fetch a release. Since we are using FreeNAS 11.1, we must select the
11.1-RELEASE option. (At least, I tried using
11-2.RELEASE and although it downloaded, it was not possible to create a jail).
root@freenas:~ # iocage fetch Setting up zpool [storage] for iocage usage If you wish to change please use "iocage activate" Creating storage/iocage Creating storage/iocage/download Creating storage/iocage/images Creating storage/iocage/jails Creating storage/iocage/log Creating storage/iocage/releases Creating storage/iocage/templates  9.3-RELEASE (EOL)  10.1-RELEASE (EOL)  10.2-RELEASE (EOL)  10.3-RELEASE (EOL)  10.4-RELEASE  11.0-RELEASE (EOL)  11.1-RELEASE  11.2-RELEASE Type the number of the desired RELEASE Press [Enter] to fetch the default selection: (11.1-RELEASE) Type EXIT to quit: Fetching: 11.1-RELEASE Downloading : MANIFEST [####################] 100% Downloading : base.txz [####################] 100% Downloading : lib32.txz [####################] 100% Downloading : doc.txz [####################] 100% Downloading : src.txz [####################] 100% Extracting: base.txz... Extracting: lib32.txz... Extracting: doc.txz... Extracting: src.txz... * Updating 11.1-RELEASE to the latest patch level... src component not installed, skipped Looking up update.FreeBSD.org mirrors... 2 mirrors found. Fetching public key from update4.freebsd.org... done. Fetching metadata signature for 11.1-RELEASE from update4.freebsd.org... done. Fetching metadata index... done. Fetching 2 metadata files... done. Inspecting system... done. Preparing to download files... done. Fetching 153 patches.....10....20....30....40....50....60....70....80....90....100....110....120....130....140....150. done. Applying patches... done. The following files will be updated as part of updating to 11.1-RELEASE-p11: /bin/freebsd-version /bin/pgrep /bin/pkill ...[snip]... /usr/share/zoneinfo/Pacific/Pago_Pago /usr/share/zoneinfo/Pacific/Tongatapu /usr/share/zoneinfo/zone.tab WARNING: FreeBSD 11.1-RELEASE is approaching its End-of-Life date. It is strongly recommended that you upgrade to a newer release within the next 2 months. src component not installed, skipped Installing updates... done. root@freenas:~ #
Now you can create a jail named
borg-jail with iocage:
root@freenas:~ # iocage create -r 11.1-RELEASE -n borg-jail
Configure iocage jail networking
Jails created through the UI seem to be automatically configured with an IP address, while iocage jails are not:
root@freenas:~ # iocage list +-----+-----------+-------+--------------+-----+ | JID | NAME | STATE | RELEASE | IP4 | +=====+===========+=======+==============+=====+ | 3 | borg-jail | up | 11.1-RELEASE | - | +-----+-----------+-------+--------------+-----+
In order for it to be useful for our purposes, we need to give it an IP address. This can be done as follows.
bge0 is the name of the network interface on the FreeNAS,
192.168.0.3/24 is the address and subnet that I am assigning to
root@freenas:~ # iocage set ip4_addr="bge0|192.168.0.3/24" borg-jail Property: ip4_addr has been updated to bge0|192.168.0.3/24 root@freenas:~ # iocage list +-----+-----------+-------+--------------+-------------+ | JID | NAME | STATE | RELEASE | IP4 | +=====+===========+=======+==============+=============+ | 3 | borg-jail | up | 11.1-RELEASE | 192.168.0.3 | +-----+-----------+-------+--------------+-------------+ root@freenas:~ #
As with the warden jail, we need to configure this iocage jail to have storage. This can be done through the CLI as follows:
iocage fstab -a borg-jail "/mnt/storage/media /mnt/media nullfs ro 0 0" iocage fstab -a borg-jail "/mnt/storage/backups /mnt/backups nullfs rw 0 0"
Now we can start the jail:
iocage start borg-jail
Unfortunately, upon starting I got this issue:
resource deadlock avoided
Again it turned out I am not the first person to encounter this.
Had this problem trying to start the jail after setting the fstab configuration:
Solving the resource deadlock issue
Fortunately, I found that by simply deleting the jail and re-creating it the problem went away…!
This time however I named the jail
fnbbu rather than
borg-jail, short for FreeNAS Box Backer Upper.
Starting the jail
This time when starting the jail I got errors about the mounted directories not existing. Fortunately simply creating the directories and then trying again seemed to fix things:
root@freenas:~ # iocage start fnbbu * Starting fnbbu + Start FAILED jail: mount.fstab: /mnt/iocage/jails/fnbbu/root/mnt/media: No such file or directory root@freenas:~ # mkdir /mnt/iocage/jails/fnbbu/root/mnt/mediaroot@freenas:~ # iocage start fnbbu * Starting fnbbu + Start FAILED jail: mount.fstab: /mnt/iocage/jails/fnbbu/root/mnt/media: No such file or directory root@freenas:~ # mkdir /mnt/iocage/jails/fnbbu/root/mnt/media root@freenas:~ # iocage start fnbbu * Starting fnbbu + Start FAILED jail: mount.fstab: /mnt/iocage/jails/fnbbu/root/mnt/backups: No such file or directory root@freenas:~ # mkdir /mnt/iocage/jails/fnbbu/root/mnt/backups root@freenas:~ # iocage start fnbbu * Starting fnbbu + Started OK + Starting services OK root@freenas:~ #
Now we can go inside our
fnbbu jail and install borg:
root@freenas:~ # iocage console fnbbu
root@fnbbu:~ # pkg search borg root@fnbbu:~ # pkg install py36-borgbackup-1.1.6
Ran through some quick borg examples just to verify that things are working this time:
root@fnbbu:~ # borg init --encryption=repokey /mnt/backups/borg-repo
Creating the repository worked this time - hooray!
I won’t post the commands here, but at this point I did manage to create backups in the repository, and very importantly, managed to retrieve files out of the backup and restore them in place.
(NOTE: At this point my plan was to use rclone to copy the borg repository to Wasabi. Later I change to instead simply using borg to create a remote backup inside a remote borg repository hosted by rsync.net, removing the need for rclone).
Install rclone in same jail as borg.
root@fnbbu:~ # pkg search rclone bomberclone-0.11.9_3 Reimplementation of Atomic Bomber Man rclone-1.38 Sync files to and from various cloud services
The one we want is rclone, not bomberclone:
root@fnbbu:~ # pkg install rclone-1.38 Updating FreeBSD repository catalogue... FreeBSD repository is up to date. All repositories are up to date. The following 1 package(s) will be affected (of 0 checked): New packages to be INSTALLED: rclone: 1.38 Number of packages to be installed: 1 The process will require 16 MiB more space. 4 MiB to be downloaded. Proceed with this action? [y/N]: y [fnbbu] [1/1] Fetching rclone-1.38.txz: 100% 4 MiB 3.9MB/s 00:01 Checking integrity... done (0 conflicting) [fnbbu] [1/1] Installing rclone-1.38... [fnbbu] [1/1] Extracting rclone-1.38: 100% root@fnbbu:~ #
Set up rclone configuration for Wasabi:
At this point I struggled to get rclone to talk to Wasabi. This prompted me to look into other options again.
There is a special discount for rsync.net users who only want to use borg, and none of the other fancy features. More about that in the next installament of these notes.
All articles in this series: