Sam Hooke

These are rough notes from whatever I was working on, interested in or thinking about at the time. They vary greatly in quality and length, but prove useful to me, and hopefully to you too!

The Important Files: Part 1

Our irreplacable family photos are stored scattered throughout the following diverse locations:

  • My phone
  • My old phone
  • My wife’s phone
  • My wife’s laptop
  • My DropBox
  • My old desktop PC
  • My old Mac
  • The SD card in our camera
  • The SD card in a box in the loft
  • The DVD of our wedding given to us by my Uncle

My old desktop PC also contains the source code for all video games I’ve developed from when I was a teenager (read: did not use version control).

I refer to all of the above as “The Important Files”. They are all files that we would like to hold on to, are irreplacable, and would feel saddened if it was lost.

In this note I begin the plan to design and assemble a solution.

Requirements

  1. Safely and securely store The Important Files;
  2. Adequate room for The Important Files to grow;
  3. Easily access The Important Files from Linux or Windows;
  4. Routine, automated, verified backups of The Important Files;
  5. Low maintenance (e.g. upgrading every 6 months is fine, installing patches daily is not);
  6. Affordable (spend money where it matters, but always consider cost/value trade-off).

What is not suitable

  • A 100% cloud-based solution:
    • Using a cloud provider as a backup is acceptable, since it is valuable to have an off-site copy, but placing everything in the cloud puts you at the mercy of the cloud provider.
  • Too much closed-source or proprietary software:
    • Want to avoid vendor lock-in.

Proposed solution

  1. Set up a home server of some description;
  2. Set up backups of home server to the cloud;
  3. Copy all The Important Files on to the server.

The two main components of this are the home server, and the backup system. Not all of the details for these can be decided now (some will be decided as we go) but following is what we can decide at this moment.

Home server

Operating System

The two main contenders are:

  • CentOS 7
  • FreeNAS

I have much experience with CentOS 7, and know it is flexible enough to solve the needs of the requirements. However it may require a quite a lot of set up compared to other solutions (e.g. FreeNAS) which are designed for this sort of task.

FreeNAS is based on FreeBSD, which I’ve never used, but in terms of curiosity that is quite enticing because I’ve been meaning to try out a BSD system for some while. From what I’ve read, it seems to come with a lot of what I’d like to set up out-of-the-box (e.g. support for RAID).

Current plan is to try and use FreeNAS, and if it doesn’t go so well, switch to CentOS 7.

Hardware

Server

In keeping with the requirement for affordability, I have an HP ProLiant MicroServer (HSTNS-5151) which I bought a few years ago but never actually used. It has space for a maximum of four 3.5” HDDs, and room for an optical drive. At a pinch it may be possible to squeeze an additional SSD where the PCIe slots are.

It’s not the latest and greatest hardware but it should be fine for our purposes.

Though it has space for four 3.5” HDDs, it only has one 250GB HDD currently. While I do have an assortment of old HDDs that have built up over the years, I would rather use new HDDs since that will directly impact the reliability of this system.

RAID

Different RAID levels require a different minimum number (or multiple) of disks. The space efficiency (i.e. how much usable storage space you get vs. total physical storage capacity) also depend upon the number of disks.

Let’s assume that we use four disks for storage (since that is how many slots the server has).

The following table compares the different RAID levels being considered assuming there are four disks used. It ignores RAID levels which are out of the question entirely.

RAID level Description Min # disks Fault tolerance (assuming 4 disks) Space efficiency (assuming 4 disks)
RAID 0 No redundancy, just combines disks 2 0 100%
RAID 1 Mirroring 2 3 25%
RAID 5 1 parity disk 3 1 75%
RAID 6 2 parity disks 4 2 50%
RAID 10 A RAID 0 of RAID 1s 4 2 if from different RAID 1s 50%
RAIDZ1 Like RAID 5 but better1 3 1 75%
RAIDZ2 Like RAID 6 but better1 4 2 50%

For more details see the Wikipedia articles on standard RAID levels, nested RAID levels and non-standard RAID levels.

FreeNAS Operating System drive

Notes

HP Proliant MicroServer N54L: https://www.amazon.com/2QR5443-ProLiant-MicroServer-704941-001-Server/dp/B00AKWUZ58

Can either get an mSATA drive in the PCIe slot, e.g.: http://www.amazon.com/Plextor-M5M-0-85-Inch-256GB-mSATA/dp/B00B5BRIF8/ref=sr_1_48?s=electronics&ie=UTF8&qid=1387929057&sr=1-48&keywords=s+sata+256+ssd

Or can simply use a USB stick which is plugged in to one of the mobo USB ports.


All articles in this series:


  1. A huge over-simplification, of course! [return]
<< Previous: Debugging a Python process not terminating due to active threads
Next: Calling iPerf3 from Python >>