You are here

Data Storage Solution: Backup & Cloud

Data Availability
Data is usually made available by two means: redundancy and reliability.

Effort has been made to buy reliable hardware. Top of the line SSD, motherboard, memory, CPU and power supplies are used in the server.

Data is made redundant by providing multiple on-line copies of data. This is achieved primarily through a RAID1 mirror and protects against bad sectors and disk failures.

At some point I should add a small UPS to protect against power outages and brown outs.

There are other options available, like having a pool of servers available so servers can be brought down for maintenance or repair. You can also add redundancy to the network and the site with co-location. These options are cost prohibitive and not necessary for a home server.

Data Integrity
If your data cannot be kept online then integrity can be maintained with backups. This helps protect against hardware failures, software bugs, catastrophe, etc. There are also cases when on-line data can lose integrity for example through accidental or unauthorized changes.

This is a summary of data presented in a previous article:

Data Type Mirror File Integrity Monitor Reverse Backup Cloud Backup
Mission Critical Y Y 60day Y
Critical Y Y N No, too large
Important Y Y 60day Y
Unessential N N N N

Reverse Backup
The reverse backup provides a secondary copy of data as well as n number of reverse backups of that data. This is handy to restore accidentally modified or deleted files. This is particularly useful for user data like documents.

Data is reverse backed daily and retention is currently 60days.

Cloud Backup
This is the primary tool for protecting data. A cloud backup provides a number of advantages: off site, off network and off system. The data is not online but it is redundant, private and provides integrity in a completely separate environment from the home network.

Data from the system is aggregated, compressed, encrypted and uploaded to the server once a month.

The biggest draw backup of a cloud backup solution is that the storage capacity of the cloud services currently available to me are currently too small to use backup all of my data. So I must be selective. I could pay for a write only backup service. There are a tremendous number available. I will consider this for the future. At some point uploading a huge amount of data over a household internet line becomes time prohibitive as well.

There are also security concerns which I attempt to mitigate with compression and encryption.

Remote Cloud Services

  • Offsite Backup - I needed a service that is already available and supported by Debian Linux 8.0.
    • I have free storage on Amazon (5G), Google (25GB) and OneDrive (100GB)
    • Native support for Google (grive2)
    • I wrote a custom plugin for backupninja that will bzip2, encrypt and upload files to gdrive on a monthly basis

Local Cloud Host
There is also a desire to provide a cloud server at home to make it easy to access and share my data. It must be secure and fast and easy to use.

I considered a number of servers, but only a few fit the bill of performance, function and trust. They can be reviewed here.

A number of these services are promising, but review quite a bit of setup and maintenance. I settled on Syncthing. It is an ad-hoc peer-peer network that is secure, robust and easy to use. It's primary function is to synchronize files in folders.

Add new comment

Filtered HTML

  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <table> <tr> <td> <ul> <ol> <li> <dl> <dt> <pre> <dd> <img> <sub> <sup>
  • Lines and paragraphs break automatically.

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
CAPTCHA
This question is for testing whether you are a human visitor and to prevent automated spam submissions.
6 + 0 =
Solve this simple math problem and enter the result. E.g. for 1+3, enter 4.