Glacial – A Scott Odyssey

I’ve been writing a lot about my tech setup lately because I’ve done quite a bit of work on it. I’ve been meaning to share my current private cloud backup setup for a while now.

The backbone of my private cloud network is still Resilio Sync. While I rely on it a bit less these days, it remains a core part of my strategy.

Right now, I’m using Resilio to replicate a full set of data from my Synology DiskStation to a Raspberry Pi 4. I also replicate a subset of this data—everything except the media center—to an SSD on my laptop. Soon, I plan to set up another Pi 4 as a backup for the same subset of data I have on my laptop.

At this point, I no longer keep any replica data at friends’ houses. I probably should, but when my last setup failed, my friend had to bring the device back to me when he visited from the States. Ultimately, it wasn’t worth buying new gear just to ship it back to him. Instead, I signed up for Amazon Glacier Deep Archive (or whatever they’re calling it now). It’s a cheap, long-term storage option where data is locked in for six months without modification or deletion options. My Synology DiskStation has a built-in client that made it easy to set up a backup of my personal data to Glacier. I still need to test a restore, but for now, I see Glacier as my remote storage solution. At about $1 per terabyte per month, nothing else comes close to that price. Setting up another Pi with a friend would cost around $150–$200, which makes Glacier far more cost-effective over a three-year period.

Because I’m still a bit unsure about restoring from Glacier, I’ve also started using Proton Drive for critical data, including my entire family photo and video library. Once I’ve uploaded the photos, that dataset stays pretty static, so Proton Drive makes sense. With our 3TB plan, I can gradually copy large, mostly unchanging files that I want securely backed up. Since there’s no automated way to sync this, it’s not my primary backup, but it adds another layer of protection.

Recently, with T in high school (or middle school if we were in the States), she’s been using the computer more often. It made sense to subscribe to the family plan of Office 365, which gives each of us 1TB of storage on OneDrive. I’m experimenting with Cryptomator encryption to securely store a subset of our backups on OneDrive. I still need to fully implement this, but it’s something I plan to sort out soon.

In addition to these replica copies, I take monthly RSYNC snapshots to a separate directory on my DiskStation. I have two scripts—one for odd months and one for even months—so I always have two recent copies. I also keep an annual copy of everything. It’s a bit less automated, but it works.

I’m also considering setting up another Pi as a remote Resilio node. Another option is to get a storage VPS again. The previous deal I had expired, so I canceled it last year. That’s partly why I’ve been relying less on remote Resilio replicas. When I got rid of my last remote Pi, I switched to a VPS running Resilio. Now, I’m debating whether it’s worth setting up another VPS instead of piecing together backups the way I have been. At around $80 per year for 2TB, it’s an option I’m keeping open.

Overall, the system works. When I had a catastrophic failure on my DiskStation before upgrading to my current one, I was able to verify that all my data was backed up somewhere. In the end, I didn’t need to restore because I managed to salvage the array on the DiskStation, but it was a valuable exercise to go through.

UPDATE: I wrote this before Christmas. Since then I have built a new Pi with a 2TB SSD and need to deploy it somewhere other than our house as a backup. I have also found a new cheap(ish) VPS storage provider. I have a 2TB VPS in Germany were I am now replicating my main Reslio shares to. I have stopped using Glacial since i haven’t been able to properly test it. It is still by far the cheapest backup option out there however without being able to verify it works to easily fully recover i was a bit concerned. The new VPS i have is a few pounds more per month but not outrageously expensive.

Last year I canceled my pay Drobo subscription. I still have the service but with a lot less storage. At the time I wanted to build my own “cloud” storage system. In reality I wanted to take my data at home and make it syncable via the internet like a cloud storage provider to my devices were ever i was. Originally I bought a Transporter that I had hopes would take a 1TB drive and let me sync data to my computers. That dream lasted about a week. Their sync software was problimatic for me. Instead I opted for a Synology NAS DS412+. The added app’s functionality and redundant storage allowed me to move all my data to the Synology and no longer rely on my aging Drobo(s) as my primary storage system. The Synology also allowed me to move some multimedia functions i did on my mac mini off to the NAS. So far I have been very happy. The Synology Cloud Sync app gives me Dropbox like functionality with my personal stuff. The other file access options the Synology offers gives me access to all my data. To do that with any other provider would be prohibitivly expensive because of the amount of data stored on the NAS.

The timeing for this change to self hosting my data was perfect. I couldn’t realistically do what I am doing now earlier because when I had Time Warner my upload speeds were horrible (1.5mb). After I moved to Fios my upload speeds jumped to between 25-35mb. I have no problems using the Synology.

The problem with the Synology is that as much as the NAS itself is designed to last and has some drive resiliance in it, I do not have an offsite backup. I used to copy my data manually for a while but now a days that is not a good idea. it is also not feasable with the amount of data i have now.

After leaving Dropbox I read more and more about the potential issues with public cloud providers. Such as Dropbox knows what you upload and wont keep a duplicate copy of a file of music if others have it. That means they can (and i believe they have) removed content if there is a DMCA request. I am also increasingly uneasy about US based hosting providers and the easy the government can get access to data. I am not a criminal and don’t really have anything to hide in what I am storing however those are not reasons why I should be ok with the ease of government access to my data. I will talk about that more in another post. In relation to cloud storage I am glad I do not use Dropbox, however I put myself in the same situation with another provider.

Because I have so much data and I need to really be thinking about a 321 backup sceme I started using Crashplan and Amazon Glacial. Crashplan I recently canceled because I was having problems with my copy of data on my Drobo that i would sync to Crashplan. The crashplan app kept thinking that the drobo was disconnected and would keep creating new copies of my data. When it takes weeks to sync a backup using crashplan having a new copy get created every so often is not good. With that issue and the fact that the Crashplan Synology app caused my Synology to perform super slow to the point that I uninstalled it I gave up on Crashplan and canceled it.

Amazon Glacial has been good so far. The challanges with Glacial is that the cost per month varries by how much i use it. It is also much more expensive than Crashplan for the amount of data i have. Another challange is getting data out is expensive in the event of a major issue. Thankfully i havent had to worry about that yet but it is a concern. The most current concern I have is that Amazon is a US based company and my data is hosted on US servers. For now I am living with that risk. My rational side says I dont do anything to warrent the government to want my data however I would rather that not even be an option.

One alternative to Dropbox I have been looking at is Mega.NZ. They don’t have any access to my data once it is uploaded. They offer 50gig free. It is good to replace dropbox for some files i have and to give my most important stuff an offsite backup but it is not enough to backup my music, videos, and photos. Their 1tb option is doable for me but even that isn’t enough to backup everything. For now I will continue with Glacial as my backup. I am investigating working with a friend or two to replica sync our Synologies between each other. The cost of that may not be worth it but it is an interesting idea.