Hi everyone. I was considering backup options to Glacier Deep Archive, and wanted to know:
- Which software do you use to encrypt client-side, obfuscate, compress and deduplicate the data before you send it to S3?
- What is the difference between
Restore Requests (bulk)andOutbound data transferand which one will I be using when I want to pull my data from AWS?
Iโll be storing approximately 8TB or so of data, which is why I was looking at inexpensive ways to back it up other than buying an HDD outright.
Thanks!
- I use Kopia (https://kopia.io/)
Iโve been using S3 but Iโm considering Cloudflare R2 as it might be a bit cheaper
Really? Their pricing is even more expensive than AWSโ S3 Glacier Archive! Iโd much rather use BackBlaze B2 than pay that much!
-
I donโt encrypt before I push to S3. Probably bad practice on my part. I just rely on AWS encryption to secure my data. My backups are low-risk (imo). That said, I lock down the bucket so that only my account can access the objects. Compression I use
tar cjf(bzip). Protip: Once the tar file is made, runtar ljf $archiveFile > archiveFile-ls.txtand store the resulting file along with the tar file in standard storage. That way you know what is in the archive. -
Both.
Restore Requestsis to copy the data out from Glacier into Standard storage. Note that I said copy. When you perform a restore, your original object stays in glacier and AWS creates a copy to somewhere in S3 that you specify. Once the restore is complete, you can then download the copied object like any S3 object, triggering theOutbound data transferfee.
deleted
Iโm using it too and even the current prices are reasonable (especially if you consider thereโs no other fees, no transfer, no ingress, no egress, โฆ). If you put it in S3 glacier and you ever have to restore a relevant chunk of your data (or god forbid, want to do periodic testing of the backed up data) then youโll be paying quite a bit of fees.
I have been recommended iDrive a lot under this post. How reliable are they?
Thanks. I was wondering about the reliability of data storage/infrastructure of iDrive specifically. For example, Iโm fairly sure that I can keep my data in AWS Glacier/B2 for 10 years or so and nothing much would happen (Assuming Backblaze doesnโt just die). Can I assume that for iDrive? Is this an old company with many years in the business? For their offerings seem amazing, itโs just the perceived risk from lack of information that is holding me back.
Lots of answers in the comment about this particular storage type/vendor. Regardless, to answer your original question, rclone. Hands down. If you spend 30-60 minutes actually reading their documentation, you are set and understand so much more of whatโs going on under the hood.
Thanks, I do know of rclone and intend to study it. I was just wondering if the likes of borg, duplicati etc could be used.
Canโt speak for those but I tried Kopia and it did the job okay. Ultimately tho I landed on rclone.
Recently I looked into the same thing, since AWS caught my eye with their apparently ridiculously low prices. Then I found this (presumably indepdenant) review, that changed my view on things: https://b3n.org/b2-vs-s3-nas-backup/
After reading that, I wonโt go with AWS. Iโm currently considering to abuse the OneDrive Office Family plan, which costs 99 $ a year for 6 TB of storage (split across 6 accounts), which comes down to 1,40 $ per month per TB. A price that I have not seen beaten by other storage / backup providers.
The problem with AWS is that one need to bring all storage from glacier to regular S3 (which is quite expensive), and then their egress costs are massive. I read more about it and completely agree that AWS is not worth it at all when pulling data in.
However, BackBlaze is quite expensive. Iโd be paying $50 for my storage, which is simply not how much Iโd like to pay. At this rate, Iโd get a 10TB Ironwolf drive in an enclosure from Amazon for my cold storage.
TBH, I was looking at other providers, and Dropbox looks much less expensive at $20 for 9TB. Itโs a lot more than the 8TB I thought Iโd be able to get away with for Glacier storage, but at least itโs not $50. I will take a look at scaleaway, but I simply trust the bigger players to be around and keep my files safe than the smaller companies.
Edit: Scaleway seems less expensive than Dropbox with their glacier option. I really hope I can trust this company because this is an excellent price and I might even be willing to pay as much if it can be kept secure and safe. Now, I just need to read more about the value proposition of Dropbox vs Scaleway Glacier.
Regarding Dropbox: Where are you seeing 9 TB for 20 $? Iโm in the EU so my pricing may vary, but all I can see is the Business plan for 16 โฌ per month per user, with a minimum of 3 users in the plan, making it cost 48 โฌ instead. Do you have access to something else?
Regarding Backblaze: Agreed that 50$ a month is a rough bill to pay, that sums up very fast if youre counting across the years. But their storage is also a lot more reliable than one single hard drive stored in a bank locker, with them always checking their arrays and replacing aged drives.
Regarding Scaleway: If im reading their pricing chart right, it would cost roughly 2 โฌ / month / TB for glacier storage, and 9 โฌ / TB when restoring from glacier to standard storage? A big questionmark for me is how ingress works. If Iโm using this for backups in case of total system failure, iโll want to upload differential backups (borg/duplicati) every couple days. How is that going to work with pricing, is that all running through standard storage driving up monthly cost, do I have to manually manage file history and deletion of older stuff or does my backup software handle that? Plus, you loose out on the instant file access that you get with Backblaze, or with something hacky like Dropbox / Onedrive. Iโm still undecided which I value more, money or fast access.
If I were to pay $50 a month for 8TB, Iโd have paid about $200 in 4 months. I can get a 10TB-12TB external drive (with Ironwolf/WD Red Pro CMR inside) for about $200.
I should have worded that properly, apologies.
Hello ! Just adding my two cents for Scaleway. Iโve used them personally for some services (and probably will add s3 storage in the near future)
Itโs seems pretty reliable in my opinion.
