Cloud Backup Costs - AWS S3 Glacier Deep Archive

Computer Hardware & Server Infrastructure Builds
1

Hi, I’ve been setting up my home TrueNAS 12 Core server to do a daily Cloud Sync to AWS S3 in the “Deep Archive” storage class. My idea was to have the lowest possible cost to have a cloud backup for my 3 TiB of files.

However, after two months of using this, I got a little bit overwhealmed with the actual bill.

My first month on AWS cost me around $52. The largest expenses were 312,077.000 S3 Glacier Deep Archive PutObject Requests ($15.6), 45,067,343.000 Requests-GDA-Tier2 ($18) and 3,258,245.000 Requests-Tier1 ($16)

I thought “It is probably the cost of the initial upload, it should get smaller”, but next month I got a $100 bill with 1,059,800.000 S3 Glacier Deep Archive PutObject Requests ($53), 42,571,946.000 Requests-GDA-Tier2 ($17) and 3,127,319.000 Requests-Tier1 ($15)

The largest cost are those “PutObject Requests” that I believe are still part of my initial upload. (I have a much lower cost on that on the current April bill), but I’m still finding the “Requests-GDA-Tier2” a bit high.

Are those “Requests-GDA-Tier2” happening when I’m checking for file changes (Cloud Sync) every day? Are there any backup recommendations to try to lower those recurrent costs? The Storage costs themselves are VERY cheap, but those request costs are really expensive!

Alternatively, if anyone here have any recommendations of a cheaper alternative to AWS S3, I would appreciate it. I was considering Backblaze B2, that would cost me $15 for storage. But I’m afraid I might also find some unexpected billing in their “transaction” costs. What are you people seeing as transaction costs on your daily backups with Backblaze?

2

Backblaze and CrashPlan are two good options that would be cheaper than AWS. I use both and haven’t seen any hidden costs.

3

Yes… Backblaze, Digital Ocean Spaces, and others rate limit the transactions instead of charging you transaction fees (you’re only allowed x transactions per minute, y per hour, z per day type of thing)…My suggestion for using Amazon Glacier as a “last resort” archive is to do a few tarballs instead of individual files to make significant savings on transaction fees.

4

Cloudflare R2 might be something to look out for when it comes out

5

So it isn’t the amount of data but the number of files that you are charged for? Or is it both?

6

AWS charges you for “transactions” on their S3 storage in addition to storage space. Listing objects (files) is a transaction. Putting (uploading) a file is a transaction. Copying is a transaction…

7

Hmmm… I see a lot of my company’s money going out the door. I know we use both AWS and Microsoft for backups, but not my area so not sure what we pay. I’ve been told we have petabytes worth of MS storage as part of our contract, but for backups I think they charge extra to move the data. If anyone knows how the educational contracts work for MS, I’d love to know for those random discussions with that part of the college. Sometimes it pays to know what the limitations are when they say something can’t be done.

For places like my work, you’d think that just putting a large NAS in a couple of buildings would give enough “off site” storage to be mostly protected. And shouldn’t cost a lot either. Only concern would be that they are all “connected” but that could be handled as well. There would need to be a really large disaster to get all of our buildings destroyed. We have around a square mile of area, all with our own fibers and about 9 buildings on main campus, one more across the street and down a quarter mile (full building generator too, and fibers to a satellite campus several miles away on leased fiber. $50,000 a year would buy a lot of Truenas storage. An ethernet operated power switch could provide and cut power to an ethernet connection at the server to isolate it during non-use times to help prevent an attacker from gaining access. With the new network going in we could have minimum of 40gbps connection and I think as high as 100gbps so time would not be a large factor, and certainly a lot less time than our 1gbps internet upload could provide.

Or maybe I just don’t trust the cloud.