We are trying to get away from AWS as our bill just keeps going up. We have had a couple old servers (Dell r410s and Dell r620) at a local datacenter and feel its time to stop using AWS and just use our own hardware. This is for a small website hosting setup for a few sites and a Saas software. We are a small company and don’t have $50k laying around so we are using some used Dell servers. Since we made the decision, we have been consuming basically every bit of Tom’s content on XCP-NG and TrueNAS. Thank you @LTS_Tom for all of your great info. Its truly a blessing for me.
HDD unknown for now. Im thinking of a couple Samsung 1.92TB enterprise SSD in raid 1
(1) Dell R730xd 2x E5-2660V3 128gb ram (NAS)
(12) 12TB WD Ultrastar drives (these are the ones from Backblaze listed as HGST)
Sonicwall firewall (unsure of model…its in our current datacenter)
Cisco switch (unsure of model…its in our current datacenter)
Microtik 16port SFP (for storage network)
Software
XCP-NG with XOA for the 3 Dell 640s
TrueNAS for the Dell 730xd (ZFS)
Ok, so the plan is for the R640s is to host our websites on VM’s. The small websites will be on a single VM running Plesk. The larger sites and Saas will be on their own VMs. We are only currently using 12 vcpu and 48gb of ram at AWS so we have a ton of expansion on this hardware. The reason we got the 3rd host is for redundancy due to running a little older hardware. All 3 hosts will be in the same pool and we are planning on paying for XOA.
The TrueNAS setup we are planning on 2 vdevs RaidZ2. According to the calculator that gives us 64.5TB of useable storage. We use less than 1TB at Amazon currently so I think we should be good to go as far as a backup NAS. We are also going to do offsite to Backblaze for the NAS backup. I know we need to put the H730 raid card into HBA mode. If we have trouble, I’ll just get a HBA330 thats flashed.
So this is our first “complicated” setup. We have run a couple servers without virtualization for the last few years but the XCP-NG and TrueNAS is new to us. If we grow, we will buy new servers and build a new pool. Anyone have any suggestions or see any issues?
All sounds good, one thing of note is that I keep my XO instance on the local storage of a host in the pool and then you can setup a continuous replication process to copy it to a local storage of another host in the pool. The reason for doing this is in case there is an issue with the shared storage you can still use XO and if the main host running XO fails you can start it up on another host from that local storage.
I might suggest smaller drives in your NAS… If you don’t need that much in the foreseeable future, then starting smaller might be a good idea. Reason is that you can always swap to bigger drives, but you can not swap to smaller drives without destroying the pool and rebuilding. Tried it twice just for a sanity check and neither time could I swap drives to make the pool smaller. Will save you a bunch of money right now, and if you need bigger, then you probably have customers paying that can cover the expense.
I go one more step past Tom, I run XO on physical hardware, again I can always get in as long as the network is up. Even just an XO from sources would be a good idea, then run your XOA where ever it makes sense. You can run XO from sources on an old HP T630 if you need a cheap source of hardware for an ultimate backup.
Thanks for the info Greg. We should have extra space and a have a couple older Dell R620s we could use to run XO on hardware vs a VM. Knock on wood but these old 410s/620s have been amazing little servers. Not a drive change or anything in 5+ years we have owned them (we put new drives in when we deployed them 5 years ago)
As far as drives, we already purchased the 12tb drives. They were close to the same prices as the 8tb we originally looked at. Heck the 4tb drives we originally wanted from the Backblaze report weren’t too much cheaper than the 12tb
It seems like you’ve done thorough research and planning for your infrastructure migration. As you deploy and start using your new setup, be prepared to iterate and adjust based on your experiences and evolving requirements. Good luck with your migration!
