I am thinking about moving away from my current storage solution to a FreeNAS solution and wanted to get some feedback as I am new to FreeNAS and the ZFS file system. I am wanting to get as close as I can to a solution that would be considered to have High Availability (HA) and also a few or no “single points of failure” (SPOF) to minimize downtime when it comes to my storage solution. I have a small home business where I do IT consulting and have recently been asked if I could support the hosting of websites and possibly email services.
Right now my limitations on my current setup other than the storage solution when it comes to HA and SPOF is that I only have 1 power source into my location (though I do have a number of UPSs installed for battery backup) and also that I only have 1 internet connection. When it comes to storage this is a different story and hence the desire to implement a better solution.
I have been using Proxmox for a few years now to run the various servers/services I have on my network and have been using QNAP NAS boxes for my storage. Right now I run 3 QNAP boxes, 1 for personal storage and VM backups, the second for VM storage for my Proxmox Nodes (I have 4 nodes currently) and the other as shared storage for my Nextcloud install and to temporarily store downloaded files as well as other random storage needs.
I am wondering which is the better way to go for my eventual move to FreeNAS as my new storage solution and this is where my question sort of splits in two. The first is hardware-based and here are the ideas I have had so far in terms of possible solutions for building 2 or 3 servers:
-
Using other enterprise-level servers such as the Dell R720XD.
-
Building systems from new parts though less enterprise-level hardware due to cost. Though they would need to have the following key features: Rack-mountable 2U or 4U chassis, 12 hot-swappable data drive bays, 2 hot-swappable OS drive bays, motherboard IPMI, redundant power supplies and at least 4 or 5 PCIe expansion slots.
The second is when it comes to installing FreeNAS, I had the following ideas and I would love feedback from the community on its viability and also on what would be the best storage solution for my needs.
-
Setup FreeNAS on bare metal hardware and use multiple servers and data replication between them to help with hardware failure and also data preservation. Though it would require manual intervention if there was an issue before services could be restored to normal operation.
-
knowing that FreeNAS does not support HA but was wondering if the following would be possible to give FreeNAS HA like feature(s) and be a suitable production setup. The idea would be to virtualize FreeNAS in say a 3 node Proxmox Cluster using that would be more of a head server configuration and then use 2, 3 or even 4 DAS or JBOD servers for the data drives.
Here is how I would see that solution working. By virtualizing FreeNAS under Proxmox and should the node FreeNAS is operating on have a failure of some kind the HA features of Proxmox would move FreeNAS to another node automatically. This, of course, does not address the storage aspect when it comes to the data drives and here is what I am thinking for that. If the multiple DAS/JBOD servers are attached to each Proxmox Node and all nodes are identical could I not use PCIe passthrough to FreeNAS. This would give FreeNAS access to the various disks for not only data storage but also data replication between the various DAS/JBOD chassis which would provide protection from a hardware failure when it comes to data storage.
Of course, if the Proxmox node were to fail and FreeNAS was moved to another node they would lose connection to the storage disks if they were only connected to the one node. What if each Proxmox node was connected to each DAS/JBOD server in such a way that it would not matter which node was hosting FreeNAS that FreeNAS would always have access to the storage disks. I know on Proxmox when It comes to network interfaces you are able to create various network bridges with the same identifier and as long as the VM that is moving to the new node as access to the same bridges with the same name they can run on a totally different node and network interface card.
I do not know if it is possible on Proxmox to even another hypervisor to create a virtual HBA while still using PCIe passthrough to an actual HBA card that would allow for this solution to work.
I know that it would obviously be more hardware than just running a FreeNAS box on bare metal but I am curious if this is possible and also if I could then extend this storage solution when the time comes by adding more DAS/JBOD servers and then expanding the existing FreeNAS setup or creating another FreeNAS install using the same Proxmox Nodes to host the FreeNAS install(s).