Requesting suggestions for appropriate OS/Configuration

The goal:

  • utilize the multiple processors and large bank of memory and storage for various VMs (NVM, Storage, Torrent, http, ftp, etc.)
  • properly utilize the storage with ZFS (RAID Z1) and network VMs to storage

The hardware:

Motherboard: ASRock Rack EP2C602-4L/D16 (4 Intel 1GB interfaces)
Processors (x2): Xeon E5-2670 - 8 cores (HT) @ 2600MHz
Memory (256GB): Kingston 16GB 2RX4 PC3-12800R DDR3 Reg ECC Server Memory SL16D316R11D4 (x16)
Storage1 (120GB ZFS): Samsung EVO 850 SSD SATA 6Gbs <---- SLOG for ZIL offloading
Storage2 (24 TB RAIDZ1): Seagate IronWolf 8TB 7200RPM SATA 6Gbs (X4)

The network:

ISP > pfSense 2.4.4 on a Qotom MiniPC with Intel i7-7500U (16GB RAM, 256GB SSD) > 2x Cisco SGE2000P L2 switches (stacked) > Server

I understand it would be ideal to have all of that storage in a NAS with just a basic MB, CPU and RAM to handle serving files, and use the Server MB with a smaller storage footprint. That will have to be phase 2.

The options I am considering include:

  1. Run Debian from a USB stick and use kvm to build the VMs and manage them
  2. Run Proxmox, uncharted territory but seems more user friendly than #1
  3. Run XCP-ng, also uncharted territory, but seems to be the popular option at LTS.
  4. Run unRAID or FreeNAS, I know very little about either but they seem to be on a “second tier” in terms of favor, versus Proxmox or XCP-ng.

Any information, suggestions or advice that anyone can provide would be very appreciated.

I want to get building this, but I would like to proceed in a sensible manner, given the hardware I am working with.

TIA!

-Darryl

FreeNAS is not a bad choice as it has excellent support / management for ZFS and you can use the IoCage Jail system for any lightweight services and only use Full VM’s if needed. The downside to FreeNAS as a VM system is that you will have less simplicity / flexibility if you were trying to move VM’s between servers the way XCP-NG or ProxMox handles it.

Thank you very much for your reply, Tom. Definitely value your input. Your amazing videos have set me on this path.

At the risk of sounding like I didn’t get the answer I was looking for, I think I will go with XCP-NG, despite your recommendation.

Reasoning:

I like to tinker, I like to experiment and I like to learn. I think FreeNAS might automate/streamline more functions than I prefer.

Who knows? I might get too deep in the weeds with XCP-NG and go to FreeNAS, as you have suggested, but as I read more about the two, I think I might enjoy how robust XCP-NG is.

Ultimately, the goal is to have a dedicated FreeNAS box with a 10GbE connection between this current system and that one. With that being quite a bit into the future, I think I can run a FreeNAS VM in XCP-NG and use PCI Passthrough to push the bare metal HBA connection for my 4 - 8TB drives to the FreeNAS VM. Once I need to move to a bare metal system, I can rebuild FreeNAS and manage that same pool.

I am also considering just running a Debian install and trying qemu-kvm. That, most likely, is biting off more than I can chew. :crazy_face:

I love Proxmox + GlusterFS. Give it a shot. Clustering for HA was pretty easy to be honest.

Thank you for the suggestion. I’m researching GlusterFS right now. Interesting stuff.

1 Like

Last time I used it was maybe back in 2013. I have been out of the game for awhile in that realm, but here is a Video I made on it all those years ago. https://www.youtube.com/watch?v=Sjy0jll0Rx4

Thank you for the link. I will check it out today.

1 Like

Definitely a very informative video for Proxmox installation. Thank you. I’ll see if I can find anything out there for the GlusterFS implementation you suggest.

1 Like

Any time! Glad it was helpful