I am trying to build a good lab for hosting VM. I have consider VMWare ESXi, XCP-NG, FreeNAS etc. I am not sure what to choose to build this lab. The lab is to use Windows Servers for testing. Any advise would be appreciated.
The hardware I have choose are:
Supermicro X9DRI-F Motherboard
Xeon E5-2670v2 2.5GHz
Kingston 16GB PC3L-10600R DDR3 EEC Memory (4)
Seagate Desktop 4TB HD (4)
WD Red 4TB HD (2)
Samsung EVO 500 GB HD (2)
LSI SAS 9210-8i 8 port
For Tom, I enjoy watching your videos and this is what you have to inspired
I would follow Tom’s recommends but also build a FreeNAS box for storage even though you are using for testing. Since you have 2 WD Reds get 2 more and use them in the FreeNAS build. Store your VMs on the FreeNAS. A simple build following the specs for a FreeNAS mini would do fine, XCP-NG best low cost solution for VM host. I am Using VMware but have had that up for quite sometime and is what I need for testing costly, but the free version lacks APIs.
Everyone loves XCP-NG but maybe I can get the ‘Proxmox Shill’ title on here
Since you’ve got 1 box, I would dump it all in one. Flash the LSI card into IT mode if possible so the system sees individual disks (important for ZFS - it needs to see the physical drives to work, not a RAID card).
I’d use Proxmox (free, open source) and create a ZFS RAID1 for the SSDs as boot + fast VM storage, then either RAIDZ2 or ZFS RAID10 the 4TB drives (~12TB usable in fast RAID 10, or ~16TB in Z2 - speed vs redundancy question based on what you want to do).
Mike: I try Proxmox, until I had to create a VM for Windows 2012R2 with Hyper-V. It will not let me load it. :(. I guess I will try XCP-NG and that will fix that .
Hyper-V as a nested VM, can.t imagine the stability. I’v seen VMWare nested in VMWare in a lab and is stable. VMware says a definite no no in production. There was an aeticle in one of the Linux mags some years back where such a setup was done on a laptop. Would like to know the results.
Yeah, it’s definitely not something I’d recommend. Containers are totally fine, but I wouldn’t attempt any nested full virt - just seem to be asking for problems.
Really depends on reason if your just wanting to have a lab (xcp/XenServer), if your looking US based @ virtualization VMware carries a good bit of weight. Either way you are fine not sure what your looking to do.
Guys, one more question, should I get a second system to separate one for XCP-NG and one for FreeNAS? Everything I read so far it gear the XCP-NG need the processing power and RAM and the FreeNAS can be a slower processor and less RAM. I think what I have posted at the beginning should be my XCP-NG box.
In my opinion, you can get away with just running FreeNAS as a virtual machine, if all you’re really looking to do is learning and monkeying around on it. This is what I’ve done to at least get familiar with the software, and you can run some light tasks to it. I just wouldn’t put data you really care about on it. Then again, maybe this my cheapness talking.
However, if you plan to actually use it, and want to keep data on it, then another physical machine is probably the way to go.
@axemaster Your original specs would make a great VM host.
My FreeNAS is a E5-2609v4 and 32GB RAM. I’m running mirrored SSDs for the OS drives and have 6 3GB HDDs in Z2 for bulk storage. You will need to add 10G networking if you want to take advantage of the SSD speed. Direct links work great for that until you get over 3 devices.
If your just wanting to learn windows and Linux, xcp-NG. I also say Hyper-V not to shabby as well. If LTS adds a buy/sell/trade section I will donate you some Ram you just pay shipping. I understand trying to learn and grow career and my focus now is working for myself and helping others.
Bob, Thanks for the offer, I have some additional RAM coming for the system. I am thinking the same thing but growing into small business and they have limited funds. I think Linux is definitely a learning curb from Windows.
Once you become comfortable with Linux you will appreciate it for its stability. Been using Linux since '94 and have found it much eaiser to deal with than MS-Windows not to mention its reliability. Same for FreeNAS and pf-Sense. Yes Windows is out there and one needs to know but Hyper-V is little used just doesn’t have the features of VMWare or Xen, so become familiar with it but don’t dwell on Hyper-V. I offer this based on what I see in the SMB market space. More Line-of-Business applications are WEB based and they run on Linux be it on prem, coloed or cloud hosted.
I am doing something very similar. I want to learn more about WinServer and Linux so I created a bunch of VMs using XCP-NG. Only reason I went with it because it’s free and plenty of guides/help online if you google xenserver and of course Tom’s videos.
The only thing I havn’t added to the VM was FreeNAS because I’ve read it’s much easier if it has direct access to the hardware but still want to try it someday.
I havn’t been able to get the XCP-NG guest tools to work on Windows because I’m using XCP Center instead of Orchestra but I’m sure that’ll change and it’s not a deal breaker.
The one thing I tend to do too much is plan, plan, plan and over-think. Do your research of course but just Go For It!! You’re just testing/learning. Pick one and if you don’t like the hyper-visor, wipe and try again till you do… this is fun stuff man, ENJOY!
IMO when it comes to hypervisors it’s Vmware than Hyper-V. I really don’t know any medium to large size companies that use Xen Server yes you have a few however overall it’s Vmware than Hyper-V. I’ve even seen cases where Vmware gets booted from the datacenter for Hyper-V. This happens because the company is pretty much getting it free with there Microsoft enterprise agreement. Also, I would focus on learning cloud platforms such as AWS and Azure because that’s where it’s going. Companies are moving everything to the cloud or they are doing hybrid configurations. When it comes to labs I want speed so I’m focusing on SSD drives not spinning disk yes I like spinning disk for storage such as additional disk to your VM’s. So I would get rid of the two additional red drives use that money for more solid state than raid 10 the 4 Seagates. One more thing learn PowerShell it’s very important and remember you can sign up for free Azure, AWS and O365 developer. The local lab combined with AWS, Azure, and O365 will allow you to learn pretty much anything.
.