I have consumed so much content in the past 2 weeks regarding building a server I am now stuck LOL. After so other discussions on here, I think I am going to build a small rack on wheels at this point.
I currently run my PfSense on a mini computer with two 2.5gb network cards. The thing runs like a champ and I plan on keeping that as my firewall.
I am looking to run a 1u or 2u server but I cannot figure out what to go with to make this work. Based off the last video, it makes sense to run TrueNAS Scale because I want to run the unifi controller, plex or jellyfin, and I will be buying a domain to host my email. Yes, I know the difficulties of running an email server, and after weighing options, I am choosing TrueNAS over Synology because I don’t like how synology forces you to buy their hardware and not allow changes without their approved hardware. I also plan on running maybe up to 10-15 virtual machines with windows, Linux, and Raspberry Pi for desktop and connect to them with maybe a thin client? Still figuring that part out.
So my initial thoughts: run DDRM 4 or 5; NVME with PCIE 4 or 5 capabilities; and 10gb cards (maybe 2?).
Don’t know what I should run as a CPU or a motherboard.
I also don’t know if I should also use this as a larger phase 2 project to build out a cloud NAS. I do plan on eventually creating a NAS back up at a completely different site so I have another back up with maybe an openVPN tunnel or tail scale. The back up will need to support photos (raw), video, etc.
What does your network look like? Do you have a 10 gbe switch? NVME drives are OK for the OS, but you don’t really need NVME drives for storage pools until you get to like 40 gigabit networking. How much storage capacity are you looking for? If pfSense is going to be limited to 2.5gbe networking, does it even make sense to have 10gb networking on your server?. Probably need more information before anyone can make reasonable recommendations.
By way of comparison, I have a n100 firewall appliance with 2.5gbe networking running pfsense, a 2.5gbe switch, 2 Proxmox hosts (both on mini PCs one is a N100 the other is a i5-12500T), 3 different NAS devices (Synology, Terramaster and Openmediavault). I run 3 public facing wordpress websites, a public facing Nextcloud instance, 12 different docker containters and 8 VMs. All of it is on 2.5gbe network cards and I don’t suffer for speed. My design aim was for low power and the entire setup, including WAP, UPS, and few other ancillary devices draws 85 watts at the wall. My ISP is really the limiting speed factor at 200mbps.
My current ISP is at 2.5gb. I intended on pushing this to 10gb once I am settled down with what I want to do in the future. The long term goal is connecting everything via fiber internally first to handle the back and forth of editing. Going to 10gb with ISP is more for future hosting of various services. The cost difference, IMO, to up from 2.5 to 10 for switches now just kind of makes sense.
What I am not sure how to calculate is the CPU performance required of TrueNAS scale. I think your set up is intriguing and curious if I could get away with using an i5 type of CPU. I would run a bunch of mini PCs like you have but I want to rack mount everything.
The only reason why I was thinking of NVME is because of faster read/write speeds so I am not bottlenecking using SATA SSDs.
The only reason why I was thinking of NVME is because of faster read/write speeds so I am not bottlenecking using SATA SSDs.
I may be wrong on this, but I believe even spinning rust in a raid 5 array will fully saturate a 10gb network connection. I just saw something on this on one of the websites I visit regularly (maybe NAS compares or serve the home?). You would need 40gbe networking to take full advantage of NVME drive speeds. Maybe Tom Lawrence (or someone else) can chime in here and correct me if I am wrong.
My little trick is like that:
On an A4 sheet of paper right side I write all I should complain,
And from right to left all I need for.
Easy.
After a dozen NAS-es it goes more smoothly.