Hey all! I’m planning a new TrueNAS server for me and my extended family, hoping to get some guidance on my hardware/infrastructure.
My Background
I’ve built personal computers for friends and family for the past 15 years, have a degree in Software Engineering, and work as a Software Engineer. I’m familiar with most Server Infrastructure concepts, as well as networking, but I’m by no means an expert. Put me slightly above novice on the experience chart.
Goals and requirements
The primary use case for this is to have a “cloud” storage solution for me and my family.
Most likely this will be achieved with a NextCloud plugin on the NAS.
We must be able to accommodate a “home” folder for each user.
We must be able to accommodate a shared “family” folder for each family unit.
We must be able to accommodate a shared “everyone” folder.
This data must have a high level of resiliency
A secondary use case is to have a “cloud” media server.
Most likely this will be achieved with a Plex plugin on the NAS
This data does not have a strict requirement on resiliency, but it is a nice to have
I don’t plan on having dedicated hardware for networking, so this NAS will also need to host the reverse proxy.
Hardware Selection
Case: Fractal Design Define R5
8 drives
Good airflow
Motherboard: SuperMicro MBD-X11SCA-F-O
8 SATA drives without an additional controller
2x M.2 interface
ECC Memory
IPMI
Processor: Intel Xeon E-2236
HyperThreading
RAM: 4x Crucial 16GB DDR4 SDRAM ECC @ 2666
ECC Memory
PSU: Seasonic FOCUS GX-550W PSU
Reliable
Supports up to 9 SATA devices without splitters
SSD: 2x Crucial P2 250GB M.2 SSD
Cheap
M.2 interface
HDD: 9x WD Red Plus 6TB HDD
Reliable
Affordable
Miscellaneous:
Cooler Master Hyper 212 CPU Cooler
2x be quiet! Pure Wings 2 140mm PWM Fan
Total After Shipping and Tax: $2748.11
Planned Configuration
I plan to use the two SSDs as a mirrored boot drive. I could be convinced to use one as the boot, and the other as a read or write cache.
I plan to mount 8 of the data drives, keeping one as an on-hand spare. I’ll put all 8 drives into a pool, and one VDev with Raid-z2.
I’ll divide this volume into datasets
jails
plex
nextcloud
Only myself and my father-in-law will have accounts in TrueNAS. Everyone else will simply use the client apps.
Networking
I plan to purchase a domain name from hover, and point the name servers at no-ip. In no-ip, I’ll configure a wildcard to send all traffic to the specified IP Address. Using no-ips DDNS feature, and the integration in my router, the IP will “always” (not sure how immediate the updates are) have my public IP address. I’ll add port forwarding for :80 and :443 to my nginx server. The nginx server will have http to https redirects, and will have a server block for media and data subdomains over port 80.A third server block for the root (mydomain.com) will redirect to the TrueNAS admin client. I don’t think I’ll be setting up VPN, but might in the future.
Questions
Do I need the mirrored boot drive? How much of an improvement is a read or write cache likely to provide.
Does the server feel right-sized?
Any other thoughts, advice, tips, recommendations, etc.?
Mirrored boot drives provide resiliency whether or not to use depends on your risk tolerance.
Should be fine for your use,
As for running Nextcloud, I always first suggest people not publicly expose things as there is greater risk of someone attacking the system or it not being patched fast enough if there is a flaw found. Using a VPN is a much better idea. As for reverse proxy, that is not something I have tested with TrueNAS but I am sure there is a way to build one in a jail.
Please do not expose the admin interface of your TrueNAS server to the public internet. Setup a VPN if your father in law needs access to it!
Whether you want to expose Plex and Nextcloud directly to the internet is up to you. Generally you can do it and both services are designed for this usecase. But as @LTS_Tom said, you have to make sure that you keep everything up to date and “everything” also includes the TrueNAS server itself, even when you not expose it’s admin interface directly to the internet.
Other than that, your configuration looks fine to me
Regarding the boot drive, I think I’ll stick with mirroring. In fact, I might grab a third drive to have an on-hand spare. Downtime really isn’t an option for this system, at least unplanned downtime anyway. If I feel we need caching in the future I can always add a riser for that.
Based on your advice and that of @bb77 (thanks by the way), I’ll be setting up a VPN for access to the server directly. Things like server admin, SMB shares, and so on will only be accessible over the VPN. This will be exclusively for my father-in-law as he and I are the only admins of the system.
The nextcloud and plex use cases really do need to be public. I think some of our family members could manage VPN connections, but many would be lost in the dark, two states over.
In another group where I was discussing the networking in more detail, it was suggested that I setup a pfsense box at the ingress of my internet connection. This way I can isolate the public stuff to a separate subnet, and can firewall it from accessing my home subnet. This also gives me more options, as I should be able to shift the reverse-proxy out of TrueNAS and into my pfsense box.
Because I’m still planning on exposing things to the internet, I’ll need to follow the advice of both of you to rigorously patch all the things. I wonder if I can set up some kind of monitoring to alert me of outdated firmware/software.
And lastly, “RAID IS NOT A BACKUP”. I’ll be setting up a backblaze sync for the data dataset, but not the media. Because I’ll encrypt the backup on the TrueNAS side, am I safe to backup the server config including encryption keys?
Again, thank you both for the advice. I’m feeling much more confident in this endeavor.
Is there a particular reason to go for so many but relatively small disks? The impact on net storage capacity with RAIDZ2 might be a factor. Wanted to check anyway …
@chris great question. According to this calculator (ZFS Capacity Calculator - WintelGuy.com) the 6TB drives gave me the best cost per usable TB out of the options WD Red Plus has to offer. I just checked again, and I could go with 4 of the 12 TB drives and achieve slightly higher total usable space, while only increasing the cost per useable TB by about $2.00. I might do that as it gives me some flexibility if I need to expand in the future.
