Please help me with my dream home network setup

Computer Hardware & Server Infrastructure Builds
1

I’m a techy, geeky person, but have never had my own space to really plan my networking. I’m also a minimalist (a vague term for almost everything) and don’t want to spend or have more than I actually need. I’ve already networked several floors and a garden area with UniFi and know how to use it, and I’ve build my own cables, PCs, etc. All in all, I’m far from an expert, but I know enough that home networking can be so much nicer with a little effort (and money). I also hate the fact that my ISP multi-function device (‘router’) doesn’t get timely security updates.

I want to repeat myself that I am between noob and expert. So I want to lay out some ideas and questions so that more experienced forum members can tell me if my plans are good or bad.

My current situation is that I live with my family on two continents. I would like to share files safely between us. I have a ton of files and a solution where all family pictures, videos and media files are stored at one location sounds fantastic. I have used Synology a bit but for planning my new setup, I have been looking at TrueNAS Core and would like to build my own NAS box. Building a PC is not difficult (and I have done it several times). However, before I build a TrueNAS Core system (the free community edition), I’d like to know the best way to make it externally accessible in a secure manner.

You may see that I want a bit more than just good WiFi. My focus is on privacy and security, and I don’t like vendor systems with registration which phone home all the time. To be flexible with my router settings and also get updates quickly, I want to stick my nose into pfSense. It doesn’t seem too complicated and seems like the perfect product for an open source enthusiast. What hardware would you recommend? The Netgate devices? I need to route 1Gbps (both directions) and also would like to run my own VPN for default encrypted connections.

I also think I’m right about this - but this is something I’d like clarified - that pfSense can be configured with WireGuard so I don’t have to open ports on my router. Is that correct? To be clear this is for accessing my networking securely and not to hide my traffic from my ISP.

Additionally, I’d like to know if pfSense has an option to easily integrate WireGuard for a VPN connection to a public VPN provider like Mullvad.

Back to the VPN for secure connections to my home network: I would also like to know how to access my WireGuard VPN (the one to connect to my home network) without a static IP from my ISP. Is DynDNS the solution for that?

As for my ISP: I have a 1Gbps connection with fiber optic cables. I think I can use my regular ISP router in a bridge mode or do a double NAT to be able to connect my other RJ45 cables.

Since I’m streaming for work, I think I’ll wire all my office equipment with a tiny switch and also provide a second switch for my second office and my NAS box (media server, Vaultwarden, etc.).

Also, I may want to play with PiHole and some other media related server applications, but that is still in the future. I’m aware of the TrueNAS jails, but I want to use the systems exclusively for storing and sharing files.

If I set up a VPN connection on my pfSense router so that I can connect from outside of my network, my family and friends would have to switch to the VPN connection. However, since I also run my own VPN server for my family and friends, I would like to ask if there are any security concerns if I do not set up a VPN on the pfSense box, but instead limit port forwarding for the NAS box to the public IP of my VPN server. That way there would be no need to change VPN and create a VPN to the NAS or VPN to something like Mullvad situation (on/off situation).

I would also like to use Vaultwarden. It’s pretty minimalistic and I think it would be something for a Raspberry Pi 4. How do I secure the instance so I can connect to the server with maximum security? The VPN solution with WireGuard and pfSense again?

Next, I would like to have good Wifi. I liked and used UniFi before, but I heard that you have to register to use the new devices since a software update. I don’t like that if it is true. Is this true? Is there a good alternative? I would like to have a mesh network with one device connected only to other AP via Wifi in an outdoor scenario. Also, I have two floors and I think one AP on each floor with cable uplink would be sufficient. Of course, I want roaming functionality so that I have a Wifi network that is powered by all APs and I can switch seamlessly as I go up and down my apartment.

In regards to TrueNAS, I would also like to know what the best backup solution is for the box itself. One solution would be to replicate the box to a second box on the other continent. Is this possible with TrueNAS performance? What advantages can I get from this? Isn’t this just redundancy and not a true backup? I would appreciate if someone could bring some clarity to this issue.

Finally, I would also like to decrease the risks by using VLANs so that all devices have to be “smart”. For example, I would like to separate devices like guest devices or my gaming consoles that don’t need to see the rest of my network topology.

Sorry for the long text, but everything is connected (literally), so please help me get out of this cyclical loop of questions.

Summary: I have a 1Gbps fiber cable that I would like to connect to a pfSense router. From there I connect two switches (one for my main office and the other for the other office and home server stuff). I would like to securely access Vaultwarden from outside of my home network. 2 wired APs and one AP connected via Wifi (mesh) is needed too. Everything should be able to be separated by VLANs. Is that plan good bad? Where can I improve or are changes needed? Please answer my questions and provice some input.

Thanks!!!

2
  • WireGuard is supported in pfsense but does not have an “Easy Setup” option for using a privacy VPN
  • WireGuard does require that you open ports and have a public IP
  • psense does support support Dynamic DNS
    Services — Dynamic DNS | pfSense Documentation
  • pfsense does support policy routing allowing you to choose where the data routes, either VPN or WAN, etc…
  • Synology is much more turnkey easy for setting up things, TrueNAS is much more to configure for sharing but does support tools such as Syncthing
  • The data on TrueNAS can be replicated to another TrueNAS
  • UniFi AP’s and Switches use the UniFi controller software do not require registration with Ubiquiti

Here are some videos where I go over some firewall rules for pfsense for a basic home setup & UniFi VLAN setup

I also have a video on policy routing with a privacy VPN

You can also find lots of other videos on TrueNAS & Syncthing on my channel.

3 Likes
3

For what it’s worth here’s my 2 pence worth …

If you’re not familiar with pfSense it will take time, dream up a number then double it. However, it does “just” work on initial install but you will need to configure it and that’s where the time goes.

If you are thinking NAS, then build your own is fine, get it the way you want it etc. One area when a NAS wins beit synology / QNAP is if you need something to deal with IP cameras, they win hands down. If you never intend to use IP cams then build your own, if you do then consider a NAS the IP camera software works for a very wide range of cameras. Any Linux alternative seems to be a bit sketchy, options exist for Windows but pony up the cash for that.

If you have unify then get their AP, they seem to be ok, you’ll need a controller. While it doesn’t have to be running all the time, if you have a cheap PC box you could install PROXMOX and run it in a vm for cheap.

I like https://eu.protectli.com/ boxes, I think you get a bit more bang for your buck, however, for your situation you might have to hunt around there are solutions that might take your fibre connections, more boxes are coming out with 2.5G WAN connections. You can also build your own box with the required nic card in the short term.

As for your VPN, I’d set up an OpenVPN site to site connection between your two homes. Then I’d setup another Remote Access Server at each site for when you are on the road. OpenVPN is well documented for when you run into trouble.

If you have a paid for VPN service, I’d use a dedicated vlan for that traffic, period, don’t faff around with the traffic, everything on that vlan goes out the VPN. You can also set up an OpenVPN RAS that exits from the VPN gateway, meaning if you are on the road, connect to your network over OpenVPN then exit out of the VPN as if you were at home.

PiHole looks pretty, but after a while you’ll never look at it. PfSense has pfBlocker which blocks a lot of crap.

With respect to security you need to do a bit of research for OpenVPN if it’s too high the speed will be slow if your processor isn’t up to the job. You can always test out different ciphers to see what works best for you.

Yeah I would setup vlans, with the ISP, VPN, Guest as the minimum.

You can always setup pfSense in a vm and take a look from there. Personally buying any hardware I would go for as many NICs as I can get, for the firewall you can put them in a LAGG, in an AP you can daisy chain other devices from it.

4

I am not very familiar with all the opensource software you mentioned so I can’t help you with questions related to that but I can give you some things to think about.

You asked if keeping two fileservers in sync will give you a backup solution. The answer is of course “no”, but you probably understood that. Sometimes people accidentally delete data or data gets corrupted. If you have everything synced up then the data is gone also at the other side within a second. So, you have to make backups and make a backup schedule. What you should keep in mind is that malware might be transferred over the VPN connection. So there is a good chance that if all your data is encrypted at one side, it will also get encrypted at the other side. So make sure that the backups are not connected to your network. Use tapes, USB drives, or a location (in the cloud) where you can’t chance or delete the data within a certain retention period.

You also stress that you want to make everything very secure. Of course I can’t tell if that is really necessary but if you come to the conclusion that your data must be protected at all times then it will be very difficult to that. If you pump up the level of security, also the level convenience will go down. You will have to use e.g. multi factor authentication every time you use your computer, laptop, VPN, etc. You have to encrypt everything including your OS, backups, etc. You have to use secure boot. Basically, if you want to do that all yourself then it will also require a lot of study to accomplish this. Also keeping everything very secure requires a lot of time or money. You have to e.g. keep all your software up to date. You can ask Tom about it. He does that for his clients. He hires employees to do that and those employees want a salary. Often they are not the cheapest employees anyway. You can of course do it all yourself but that will take time and it will take a lot more time for you per device then for a professional who does those updates for a lot of clients at the same time.

If it is also sort of you hobby to do this then that is great. If you are in the IT business and want to get some extra experience then it is also a very nice project. If it is however just a hassle then just make use of the normal security that is build into pfSense, your OS, and make from time to time a backup on a USB drive from only the most important files.

5

@janet2, there’s going to be a lot of ways for you to accomplish more-or-less what you want, but as the others have alluded to and depending on your appetite for experimentation - it might take a couple iterations and some time to get to exactly or within the vicinity of where your want. Luckily, between his forums and videos, @LTS_Tom should have you covered for most scenarios.

Just to touch on some of the topics you mentioned:

  1. Yes - pfSense can be configured (with WireGuard, OpenVpn & others to be honest) in various scenarios, including: Client, Remote Access, Site-to-Site, Site-to-Multisite.

  2. You’re also correct in that you’ll require some form of Dynamic DNS to help your VPN clients find your VPN server. Plenty such services abound, I’ve used dyndns.org (now dyn.com) since the early days, but not sure if they offer any free services anymore. Nowadays, if you own a domain with a half decent provider, they’ll typically allow you to keep a sub-domain “dynamically” in sync through an API as well.

  3. Definitely first try your ISP modem/ONT in bridge mode. Failing that, yes you’ll have Double NAT which will present you with a number of issues. Many modems have a DMZ setting (bit of a legacy misnomer) you can employ, which essentially forwards all ports to say your pfSense WAN, but it’s still not ideal.

  4. Only ever expose or port forward to a service that’s actually meant for public consumption and that stands up to the security implications thereof. So your VPN & Web Servers are typically ok, but most NAS and File Servers are a definite no-no.

  5. Above point limits your “externally accessible” file server options somewhat, consider dropping a pfSense remote side as well and make use of a Site-to-Site VPN - keeping all your file services “private”. Or use something like nextcloud - with storage backed by your NAS box.

  6. It’s not a bad idea to separate things into VLANs, but take some time to plan your various subnets & VLANs, both sides of the family tree so to say, and separate by distinct security concerns or roles. @neogrid gave you a good starting point here, but don’t get too carried away with the VLANs - it’s pointless creating 20 different VLANs only to have to open all your rules because you actually need things to be able to talk to each other. And remember that all inter-VLAN comms have to be routed, through pfSense in this case, so you typically want to keep local heavy use traffic - such as storage - within the same broadcast domain.

  7. Again with @neogrid, consider building yourself a VM Host machine, whether it be XCP-ng, Proxmox, or whatever - for all your Vaultwarden, PiHole, etc, experimentation.

6

Awesome man! Thanks a lot for this! Of course, I have already seen some videos. It’s more like this: I’ve already watched some videos, looked into the documentation, played with UniFi, etc., but I ask so I don’t catch the one point I missed.

I have still some questions and some follow up questions.

  1. Regarding file sharing, as long as all my friends and family have access to the TrueNAS box via a pfSense Wireguard configuration and I only want to share with them, TrueNAS is good to go, right?

  2. What Wireguard endpoint do I specify for the pfSense gateway (I don’t have a static IP from my ISP)? Of course I have to open a port for Wireguard, but that seems more secure than opening Vaultwarden/TrueNAS directly, right?

  3. What do you think about my Vaultwarden plans? Vaultwarden caches the encrypted blob (password DB) for some time even after my home server is done? Is that right? How long does Vaultwarden (or more precisely the Bitwarden client) store the database? I don’t want to lock myself out, especially since I live more or less on two continents and can’t just hop on a plane every day.

  4. I want to replicate my TrueNAS to a second TrueNAS. Is it possible to upload files to the second box so that I have replication in both directions? To clarify: Internet speeds are slow, so it would be super nice if I could push files locally to both boxes and have them all merge files accordingly. Otherwise, I would have a master/slave setup where I have to send all my files to box 1 and then replicate back to box 2.

  5. UniFi does not require any form of registration? If I’m not mistaken, UniFi was open source but is no longer? Is there any open source equivalent solution out there?

  6. Regarding pfSense and privacy VPN: Is pfSense just hard to configure or is the WireGuard setup still experimental? If I can do the same with OpenVPN I am fine with it. Please clarify that for me. I also would like to know if there is support for more exotic VPN solutions so I can break through Internet censorship (I live in an censored area and would like to have the option for ShadowSocks or V2Ray, etc.).

  7. Regarding TrueNAS box replication security, can I replicate to box 2 in such a way that whoever has control of box 2 cannot access it?

Thanks again for your amazing help!

7

Thank you for letting me know about the cameras! No, I did not plan to run cameras.

Do I get any other benefits from PROXMOX than the fact that I don’t need the UniFi controller hardware?

I think 4 ports are fine. What model do I need to transfer 1Gbps with a VPN? Maybe this is good: Vault 4 Port - Protectli Having Coreboot is awesome! I’m a big fan of Purism’s PureBoot and Coreboot on a router sounds fantastic!

Buying a box with 1Gbps is fine. I can always upgrade to 2.5Gbps later. All I want is 1Gbps, good quality and no backdoors. Are the Protectli devices suitable for pfSense (I don’t want a super hacky option that causes a lot of problems just to save $50 or so)?

OpenVPN seams old and clunky. Can I do the same with WireGuard? It is so much simpler what I like a lot.

Can I use the pfSense box also for local DNS so that I can have my media server at something like media.box? Otherwise I would use PiHole for it.

Intel AES-NI is a must, right?

Thanks, I would do the same! It’s always get a second opinion even if you just agree. Thanks!

This one is not clear to me. What do you mean?

Thanks a ton for your help! Super happy about how nice this community is!

8

Is there any other solution? Isn’t TrueNAS with shadow copies more resilient to ransomware? I understand the importance of backups but my NAS build tries to reduce the mess of external hard drives. So I start again to copy manually files all day and unplug it from the system?

I am aware of that. Yes, security is important in my case.

I am aware of that. I am a QubesOS user…

That is 100% clear to me, but one part of my security is that I want to be 100% in control over my own data and setup. Time and money is the hard way but I am happy to learn and invest (and I already know some stuff).

Yes!

What is “normal” security? You mean the private by default settings after installation? Basically you say I should just install pfSense and do updates from time to time. That’s it?

9

Like I said, I don’t know all the products that you mention. I asume that TrueNAS is an opensource product and I have never used it so I don’t know if it is a good product or not for making backing backups. Since the word “NAS” is in it, I asume it is a NAS and that is not a backup product. A NAS is more like a file server. I use Veeam as my backup solution. What you have to ask yourself if anything goes wrong with your data, can you then still recover from your backup. So if a bugler enters your house and steals al your computer equipment, or a fire destroys all your hardware, can you then still recover from a backup. In your case, yes because you have a copy in an other country. If malware encrypt all your data, waits a month and then locks you out, can you then still go back a month? If you can then it is good, if you can’t then your backup solution is not good enough. I you still have a disconnected USB drive somewhere, then you might be able to recover. So if you want to know if your backups are good then think about all the things that can happen with your data and if you can recover.

What I meant with normal security with pfSense is just using strong usernames and passwords. Keep in mind that strong username and password combinations that are not used twice are almost unbreakable. So if you are the only person who is using that system and you know you have a good username and password then that is fine. Better is however multifactor authentication. To use that you have to set that up. I don’t use pfSense yet but typicaly you have to setup a radias server to do so and use some cloud service to set that up.

Some things that you also have to understand, because you are stressing that you want to have everything in your own hands. If you use opensource software then that does not automatically mean that it is suddenly (more) secure. Yes, more people can correct the source code if they want to but will they actually do that? Sometimes they will but sometimes they won’t. Yes, the concept of open source is better then closed source but it does not do miracles by itself.

Typically the user is the weakest link. I work in a relatively small business with less then 100 users and we are very strict with giving access to users with VPN. That has two reasons. The first reason is that our management want people to be free to do whatever they want when they step out of the office. So they don’t have access to email and people are typically not being called in their free time. This is to keep people away from a burnout.

The other reason is of course security. Although we use multifactor authentication it can still be a risk. We have seen that an user approved a VPN connection for a collegae. In a lot of offices it is almost common practice to share usernames and passwords with collegaes. I am not saying that you work insecure on purpose but a mistake is easily made when it comes to security.

10

Yes, I am aware of it. It’s not my first iteration and Tom’s videos are amazing!

Besides my double box merge/replication question (check out my reply to Tom) I just want to have my Vaultwarden, TrueNAS and media server stuff accessible. On the other location I just want to replicate to the other NAS box. So do I need Remote Access or a Site-to-Site VPN? Please help me out on that.

Because you have mentioned other VPN solutions. Is it possible to use something like ShadowSocks or V2Ray on the pfSense router so that I can break through censorship?

I am a friend of privacy. Is there any DynDNS provider I can pay with cryptocurrency?

Thanks for the warning. If I use WireGuard on my pfSense I only have to open a port for WireGuard and everything else is protected right?

I don’t really feel comfy with having a ton of software on my TrueNAS (I know about jails). If I need Nextcloud for sharing files with strangers (who don’t have an WireGuard setup for Remote Access) I think I would run Nextcloud on a second mini PC (maybe even a Raspberry Pi?) and separate it to it’s own VLAN and open a port for it. Does it make sense? Site-to-Site VPN would limit the userbase to the groups to the users on the two local networks, right? I would like to have some followup on this. Thanks!

So my media server which stores files on TrueNAS or my laptop I send a ton of files from should be in the same VLAN for speed?

That’s a cool idea, but I think I want to work with some small server hardware boxes first. That’s a good idea to improve over time and raise my game in the future.

11

Thanks for the info.

12

Well it’s just a hypervisor, so you don’t waste any resources (RAM) needing an OS. Pays your money, takes your pick. Obviously you can run a vm on your laptop but I find for things like Nextcloud it’s super convenient to have a box.

You need to look on their site, to see sorts of speeds can be achieved on the box, I’ll have to admit it’s not super clear to me, I know Negate also has a page comparing WAN speeds for devices. I don’t even come close to 1Gb on my WAN.

LOL old and clunky ! Might be. I don’t run it myself but for sure it looks like it has fewer options than OpenVPN. You can compare security and access privileges, see if it gives you what you want. With pfSense you can also use FreeRADIUS and combine this with OpenVPN if you want 2FA on a RAS, don’t know if you can do that with Wireguard.

You definitely want more than one way to access to your sites, at least for me, having a site-to-site and RAS will easily give you that.

Yes you can use pfSense for local resolution, I don’t, but I do believe there is a difference between using either the Forwarder or Resolver.

I think you won’t be able to buy a recent Intel processor without it.

Maybe it’s just me but I try to use a lagg connection wherever I can. Between my pfSense box and switch I have 4 links in a lagg, if one goes down I still have the other 3 available. Between my Proxmox box and switch the quad NIC is in a lagg, again is one goes down I still have the other 3 as well as the onboard which I use as a management port.

Might be overkill but the price difference for a 48 port switch and 24 wasn’t 50%, so I have plenty of ports for redundancy.

I have 2 networks in two countries so I’ve tried to set things up such that there is redundancy in the event of failure, speed is secondary for me.

13

Thanks for the answer!

How would you set up the media server (Jellyfin and some tracker apps…)? Virtualized or on a physical box? I’ve seen that a Raspberry Pi can transcode a single 4K stream with hardware acceleration, but I’d like to have more, like 3 parallel streams.

How would you set up Vaultwarden?

I end up with PiHole, Vaultwarden, Jellyfin, Jackett, Sonarr, Nextcloud, the UniFi controller, etc. Can I have it all running on a single server or is that bad for ease of setup or security?

Can I visualize all of that with Proxmox?

Additionally what is your opinion on PiKVN? What does it make sense for maintenance especially if I am not physically close to my server?

14

I have looked at Jellyfin in the past, at that time it could not read files held remotely, perhaps it has since changed.

Have ripped a few of my blurays on my NAS. Personally I don’t see the point/need for transcoding, on my home network I can get my devices to access the “full-fat” files without any issues. I’ve used various devices, RaspberryPi’s, Android TV boxes etc, but I find them all a bit crappy for watching media sooner or later. Instead I bought a cheap Lenovo m900 tiny, and use that with linux and kodi, no issues.

If you are streaming from one network to another, you ought to be able to buffer enough of the file for it to stream smoothly on say your phone or tablet. When transcoding you then need more processor power, perhaps it doesn’t have to be that high as my NAS celeron does it but I don’t use it.

I don’t use Vaultwarden, I use KeePassXC mainly because it has a feature for using TOTP, I also keep a copy of this application on a vm just in case of a disaster.

Can’t see any reason why you can’t virtualise those applications on Proxmox, how well they work will depend on the kit you have. I use an i7 desktop and 64GB of ram. Just backup the vm to your NAS.

As for security who knows until something goes wrong.

I’ve seen various expensive solutions to access the BIOS machines and the Pi solution is the cheapest. As a rule, I do no maintenance unless I am at the location. I’ve knocked out my router on a couple of occasions playing :rofl: so that device won’t help.

I think if you basically get some half decent kit you can work the rest out, I try to stay away from ARM and android as they always eventually disappoint even if they are cheap.

15

Go with the TrueNAS and pfSense. With your pfSense install the add on pfBlocker-ng it allows for geo, IP and DNS blocking all done from within pfSense. Keep your TrueNAS on a vlan access remotely via VPN then SSH or SHTML. TrueNAS has good access controls that are very granular down to the file level, so access control is not a big issue. Last use the encryption available in TrueNAS.

Backup and replication are two very distinct things. Do both and practice 3-2-1. Three copies of your data, (1 in production use, 1 on pren backup) and one offsite. I would recommend Backblaze for your offsite BU.

16

What do you mean with “pren” backup?

17

Please excuse the typo should have been prem short for premises. Extra copy of you system(s) OS, applications and data that is close at hand.