So recently, I upgraded my Netgate SG-3100 to a 6100. The main goals I wanted to accomplish with this upgrade were to move to a supported platform since the SG-3100 is EOL, upgrade to 2.5G LAN, and reduce my dependence on daisy chaining switches. One problem is that I live in an apartment and my landlord won’t let me do any upgrades (I asked for permission as the leasing agreement tells us to do; and I got really a agressive response from the leasing manager). Yet the larger problem is that I didn’t really understand what “unswitched” meant when I bought the 6100. I honestly thought that it meant the ports can handle more throughput. In reality, the unswitched ports basically put devices connected to them on different subnets. This is somewhat annoying because of how my apartment is wired up. The networking equipment is installed in the laundry room. The laundry room has three cables going to different rooms in my apartment: a CAT5e cable that goes to bedroom, another CAT5e cable that runs to the living room, and the final one that I ran along the walls and ceilings to give me a CAT6 drop to the living room. I wonder if I should get a switch and connect it to the 6100’s 10g port that I have marked as LAN, or if I should bridge some of the other ports to that one. If I get a switch, do you guys have any good recommendations? I want to be able to rack mount it in the future of possible. This has also created problems with ny HomeLab itself, but I hope I can solve most, if not all of those problems by port forwarding from my LAN interface to the interface that I’ve aptly named HOMELAB. I think the biggest problem is that I’ve unintentially isolated HomeAssistant from the Zigbee and ZWave networks too. Obviously, I could put the HomeLab on a switch or bridge to the LAN, but I kind of like the separation; and I am willing to sacrifice some short-term convenience to figure things out. But recommendations and pointers from those of you who are more knowledgeable about networking would be amazing.
If you are looking to setup VLANs then I really like the UniFi switches and ecosystem but they do require running either a cloud key to manage the switch or running their free UniFi controller/server software. There are options out there that work well such a MikroTik but the learning curve for setting them up is much higher.
If you ARE NOT looking to use and VLANs then there are lots of inexpensive options out there.
a few remarks that let you find the direction
-
even if the netgate ports are not on a hardware switch internally I think you can still configure a software bridge across several ports A, B, C and all cables connected to these ports would see the network(s) configured on the bridge. You could define tagged traffic for various VLANs on the bridge, so you can have all the same networks on all the cables connected to A, B, C. This setup does NOT require a switch device at all.
-
the other solution is having a switch device that you connect to the port of your netgate that is carrying this network and then connect several CAT cables to the switch. You could configure several VLANs on the netgate interface that connects to the switch, and also configure the same VLANs for the connected switch port. Then you can allow some of these VLANs on the other switch ports.
If you don’t want VLANs, buy a cheap unmanaged switch. If you want VLANs, I’d try the Unifi Switch Flex Mini (it is the least expensive VLAN capable switches in the Unifi lineup).
Since both you and @LTS_Tom brought them up; what exactly would VLANs allow me to do?
given the proper use of network addresses and network masks you can have more than 1 network on the same given CAT cable / or on the same unmanaged switch. I don’t know how common such a setup is these days. There is nothing that would prevent any device on the same siwtch to receive/send in all of these networks.
what nowadays usually is done if you can several networks on the same cable/switch is that you have a managed switch (which means that it has additional functionality, including the VLAN capability). You can then define for each port, which VLANs are carried on the connected cable. A device connected to a port can only see the VLANs that are allowed on that port, and CANNOT receive/send to other VLANs without going through the router firewall.
VLANs allow you a stronger separation of your networks, which may or may not be a useful feature in your home network.
For me personally it is a nobrainer, given the low price of the Flex mini switches. Having VLANs does not hurt and later you may want to have them anyways.
Hmm. That adds some options it seems, but if not for security why would you need separation of networks? It seems like VLANs don’t provide any security benefits. One thing that I’ve considered doing is isolating my Wireless network from the rest of the LAN, but doing so has some key drawbacks - namely allowing my phone to cast to or control the TVs. Could I connect my WAP to it’s own interface on the PFSense firewall, and then create a VLAN on the LAN interface that gives the LAN access to devices on the wireless network? I hope that makes sense; but thinking about it as I try to type it out, it doesn’t seem like it would be a valid use-case for VLANs.
I may not correctly understand your question. But these are true:
- DLNA, AirPlay and Chromecast, Musiccast can be made to work across VLANs. The segregation of those VLANs is somewhat questionable as it requires opening large port ranges.
- a decent WAP maps SSIDs to VLANs, so if you have more than SSID the device connected to a SSID is placed in the associated VLAN
- VLANs definitely do provide security benefits as long as you do not overprovision the allowed VLANs per switch port. all devices connected to a switch port can access all VLANs provisioned to that port - and they CANNOT access VLANs that are not provisioned on that port without going through the router firewall (I think I now repeat myself, and if that does not seem to be a security benefit to you then you may have a different understanding of network security).
So I have another question. Here are my interfaces now:
My network is sort of a “it just works” kind of state. I haven’t even taken the time to configure this device with many of the configurations on my old SG-3100. It had a built in switch interface that I used for the LAN, so the setup was much simpler. As a result, I’ve not made any course of actions to get things that aren’t absolutely necessary up and running.
But I need to solve a few problems. First, my HomeLab is completely isolated from the rest of the LAN. I kind of like this for security, but that means I cannot SSH into it. I can do a Wireguard tunnel for most SSH tasks; but for any large network writing operations, I want something faster because this was the main reason I went for 2.5G (during my time using Wireguard, it seems to show up to the operating system as a gigabit interface). Are my assumptions about Wireguard’s speed limits false? What else could I do to provide access? Could I forward traffic from the LAN to the HOMELAB interface using the firewall’s port forwarding rules?
Also, on the topic of VLANs, could I use them to create a separate VLAN for my wireless network and IoT devices? The IoT devices present a problem too because the homelab has to communicate with the ZWave controller and this Zigbee coordinator. I can probably easily give it access to the ZWave controller, but for the Zigbee coordinator, it’s just an ESP32 microcontroller, so idk if there is a port to forward, and I definitely cannot get a Wireguard tunnel set up on this thing.
What kind of uses cases use these large network transfers?
Usually these would be large file transfers,
or a large volume of small file IOs (virtual machine disk access)
Both of these have a filer / NAS system as one endpoint. You would design your network in a way that NAS file traffic is not routed, i.e. that traffic would not touch the pfsense at all.
Yes you can even NAT the address space of each VLAN and then forward ports for the services that are used across VLANs. That way the IP addresses in a given VLAN are hidden from all other VLANs. You can use a jump host concept to jump into a VLAN and from there further. You can implement this using SSH plus VNC, or Teleport, or KASM Workspaces.
Yes, you can have a dedicated VLAN for IoT and a corresponding SSID for IoT, all IoT wireless and wired devices would be in that VLAN. You then havea different VLAN and different corresponding SSID for your non-IoT devices. Normally prosumer WAPs can handle something around 5 SSID-VLAN pairs.
You would be using Z2M addon on HA for this and define the IP:port pair of the coordinator as the serial device in the Z2M config. I am using a similar coordinator, also based on ESP32. You could e.g. forward this port through the pfsense through to the coordinator IP.
Well, what I refer to as my HomeLab is one x86 machine running RHEL. I like building PCs, so it’s built from a mixture of consumer and prosumer parts. Most services that run on it are Podman containers, others aren’t containerized, and 1 KVM virtual machine (for now). My self-hosting setup is actually very similar to the one Tom made in this recent video:
Anyway, it’s not like an off-the-shelf NAS or anything with all the goodies that would come with that. I just run an NFS server on it. However, when I go to upload large files now, I’ve started preferring SFTP so that the NFS server just acts as read-only network storage. My other use-case is for backups. I want to start doing automated backups. I also use the server as a Syncthing hub, but that rarely results in a bunch of file transfers all at once. Just out of curiosity, what do most people use their NAS for? I currently just back up media and stream it, primarily.
I don’t understand what the jump host concept is tbh. Also, can you have a VLAN operate on more than one network? Maybe this is what you’re getting at, but fundamentally, the IoT VLAN will need to run on the LAN interface. However, the HomeLab is the IoT hub, so it will also need to be able to see the IoT VLAN.
Right now, I have a Unifi 6 Lite; but I installed OpenWRT on it, though, and I currently run it as a dumb AP because I’ve been lazy 
. Here’s the Wireless Overview with the OpenWRT firmware. It looks like I can add more SSIDs to the radios? I’m not sure. I ran OpenWRT on my router before I moved to PFSense in 2021, so that’s a big reason I installed it on here. If I need to move back to Ubiquiti’s firmware, I will. Honestly, I don’t expect you to to tell me how to do it; I can do some research when I get ready to tackle this.
Also, can I just say thank you @LTS_Tom for all of the information you put out on YouTube. I moved to PFSense because of Wendell from Level1Techs, but your channel is honestly my number one resource for trying to do X thing on PFSense. I’ve honestly set up a port forward so few times that I cannot remember how to do that. (Honestly, that’s mostly thanks to Wireguard because I use it like everywhere).
I just made this rule using your old port forward video.
But anyway, I just tried ssh 192.168.15.1 from my desktop which currently has the address 192.168.57.145 and it worked. This honestly seems like magic to me. Why does this work although the desktop is on a different subnet. Is it NAT?
You use cases have pretty low speed requirements, only if you copy some large media files from A to B you would profit from a fast network.
I cannot speak for others, but I already mentioned what use cases I have: transfer files to/from network file shares on a NAS (bulk transfer), and use shared storage for virtual disks of virtual machines running on hypervisors (lots IOPS), and warm migration of virtual machines (bulk transfer).
I suggest that you use the search engine of your preference.
I am unsure what you mean by network in the context of your question. Thus it is hard to answer.
You can have several VLANs assigned to a single interface.
You can have your IoT VLAN assigned to the same interface as you other networks. Then you need to connect a VLAN-capable switch to this interface to break out the different VLANs to different ports of the switch.
To be able to access resources in the IoT VLAN from other VLANs you have some options. I already mentioned using a jump host / jump server. Another option is just having firewall rules that allow selected accesses across selected VLANs. You could also work with a reverse proxy on the pfSense.
I used OpenWRT for 10+ years but only as a router, never the AP functionality. I cannot be of any help for that. Personally I would consider running OpenWRT on Unifi hardware a unfamiliar move because usually one buys the more expensive Unifi hardware to be able to use the Unifi controller software to manage all devices in one spot. My recommendation would be to put back the Unifi firmware. This will highly raise you chances of getting help in this forum. Unless you want to try the OpenWRT forum of course.
For the part that you quoted, yes. That’s a low speed requirement.
[quote=“CodeDragon57, post:10, topic:23795”]
However, when I go to upload large files now, I’ve started preferring SFTP so that the NFS server just acts as read-only network storage.
[/quote]
I tend to upload large files over SFTP to the NFS server. I probably wasn’t clear about this part. A lot of this has been me slowly working on upgrades, and planning for future expansion too. Sure, I don’t have a lot of use cases for 2.5G, right now, but I have a few and I expect to have more and more as my homelab project grows.
Oh yes, it was quite clear. Let’s be honest, it is not crucial whether a media file takes a bit less or a bit more time to transfer once from the ingest location to the media server location. It is rather nice to have if that is faster but normally that would happen in an automated way in the background.
If you compare this to a VM taking long for a warm migration from one hypervisor to another and you have to actively wait for the migration to conclude then you see the difference.
But to make this more convenient in your case i’d have the internal NAS mount a share from the media backup ingest machine using SMB and then some process on the NAS initiates a copy operation. I assume the NAS and the other machine reside in different VLANs where the media ingest VLAN is untrusted or less trusted than the NAS VLAN. So the copy operation would go through the router firewall. Be aware that the faster your network is the more potent the router needs to be. pfSense does not really run at 10G line speed for single data streams, so you’d not get a 10G speed file transfer. This is ultimately a limitation of the CPU architecture and carrier grade routers use ASICs to do the routing in line speed.
This is why it is normally recommended to have the NAS with a virtual interface in all VLANs where it has clients so that the file transfers do not need to be routed. From a security perspective that is a bad idea if there are clients in untrusted VLANs. In that case the solution described above is viable but performs worse.
Oh I misunderstood. 10G is 100% overkill for my needs, but when I was searching for 2.5G switches; I found this switch on sale and couldn’t resist. So I bought two of them and figured I’d connect those two switches to the PFSense box at 10G, but my ultimate goal is 2.5G. In my mind, a 10G backbone just makes it sweeter. I thought you were arguing against me going for 2.5G.
The purpose of the two QNAP switches I bought there is for the drops in my living room and bedroom. I generally prefer ethernet connections over WiFi.
Makes sense to me. All the best.
Well, I just found out that one of our hardware products we’re making/selling at work uses OpenWRT as its firmware. That’s pretty cool, so I may continue using OpenWRT at home because of this. Though I doubt its wireless features will translate into the work hardware.
So, I followed this PFSense docs page to configure additional LAN interfaces and it helped me clear up some misconceptions I had. It turns out that configuring a LAN interface as an Open LAN in PFSense’s firewall settings allows hosts on that interface to communicate with any other LAN subnet in the Firewall. I figured that one could not communicate across subnets at all, but PFSense turns out to be pretty awesome. As a result, my 10Gb wire is on an Open LAN. My landlord’s built-in Cat 5e wires are on an isolated IoT subnet, as well as the HomeLab subnet. I know that the separate LAN wire would be made redundant by VLANs, but I already routed the wire along the outside of the walls using cordhiders from Lowes because my landlord won’t let me upgrade the Cat 5e. 
Also, something else I’ve wondered for awhile is if I can get a LetsEncrypt certificate or something (prefer or something) to get rid of the self-signed certificate warning when I try to access my PFSense GUI.
Use acme package to get lets encrypt ssl certificates