When I started the process of buying a new house, I’ve dreamed about how I was going to set things up. I was dealing with many theoreticals, but I’ve been in the new house for nearly 3 weeks now and I’ve nearly worked out all the kinks to my networking, except for one frustration.
The primary hardware in my system is a pfSense router, three stacked Aruba S3500-24P network switches (interconnected with 10 Gbps fiber links), and three Ubiquiti Unifi UAP-AC Pro access points. I have had no issues with my primary LAN network but having issues with my VLANs. I created three VLANs; Guest, IoT, and Cameras. I followed instructions seen on Lawrence’s YouTube videos for setting up my VLANs. I gave “Guest” a VLAN of 20 and a subnet of 172.19.0.0/20, “IoT” a VLAN of 30 and subnet of 172.19.16.0/20, and “Cameras” a VLAN of 40 and subnet of 172.18.16.0/20. My primary LAN happens to be on the subnet of 172.18.0.0/20. I set up all the rules for the firewall as instructed.
I created a switching-profile on my Aruba switches that when assigned to a port, makes the port a trunked port, tagged with VLAN 1, 20, 30 and 40, with VLAN 1 being the access/native VLAN. I have applied this profile to the LAN port from pfSense, each of the UAP-AC Pros and to my XCP-ng server.
I have a VM on my XCP-ng server running Ubuntu server and Unifi Controller. In Unifi Controller, I set up my primary LAN as a “Corporate LAN”, then I created a Guest network on VLAN 20, IoT on VLAN 30 and Cameras on VLAN 40. Under “Wireless Networking”, in addition to my primary LAN, which has always worked as expected, I set up an SSID for each of my VLANs. IoT and Cameras are both set as the default user group while Guest is set to the Guest user group.
Now, I can log into each of the SSIDs from my cellphone but initially, I could not get out to the Internet on any of the VLANs. I would look at the network setup and I was issued an appropriate IP address from the DHCP server for each of the SSIDs, including all the correct routing information but no Internet. I tried modifying my firewall rules on pfSense but nothing would change.
Yesterday, while at work (VPN’d from my laptop at work to my network at home), I got a brainstorm to play with VLANs in XCP-ng. I created a network interface for each of VLANs, in addition to creating a new interface of my LAN, giving each an easy to understand name. I then opened up a VM I call my “Sandbox”, where I do experiments. I then went through and activated each interface, one at a time. I d I needed to make a few rule changes on my firewall settings, but ultimately, every VLAN was working exactly as it was supposed to. I would be issued an appropriate IP address and had full access to the Internet but no access from a VLAN into either any other VLAN or my primary LAN. I also tested and was able to SSH from my LAN to my VLAN but unable to initiate an SSH connection from the VLAN to the LAN. This was exactly what I was expecting it to do. I was anxious to get home to test everything out, once I got home.
When I got home, I immediately tried to connect to my Guest network. I could see that I was given an appropriate IP address, but alas, I couldn’t actually do anything… no Internet connection. I then changed to my IoT network. WHOA! I have Internet! I was able to do everything I expected I should be able to do and not do what I expected I couldn’t do. I then tried my Camera network… no dice. The Camera network was just like the Guest network; it would give me all the proper DHCP information but no network connectivity. I pulled out my laptop and found the same results as my phone. I then pulled up everything I could think of remotely related to VLANs, from pfSense, my switches and Unifi. The only difference anywhere that I could see was that on Unifi, the Guest network was assigned to the Guest group, but I had the same issues with the Cameras network and it didn’t belong to the Guest group but the default group like the LAN and IoT networks. I pulled up the logs on Unifi and saw nothing different in the logs when connecting via one SSID versus another. I pulled up the logs in pfSense and again saw nothing alarming. I could see the DHCP log issuing addresses and nothing unusual between connecting to a network that worked versus one that didn’t. The only thing that I found that was somewhat unusual was in the DHCP leases. While I could see the different DHCP leases created by either my phone or laptop, as well as my “Sandbox” VM, I found it a bit odd that each of my UAP-AC Pro units had an IP address assigned by the Guest network DHCP server in addition to the LAN DHCP server, but no IP address for IoT or Cameras. I’m not sure WHY the access points would have an IP address from the LAN and Guest network but not the other two VLANs. I would have expected that it would ONLY have IP addresses from the LAN and none of the VLANs.
Anyway, I have a partial success, in that I have one VLAN WiFi SSID working, but stumped why that one works and the other two do not. It appears that the issue is NOT with pfSense or the network switches, as a wired network has no issues between the different VLANs, leaving the issue somewhere within the Unifi sphere-of-influence. I’m baffled.