VLAN headache from switch config

Networking & Firewalls
1

I apologise for being utterly ignorant, but I don’t understand what I don’t understand.

I’ve a UCG Fibre, a Cisco CBS 220 POE and a Proxmox server.

I had a plan to segregate the proxmox VMs into a separate VLAN, with their own address range(s).

I created the net VLAN on the UCG. That seemed striaghtforward. Assigned VLAN ID2 and the DHCP of 192.168.2.1 (router) and 5-10 for addresses (this was only a proof of concept).

On Proxmox I set the bridge interface, VMBR0 to be VLAN aware.

I assigned VLAN 2 to one of my VMs.

And the VM (Linux Mint) wouldn’t pick up an IP address in the 2.x range. No command line shenanigans or UI button mashing made a difference. It simply wouldn’t get the address.

This had me looking at the switch (Cisco Business 220 Series Switches Administration Guide - VLAN Management [Cisco Business 220 Series Switches] - Cisco)

And I created a complete pig’s breakfast of everything - everything broke - because I don’t really know what the blinking heck I’m doing.

To my ignorant mind I want the vlan tag to simply pass over any port, to any other port and have the router then work out which VLAN it is in and hand out an address on that basis.

I don’t want to define specific vlans to specific ports as that seems backward and I plug and unplug devices, forget where they go and generally mess about - it’s a homelab, not a production environment.

In my ignorant head, each port is configured as a sort of ‘super highway’ where each vlan has it’s own lane and never the twain shall meet - unless there is no tag and then the traffic goes into vlan 1. Is this possible? Does anyone have this switch/similar and has faced and overcome the problems? Is there a better way to do this by replacing hardware? Would a Ubiquiti switch better integrate and know what port settings to apply?

What do I not know, where can I start to learn?

2

I am not saying that my way is the right way, but it works for me. Here’s what I do: I have only ONE instance of pfSense, not one for each VLAN. I think that is a giant mistake. I have my six VLANs set up in my pfSense box: Trusted, Untrusted, Guest, IOT, Television, Server management. I put all my internet facing services in untrusted, my kids and their friends get the guest network. IOT is for all of my Ring devices, Television is for the smart TVs and Roku sticks, and server management is the web UIs for all of my hardware. Truested is just 2 computers, mine and my wife’s. All the of the VLANs are separated by firewall rules set up in pfSense. The only VLAN that can access the other VLANs is the Trusted VLAN. Its one way access only.

Inside of pfSense, I have set up a DHCP server and unique IP address ranges for each VLAN.

I have the following devices connected directly to my pfSense: Upstream I have two separate WAN interfaces, one by cable modem and one by 5G home internet (Tmobile). They are set up in a failover group. Downstream I have my managed switch. My managed switch is connected via a trung/tagged port on the switch, so that it passes packets for all VLANs to the switch. My switch is configured with a mix of tagged/trunk ports and untagged access ports, assigned to specific VLANs.

My wirelass access point is VLAN aware, and can have up to 8 SSIDs, each assigned to one unique and specific VLAN. So I have SSIDs for each VLAN basically except the untrusted VLAN, which doesn’t need wireless access.

My four proxmox servers are all connected by trunk/untagged ports, so that they pass all VLANs to each proxmox host, and I can decide for each VM which VLAN to attach it to in the hardware configuration menu.

All the rest of my stuff (ring alarm base station, Synology NAS, Pi Star ham radio hot spot, etc.) all connect by untagged/access ports, and only have access to the one VLAN I designate in the switch configuration.

It sounds to me like either your pfSense box or your UCG are redundant and not needed, and will mess you up if not configured correctly. I think you need to choose which one you want as your router/firewall.

3

Hello Lou, thank you for replying. I’m not sure I mentioned a pfsense box. I did have an opnsense router but I have removed that. I’ve only the router (UCG Fibre) to switch (CSB 220) to the proxmox device.

Do I need to configure trunk ports on the router and proxmox ports : and any other port I want to pass vlan traffic of any tag over?

There’s a PVID setting as well - do I need to put the port ranges in that field?

4

My bad, I must have misread Proxmox for pfSense. Old age sucks!

OK, now that I re read it, here’s my thoughts. I wasn’t too far off. :laughing:

Do I need to configure trunk ports on the router and proxmox ports : and any other port I want to pass vlan traffic of any tag over?

I am not sure about the router. On mine there is no special config on the router side. On the switch you need a trung port (untagged port) for all devices that you want to pass all VLAN traffic, so for sure you need that for your Proxmox port(s). My switch uses PVID of 1 (the default VLAN ID) for trunk/untagged ports, and on tagged/access ports (i.e., not trunked ports) I set the PVID to the VLAN I want that port to be assigned to.

Do you have DHCP on each of the VLANs?

For sake of reference this is what my /etc/network/interfaces file looks like on my Proxmox machines

auto lo
iface lo inet loopback

iface enp1s0f0 inet manual

auto vmbr0
iface vmbr0 inet static
bridge-ports enp1s0f0
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 2-4092

auto vmbr0.10
iface vmbr0.10 inet static
address 10.10.10.2/24
gateway 10.10.10.1

Note that in Proxmox, you only need one gateway that points to your router, not one for each VLAN. In this example, the IP address is the web UI of Proxmox.

5

Hullo Lou, you were entirely right on the trunk ports. I configured those connected to VLANs as trunks and picked up an IP address.

I also rather cheekily compared my config to yours and they’re very similar, especially around the VLAN section on the bridge interface.

However, much to my ‘delight’ I then found that vlan traffic couldn’t get to the other vlan for servers, such as my Mint aptcache and pihole.

I’m not sure why, but that’s for another day when I don’t have a headache!

Many thanks for your pointers, I really appreciate them.

1 Like
6

Firewall rules. You may have to make an explicit firewall rule to allow traffic to traverse VLANs. If you have an L3 switch, you can configure inter-vlan routing there, or if not (like I have an L2 switch) then it has to be done in the router. You can make a very lose rule at first to get it working and for testing, but for production, I like to allow specific IP addresses only for specific services and specific ports to cross VLANs. Make the exception as tight as possible.