Have been using PFSense and OpnSense for years but only with a WAN and LAN.
OpnSense is virtualized on a newly installed Proxmox 8 with the latest updates.
4 Port Intel NIC with a Linux Bridges setup for each port.
Each bridge is assigned to the OpnSenseVM as a an ET1000 adapter and the Firewall is disabled on all but the WAN bridge.
RFC1918 and Bogons are not enabled as the WAN is an RFC1918 network as well.
Disabled IPV6 on all interfaces
I may switch to Hardware passthrough if I can get the routing to work.
WAN is network 10.0.0.x
LAN is 10.1.1.x/24
PVT is 10.20.20.x/24
DMZ is 172.168.10.x/24
Default Firewall rules in LAN allow access the Internet and work fine.
Cloned the default IPV4 Firewall rules from the LAN to both PVT and DMZ
LAN, PVT and DMZ can access the Internet but cannot traverse OpnSense to talk with hosts on other networks.
those are the only rules…it is really puzzling that I could connect to the hosts 10.20.20.28 and 10.20.20.27 hosts yesterday but now. The .27 host was decommissioned and a different host was added as .30 which is accessible from the 10.1.1.x network but I can no longer connect to the .28 host. I have even rebooted OpnSense.
Maybe it has something to do with the fact that OPNsense is virtualized. Unfortunately I can’t help with that since I have no experience with such a setup.
So I think i finally found the problem after reinstalling all systems…swapping out switches and NICS…
Pretty sure it was a faulty network cable. It worked intermittently then would fail at random times. Replacing the cable also improved overall throughput on the PVT network and reduced CPU spikes of 20% to 4-5%. All seems to be good now