Hello everyone!
I’ve completed a new server build with XCP-ng on a Ryzen 5650G. I have a 10Gbit Mellanox card in the hypervisor and vlan virtual interfaces. I have virtualized all interfaces of pfSense instead of passing through a nic because I’m limited on pci-e slots on this board - both the WAN and LAN are vnics. Tx checksumming is turned off on all interfaces. The pfSense instance only routes to the internet - no inter-vlan routing.
For whatever reason the maximum speed I can achieve to the internet is around 200 mbps. The CPU in pfSense is pegged with interrupts, limiting the flow. More vCPU’s doesn’t help, as whatever the bottleneck is seems to be single threaded. Changing the NIC from e1000 to RTL makes a minor difference in speed but still pegs the CPU.
The box was migrated from another host, so thinking that might be part of the issue, I’ve remade a fresh pfSense and OPNsense box. The highest I’ve seen was OPNsense at ~270 mbps, but it had no firewall rules or services running. It’s nowhere near my line speed of 900 mbps.
Other Linux VM’s have no issue emulating the nics to well beyond 1Gbps. I’ve seen up to 3Gbps on my virtualized TrueNAS install which I believe is still based on BSD. I had no issues with pfSense and speed on my old server with physical NIC’s passed through.
Anyone have any ideas? I’ve done my best to follow all threads with similar issues so far to find no difference.