WireGuard issues with pfsense Chelsio nics on dell r210 II

By any chance has anyone had issues with this combination of nics with Wireguard… Repeatedly defaulted on 2 different boxes and on the one with chelsio and the same config it never works. but the small dell with 2 intel nics doesnt have issues.

Or with bce nics

Pfsense hardware is designed for intel nics and prefers them, as you have found out from experience.
"The default installation includes a well-rounded set of values tuned for good performance without being overly aggressive. There are cases where hardware or drivers necessitate changing values or a specific network workload requires changes to perform optimally.

The hardware sold in the Netgate Store is tuned further since Netgate has detailed knowledge of the hardware, removing the need to rely on more general assumptions."

If you take the time to tune the cards correctly it’s easy to max out the connection, including the intel nics. Even though you can plug and play with intel nics, if you tune the hardware you can definitely increase the throughput.

Disable hardware checksum offloading, hardware tcp segment offloading, hardware large receive offloading, as well.

Don’t forget the GENERAL ISSUES as well as the specific cards.

Ah, it’s to early in the morning, so many edits to try to make the grammar correct and paragraphs flow properly.



The settings for Hardware TCP Segmentation Offload (TSO) and Hardware Large Receive Offload (LRO) under System > Advanced on the Networking tab default to checked (disabled) for good reason. Nearly all hardware/drivers have issues with these settings, and they can lead to throughput issues. Ensure the options are checked. Sometimes disabling via sysctl is also necessary.

Card-Specific Issues

Broadcom bce(4) Cards

Several users have noted issues with certain Broadcom network cards, especially those built into Dell hardware. If bce interfaces are behaving erratically, dropping packets, or causing crashes, then the following tweaks may help.

Add the following to /boot/loader.conf.local:

kern.ipc.nmbclusters=“1000000” hw.bce.tso_enable=“0” hw.pci.enable_msix=“0”

That will increase the amount of network memory buffers, disable TSO directly, and disable msix.

Packet loss with many (small) UDP packets

If a lot of packet loss is observed with UDP on bce cards, try changing the netisr settings. These can be set as system tunables under System > Advanced, on the System Tunables tab. On that page, add two new tunables:

net.isr.direct_force=“1” net.isr.direct=“1”

Broadcom bge(4) Cards

See above, but change “bce” to “bge” in the setting names.

Chelsio cxgbe(4) Cards

It is possible to disable the allocation of resources that are not related to the router so that the network adapter can use its entire set of resources for the corresponding functions:

Add the following to /boot/loader.conf.local:

hw.cxgbe.toecaps_allowed=“0” hw.cxgbe.rdmacaps_allowed=“0” hw.cxgbe.iscsicaps_allowed=“0” hw.cxgbe.fcoecaps_allowed=“0”

I will mention that the Intel NICs built into my Supermicro A1SRi-2758F have some problems with TCP offload, I ended up buying a used Intel card to get that working. An i350 card is probably the choice going forward for me (assuming they don’t reassign my job duties).

Thank you for everything greg it looks like what i discovered was that when gateway monitoring was enabled and the interface was set to dynamic the tunnel would shut down. But when i disabled monitoring it began to work! im still buying Intel Niccs tho lol