Site 1 and Site 2 used to be connected to each other over wireguard.
Site 1 is a protectli box and was running OPNsense. Prior to loading pfsense on it, I was running Wireguard on OPNsense and iPerf tests were giving me around 180Mbps. Site 2 is a 200Mbps line so this was perfect.
Site 1 is moved over to PFsense and now I cant go above 20Mbps throughput while on Wireguard. I made the changes in System > Advanced, and Enabled PowerD and set it to HiAdapative. Also enabled AE-NI CPU-based acceleration. Same settings I made when it was on OPNsense. No change to speeds.
I decided to change to IPsec just to test if maybe, somehow, the wireguard implementation was at fault. No change and in fact I get the same speeds - 20Mbps give or take.
To make sure I wasn’t crazy, I ran a speedtest.net test at Site 2 running OPNsense and the results are normal - 200/30. So I know its not trouble with the circuit. Same goes for Site 1. I get my 500/500.
The only change made was pfsense so my focus is on that being the item at fault. Anyone have any ideas ?
For what its worth I went through the following guide:
Hey just wanted to update everyone here.
Dont think I’m facing an MTU issue as I’m able to send up to 1472 bytes over the tunnel.
ping -f 192.168.70.26 -l 1472
Pinging 192.168.70.26 with 1472 bytes of data:
Reply from 192.168.70.26: bytes=1472 time=39ms TTL=62
MSS clamping is set on the pfsense to 1400 bytes by default. Im honestly don’t know where the performance drop occurred but the strangest thing is that its only Site2Site that’s impacted. Remote Access coming in via OpenVPN or Wireguard I can download around 60Mbps.
I think the packets are being fragmented but the fragmentation is being hidden from you.
I don’t use wireguard on pfsense, but if your ethernet MTU is 1500, how could a full size packet go through the tunnel without fragmentation, since there is encapsulation overhead?
google clear don’t fragment bit vpn
You may need to look at what is happing at the ethernet interface level with tcpdump to know if the packets are being fragmented. Or perhaps there is another way. I am a pfsense neophyte, as I use an ER-X at home.