Hi! I borked something after the update to 23.01 and took the chance to test Untangle 16.6.2. With the same vanilla configuration, I was getting 620 Mbps from my Omada AP with Untangle, vs 500 Mbps from the same AP with pfsense. I have a 1 Gbps up/down connection. I couldn’t believe it at first, so I installed pfsense CE 2.6 and using stock configuration, tested my WiFi again. Went back to getting 500 Mbps from the same AP. I’ve always thought that getting 500 over WiFi was the best my AP could do. Any ideas on why this is the case? Is there a setting in pfsense I can change to achieve the 620 Mbps I’m getting in Untangle? Thanks in advance.
Personally I think WiFi is always hit or miss at best. Make sure to install the iperf package when testing. That way your not seeing network overhead from the ISP and thinking it is your firewall and you’ll get more consistent results. I’m not saying it couldn’t be pfsense because it would depend on hardware you are running it on that could be a bottleneck.
Just to further isolate have you tried testing it through wired connection? Check if you still have a bad result compared to Untangle. If its the same or better with Untangle on a wired connection, it is possible there is some configuration you needed to change/set between pfsense and your access point.
Thank you for your replies, @xMAXIMUSx , @reymond070605 .
Make sure to install the iperf package when testing.
Just to further isolate have you tried testing it through wired connection? Check if you still have a bad result compared to Untangle. If its the same or better with Untangle on a wired connection, it is possible there is some configuration you needed to change/set between pfsense and your access point.
Yes, I tried iperf. I’m getting 940 Mbps using iperf using pfsense as server to a host. Which is about right. I don’t There’s no iperf package in Untangle I think. But I ran iperf on two free interfaces on the box after bridging and got 940 again. Another thing I tried was fq_codel SQM on both. Setting this in Untangle is a breeze. I set up/down limits at 800 Mb and was getting the correct speed in speed tests. I followed the Netgate documentation on setting this up in pfsense. I was getting ~800 down but about 15% less up.
I’m not saying it couldn’t be pfsense because it would depend on hardware you are running it on that could be a bottleneck.
My device is a Qotom box with Intel N5105 2 GHz processor, 8 GB RAM, 128 GB M.2 drive, and 4-port Intel I225-V(3) network card. I think it could be my hardware + FreeBSD. I already tested this twice. I installed Untangle again, did wired, WiFi tests, fq_codel, wired tests again. Same result as before (wired=~940, WiFi=~620, fq_codel=~800). Then installed pfsense and tested again (wired=~940, WiFi=~500, fq_codel=~800 down/~700 up). So those two things (WiFi and fq_codel up speeds) seem peculiar to me. I also tried OPNsense on the same box. Again I was getting only about 500 down on the same AP. OPNsense was running warmer though (+10°C).
I’m gonna keep Untangle running until the trial ends, but that looming $150 license for the Untangle Home Pro is making me anxious. I’m not prepared to pay that much.
Is this an issue with the i225/i226 drivers under BSD.
Not really sure. I did another iperf test using a host with a 2.5 Gb network card. I was getting 2.35 Gb from the pfsense appliance and 2.20 Gb to it. So I don’t know.
Just an update. I did a bunch of testing yesterday. Omada AP (EAP653) is 2 feet away with no other device connected to the network. Test server used is the nearest one and offered by my provider. It has been consistent.
Red - pfSense Plus 23.01 without traffic shaping/limiters
Yellow - pfSense Plus 23.01 with traffic shaping/limiters (650 up/down fq_codel limits)
Green - Untangle 16.6.2 with traffic shaping/limiters (650 up/down fq_codel limits)
Blue - Untangle 16.6.2 without traffic shaping/limiters
For the yellow results, jitter for both download and upload is less than 10 ms. But when speeds hit more than 600 in any of the results, jitter increases to ~15 ms, both for pfSense and Untangle.
The results are compelling. I’ve always thought that getting 500 Mbps over WiFi was the best my AP could do. It seems like the combination of my hardware appliance (Qotom N5105, 8GB RAM, 128 GB M.2 SSD, 4-port Intel I225-V(3)) and my Omada hardware gives slower WiFi when using pfSense. Could it be because Omada also runs in Linux?
I have a hard decision to make.
I would remove the hardware checksums in pfsense and see if that make an improvement.
The AP should have no bearing on the firewall, if the AP is able to pass 700mbps, then it should pass 700mbps.
It still seems like a driver issue with the i225/i226 NIC. Does the latest version fix this or do you still need to install updated drivers for this NIC with BSD?
I’d still want to see what kind of speed you get from a wired port, especially from the same port the AP is connected, you could have a bad port or bad cable.
Thanks, @Greg_E . Just to add, here’s the wired test results.
pfSense, bottom 3 without shaping, top 3 with shaping (650 Mbps both ways). I tried configurations from the Netgate documentation, Tom’s guide, and SANS Internet Storm Center. They give similar results.
Untangle results, bottom two without QoS, top 3 with QoS.
I also tested using different AP unit (same brand and model EAP653) and using a different port in the PoE switch, I get the same results as the I previously posted. It’s the consistent difference in the test results between Untangle and pfSense (single AP, single client) that I want to understand.
It still seems like a driver issue with the i225/i226 NIC. Does the latest version fix this or do you still need to install updated drivers for this NIC with BSD?
It must really boil down to this, i.e. the difference in drivers. I don’t think it’s an issue with the i225 as I’m getting 2.35 Gbps in iperf tests. I wasn’t even aware there was an issue, I just assumed it’s already fixed in 23.01.
I’m not sure where to go from here as you are definitely seeing a big difference between the two.
When you are running speed tests, have you looked at the stats page on pfsense to see if the processor is spiking or some other thing that might give an indication of why it is slower? There might be some more advanced tools out there, but I’m not sure what we can monitor.
All I can say is that with my old Atom 2750 I can get 900mbps each way even with Suricata running, your computer should be much faster than mine. I would go through and check to disable all forms of offloading and see if you get an improvement, I had to do this on my NIC at work because it is a little too old and it was definitely causing problems. Let the CPU take all the load and give it another try.
Also make sure you have hardware AES-NI checked, your processor should support it and maybe this makes a difference.
2 Likes
Thanks!
When you are running speed tests, have you looked at the stats page on pfsense to see if the processor is spiking or some other thing that might give an indication of why it is slower?
It jumps to about 8% and temperature could jump to 75°C.
I would go through and check to disable all forms of offloading and see if you get an improvement
Will try these in the weekend. Although I did the tests using stock configuration of 23.01 with AES on and off.