pfSense WAN High Packet Loss, dropouts

Networking & Firewalls
1

Hey guys

Running 2.7.2-RELEASE (amd64) on a Dell Optiplex with an i5-4590 CPU and using an Intel i350-t4 NIC.

My pfSense box WAN port is connected to a Arris CM8200 Cable Modem (NBN in Australia), I don’t have access to any config on this modem. Speeds I get are 100mb down /50 mb up. I’ve followed Toms bufferbloat youtube guide for traffic shaping which I suspect isn’t causing this.

So far I have tried the following:
IPV6 is completely disabled.
In Wan DCHP Config under DHCP Client Configuration changing the advanced config preset to ‘FreeBSD default’
Adding in supersede dhcp-server-identifier 255.255.255.255 under Option Modifiers
In the Gateway I have enable and disabled “Disable Gateway Monitoring” and “Disable Gateway Monitoring Action” in different configurations/
Have changed the Monitor IP from my ISP provided IP to 1.1.1.1 etc

Essentailly all steps in these threads:

The only thing I haven’t tried is “Use non-local gateway” as I’m not entirely sure if this is safe?

My logs when the packet loss occures:

Under Status > System > Gateways

Sep 27 11:47:57 	dpinger 	13221 	WAN_DHCP 1.1.1.1: Alarm latency 11868us stddev 7254us loss 21%
Sep 27 11:47:57 	dpinger 	13221 	WAN_DHCP 1.1.1.1: sendto error: 65

Lots of “sendto error: 65” lines

Then

Sep 27 11:51:06 	dpinger 	13475 	exiting on signal 15
Sep 27 11:51:06 	dpinger 	13221 	exiting on signal 15

Under Status > System > Logs > DHCP

Sep 27 11:45:38 	dhclient 	60138 	DHCPREQUEST on igb0 to 255.255.255.255 port 67
Sep 27 11:46:11 	dhclient 	60138 	DHCPREQUEST on igb0 to 255.255.255.255 port 67
Sep 27 11:47:43 	dhclient 	41047 	EXPIRE
Sep 27 11:47:43 	dhclient 	41842 	Deleting old routes
Sep 27 11:47:43 	dhclient 	43812 	PREINIT
Sep 27 11:47:43 	dhclient 	60138 	DHCPDISCOVER on igb0 to 255.255.255.255 port 67 interval 22
Sep 27 11:48:05 	dhclient 	60138 	DHCPDISCOVER on igb0 to 255.255.255.255 port 67 interval 39
Sep 27 11:48:44 	dhclient 	60138 	No DHCPOFFERS received.
Sep 27 11:48:44 	dhclient 	60138 	No working leases in persistent database - sleeping.
Sep 27 11:48:44 	dhclient 	91423 	FAIL
2

If you connect a computer directly to the cable model does it have the same issues?

3

The one thing I haven’t tried! Will report back once I do.

4

Issue finally reoccured during waking hours. Short answer Yes.

Plugged in a PC directly into the cable modem and had full internet access, assigned it an ipv4 address.

The WAN_DHCP Gateway showed 100% loss, then after I unplugged the PC and reconnected the pfSense box it stuck on a Status of ‘Unkown’ for a few minutes. I tried restarting the gateway service but it still stayed as ‘Unkown’. So I rebooted pfSense and the gateway went back to Online. The logs are quiet long and I didn’t remember to note the times I did all of this inbetween running running between rooms, but can post if needed.

5

So you are saying that the issue was occurring even when you had a PC attached?

6

You can access the modem status with http://192.168.100.1
(user: admin, default pw: last 8 digits of serial#)

The first tab is status, click on Downstream/Upstream status.

image
image1920×868 72.5 KB

1 Like
7

Occuring on pfSense yes.

PC Directly into modem internet works fine.

(Edited to clear up confusion. Apologies.)

8

Thanks.

There is a a caveat with NBN provided modems is that you need to hard reset them and have a 1-2 min window to access it via this guide.

I’m not entirely certain this is a modem issue.

9

I’ve enabled the “Use non-local gateway through interface specific route.” in the WAN_DHCP. The packet loss is still there and the connection still drops out, but only for about 45 seconds as opposed to 5 minutes.

I suspect this is because I have “Disable Gateway Monitoring Action” unchecked and the gateway is restarting based on the below log entry under System > General

Nov 5 12:36:28 	rc.gateway_alarm 	54123 	>>> Gateway alarm: WAN_DHCP (Addr:1.1.1.1 Alarm:0 RTT:10.827ms RTTsd:4.510ms Loss:5%)
Nov 5 12:36:28 	check_reload_status 	431 	updating dyndns WAN_DHCP

If so, I am hoping that checking “Disable Gateway Monitoring Action” will stop this. Will it affect anything else?

10

Update after a month or so of troubleshooting and testing for anyone else that might have the same issue, specifically Australians with HFC NBN connections.

Tried various cables/NIC’s in pfsense, different router(s), direct cable connections to a PC. Changed ISP from Optus (useless insisted it was a ‘you’ problem) and switched over to Aussie Broadband. Fortunately they have a decent account/app that has a few diagnostic tools, one of these records ‘flaps’.

ABB notcied this was an issue and got an NBN tech out to me house and he identified pretty quickly that the issues were caused by a low signal coming into the Arris Modem / NTD. This was cuased by a left over coax splitter from Foxtel (cable tv) in the NBN unit at the front of the house. He’s removed it replaced the modem with a new one, just incase. 24 hours later and I’ve noticed no signifigant packet loss in monitoring.

TLDR; NBN HFC signal loss caused by foxtel splitter in the NBN panel, which being NBN property can only be accessed by NBN approved techs, was cuasing signal loss.

1 Like
11

Whenever I have an Issue like this it’s always most likely a fault in the network of my ISP. I would say most of the time it’s either a shitty modem (those ISP at least here in Europe cheap out hard with those) or something else in their network.

Here in Austria we have a pretty big coax Network with tons of repeaters, and it just needs one unplugged line that has no end resistor on it and you get ton’s of interference into the network.

Already talked with a technician about that and those coax networks are extremely notorious for such faults.

1 Like