Packet Loss and Latency Monitoring

Networking & Firewalls
1

I just watched the last YouTube video about Packet Loss and Latency Monitoring with pfSense and want to make sure I am measuring the right thing.

I have synchronous 1 Gigabit AT&T Fiber Internet at my house. The fiber that comes to my house runs between what I believe is a passive optical splitter that is situated in an underground box a few houses away. It comes to my house and plugs into a small box that plugs into my home’s power and has an Ethernet cable coming out of it. Unfortunately, I am told that I cannot plug that straight into my pfSense router but unfortunately have to plug it into an AT&T-provided “modem”, which has been bridged to my pfSense machine. The AT&T modem and pfSense box are next to each other on a shelf in my computer rack and connected with a 1-foot Ethernet cable.

I am not sure how I found the particular IP address I put as my “IP Monitor” for my Gateway in pfSense, but if I do a traceroute to that IP address, I see my gateway as the first hop and the address I have entered as “IP Monitor” as the second hop. I am not sure if this “IP Monitor” is before or after it has gone from copper to fiber. If I try to traceroute to anywhere else, the second hop always shows “* * *”.

I was following along with Tom as he went into Monitoring quality and created several different views… 1-Day, 1-Week, 30-Days, 3-Months, and 1-Year. By looking at the year, I believe I must have changed from the older pfSense CE to pfSense+ on a ZFS filesystem on February 25th of this year, as that is the first I have any information.

Year
Year1024×597 79.9 KB

I see that there is very little packet loss for me, but it appears that there have been a few occasions. Looking at the past month, I see some delays, including a several-day period of delays between October 14th and October 17th.

30-Days
30-Days1024×597 85.7 KB

From that monthly view, I saw a drop in packets, but that was in the past week, so here is the past week:

Week
Week1024×597 76.5 KB

Apparently, on October 31st at 3:00, there was a drop in packets, followed by a short bit of increased delay averages.

Any ideas as to what may be causing this? I’m not complaining about my service, just curious and want to know what I might do to better understand what is happening.

2

There is not really any way to know the “Why” if it is happening on the service provider side and they would have the logs. As for testing, ping the first hop, unplug the fiber from the device, can you still get a ping response. If so, choose another IP within their network.

3

As a long time network engineer at a large telco/ISP (now retired, unless some wonderful opportunity pops up), you should be careful about assigning too much significance to ping (and traceroute) results, especially if you’re pinging a device outside of your control. ISP routers pretty much always have configurations that throttle the amount of non-essential traffic to the device itself (Cisco calls it “Control Plane Policing”), and this can cause apparent packet loss if the device being pinged is experiencing even slightly higher than expected levels of ping traffic. These devices also often consider ping traffic to be of a lower priority, which can often result in higher levels of intermittent delay (i.e. jitter) to ping traffic. The ping utility is, first and foremost, a connectivity test - is the destination device reachable at all. The fact that it can report response times and packet loss rates adds utility, but unexpected results should be verified with additional, more robust testing rather than just being blindly accepted as an indication of trouble. Again, this applies mostly to situations where you’re pinging someone else’s device. If it’s your own endpoint (e.g. server) across a network then you can be more confident that the results truly represent the behavior of the intervening network.

Hope this helps!

4

I think I see this on my Internet connection. I have a Netgate 2100 which is mostly fine. When I download a large dataset >4G (up to 40G) I see a huge spike in packet loss going from 0.0% to 10% +, all the while I still transfer data at about ~520Mbps. I think the gateway or router is sacrificing ICMP packets since at 10-20% packet loss I would expect to see some network slowdown. What’s really odd is if I run a long fast.com speed test I get the same transfer rate, but no packet loss. I think because Netflix often puts edge equipment in ISPs dataceters. Weird.

I think the only way I’ll know for sure is to isolate the 2100 and see if it can do the advertised speed without loss.