I recent got a shiny new fiber connection (pun intended) that replaced my old and much slower DSL connection. Since then i got interested in the performance of my new connection. Now troughput is great and latency seems to be good as well. But i keep struggeling to understand the RTTsd number pfSense is reporting to me. Let me explain. I have my default WAN gateway just setup as default and also monitor the default gateway. I get a RTT of about 3ms which seems great to me, as alternative i set monitoring to 1.1.1.1 for some time to have something else to compare. Then i get a RTT of about 5,5ms also great. But the RTTsd that the monitoring reports is all over the place minimum of about 1ms up to 12ms, and that is when there is hardly any traffic going trough. Ok so that made me think my connection is fast but not that stable. But then i tryed Diagnostics/Ping ping the same address as i use to monitor set the number of pings to 10 and run that like a 100 times in a row and check the last number (stddev) that it reports and that is always 0,0xx ms. Why is RTTsd and the STDDEV of the ping function so much apart i did think they represent the same thing?
I assume you know that RTTsd is RTT standard deviation.
Maybe an image would help understand your issue.
my gateway has a RTT of 9.2 with RTTsd 2.1 it may be [9.2 - 2.1 , 9.2 + 2.1] (one standard deviation)
Yes i know that RTTsd means Round Trip Time Standard Deviation. Round trip is most of the time measured with a icmp (ping) request. So that is why i don’t understand why the icmp requests of the gateway monitoring gives different results as the icmp request from the ping option you can trigger from the menu Diagnostics/Ping. As you can see in the image the RTTsd says 5.2ms and the ping stddev says 0.096 ms as the standard deviation of 10 pings to the same address.
The mathematical calculation of the standard deviation i know as well, but for easy understanding for people less good in math the following approximation is good enough.
- Around 68% of scores are within 1 standard deviation of the mean,
- Around 95% of scores are within 2 standard deviations of the mean,
- Around 99.7% of scores are within 3 standard deviations of the mean
Or use a online calculator: Standard Deviation Calculator
The online calculator show that the calculated standard deviation of the Ping function is correct so my first guess is that the RTTsd value in pfSense is somehow broken. However because you do not have all the samples that makes the magic RTTsd number you have no way to verify the result. Which is bad because any guide always tells you to look at the RTTsd and almost never to the manual ping stddev. If people have been staring at a number that is somehow flawed and without the sample data you cannot see that it’s flawed.
I started to veryfy the RTTsd of pfSense with other tools from behind my pfSense box i used CFOS hrping and also setup a smokeping server. Both of these confirm my previous finding that the RTTsd of pfSense is incorrect and the stddev of the ping function in pfSense is correct. Can others confirm my findings?
Ok has been a while since i did a followup to this strange problem i had with RTTsd. The problem is that in all OS variants that default ping has a payload Windows has 32, Linux has 64. The standard ping the gateway monitoring sends has no payload. When i set the payload to 56 (56 is the default payload for a ping in FreeBSD) my problems are gone. I have no clue why the default payload is set to 0 in pfSense gateway monitoring. Before people jump on the bandwagon that you try to save every byte when monitoring, minimum ethernet frame length is 64 and sending pings with no payload means only sending a header that has a size of 20 so the remainder will be padded anyway to get to the minimum of 64. If the makers of the OS pfSense say 56 is the best for this OS and all other OS makers do not use zero either i have no clue why Netgate does it different then the rest of the world.
