OpenVPN RTT Way Too High

Networking & Firewalls
1

So, I’ve been having this peculiar issue lately. About a couple of weeks ago, I started noticing that my real IP was leaking on Apple TV as I tried to watch shows across various streaming services and would get told that it was unavailable in my region. So, I’d run a quick SpeedTest app on Apple TV, and it’d show me my IP. It made no sense why it’d do that since pfSense is configured to route only VPN traffic to Apple TV (but with no killswitch).

I started investigating, went to the main page of pfSense, and enabled Gateway monitoring. And I noticed that my OpenVPN RTT was going way too high, in a few hundred to even thousands — which would cause the gateway to go offline, let my WAN traffic go through to Apple TV, and leak my IP.

I’m attaching a couple of screenshots so you can see how bad the ping goes, and it only starts to go up when I try to use any device with VPN routing on.

image1
image2

I have absolutely zero issues using the internet through pfSense when it’s not routed through OpenVPN, as I have an alias list of IPs that includes my desktop and that I exclude from using any VPN traffic. I am on my computer practically all day doing various things, including gaming, and I never face any issues. It is only happening on the VPN gateway.

Now, I’ve done ping tests to this VPN IP overnight, over hours, and so on and on, and the avg ping never goes beyond 300 even, which is fitting because an avg ping from my location to that VPN’s location is supposed to be around 280 anyway. I’m also attaching a screenshot of WinMTR of that result, so you can see there isn’t any issue with ping when pinging that VPN server IP directly from my ISP internet in a desktop environment.

image

This started happening about a couple of weeks ago; it was totally fine before that. After running into these issues, I updated my pfSense+ to the latest firmware, but before that, it was running fine for hours on end, streaming for about a month.

One more thing I should note if it helps someone understand why I’m having this issue is that whenever I do a SpeedTest of the VPN via OpenVPN environment through pfSense, as in if I’m doing it from Apple TV or desktop, my ping shows about double on SpeedTest like 500-600. But if I connect to the same host via L2TP/IPSec via Windows PC, the ping stays around 240ms or so for SpeedTests.

So, as you can see, the server itself is fine; I can ping it, and the service is good and all. But when using OpenVPN via pfSense, it causes a severe spike to RTT, and it only does that sometimes, it doesn’t go too high hall the time, sometimes it’ll go and chill around 400, but sometimes it’ll go 600, 800, 1000 and more.

Any help for this would be greatly appreciated, I’ve been trying to troubleshoot this for about two weeks now and spent quite a bit of time with my ISP to figure out if they were throttling my server IP for any reason, but that’s not the case. I even thought maybe my ONT was the issue, but we investigated and ruled that out, too.

Thank you!

p.s. the following info might also help.

Client: pfSense+ 23.05.1-RELEASE running on HP t730 Thin Client - AMD R-Series RX-427BB
OpenVPN Server: UniFi setup at my office in another country.

2

I’ve had similar issues with my VPN services too, though I believe it’s pfSense, that is the configuration.

Some things you can test out:

  • I’ve found that it’s most stable on 5.2, roll back to that and inspect the results.
  • Try TCP instead of UDP on the client, I’ve found that to be more stable on some versions of pfSense. With 23.05.1 I’m using UDP. Right now I’ve found 23.05.1 to be more stable than 2.7.
  • Swap to other VPN servers
  • You can setup a Gateway Group, put say three clients in there, if the latency is too high it should switch over to another faster one. I have this setup, it does switch over … eventually. Under the Gateway config you can tweak the thresholds.
  • Under Advanced Configuration for the openVPN client, there are a bunch of settings that you can try tweaking.
  • Under System → Advanced → Networking → Network Interfaces there are a bunch of settings you can try tweaking.

If you have a DNS leak then you obviously have a config error, to make life simpler I would setup vlans, have one for ISP and the other for VPN traffic, then you don’t have to think about it.

3

Thanks for your help, mate. And sorry for the late response, was busy with work.

  • Do you think it’s worth it even if it was working fine all this time? I did update to the latest firmware after I first faced the issue, but just want to make sure your reasoning for it. Also, can you tell me the exact version/link of it? I’m on 23.05.1 rn for pfSense+. Are you talking about 2.5.2? And does that version have an export wizard feature for OpenVPN files?

  • It’s already on TCP, it won’t even work on UDP since the VPN is set up that way.

  • Not really an option to swap to other VPN servers since this VPN server in particular isn’t a public one, but a private one. So, only one IP.

  • So, for this gateway group scenario, it’d be ideal if it’s something like a public VPN or when I have multiple servers setup, right? So, if one node of NordVPN goes down, it moves to another, and so on and on? I can see that as a band-aid solution if my ping goes high on even public servers. But even other than that, seems like a really neat feature to have failover. I’ll definitely look into that for future, but don’t believe it’ll help in this scenario.

  • I’ve peeked around in Advanced Config for OpenVPN client, but found it to no avail. Tweaking in there as well as in Networking Interfaces, it just wouldn’t do anything to stop the latency from going that high. It just makes no sense to me why it goes that high only in pfSense, when in any other environment, my ping to that server stays stable under 300, as it should.

I’ll look into setting up VLANs, but do you think a DNS leak could cause such a high ping?

As of right now, I really just want to be able to stream content over Apple TV using this server, until Apple TV adds native VPN support next month.

Once again, really appreciate you sharing your help and experience with similar issues. It helps a lot!

4

If none of those recommendations affect your config then sounds like you’re stuck. Inspect the logs see what it’s saying. DNS leak won’t affect latency, that’s your config.

Unless you own the server, you have no idea if anything has changed on it that is affecting the connection.

5

Well, swapping out to other VPNs isn’t really a solution, but I guess I can rule out if certain settings on any side of this private VPN are causing the issue.

That’s correct on owning the server; that’s why I was insistent that no changes had been made.

I’ll try this. First, I’ll try to use some public VPNs and monitor their results to see how that affects it all. After that, I’ll roll back to 5.2 and use private VPN and public both and compare the results. Cause if neither of them helps me with that, I really am stuck, and that’s a bitch.

About the version, though. Just to clarify, again, you meant to rollback and see how 2.5.2 does, correct?

6

And just now, I had this shoot up this high as well.

image
image719×224 13.4 KB

I was watching an episode on Paramount+, and all of a sudden, it started buffering, and when I looked at my gateway, my ping was going up and up.

p.s. after pausing the episode, ping went down to 400 something, tried to play it again and now it’s back high.

image
image717×229 13.9 KB

It’s just so annoying.