Hello, I’ve been having issues with all my gateways in pfSense. They all point to different IP addresses in the 192.168.10.0/24 network, and all those gateway IP’s are valid and have internet connectivity through them
My issue is when I reboot some of those gateways for whatever reason (maintenance, etc…) pfSense correctly and quickly detects that the gateway went down and it becomes red, indicating it’s offline. But it fails to recover when the gateway comes back online. I suspect this is some issue or bug in pfSense, because all I need to do for those gateways in red to be come green again is to change anything in the gateway config in pfSense and it immediately comes back as green, so something tells me that it’s pfSense’s monitoring that’s causing this.
In Advanced settings, I use the “kill states for all gateways which are down” option in the “State killing on gateway failure” setting - and I do not modify this default behavior in any of the configured gateways. All gateways use default dpinger configs.
Anyone have any clues as to why this is happening and how to fix it?
Update: was able to solve it and just posting in case anyone has a similar issue.
It turns out that when I rebooted the gateways, pfSense’s dpinger continued to try to ping the external IP address (e.g. 8.8.8.8) through the gateway (this is the expected behavior). However, during the reboot process of the gateways there were a few seconds while the VPN connection in them was not yet established, but the ping attempts created states in the gateway that, after the VPN tunnel got established, remained “stale” and could not go through the VPN interface. Basically the states remained there “forever” until dpinger was restarted and it created a new ping state that did go through the right VPN interface.
The way to solve this problem was to create firewall rules in the gateway’s iptables that effectively dropped all packets that needed forwarding EXCEPT for packets forwarded through the VPN network interface. With this setting in place, the ping state from the dpinger attempts never gets created while the VPN interface is down and the issue is resolved.
