I’m quite new to the homelab scene. My background is mostly C++ programming and I’ve started to develop an interest in network security. Tom’s videos have been a massive help getting started with pfSense and getting a basic lab up and running.
Currently I’m having some trouble accessing my NextCloud instance if I have an active OpenVPN client connection on pfSense. Currently my setup consists of
- 4 Networks (WAN, LAN, IoT, Isolated).
- WAN - straight forward fibre ONT from ISP
- LAN is the private internal network with the usual rules of having access to everything.
- IoT has access to everything on that particular subnet and to WAN
- Isolated can only access WAN (this is a subnet for testing / learning purposes).
- 1 VPN server (currently inactive)
- 1 VPN client (Express VPN Chicago)
I have a single client on my Isolated network running NextCloud. I have setup port forwarding rules under NAT to port 443 to that particular client and this works fine when trying to access it externally.
However as soon as I enable my Express VPN client in pfSense I am no longer able to access my NextCloud instance. I have setup some policy based routing on my IoT network (there are some clients I want to route through ExpressVPN) but I have no rules on my Isolated network to make use of that VPN.
As for the port forwarding rules (and associated firewall rules)
I also have 2 outbound NAT rules for the Isolated network
I’m not sure what else is missing. My first thought was that I might be missing some rules on the ExpressVPN client firewall rules - but the firewall logs doesn’t show anything (I have logging enabled for default blocked rules).
Could this be a missing NAT rule or some additional Firewall rules that I should be adding?
Being new to this I’m not sure where to start looking / debugging.
(Apologies for the external image links - new forum users can only embed 1 image.)
My guess would be that when turn on the Express VPN it is setting the default route to be the VPN therefore when request come in on the WAN to NextCloud it is responding out the VPN instead of the WAN. You can use pfTop under “Diagnostics” to track the connected and then make the correction in the policy routing.
Thanks for pointing me in the right direction. That does seem to be the case. Looking in pfTop I can see a connection is trying to be established. Inbound the correct gateway is being used - but Outbound there is no gateway being logged. Im not too sure what to update though. Based on my own testing and various documentation I could find - one shouldn’t set the gateway on WAN rules. I also couldn’t see any additional settings to set on the Outbound rules under NAT to force a specific gateway.
Made somewhat progress on this (I think). If I enable the “Don’t pull routes” option on the VPN client - then I am able to access my NextCloud Instance even while the VPN client is connected. However the clients I want to route through the VPN no longer routes properly (its going via the WAN).
IoT Interface Firewall Rules
and the NAT rules
Edit: Taking a step back I think the main issue is that my VPN client is messing with the gateway routing somehow. I’ve enabled the VPN client and then tried to do a TraceRt from pfSense via the LAN interface and it times out (which I expect as I have no NAT rules for LAN → Express VPN). What I don’t understand though is why is my firewall rule being ignored on the LAN interface (gateway is explicitly set to use WAN)?
I’ve finally managed to get everything working (to the credit of Tom and all the authors of the various posts I read scattered over the web). I had to check “Don’t pull routes” under the client config for ExpressVPN and “Disable Gateway Monitoring” under Routing. With “Disable Gateway Monitoring” unchecked my IoT devices would simply route through WAN.
While I am happy that all my use-cases are working - I would very much like to understand why this works. If anyone is able to provide an explanation I would be very grateful.
