pfSense site-to-site mesh

Hi all,

I’m setting up network between 3 physical locations. All of them are using pfSense as main router with couple of servers available in each LAN. What I try to build can be captured in few points:

  • have remote access gateway in each location (all network traffic should be tunneled)
  • remote vpn to any location should give access to servers from all 3 networks
  • any location going offline should not affect operations of rest of the network
  • each server has it’s DNS name defined in router in location where it’s hosted

Simplified diagram:
network

Where I am at the moment:

  • 2 locations online (london, berlin)
  • site-to-site openvpn between them (servers in both LANs can see each other)
  • openvpn remote access to both (client can access only LAN of location it’s connection to)

I hope this description is not to convoluted :slight_smile: I’m not really attached to openvpn so if you guys have better ideas I’m open to suggestions.

This can be done but it will take a lot of configuration to get all the sites talking to each other and of course this will make pfsense London a sing point of failure. This is where solutions such as ZeroTier and Tailscale can make this easier to manage, but of course they come with their own risks.

1 Like

To avoid London being single point of failure I would need to configure site-to-site between each site right? How that would play out later? What would need to be set up on each site to let all of them to talk?

ZeroTier and Tailscale sounds very good, but then you need to configure each server. Doing that on routers level is transparent for all servers in the network.
I’m also aiming to have all the parts self hosted. ZT has option to do that right? But is it still paid then? If so then you pay for AWS and ZT witch is even ‘better’ deal :stuck_out_tongue:

I do believe that ZeroTier is experimental on pfSense … seems like OpenVPN is the way to go.

Am curious, why don’t you have a route from Berlin to Paris ? Seems that the routing would then be straight forward, the routing is still workable with your topology.

I didn’t find any material on putting several OpenVPN servers in a group so that there was some failover and/or redundancy, do you have some kind of plan for that ?

Sorry, I didn’t put that into drawing. I would put tunnel between Berlin and Paris. That I hope would solve problem with London as single point of failure? But Paris does not exist yet.

As for OpenVPN redundancy - no, no plans for anything more fancy than single OVPN server in main pfSense in each location.

So assuming I can just start with simple site-to-site OpenVPNs. Anything I might be missing that prevents remote client connected to London to reach IP in Berlin?

Should all work in theory.

I have two sites connected over openVPN, however, I don’t use a site-to-site, I’ve used servers and clients on both sides, so that I basically have two connections rather than the one with a site-to-site. My theory is if one tunnel goes down the other will still be up. So I might be able to then do something to bring up the other side (admittedly I’m not sure what).

Though I now have 3 such connections between different vlans, so that I have some redundancy.

I’d advocate such an approach in the event of a tunnel going down only 50% of people will be screaming at you instead of 100%.

BTW you realise that it helps to have different IP address ranges in use at each site, I do recall there is a solution on pfSense’s site outlining an approach if it’s the same.

So I do have remote access to both locations in case anything would go wrong with site-to-site tunnel.

LN ranges:
172.20.10.0/24 - lan
172.20.100.0/24 - access
172.20.101.0/24 - sts (server)

BE ranges:
172.22.10.0/24 - lan
172.22.102 - access

and just got that working LOL :smiley: Not sure if that’s best way of doing that.
In London site-to-site server I’ve added 172.22.102.0/24 to “IPv4 Remote network(s)”…
Just spotted that it might be it as I was double checking if all ranges are correct :slight_smile: