Need help setting up Multi-Building Networking with one main WAN and failover at each

Hey,

So I have a question for you guys. I’m currently helping my job redoing their entire network. I’m by no means a network engineer. I just volunteered my way into this mess.

TLDR: Multi-Location; sharing primary WAN with Both over Wireless Backhaul, and Sharing or if possible load balancing independent locations backup connections if primary goes down.

Anyways, they have two different properties about a km apart. We don’t owe the property in the middle though we do have an easement on it that we could run a fiber line on if push came to shove. (This isn’t a great option atm since the cost would be high for the reward and would be a hard sell)

They currently use two starlinks at Location A facing different directions, and at Location B they have two starlinks, and one regional wireless internet provider.

My current plan is to get fiber at Location A and keep a starlink as a backup on the lowest plan at Location A, and the regional wireless provider at Location B. I want to use some form of wireless backhaul to share the fiber connection between the two buildings. (We have a fully clear LOS, and the property in the middle won’t be building anything on it.) Having fiber at both isn’t a realistic option because of the rural nature of the locations. The only reason one works out is because of where the lines run between the towns that’s being built out currently.

Currently the Cloud Key (at Location B) manages both networks, though this makes the Guest Login portal buggy at the location A (Haven’t dived into fixing this as they didn’t have a portal before and I just left it disabled. It is something I want to get working in the future but it would only be at Location A because of certain devices don’t play nice with those portals, and we just have a locked network over at Location B for those guests)

During our busiest season we have about around 400 people at Location B, and around 150 at Location A we expect this to increase to around 250-300 within the next two years though, So I’d like to make sure the network can handle that load when it’s the busiest. Location B will likely have a significantly higher bandwidth usage due to the cliental there, it will be mostly 4k video streaming, likely all at the similar times.

So to the root of the issue, I would like to ensure proper setup of this network. We don’t have anything on site that’s required access at both locations other than the cloud key (and I’m just using our static IP address for the inform currently for Location A) and I don’t expect this to change within the next few years. I would however like to have it so even the backups internet connections can share with each site, or even load balance the backup connections if the primary goes down. I know I’m digging into more enterprise stuff with this, but if we’re paying for the connections anyways it’d be nice to have them available for both sites. I’m willing to push to move to Opensense if needed though I do enjoy the benefits the UXG-Pro provides for easy tracking of bandwidth usage, and notifications of issues.

Oh last by not least I’ll have 5 static IPs available to me.

In terms of gear currently available to me

Cloud Key G2 Plus
Gateway: UXG-Pro, Edge Router X & 12, Several consumer routers
APs: ~ 5 AC Mesh Pros, ~ 25 AC Mesh
Switch: 2 US 8 60W, 4 USW Flex Mini, 5 port unmanaged Switch, 12 port unmanaged Swith.
PtMP: ~ 3 Wave AP Micro
PtP: ~ 2 airMAX Lite AP GPS, ~ 12 Wave Nano

If you’re wondering about the odd gear list, we have a roughly 200 acre outdoor area that we want to have full wifi coverage in, and this what was included in a quote from a 3rd party company that wanted to manage it. (He bought it thinking that he’d have the time to do it himself…)

Any help would be appreciated.

You can do more advanced policy routing and WAN load balancing in pfsense than you can with the UniFi equipment so that would be a better choice for the firewall. I would start with mapping all the routing and switching devices out using a tool such as https://www.drawio.com/ to get a better understanding of your setup.

This is what I currently have I believe, I’m not currently there so Its just what I can recall. But this is the jist of it. There’s several devices that connect wirelessly that I will in the future be hardwiring I just have to find the time to trace old cat 5e cables or run new ones. And while it may not be best practice I’ll likely use some flex switches at certain places to reduce the number of cable runs (Theres no way to hide the cables, Location A is an old brick building, and right now all the Fire system cables are just hung on the ceiling in a poorly managed bundle I plan to comb it, and use some cable sleeves once I get the time)

I am not sure if the UniFi system could do the routing for that but pfsense can. It could treat the PTP as a second WAN for failover.

I’ve asked around about this before on reddit and one person was telling me that doing a HA gateway is the easiest way to do this. Maybe I just need to spin up a few vms and really dive into testing it out. If get can get the fiber installed somewhat quickly I’ll likely just use an unmanaged switch between the ONT and the gateway and use one of the static IPs over at the other site for the WAN. It just looses the option of extra redundancy.