pfSense HA, redundant switches for fault tolerance with 2 ISPs?

Hey everyone, I’ve hit a roadblock and hoping someone could help me out. I am attempting to configure High Availability with 2 pfSense boxes and 2 ISPs for failover while having redundant switches between the ISP and firewalls (diagram below). Basically trying to combine THIS and THIS. (HA + fault tolerance)

I don’t have a second ISP to test with so I was using a router to simulate 2 ISPs using 2 different subnets. I only have dumb switches laying around and that ended up causing a loop.

Will the diagram below work or am I complete failure?

Also I haven’t given much thought to the managed switches that clients will be connecting to. do i need to interconnect them to both pfSense boxes or is the diagram accurate?

To do this properly you need managed switches and 2 LAGG connections from each pfsense.

1 Like

After thinking a bit, I think I might have the solution set up or at least getting closer.

Some of the things I’m questioning:

  1. Do I need to connect managed switch A to B anymore? Same goes for C and D?

  2. Do I need LAGG on Managed Switches A - D where the WAN connections are?

  • MS A - ports 5 & 6
  • MS B - ports 3 & 4
  • And so on
  1. Total number of WAN IPs
  • pfSense Main - 6 total, 3 from each ISP?
  • pfSense Failback - 6 total, 3 from each ISP?
  • Do I need to take into account any IPs for managed Switches A - D?

Why do you need those 4 managed switches (A, B, C, and D) Just connect ISP1 and ISP2 to each pfSense box and eliminate those switches entirely. Also, very unlikely those switches support MC-LAG (Multichasis Link Aggregation)…$$$$$

Aren’t the switches the required for High Availability? (CARP addresses?)

You don’t need switches ABCD, in fact having those is not contributing positively to your fault tolerance…because they aren’t necessary. Also, very unlikely your clients are going to have two NICs each to connect to two separate switches. The only reason to have two switches on the LAN side would be for redundant distribution to access switches. You might be biting off a bit more than you can chew tbh. No tutorial or guide on this subject is suggesting what you’re asking.

https://docs.netgate.com/pfsense/en/latest/recipes/high-availability.html

image

Thanks for spit balling with me. I’m trying to eliminate any single point of failure so I started looking into High Availability — Layer 2 Redundancy | pfSense Documentation which adds that redundant switch in case the main WAN switch goes down. From there I thought, well if ISP 1 goes down, I need a backup ISP so I was attempting to duplicate redundant switches for ISP 2.