Hi All,
So i’ve been wanting to setup CARP Failover for a while with a companhy i maintain within the UK. I have followed the video guide written by Tom but cant seem to get it working, the system does not hand out DHCP but if a static address is entered it works.
I can confirm that the system syncs as i only setup all setting on the first and they instantly duplicated over where needed.
If I set a static ip fom a connected VM to either of the PF Boxes LAN ports they get a connection
Any Advise / would be very much appreciated.
My Hardware setup
I have a proxmox cluster constiting of 2 Dell R630 PVE Servers & a QDevice to keep quorum ( Again in know not recomended but this will be changing in the next 6 months to a quad cluster).
- OS - Linux 6.5.11-8-pve (2024-01-30T12:27Z)
- pve-manager/8.1.4/ec5affc9e41f1d79
- 128GB ram
- 2x 1tb sas ssd [raid 1 for redundency]
- 6x 1.2Tb sas 10Krpm
- 56 x Intel(R) Xeon(R) CPU E5-2660 v4 @ 2.00GHz (2 Sockets)
Each Dell R630 has a quad onboard GBe nic & a Quad GBe nic card in pcie slot 1 giving a total of 8 GBe.
(1 management, 3 issued to PFsense & 4 use for vairious VM on the system).
The onboard nic has been assigned to the management port & PFSense VM.
PFSENSE - 1 port WAN, 1 port LAN, 1 Port SYNC the setup has been duplicated through both machines in the cluster.
Starting from the begining here is my setup
ISP Virgin media Business - GRE Tunnel to Hitron Modem (Rather C**P but unfortionatly that what i have to work with)
They have a block of 5 static IP addresses 1 is assigned to the HiTron as the gateway leaving me 4 to play with.
— Uses for VM / Modem Gateway —
- 8!.!!.!!.248 - GRE Tunnel via Virginmedia Business
- 8!.!!.!!.249 - Hitron Gateway on modem
— Addresses usabele for LAN —
- 8!.!!.!!.250 - First free address
- 8!.!!.!!.251 - Second free address
- 8!.!!.!!.252 - Third free address [PFSense - pveB]
- 8!.!!.!!.253 - Forth free address [PFSense - pveA]
Proxmox has been setup to use port physical eno2, eno3, eno4 on both machines in cluster.
This has been configured with linux bridges. vmbr1, vmbr2, vmbr3 on both PFSense VM.
- vmbr1 - SYNC
- vmbr2 - WAN
- vmbr3 - LAN
— SYNC —
pveA - vmbr1 - has been directly linked to pveB - vmbr1. for the sync line in PFSense.
— WAN —
pveA - vmbr2 - has been linked direct to HITron Modem & given address 8!.!!.!!.253
pveB - vmbr2 - has been linked direct to HiTron Modem & given address 8!.!!.!!.252
— LAN —
pveA - vmbr3 - has been pluged directly into a unmanages switch [for testing]
pveB - vmbr3 - has been pluged into the same switch [for testing]
PFSense Setup - PFA…IP.253
The first PFSense box has been set as master,
— interface setup —
SYNC eno2/vmbr1/vtnet2 - STATIC ip - 10.20.250.253/24
WAN eno3/vmbr2/vtnet0 - STATIC ip - 8!.!!.!!.253/29
LAN eno4/vmbr3/vtnet1 - STATIC ip - 10.20.10.253/24 [DHCP ENABLED]
— Rules —
Lan pass to all
Sync Pass to all
— Virtual IP Settings —
WAN CARP - 8!.!!.!!.251/29
LAN CARP - 10.20.10.254/24
— HA Settings —
interface - SYNC
config ip - 10.20.250.252/24
username - admin
password - password. [NOT ACTUAL PASSWORD]
all sync options toggled
— NAT Settings —
Outbound Hybrid
Mapping -
- interface - WAN
- Source - Network / alias - 10.20.10.0/24
- translation address - 8!.!!.!!.251/29 [CARP ADDRESS WAN]
— DHCP Server Settings ----
Other DHCP Options
- Gateway 10.20.10.254 [CARP LAN IP]
- Failover IP - 10.20.10.252
PFSense Setup - PFB…IP.252
The second PFSense box has been set as backup,
— interface setup —
SYNC eno2/vmbr1/vtnet2 - STATIC ip - 10.20.250.252/24
WAN eno3/vmbr2/vtnet0 - STATIC ip - 8!.!!.!!.252/29
LAN eno4/vmbr3/vtnet1 - STATIC ip - 10.20.10.252/24 [DHCP ENABLED]
— Rules —
Lan pass to all
Sync Pass to all
— Virtual IP Settings —
WAN CARP - 8!.!!.!!.251/29
LAN CARP - 10.20.10.254
— NAT Settings —
Outbound Hybrid
Mapping -
- interface - WAN
- Source - Network / alias - 10.20.10.0/24
- translation address - 8!.!!.!!.251/29 [CARP ADDRESS WAN]
— DHCP Server Settings ----
Other DHCP Options
- Gateway 10.20.10.254 [CARP LAN IP]
- Failover IP - 10.20.10.253
