The goal is to be able to host services behind a CGNAT. I’ve setup HAProxy on pfsense if I’m on my LAN, HAProxy is working after I remove accept-proxy in the Frontend HAProxy external listener… Wireguard tunnel between the pfsense and the cloud vpc is also working. From the VPC I can ping and connect to local servers thru the terminal.
I’m testing on my mobile thru my phone. I’ve tested with Chromium, Firefox & Brave
when I try http://gitea.MYDOMAINNAME I get error
503 service unavailable
No serve is availabe to handle this request
which is expected because I’m not using port 80
when I try https://gitea.MYDOMAINNAME I get error
This site can't be reached
gitea.MYDOMAINNAME unexpectedly closed the connection
Try:
Checking the connection
ERR_CONNECTION_CLOSED
I’ve also tried adding DNS = 192.168.22.222 #pfsense wireguard ip to my oracle cloud VPS wireguard and then the error changed to
This site can't be reached
gitea.MYDOMAINNAME took too long to respond.
Try:
Checking the connection
ERR_TIMED_OUT
Those are the errors I’m getting and I guess HAProxy is not working properly proxying the traffic to the local haproxy thru the wireguard tunnel . So please help 
Cloud
I’ve got Oracle clouds always free tier account and setup an ubuntu 22.04 instance there.
-
on my cloud virtual network I’ve opened ports:
- 51820/udp
80/tcp
443/tcp
ssh/tcp
- 51820/udp
wireguard cloud part
- in my instance I’ve installed wireguard
sudo -i
apt update && apt upgrade && apt install wireguard wireguard-tools resolvconf -y
- created my keys
cd /etc/wireguard umask 0077; wg genkey | tee privatekey.key | wg pubkey > publickey.pub - configured my interface
vim /etc/wireguard/pfsense_lab
[Interface]
Address = 192.168.22.22/24 # oracle_wireguard tunnel IP
MTU = 1500
SaveConfig = true
PostUp = iptables -A FORWARD -i %i -j ACCEPT;
PostUp = iptables -A FORWARD -o %i -j ACCEPT;
PostUp = iptables -t nat -A POSTROUTING -o enp0s3 -j MASQUERADE
PostDown = iptables -A FORWARD -i %i -j ACCEPT;
PostDown = iptables -A FORWARD -o %i -j ACCEPT;
PostDown = iptables -t nat -D POSTROUTING -o enp0s3 -j MASQUERADE
ListenPort = 51820
PrivateKey = REDACTED # privatekey of the oracle_wireguard
[Peer]
PublicKey = 76iLP1IxfvPeGmqn0GbaS7b9i5doIrcJcUhOUoNUUhA= #pfsense wireguard
PresharedKey = REDACTED # preshared key from my pfsense wireguard
AllowedIPs = 192.168.22.0/24
- enabled the wireguard tunnel in systemd
systemctl enable --now wg-quick@pfsense_lab - enabled the ports in the firewall
ufw allow 22/tcp
ufw allow 80/tcp
ufw allow 443/tcp
ufw allow 51820/udp
ufw enable
- added the ip_forward for ipv4
echo "net.ipv4.ip_forward = 1" > /etc/sysctl.d/100-ip_forward.conf
HAProxy cloud
- after that I’ve installed haproxy
apt install haproxy -y - this is my haproxy config:
vim /etc/haproxy/haproxy.cfg
global
log /dev/log local0
log /dev/log local1 notice
chroot /var/lib/haproxy
stats socket /run/haproxy/admin.sock mode 660 level admin expose-fd listeners
stats timeout 30s
user haproxy
group haproxy
daemon
maxconn 10000
defaults
log global
mode http
option httplog
option dontlognull
timeout connect 10m
timeout client 10m
timeout server 10m
timeout http-request 10m
# I know you can use front and backend separate but I've gone with the listener and kept all in one place
listen https
bind *:443
mode tcp
option tcplog
server default 192.168.22.222:443 send-proxy
listen http
bind *:80
stats enable
stats show-node
stats uri /REDACTED #uri for the stats to be shown
stats auth $USERNAME:$PASSWORD # set your own username and password for haproxy stats
server default 192.168.22.222:80 check
- check to see if there are any problems with the configuration:
haproxy -f /etc/haproxy/haproxy.cfg -c - start and enable haproxy on boot
systemctl enable --now haproxy.service
If needed I can provide everything that I’ve done on the lan pfsense part also.