HAProxy + SSL within LAN: Layer6 Handshake failure – confused on this one. Help, please?!

Networking & Firewalls
1

Hello Lawrence Systems folk!

First of all — Tom’s YouTube guides have been INVALUABLE. What a phenomenal body of work for the internet. Thank you.

Of course however I’m here because I’m stuck!

I’ve spent quite a bit of time between a rock and a hard place trying to be able to connect via SSL to one particular server running on a proxmox VM on my home LAN using HAProxy on Pfsense.

Current setup:

  • I have a a few servers running in a couple of different VLANs for different purposes: pihole, proxmox, etc.
  • Created wildcard certificate for mydomain.com / *.mydomain.com in ACME
  • Set up HAProxy backends for these servers + frontend with SSL offloading
  • I can access https://pihole.mydomain.com, https://proxmox.mydomain.com, etc. all with zero issues — happy me… except for this new problem:

On Proxmox (which I’m recently converted to), there is 1 VM (Ubuntu Server) that’s running a database index with no webUI front-end (you connect with apps).

I can use my desktop software to connect to the index today by typing the IPv4 ssl://192.168.10.3 on port 50002 and the connection is successful + traffic is encrypted using a self-signed certificate on the VM.

However, I want to connect to ssl://index.mydomain.com:50002via a reverse proxy for SSL termination, as the mobile apps I use to connect require a valid SSL certificate.

I’ve set up HAProxy backend to point index to 192.168.10.3 on port 50002 with Encrypt (SSL) checked and SSL checks unchecked.

On the frontend, I already have the Listen Address as the Proxmox VLAN interface for port 443 (for the services with webUIs), so I added a new row, same interface, with port 50002 and SSL offloading checked.

Access control list and backend are just as the webUI services, and DNS resolver is configured also.

Using this, I could never get the service to connect and 50 youtube videos later was giving up. In the end, I added a NAT rule to port forward 50002 between HAProxy on the VLAN interface (192.168.10.1) and the IP of the VM itself (192.168.10.3) — amazing! I can connect.

EXCEPT — I get man-in-the-middle warnings, handshake failure warnings… always warnings. And the mobile apps won’t connect.

Using openssl I think I have identified the problem, but I’m not sure what I’ve done wrong:

When I run openssl s_client -servername index.mydomain.com -host 192.168.10.1 -port 50002 with the port forwarding enabled, I receive confirmation of the self-signed certificate. I can connect to the server with SSL warnings that I can ignore on desktop, but not on mobile.

With port forwarding disabled & running the same query, I receive confirmation of the wildcard certificate (what I expected), but I cannot connect to the server.

It’s been several days/weeks that I’m fiddling around here with this. Any help would be much appreciated!

2

I have never tried putting a reverse proxy in front of a database connection but my guess would be that is probably needs some extra parameters either in the database setup or HAProxy.

3

Thank you for the reply, Tom!

I’ve dug into this more and it looks like, were I using nginx, this would be a stream and you’d place the conf file in streams-available, etc.

I’ve been able to verify this works from a VPS with a wireguard tunnel into the router.

Any idea on how I achieve the equivalent with HAProxy for my LAN? I suspect it’s most likely I’m missing something obvious vs. something difficult.